Data processing method and device, storage medium and electronic equipment

Abstract

The application discloses a data processing method and device, a storage medium and an electronic device. It relates to the field of cloud computing, and the method comprises the following steps: a first process is used to listen to updated data in a first database of a network controller; in the case that the updated data is listened to, the first process is used to update a second database based on the updated data, the second database being a database arranged outside a network service component; in the case that a network resource update request is received, a second process is used to obtain target network data associated with the network resource update request from the second database, and the first database is updated according to the target network data and the network resource update request. Through the application, the problem of low processing efficiency of the component caused by the fact that a single process of a network service component in a cloud computing platform is used to perform real-time data caching and request processing in the related art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of cloud computing, and in particular, to a data processing method and device, a storage medium and an electronic device. BACKGROUND

[0002] In a cloud computing environment, a network service component plays an important role, which is responsible for managing the dynamic configuration and scheduling of virtual network resources. The network service component realizes the control of the virtual network by interacting with the network controller. In the traditional architecture design of the network service component, a single process of the network service component is responsible for both real-time data caching and request processing functions. This design faces challenges in large-scale and high-density cloud computing scenarios.

[0003] In large-scale deployment, a cloud platform can need to process thousands of virtual machines and network resources, resulting in the need for the single process of the network service component to cache a large amount of data in the network controller database for high-speed access and update. At the same time, the network service component also needs to receive a large number of network resource update requests, resulting in a large number of transactions processed by the process, and the problem of low processing efficiency of the component.

[0004] In view of the above problems in the related art, no effective solution has been proposed so far. SUMMARY

[0005] The main purpose of the present application is to provide a data processing method and device, a storage medium and an electronic device to solve the problem of low processing efficiency of the component caused by the single process of the network service component in the cloud computing platform for real-time data caching and request processing in the related art.

[0006] In order to achieve the above purpose, according to one aspect of the present application, a data processing method is provided. The method is applied to a network service component in a cloud computing platform, the network service component comprising at least a first process and a plurality of second processes, comprising: listening to update data in a first database of a network controller through the first process, wherein the network controller is used to manage a virtual network according to network data in the first database, and the virtual network is created by the cloud computing platform; in the case of listening to the update data, updating a second database based on the update data through the first process, the second database being a database set outside the network service component; in the case of receiving a network resource update request, obtaining target network data associated with the network resource update request from the second database through the second process, and updating the first database according to the target network data and the network resource update request.

[0007] Further, the data processing method further comprises: determining a table name of a data table to which the update data belongs in the first database; combining the data identifier of the update data and the table name to obtain a first identifier, and determining a first key-value pair based on the first identifier and the update data; determining a second key-value pair according to the table name and the data identifier of the update data; updating the metadata table in the second database according to the first key-value pair, and updating the index table in the second database according to the second key-value pair.

[0008] Further, the data processing method further comprises: determining a table name of a data table to which the update data belongs in the first database; combining the data identifier of the update data and the table name to obtain a first identifier, and determining a first key-value pair based on the first identifier and the update data; determining a second key-value pair according to the table name and the data identifier of the update data; updating the metadata table in the second database according to the first key-value pair, and updating the index table in the second database according to the second key-value pair.

[0009] Further, the data processing method further comprises: determining a table name of a data table to which the update data belongs in the first database; combining the data identifier of the update data and the table name to obtain a first identifier, and determining a first key-value pair based on the first identifier and the update data; determining a second key-value pair according to the table name and the data identifier of the update data; updating the metadata table in the second database according to the first key-value pair, and updating the index table in the second database according to the second key-value pair.

[0010] Further, the data processing method further comprises: determining a table name of a data table to which the update data belongs in the first database; combining the data identifier of the update data and the table name to obtain a first identifier, and determining a first key-value pair based on the first identifier and the update data; determining a second key-value pair according to the table name and the data identifier of the update data; updating the metadata table in the second database according to the first key-value pair, and updating the index table in the second database according to the second key-value pair.

[0011] Further, the data processing method further comprises: determining a table name of a data table to which the update data belongs in the first database; combining the data identifier of the update data and the table name to obtain a first identifier, and determining a first key-value pair based on the first identifier and the update data; determining a second key-value pair according to the table name and the data identifier of the update data; updating the metadata table in the second database according to the first key-value pair, and updating the index table in the second database according to the second key-value pair.

[0012] Further, the data processing method further comprises: determining a table name of a data table to which the update data belongs in the first database; combining the data identifier of the update data and the table name to obtain a first identifier, and determining a first key-value pair based on the first identifier and the update data; determining a second key-value pair according to the table name and the data identifier of the update data; updating the metadata table in the second database according to the first key-value pair, and updating the index table in the second database according to the second key-value pair.

[0013] In order to achieve the above object, according to another aspect of the present application, a data processing apparatus is provided. The apparatus comprises: a processing module configured to listen to update data in a first database of a network controller by a first process, wherein the network controller is configured to manage a virtual network according to network data in the first database, and the virtual network is created by a cloud computing platform; a first updating module configured to update a second database based on the update data by the first process in a case that the update data is listened to, and the second database is a database arranged outside a network service component; and a second updating module configured to obtain target network data associated with a network resource update request from the second database according to the target network data and the network resource update request by a second process in a case that the network resource update request is received, and update the first database according to the target network data and the network resource update request.

[0014] Further, the processing module further comprises: a first determining submodule configured to determine a table name of a data table to which the update data belongs in the first database; a second determining submodule configured to combine a data identifier of the update data and the table name to obtain a first identifier, and determine a first key-value pair based on the first identifier and the update data; and a third determining submodule configured to determine a second key-value pair according to the table name and the data identifier of the update data; and a first updating submodule configured to update a metadata table in the second database according to the first key-value pair, and update an index table in the second database according to the second key-value pair.

[0015] Further, the second determining submodule further comprises: a first judging submodule configured to judge whether the data content of the update data comprises a first attribute, wherein the first attribute refers to an attribute whose attribute value is the same as a preset value; a deleting submodule configured to delete the attribute value of the first attribute in the update data to obtain updated update data in a case that the data content of the update data comprises the first attribute; and a fourth determining submodule configured to determine the first key-value pair based on the first identifier and the updated update data.

[0016] Further, the second updating module further comprises: a second judging submodule configured to determine target update data according to the network resource update request, and judge whether network data with the same table name and data identifier as the target update data exists in the second database; a fifth determining submodule configured to determine the network data as the target network data in a case that the network data with the same table name and data identifier as the target update data exists in the second database; and a first storing submodule configured to store the target network data in the second database to a first storage area in the second process.

[0017] Further, the second updating module further comprises: a third judging sub-module, configured to judge whether the data content of the target updating data and the target network data is same; a second storing sub-module, configured to store the target updating data into a second storage area in the second process in the case that the data content of the target updating data and the network data is different; and a second updating sub-module, configured to update the first database based on the data in the second storage area.

[0018] Further, the third judging sub-module further comprises: a first judging unit, configured to judge whether there is a second attribute in the target network data, wherein the second attribute refers to an attribute with a null attribute value; an updating unit, configured to update the attribute value of the second attribute in the target network data according to a preset value to obtain updated target network data in the case that there is the second attribute in the target network data; and a second judging unit, configured to judge whether the data content of the target updating data and the updated target network data is same.

[0019] Further, the data processing apparatus further comprises: a third updating module, configured to update target data in the third database based on the data in the second database which is updated in the case that the second database is updated through the third process, wherein the target data and the data in the second database which is updated have a dependent relationship.

[0020] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer readable storage medium is provided, which comprises a stored executable program, wherein the computer readable storage medium controls the device where the computer readable storage medium is located to execute the above-mentioned data processing method when the executable program is running.

[0021] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an electronic device is provided, which comprises a memory storing an executable program, and a processor configured to run the program, wherein the program executes the above-mentioned data processing method when running.

[0022] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer program product is provided, which comprises computer instructions, and the computer instructions are executed by a processor to implement the steps of the above-mentioned data processing method.

[0023] In the embodiment of the present application, by setting the first process to listen to the update data in the first database of the network controller, updating the second database in the case of listening to the update data, and setting the second process to update the first database based on the content in the second database and the network resource update request in the case of receiving the network resource update request, compared with the related art, the interaction mode between the network service component and the first database of the network controller in the cloud computing platform is changed, and the mutual independent processing of the data listening and caching and the request transaction processing is realized. Therefore, the work content of the single process can be reduced, and the memory occupation amount when each second process respectively caches the data can be avoided, and thus the processing efficiency of the network service component can be effectively improved.

[0024] Therefore, the method provided in the present application achieves the purpose of data caching and request processing by different processes, realizes the technical effect of improving the processing efficiency of the network service component, and solves the technical problem of low processing efficiency of the component in the related art caused by the single process of the network service component in the cloud computing platform for real-time data caching and request processing. BRIEF DESCRIPTION OF DRAWINGS

[0025] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and the illustrative embodiments of the present application and their description serve the purpose of explaining the present application. The accompanying drawings in conjunction with the present application provide a further understanding of the present application. In the drawings:

[0026] Figure 1 is a hardware structure block diagram of a computer terminal according to an embodiment of the present application;

[0027] Figure 2 is a flowchart of a data processing method according to an embodiment of the present application;

[0028] Figure 3 is a working schematic diagram of a network service component according to an embodiment of the present application;

[0029] Figure 4 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;

[0030] Figure 5 is a structure block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to enable personnel in the technical field to better understand the scheme of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0032] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0033] It should be noted that the collected information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present application are information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, necessary security measures are taken, do not violate public order and good customs, and provide corresponding operation portal for user to choose authorization or refusal. For example, the system and the interface between the related users or institutions provide the user with a corresponding operation portal for the user to choose to agree or refuse the automatic decision result; if the user chooses to refuse, the expert decision process is entered.

[0034] Embodiment 1

[0035] According to the embodiments of the present application, an embodiment of a data processing method is also provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0036] The method embodiment provided by the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a data processing method is shown. As shown inFigure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0037] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0038] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the aforementioned data processing method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0039] The transmission device 106 is configured to receive or send data via a network. The network can include a wireless network provided by a communication provider of the computer terminal 10. In an example, the transmission device 106 includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In an example, the transmission device 106 can be a radio frequency (RF) module that is configured to communicate with the Internet wirelessly.

[0040] The display can be a liquid crystal display (LCD) that is touch screen, for example, which can enable a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0041] In the above operating environment, the present application provides a data processing method as shown in Figure 2 Figure 2 is a flowchart of the data processing method according to Embodiment 1 of the present application. The method is applied to a network service component in a cloud computing platform. The network service component includes at least a first process and a plurality of second processes, as shown in Figure 2

[0042] In step S201, the first process is configured to listen to updated data in a first database of a network controller. The network controller is configured to manage a virtual network according to network data in the first database. The virtual network is created by the cloud computing platform.

[0043] Optionally, the cloud computing platform can provide an infrastructure as a service function to build a private cloud and a public cloud. It can manage a large number of computing, storage and network resources, and abstract these resources through virtualization technology, so that users can obtain and release resources on demand. For example, a user can quickly create a virtual machine (computing resource) through an interface of the cloud computing platform, allocate a storage space (storage resource), and configure a network connection (network resource).

[0044] Optionally, the cloud computing platform includes a plurality of components, for example, a component responsible for computing services and managing the life cycle of a virtual machine; a component providing block storage services for persistently storing data of a virtual machine; and a network service component. The network service component is responsible for network services, which can create and manage a virtual network, and enable communication between virtual machines and connection between a virtual machine and an external network.

[0045] ​​Optionally, the network controller aims to provide a high-performance, distributed network control plane for virtualization / containerization environment. The network controller provides a distributed way to manage and control virtual networks. The first database of the network controller can be a database for storing long-term unchanged network configurations, or the first database of the network controller can be a database for storing short-term network state information. The network controller manages virtual networks relying on network data in the first database, and the virtual networks are created by a cloud computing platform. The network controller is independent of the network service component.

[0046] In the related art, a plurality of API (Application Programming Interface) processes are included in the network service component, each of which is responsible for processing API requests (i.e., network resource update requests) and updating the first database of the network controller according to the requests. In order to realize differentiated updating of data in the first database, the API processes adopt a way of listening to changes in the first database and caching data in the first database to the memory, so as to realize fast acquisition and comparison of related data in the process of processing network resource update requests. In a large-scale high-density scenario, the network service component needs to configure a large number of API processes to improve the concurrent processing capability of the service, but each API process needs to cache data in the first database, which occupies a large amount of memory. When the data in the first database changes, each API process is notified to process, which occupies a lot of CPU. And each API process is a client of the first database, and with the increase of the API processes, the processing pressure of the first database will increase, causing some cluster message processing delay and affecting the stability of the network controller. In addition, the API processes are also responsible for listening to data processing after the data in the first database is updated, which affects the service of processing the first database and causes slow response of the API processes.

[0047] Therefore, in the present application, a plurality of processes are arranged in the network service component, and the caching function is externalized to improve the data processing efficiency of the network service component.

[0048] Optionally, the first process in the embodiment is used to listen to data updates in the first database. The first process continuously monitors the updated data in the first database, which can come from events such as creation, deletion, and network configuration change of virtual machines. When the data in the database changes, the first process will immediately capture the updated data to ensure that the network service component can respond in time.

[0049] Step S202, in the case of listening to the updated data, updating the second database based on the updated data through the first process, the second database being a database arranged outside the network service component.

[0050] Optionally, once the first process listens to the update data, the first process synchronizes the data into a second database. The second database can be an in-memory type database, and the second database is a database that is set outside the network service component. In fact, the first database belongs to the network controller, and the first database is also a database that is set outside the network service component.

[0051] Optionally, the second database can be set on the same physical node (i.e. physical server) as the network service component.

[0052] In step S203, the second process acquires target network data associated with the network resource update request from the second database in a case where the network resource update request is received, and updates the first database according to the target network data and the network resource update request.

[0053] Optionally, the network resource update request can be sent by a system other than a user or a cloud computing platform. In a network virtualization and cloud computing environment, the network resource update request is a request for modifying, adding or deleting a network resource, and the network resource includes but is not limited to a virtual network, a virtual router, a virtual switch, a port, a firewall rule, a load balancer, a network address translation rule, a network service, etc. For example, the network resource update request is for adding or deleting a virtual port, modifying an IP address, etc.

[0054] When the second process receives the network resource update request, the second process acquires target network data associated with the network resource update request from the second database, for example, acquires a current value of data requested to be updated by the network resource update request as the target network data. Then, the first database is updated according to the target network data and a target value of data requested to be updated in the network resource update request.

[0055] Optionally, after the first database is updated, the network controller can manage the virtual network according to the updated data in the first database.

[0056] Optionally, the above-mentioned manner can not only improve the processing efficiency of the network service component, but also can avoid the problem that each second process needs to interact with the first database when each second process respectively caches data, which leads to a large processing pressure of the first database, thereby also improving the stability of the first database.

[0057] In the embodiment of the present application, by setting the first process to listen to the update data in the first database of the network controller, updating the second database in the case of listening to the update data, and setting the second process to update the first database based on the content in the second database and the network resource update request in the case of receiving the network resource update request, compared with the related art, the interaction mode between the network service component and the first database of the network controller in the cloud computing platform is changed, and the mutual independent processing of data listening and caching and request transaction processing is realized. Therefore, the work content of a single process can be reduced, and the memory occupation amount when each second process respectively caches data can be avoided, and thus the processing efficiency of the network service component can be effectively improved.

[0058] Therefore, the method provided in the present application achieves the purpose of data caching and request processing by different processes, realizes the technical effect of improving the processing efficiency of the network service component, and solves the technical problem of low component processing efficiency caused by the single process of the network service component in the cloud computing platform for real-time data caching and request processing in the related art.

[0059] Optionally, in the data processing method provided in the embodiment of the present application, the first process updates the second database based on the update data, including: determining the table name of the data table to which the update data belongs in the first database; combining the data identifier of the update data and the table name to obtain a first identifier, and determining a first key-value pair based on the first identifier and the update data; determining a second key-value pair according to the table name and the data identifier of the update data; updating the metadata table in the second database according to the first key-value pair, and updating the index table in the second database according to the second key-value pair.

[0060] Optionally, the first identifier can be composed of the table name and the data identifier of the update data, so that the first identifier is used as a key and the update data is used as a value to form the first key-value pair for locating specific metadata records. For example, the first key-value pair can be: “Logical_Switch_757e906f…:{“acls”:””,“external_ids”:””…}”, wherein “Logical_Switch_757e906f…” is equivalent to the key, and “{“acls”:””,“external_ids”:””…}” is equivalent to the value.

[0061] Optionally, the table name is used as a key and the data identifier of the update data is used as a value to form the second key-value pair for index fast search. For example, the second key-value pair can be “Chassis_Private:(99f9…,15e377…)”, wherein “Chassis_Private” is equivalent to the key, and “(99f9…,15e377…)” is equivalent to the value.

[0062] Optionally, the metadata table in the second database is updated according to the first key-value pair. For example, if there is data in the metadata table that matches the key of the first key-value pair, the data is updated; if there is no data in the metadata table that matches the key of the first key-value pair, the first key-value pair is added as new data in the metadata table.

[0063] Optionally, the index table in the second database is updated according to the second key-value pair. For example, if there is data in the index table that matches the second key-value pair, the second database is not updated; if there is no data in the index table that matches the second key-value pair, the second key-value pair is added as new data in the index table.

[0064] Optionally, the purpose of updating the index table is to maintain the index structure of the metadata, ensuring that when a global query or all data in a certain table is needed, the data can be quickly located and obtained, thereby improving the overall processing capacity of the component.

[0065] It should be noted that by separating the metadata update and the index update, the storage space requirement of the in-memory database can be effectively reduced, and the performance of data retrieval can be optimized, thereby further improving the processing efficiency of the component.

[0066] Optionally, in the data processing method provided in the embodiments of the present application, the first key-value pair is determined based on the first identifier and the update data, including: determining whether the data content of the update data includes a first attribute, wherein the first attribute refers to an attribute whose attribute value is the same as a preset value; in the case where the data content of the update data includes the first attribute, removing the attribute value of the first attribute in the update data to obtain updated update data; and determining the first key-value pair based on the first identifier and the updated update data.

[0067] Optionally, after determining the update data, the first process can determine whether it includes the first attribute. For example, in the case where the updated value of an attribute is the same as the preset value corresponding to the attribute, it is determined that the attribute is the first attribute; in the case where the updated value of an attribute is different from the preset value corresponding to the attribute, it is determined that the attribute is not the first attribute.

[0068] Optionally, in the case where the update data includes the first attribute, the first process can remove the values of these attributes to obtain updated update data, so as to take the first identifier as the key and the updated update data as the value to obtain the first key-value pair.

[0069] Optionally, the second process no longer caches data in the first database by itself, so the method of the second process originally looking for data and submitting data needs to be changed. In order to reduce the changes of the original code and avoid introducing problems due to modifying the logic of the original code, some new mechanisms are designed to achieve the scheme design without modifying the original business processing logic. For example, before the actual application of the second process, the method ovsdbapp.backend.ovs_idl.idlutils.row_by_value used to look up data from the memory of the second process itself is replaced by a method for obtaining data from the second database through a related protocol. The obtained data (for example, the target network data described above) is instantiated as a new MemRow object. The MemRow object inherits from the class ovs.id.Row used to obtain data from the memory of the second process itself, and the data will be cached according to the data structure of the class. And the following changes are made:

[0070] I. After the MemRow object is instantiated, it is automatically added to the class variable rows of the newly designed transaction class MemTransaction, which is used to define how to submit the data content to be updated to the first database.

[0071] II. The attribute_data is added to save the original data (i.e., the target network data) obtained from the second database.

[0072] III. The attribute_updates is added to save the data to be updated to the first database.

[0073] IV. The method__getattr__ is overloaded. When querying the data cached in the local memory of the second process, first query whether the data has a value in_updates, and if not, query whether it has a value in_data. If the queried value is a UUID (Universally Unique Identifier) resource type, first query whether it is cached in MemTransaction.rows, and if it is cached, return directly, and if it is not cached, obtain it from the second database. If the queried value is of other types, return the data in_data directly.

[0074] It should be noted that by excluding those attribute values in the default state and then storing the updated update data to the second database, the data storage strategy is effectively optimized, and unnecessary data redundancy and memory occupation are reduced.

[0075] Optionally, in the data processing method provided in the embodiments of the present application, the target network data associated with the network resource update request is obtained from the second database, including: determining the target update data according to the network resource update request, and judging whether the network data with the same table name and data identifier as the target update data exists in the second database; in the case that the network data with the same table name and data identifier as the target update data exists in the second database, the network data is determined as the target network data; and the target network data in the second database is stored into the first storage area in the second process.

[0076] In an optional embodiment, when the second process receives the network resource update request, it first parses the request content, determines the network resource to be updated (i.e. the target update data) and its corresponding position in the first database, the corresponding position being the table name and data identifier of the target update data. Then, the second process detects whether the corresponding cache record of the target update data exists in the second database, i.e. judges whether the network data with the same table name and data identifier as the target update data exists in the second database.

[0077] Optionally, if the corresponding cache record of the target update data exists in the second database, the cache record is determined as the target network data, so as to store the target network data in the second database into the first storage area in the second process.

[0078] In an optional embodiment, when the data of a certain network resource is updated, the data of other network resources may also need to be updated based on the update of the former. For example, when the data (e.g. output port) of network resource A is updated, the data (e.g. the port number of A recorded when A establishes a communication connection with B) of network resource B also needs to be updated. And in some business scenarios, the update of the data of other network resources not only depends on the update of the former, but also depends on the update of other network resources. For example, when the data of network resource A is updated, not only the update content of A needs to be known, but also the data of network resources C and D need to be known, so as to update the data of network resource B based on the update content of A and the data of C and D.

[0079] Therefore, based on the diversity of business scenarios, optionally, the second process can also determine the target update data according to the network resource update request, and determine whether the network data associated with the target update data exists in the second database, for example, the data required for updating other network resources (such as the data of B described above) based on the target update data (such as the update content of A described above) is determined as the network data (such as the data of C and D described above) associated with the target update data. Other network resources refer to network resources whose data update depends on the target update data, and the data of the other network resources has been recorded in the first database. Optionally, in the case where the network data associated with the target update data exists in the second database, the network data associated with the target update data is also determined as the target network data.

[0080] That is, the target network data can be only the historical cache of the target update data, or the historical cache of the target update data and the network data associated with the target update data.

[0081] Optionally, in the case where the network data with the same table name and data identifier as the target update data does not exist in the second database, the target update data is determined as a new data, and in this case, the first database can be directly updated based on the target update data, such as directly writing the target update data into the first database.

[0082] It should be noted that by the above method, the target network data is accurately obtained, thereby improving the reliability of updating the first database.

[0083] Optionally, in the data processing method provided in the embodiments of the present application, the first database is updated according to the target network data and the network resource update request, comprising: determining whether the data content of the target update data is the same as that of the target network data; in the case where the data content of the target update data is different from that of the network data, storing the target update data into a second storage area in the second process; and updating the first database based on the data in the second storage area.

[0084] Optionally, the update requested in the network resource update request is sometimes used to set the attribute value to a certain fixed value, in which case, the update requested in the network resource update request can be the same as the existing historical cache, and if the two sets of data are completely the same, it means that the first database does not need to be updated, so as to avoid unnecessary write operations. Therefore, in the present embodiment, it can be determined whether the data content of the target update data is the same as that of the target network data.

[0085] Optionally, in the case where the data content of the target update data is different from that of the network data, the target update data is stored into a second storage area (such as the updates described above) in the second process.

[0086] In the updating, the first database is updated based on the data in the second storage area.

[0087] Optionally, in the case that the target update data is identical in data content to the network data, the target update data is prohibited from being stored into the second storage area in the second process. Thus, in the updating, if there is no data in the second storage area, the first database is not updated.

[0088] In an optional embodiment, if the data of other network resources is updated in dependence of the update of the target update data, the second process can determine the update data of the other network resources based on the target update data, and then store the update data of the other network resources into the second storage area in the second process, so as to realize the update of the data of the other network resources together in the updating.

[0089] In an optional embodiment, before the second process is applied, a transaction class MemTransaction can be designed to inherit from ovs.db.id.Transaction, and the transaction class MemTransaction is a class for transactions of updating the first database. And the following changes are made:

[0090] I. A class variable rows is added. The class variable is a thread and coroutine safe variable, and is a HASH table. The class is used to save the data in the Updates. The variable is reset each time a MemTransaction is created and the MemTransaction is committed.

[0091] II. The do_commit and commit methods are overloaded. The do_commit and commit methods are used to traverse all the data in the class variable rows, assemble the data to be updated of the first database according to the changed data, and send the data to the first database.

[0092] It should be noted that, through the above method, only the data that is really changed is updated when the first database is updated, and repeated writing of unchanged data is avoided, so that the processing efficiency of the component can be improved.

[0093] Optionally, in the data processing method provided in the embodiments of the present application, the method of judging whether the data content of the target update data is identical to the target network data includes: judging whether the second attribute exists in the target network data, wherein the second attribute refers to an attribute with a null attribute value; in the case that the second attribute exists in the target network data, updating the attribute value of the second attribute in the target network data according to a preset value to obtain updated target network data; and judging whether the data content of the target update data is identical to the updated target network data.

[0094] Optionally, since in the second database, in order to reduce the memory occupation, the attribute value same as the preset value of the attribute is set as null, therefore, after obtaining the target network data, the second process can determine whether the second attribute exists in the target network data.

[0095] Optionally, in the case that the second attribute exists in the target network data, the second process can determine that the attribute value of the second attribute should actually be the preset value, therefore, the second process can update the attribute value of the second attribute in the target network data according to the preset value, to obtain the updated target network data.

[0096] Optionally, after obtaining the updated target network data, it is determined whether the data content of the target update data and the updated target network data is same.

[0097] It should be noted that, through the above manner, the accuracy of the determination result can be effectively improved, thereby improving the accuracy of the update of the first database and improving the component processing efficiency.

[0098] Optionally, in the data processing method provided in the embodiments of the present application, the network service component further comprises a third process, and the method further comprises: updating, by the third process, target data in the third database based on the data updated in the second database in the case that the second database is updated, wherein the target data has a dependency relationship with the data updated in the second database.

[0099] Optionally, the third process is used for external linkage processing. That is, for the data outside the network service component and the network controller, a part of the data can also need to be updated due to the update of the data in the first database, therefore, the data in the third database which has a dependency relationship with the data updated in the second database is determined as the target data, and the data updated in the second database is essentially same as the data updated in the first database. For example, the third database can be used for storing the data of the network resource in the system outside the system where the network service component and the network controller are located.

[0100] Optionally, the third process updates the target data in the third database based on the data updated in the second database in the case that the second database is updated, to realize the linkage processing. The processing logic of the linkage processing can be preset in the third process.

[0101] It should be noted that, through the above manner, the external linkage is realized through the independent process in the network service component, thereby the component processing efficiency can be further improved.

[0102] In an optional embodiment, Figure 3is a working schematic diagram of a network service component provided by an embodiment of the present application, as shown in Figure 3 The first process in the network service component listens to update data in the first database of the network controller, and updates the second database based on the update data through the first process in the case of listening to the update data. The second process obtains target network data associated with the network resource update request from the second database in the case of receiving the network resource update request, and updates the first database according to the target network data and the network resource update request. The third process obtains relevant data from the second database in the case of listening to the second database update, and updates target data in the third database based on the data updated in the second database, to realize the linkage update.

[0103] Therefore, the method provided by the present application achieves the purpose of data caching and request processing through different processes, realizes the technical effect of improving the processing efficiency of the network service component, and solves the technical problem of low processing efficiency of the component in the related art, in which a single process of the network service component in the cloud computing platform performs real-time data caching and request processing, resulting in that the process handles many transactions, thereby existing the technical problem of low processing efficiency of the component.

[0104] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0105] Embodiment 2

[0106] The embodiment of the present application also provides a data processing apparatus. It should be noted that the data processing apparatus of the embodiment of the present application can be used to execute the data processing method provided by the embodiment of the present application. The data processing apparatus provided by the embodiment of the present application is introduced as follows.

[0107] According to the embodiment of the present application, a device for implementing the above-mentioned data processing method is also provided, as shown in Figure 4 The device comprises:

[0108] The processing module 401 is configured to listen to update data in the first database of the network controller through the first process, wherein the network controller is configured to manage a virtual network according to network data in the first database, and the virtual network is created by a cloud computing platform.

[0109] The first updating module 402 is configured to update the second database based on the update data through the first process in the case of listening to the update data, and the second database is a database arranged outside the network service component.

[0110] The second updating module 403 is configured to, in the case that the network resource updating request is received, acquire target network data associated with the network resource updating request from the second database through the second process, and update the first database according to the target network data and the network resource updating request.

[0111] In the embodiments of the present application, the first process is arranged to listen to the update data in the first database of the network controller, the second database is updated in the case that the update data is listened to, and the second process is arranged to update the first database based on the content in the second database and the network resource updating request in the case that the network resource updating request is received. Compared with the related art, the interaction mode between the network service component and the first database of the network controller in the cloud computing platform is changed, the mutual independent processing of the data listening and caching and the request transaction processing is implemented. Therefore, the work content of a single process can be reduced, and the memory occupation amount when each second process respectively caches data can be avoided, and thus the processing efficiency of the network service component can be effectively improved.

[0112] It can be seen that the method provided in the present application achieves the purpose of data caching and request processing through different processes, and achieves the technical effect of improving the processing efficiency of the network service component, and solves the technical problem of low component processing efficiency caused by the real-time data caching and request processing of the single process of the network service component in the cloud computing platform in the related art.

[0113] Optionally, in the data processing apparatus provided in the embodiments of the present application, the processing module further includes: a first determining sub-module, configured to determine the table name of the data table to which the update data belongs in the first database; a second determining sub-module, configured to combine the data identifier of the update data and the table name to obtain a first identifier, and determine a first key-value pair based on the first identifier and the update data; a third determining sub-module, configured to determine a second key-value pair according to the table name and the data identifier of the update data; and a first updating sub-module, configured to update the metadata table in the second database according to the first key-value pair, and update the index table in the second database according to the second key-value pair.

[0114] Optionally, in the data processing apparatus provided in the embodiments of the present application, the second determining sub-module further includes: a first judging sub-module, configured to judge whether the data content of the update data includes a first attribute, wherein the first attribute refers to an attribute whose attribute value is the same as a preset value; a deleting sub-module, configured to delete the attribute value of the first attribute in the update data to obtain updated update data in the case that the data content of the update data includes the first attribute; and a fourth determining sub-module, configured to determine the first key-value pair based on the first identifier and the updated update data.

[0115] Optionally, in the data processing apparatus provided in the embodiment of the present application, the second updating module further comprises: a second judging submodule, configured to determine target updating data according to the network resource updating request, and determine whether network data with the same table name and data identifier as the target updating data exists in the second database; a fifth determining submodule, configured to determine the network data as target network data in the case that network data with the same table name and data identifier as the target updating data exists in the second database; and a first storing submodule, configured to store the target network data in the second database to the first storage area in the second process.

[0116] Optionally, in the data processing apparatus provided in the embodiment of the present application, the second updating module further comprises: a third judging submodule, configured to determine whether the data content of the target updating data is the same as that of the target network data; a second storing submodule, configured to store the target updating data to the second storage area in the second process in the case that the data content of the target updating data is different from that of the network data; and a second updating submodule, configured to update the first database based on the data in the second storage area.

[0117] Optionally, in the data processing apparatus provided in the embodiment of the present application, the third judging submodule further comprises: a first judging unit, configured to determine whether a second attribute exists in the target network data, wherein the second attribute refers to an attribute with a null attribute value; an updating unit, configured to update the attribute value of the second attribute in the target network data according to a preset value to obtain updated target network data in the case that the second attribute exists in the target network data; and a second judging unit, configured to determine whether the data content of the target updating data is the same as that of the updated target network data.

[0118] Optionally, in the data processing apparatus provided in the embodiment of the present application, the data processing apparatus further comprises: a third updating module, configured to update target data in the third database based on the data updated in the second database in the case that the second database is updated through the third process, wherein the target data has a dependent relationship with the data updated in the second database.

[0119] It should be noted that the processing module 401, the first updating module 402 and the second updating module 403 correspond to steps S201 to S203 in Embodiment 1, and the three modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware components or software components stored in the memory (for example, the memory 104) and processed by one or more processors (for example, the processors 102a, 102b, …, 102n), and the above modules can also be run in the computer terminal 10 provided in Embodiment 1 as a part of the apparatus.

[0120] Embodiment 3

[0121] Embodiments of the present application can provide an electronic device, Figure 5 is a structural block diagram of an electronic device according to an embodiment of the present application. As shown in the figure, the electronic device can include one or more (only one is shown in the figure) processors 1002, a memory 1004, a storage controller, and a peripheral interface, wherein the peripheral interface is connected with a radio frequency module, an audio module, and a display. Figure 5 Figure 5 The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functions and data processing by running the software programs and modules stored in the memory, that is, implements the above-mentioned methods. The memory can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the terminal through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0122] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: listening to update data in a first database of a network controller by a first process, wherein the network controller is used to manage a virtual network according to network data in the first database, and the virtual network is created by a cloud computing platform; in the case of listening to the update data, updating a second database based on the update data by the first process, the second database being a database arranged outside a network service component; in the case of receiving a network resource update request, obtaining target network data associated with the network resource update request from the second database by a second process, and updating the first database according to the target network data and the network resource update request.

[0123] The processor can further call information and application programs stored in the memory through the transmission device to perform the following steps: determining a table name of a data table to which the update data belongs in the first database; combining a data identifier of the update data and the table name to obtain a first identifier, and determining a first key-value pair based on the first identifier and the update data; determining a second key-value pair according to the table name and the data identifier of the update data; updating a metadata table in the second database according to the first key-value pair, and updating an index table in the second database according to the second key-value pair.

[0124] The processor can further call information and application programs stored in the memory through the transmission device to perform the following steps: determining a table name of a data table to which the update data belongs in the first database; combining a data identifier of the update data and the table name to obtain a first identifier, and determining a first key-value pair based on the first identifier and the update data; determining a second key-value pair according to the table name and the data identifier of the update data; updating a metadata table in the second database according to the first key-value pair, and updating an index table in the second database according to the second key-value pair.

[0125] ​The processor can further call information and application programs stored in the memory through the transmission device to execute the following steps: determining whether the first attribute is included in the data content of the update data, wherein the first attribute refers to an attribute with a same attribute value as a preset value; in a case where the first attribute is included in the data content of the update data, deleting the attribute value of the first attribute in the update data to obtain updated update data; and determining the first key-value pair based on the first identifier and the updated update data.

[0126] The processor can further call information and application programs stored in the memory through the transmission device to execute the following steps: determining target update data according to the network resource update request, and determining whether network data with a same table name and data identifier as the target update data exists in the second database; in a case where network data with a same table name and data identifier as the target update data exists in the second database, determining the network data as target network data; and storing the target network data in the second database to the first storage area in the second process.

[0127] The processor can further call information and application programs stored in the memory through the transmission device to execute the following steps: determining whether the data content of the target update data is same as that of the target network data; in a case where the data content of the target update data is different from that of the network data, storing the target update data to the second storage area in the second process; and updating the first database based on the data in the second storage area.

[0128] The processor can further call information and application programs stored in the memory through the transmission device to execute the following steps: determining whether the second attribute exists in the target network data, wherein the second attribute refers to an attribute with a null attribute value; in a case where the second attribute exists in the target network data, updating the attribute value of the second attribute in the target network data according to a preset value to obtain updated target network data; and determining whether the data content of the target update data is same as that of the updated target network data.

[0129] The processor can further call information and application programs stored in the memory through the transmission device to execute the following steps: based on the data updated in the second database, updating target data in the third database through the third process in a case where the second database is updated, wherein the target data has a dependency relationship with the data updated in the second database.

[0130] In this embodiment, by configuring a first process to monitor updated data in the first database of the network controller and update the second database upon detecting updated data, and by configuring a second process to update the first database based on the content in the second database and the network resource update request upon receiving a network resource update request, the interaction mode between the network service component and the first database of the network controller in the cloud computing platform is changed compared to related technologies. This achieves independent processing of data monitoring and caching, and request transaction processing. This reduces the workload of a single process and avoids the memory consumption when each second process caches data separately, thereby effectively improving the processing efficiency of the network service component.

[0131] Therefore, the method provided in this application achieves the goal of data caching and request processing through different processes, thereby improving the processing efficiency of network service components. It solves the technical problem in related technologies where a single process of the network service component in the cloud computing platform performs real-time data caching and request processing, resulting in the process handling many transactions and thus low component processing efficiency.

[0132] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0133] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0134] Example 4

[0135] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the data processing method provided in Embodiment 1.

[0136] Optionally, in the embodiment, the storage medium can be located in any one of computer terminals in a computer terminal group in a computer network, or in any one of mobile terminals in a mobile terminal group.

[0137] The application also provides a computer program product, which is suitable for executing the steps of the data processing method when executed on a data processing device.

[0138] The above-mentioned serial numbers of the embodiments of the application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0139] In the above-mentioned embodiments of the application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0140] In the several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the unit described as the division is only a logical division, and there can be another division way during actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or communication connection between each set can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0141] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

[0142] In addition, each functional unit in each embodiment of the application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0143] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0144] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A data processing method, characterized in that, A network service component applied in a cloud computing platform, the network service component comprising at least a first process and multiple second processes, the method comprising: The first process monitors updated data in the first database of the network controller, wherein the network controller manages virtual networks based on network data in the first database, and the virtual networks are created by the cloud computing platform. Upon detecting the updated data, the first process updates the second database based on the updated data. The second database is a database set outside the network service component. Upon receiving a network resource update request, the second process retrieves target network data associated with the network resource update request from the second database and updates the first database based on the target network data and the network resource update request.

2. The method according to claim 1, characterized in that, Updating the second database based on the updated data through the first process includes: Determine the table name of the data table in the first database to which the updated data belongs; By combining the data identifier of the updated data and the table name, a first identifier is obtained; and a first key-value pair is determined based on the first identifier and the updated data. The second key-value pair is determined based on the table name and the data identifier of the updated data; Update the metadata table in the second database according to the first key-value pair, and update the index table in the second database according to the second key-value pair.

3. The method according to claim 2, characterized in that, Determining the first key-value pair based on the first identifier and the updated data includes: Determine whether the updated data content includes a first attribute, wherein the first attribute refers to an attribute whose attribute value is the same as a preset value; If the updated data includes a first attribute, delete the attribute value of the first attribute in the updated data to obtain the updated data. The first key-value pair is determined based on the first identifier and the updated data.

4. The method according to claim 1, characterized in that, Retrieving target network data associated with the network resource update request from the second database includes: The target update data is determined based on the network resource update request, and it is determined whether there is network data in the second database that has the same table name and data identifier as the target update data; If network data with the same table name and data identifier as the target updated data exists in the second database, then that network data is identified as the target network data. The target network data in the second database is stored in the first storage area of ​​the second process.

5. The method according to claim 4, characterized in that, Updating the first database based on the target network data and the network resource update request includes: Determine whether the data content of the target updated data is the same as that of the target network data; If the target update data differs from the data content of the network data, the target update data is stored in the second storage area of ​​the second process; The first database is updated based on the data in the second storage area.

6. The method according to claim 5, characterized in that, Determining whether the target updated data and the target network data have the same data content includes: Determine whether a second attribute exists in the target network data, wherein the second attribute refers to an attribute whose value is empty; If the second attribute exists in the target network data, the attribute value of the second attribute in the target network data is updated according to a preset value to obtain the updated target network data. Determine whether the data content of the target updated data is the same as that of the updated target network data.

7. The method according to claim 1, characterized in that, The network service component also includes a third process, and the method further includes: When the third process detects an update in the second database, it updates the target data in the third database based on the updated data in the second database, wherein the target data has a dependency relationship with the updated data in the second database.

8. A data processing apparatus, characterized in that, A network service component applied in a cloud computing platform, the network service component comprising at least a first process and multiple second processes, the apparatus comprising: The processing module is used to monitor updated data in the first database of the network controller through the first process, wherein the network controller is used to manage virtual networks according to network data in the first database, and the virtual networks are created by the cloud computing platform; The first update module is used to update the second database based on the update data through the first process when the update data is detected. The second database is a database set outside the network service component. The second update module is used to retrieve target network data associated with the network resource update request from the second database when the second process receives a network resource update request, and update the first database according to the target network data and the network resource update request.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the data processing method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the data processing method according to any one of claims 1 to 7.

Images