Interface file processing method and device, electronic equipment and storage medium

Abstract

The invention provides an interface file processing method and device, electronic equipment and a storage medium, and belongs to the field of data processing.The interface file processing method comprises the steps that a micro-service center stores a configuration description file to a network attached storage (NAS) device, and the configuration description file is mounted to each instance of an application programming interface (API) micro-service through the NAS device; the micro-service center receives an interface file monitoring request; the micro-service center suspends the monitoring request and scans a message table in a database of the micro-service center according to a first period; the scanning result is used for the API micro-service to judge whether the configuration description file needs to be reloaded or not; if the scanning result shows that the asynchronous notification message exists in the message table, the micro-service center sends a monitoring response message to the API micro-service; the monitoring response message indicates that the asynchronous notification message exists in the message table, so that the API micro-service loads the changed configuration description file from the NAS equipment; the application ensures that the system can keep service continuity and performance stability after the nebula Apollo interface is closed.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to an interface file processing method, apparatus, electronic device, and storage medium. Background Art

[0002] With the rapid development of the distributed system architecture, as a core component supporting the operation of the system, the stability and security of the configuration center have received wide attention. The currently widely used Apollo configuration center system in the industry provides storage management and change notification services for the configuration description files of the big data platform data API system through its open application programming interface (Application Programming Interface, Open API). Specifically, the existing technical solutions mainly rely on Apollo to implement two core functions: firstly, the persistent storage of API description files is carried through a key-value pair storage mechanism; secondly, a real-time push mechanism for configuration changes is implemented based on the publish-subscribe mode.

[0003] However, with the upgrade and transformation of the Nebula Apollo system, its external service interface policy has undergone major adjustments: on the one hand, the Open API access channel will be gradually closed, and on the other hand, strict access standards are proposed for system access parties. Through technical verification, it is found that the data volume of API description files generated during the operation of the existing big data platform has reached the TB scale level, and the peak value of the configuration change frequency exceeds 500 times per second. These two technical indicators both exceed the threshold requirements of the new access standards of Nebula Apollo. This technical contradiction has led to a major compatibility challenge for the existing architecture: neither can the original Apollo interface implementation solution be continued to be used, nor is there an alternative technical means to ensure the efficient storage of massive configuration data and the high-frequency change push requirements. Therefore, it is urgent to develop a new configuration management architecture to reconstruct the configuration storage engine and change notification mechanism without relying on the Apollo native interface, so as to ensure that the system can still maintain service continuity and performance stability after the Nebula Apollo interface is closed. Summary of the Invention

[0004] Embodiments of this application provide an interface file processing method, apparatus, electronic device, and storage medium, which can ensure that the system can still maintain service continuity and performance stability after the Nebula Apollo interface is closed.

[0005] The technical solution of the embodiments of this application is implemented as follows: In a first aspect, embodiments of this application provide an interface file processing method, and the method includes: The microservice center stores the configuration description file in a Network Attached Storage (NAS) device, and the configuration description file is mounted to each instance of the API microservice through the NAS device; The microservice center receives an interface file listening request sent by the API microservice; The microservice center suspends the listening request, scans the message table in the database of the microservice center at a first period, and obtains a scan result; the scan result is used by the API microservice to determine whether to reload the configuration description file; If the scan result indicates that there is an asynchronous notification message in the message table of the database, the microservice center sends a listening response message to the API microservice; The listening response message is used to indicate that there is an asynchronous notification message in the message table of the database, so that the API microservice loads the changed configuration description file from the NAS device.

[0006] In some embodiments, the method further includes: if the scan result indicates that there is no asynchronous notification message in the message table of the database, the microservice center sends a response code to the API microservice after a preset duration from receiving the listening request; the response code is used to indicate that the configuration description file has not changed.

[0007] In some embodiments, before the microservice center stores the configuration description file in the Network Attached Storage NAS device, the method further includes: The microservice center receives an API service management request, which includes at least one of API publishing, API migration, API archiving, and API hot editing; The microservice center performs API service management based on the API service management request; The microservice center updates the configuration description file based on the result of the API service management.

[0008] In some embodiments, before the microservice center stores the configuration description file in the Network Attached Storage NAS device, the method further includes: The microservice center writes an asynchronous notification message for updating the configuration description file into the message table of the database.

[0009] In some embodiments, the microservice center has the permission to modify the configuration description file; Microservices other than the microservice center only have the permission to read the configuration description file.

[0010] In some embodiments, the method further includes: the microservice center scans the configured description files stored in the NAS device at a second period; If the configured description files stored in the NAS device are different from the configured description files stored in the memory of the API microservice, the API microservice reloads the configured description files stored in the NAS device.

[0011] In a second aspect, an interface file processing apparatus provided by an embodiment of the present application is characterized in that the interface file processing apparatus at least includes a microservice center; the microservice center includes: A storage module, configured to store the configured description files to a NAS device, and the configured description files are mounted to each instance of the API microservice through the NAS device; A receiving module, configured to receive an interface file listening request sent by the API microservice; A scanning module, configured to suspend the listening request, scan a message table in the database of the microservice center at a first period, and obtain a scanning result; the scanning result is used for the API microservice to determine whether to reload the configured description files; A sending module, configured to if the scanning result indicates that there is an asynchronous notification message in the message table of the database, the microservice center sends a listening response message to the API microservice; the listening response message is used to indicate that there is an asynchronous notification message in the message table of the database, so that the API microservice loads the changed configured description files from the NAS device.

[0012] In some embodiments, if the scanning result indicates that there is no asynchronous notification message in the message table of the database, after a preset duration from receiving the listening request, the sending module is further configured to send a response code to the API microservice; the response code is used to indicate that the configured description files have not changed.

[0013] In some embodiments, before the microservice center stores the configured description files to a network attached storage (NAS) device, the receiving module is further configured to receive an API service management request, and the API service management request includes at least one of API publishing, API migration, API archiving, and API hot editing; The microservice center further includes a processing module, configured to perform API service management based on the API service management request; update the configured description files based on the result of the API service management.

[0014] In some embodiments, the storage module is further configured to write an asynchronous notification message for updating the configured description files into the message table of the database.

[0015] In some embodiments, the microservice center has the permission to modify the configured description file; Microservices other than the microservice center only have the permission to read the configured description file.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, which includes: a processor and a memory communicatively connected to the processor; wherein, the processor memory stores instructions executable by the processor, and when the instructions are executed by the processor, the processor can execute the above interface file processing method.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, storing executable instructions, which are used to implement the above interface file processing method when executed by a processor.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, the above interface file processing method is implemented.

[0019] The interface file processing method provided by the embodiment of the present application includes: the microservice center stores the configured description file in the NAS device, and the configured description file is mounted to each instance of the microservice through the NAS device; the microservice center receives an interface file listening request sent by the API microservice; the microservice center suspends the listening request, scans the message table in the database of the microservice center according to the first period, and obtains a scan result; the scan result is used for the API microservice to determine whether to reload the configured description file; if the scan result indicates that there is an asynchronous notification message in the message table of the database, the microservice center sends a listening response message to the API microservice; the listening response message is used to indicate that there is an asynchronous notification message in the message table of the database, so that the API microservice can load the changed configured description file from the NAS device. In the embodiment of the present application, the configured description file is stored in the NAS device, and the file system of the NAS device is connected to the system of the API microservice of the server by using the mount point, so that the API microservice can access the folder of the NAS device through this path. Therefore, for the storage and change of the configured description file, the embodiment of the present application does not need to rely on Nebula Apollo and is decoupled from Nebula Apollo; instead, it is implemented based on the NAS device, ensuring that the system can still maintain service continuity and performance stability after the Nebula Apollo interface is closed. Description of the Drawings

[0020] Figure 1 is an optional processing flow diagram of the interface file processing method provided by the embodiments of the present application; Figure 2 is an architecture diagram of the configuration description file mounted to the API microservice through the NAS device provided by the embodiments of the present application; Figure 3 is a processing flow diagram of the microservice center updating the configuration description file provided by the embodiments of the present application; Figure 4 is a schematic diagram of a composition structure of the interface file processing device provided by the embodiments of the present application; Figure 5 is a schematic diagram of the hardware composition structure of the electronic device provided by the embodiments of the present application. Detailed implementation manners

[0021] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be construed as limiting the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0022] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0023] In the following description, the terms "first / second" are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0025] It should be understood that in various embodiments of the present application, the magnitudes of the serial numbers of the various implementation processes do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0026] An optional processing flow diagram of the interface file processing method provided by the embodiments of the present application, as Figure 1 shown, includes at least the following steps: In step S101, the microservice center stores the configured description file in the NAS device, and the configured description file is mounted to each instance of the API microservice through the NAS device.

[0027] In some embodiments, in the field of microservice technology, the configured description file may also be referred to as an API interface file or a configured API description file. Among them, the configured API description file is a structured document used to define various aspects of an API in a declarative manner, including endpoints, request / response formats, parameters, authentication methods, etc. The API configured description file may include: request fields, return fields, query sercle, and data sources, etc.

[0028] In some embodiments, the microservice center (ServiceCenter-Micro) is one of the core components in the microservice framework ServiceComb, mainly used for service registration and discovery, service governance, and metadata management in the microservice architecture.

[0029] In some embodiments, the architecture diagram of the configured description file mounted to the API microservice through the NAS device is as Figure 2 shown. Based on Figure 2 the architecture diagram shown, the implementation process of the configured description file mounted to each instance of the API microservice through the NAS device may be: each API microservice corresponds to a NAS mount point, and each instance of each API microservice reads the configured description file stored in the NAS device through the NAS mount point.

[0030] In step S102, the microservice center receives the interface file listening request sent by the API microservice.

[0031] In some embodiments, the API microservice sends an interface file listening request to the microservice center, and the interface file listening request is used to listen for whether the interface file has changed.

[0032] In step S103, the microservice center suspends the listening request and scans the message table in the database of the microservice center at the first period to obtain a scan result.

[0033] In some embodiments, the first period can be flexibly set as needed, such as set to 3 seconds or 5 seconds, etc. The database of the microservice center can be OceanBase. In the microservice architecture, the message table in OceanBase (usually referred to as the EventLog Table or Outbox / Inbox table) is the core component for implementing reliable event-driven communication. Its core function is to record asynchronous notification messages (such as events, commands, etc.) that need to be transmitted between services, ensuring the reliable storage and eventual consistency of messages in distributed transactions. The message table can include field names, data types, and descriptions. For example, the field name is: message_id, the data type is: UUID, and the description is: the unique identifier of the message, which is used for idempotency processing and message tracking.

[0034] In some embodiments, the scan result can be: the scan result obtained by the microservice center when scanning the message table in the database for the information carried in the listening request. For example, if the listening request carries a UUID, the scan result is the content in the message table corresponding to the UUID.

[0035] In some embodiments, the scan result is used for the API microservice to determine whether to reload the configured description file; if the scan result indicates that there are asynchronous notification messages in the message table of the database, the API microservice determines that it needs to reload the configured description file; if the scan result indicates that there are no asynchronous notification messages in the message table of the database, the API microservice determines that it does not need to reload the configured description file.

[0036] Step S104, if the scan result indicates that there are asynchronous notification messages in the message table of the database, the microservice center sends a listening response message to the API microservice.

[0037] In some embodiments, the listening response message is used to indicate that there are asynchronous notification messages in the message table of the database; that is, the configured description file for the listening request has changed, and the API microservice needs to load the changed configured description file from the NAS device.

[0038] In some embodiments, if the scan result indicates that there are no asynchronous notification messages in the message table of the database, the microservice center sends a response code to the API microservice after a preset duration from receiving the listening request; the response code is used to indicate that the configured description file has not changed. Among them, the response code can be a 304 status code, which is used to inform the API microservice that the requested configured description file has not been modified and can directly use the configured description file cached locally by the API microservice.

[0039] In some embodiments, the microservice center has the permission to modify the configured description file; microservices other than the microservice center (such as any API microservice) only have the permission to read the configured description file.

[0040] In the embodiments of the present application, the configured description file is stored in a NAS device, and the mount point is used to connect the file system of the NAS device to the system of the API microservice of the server, so that the API microservice can access the folder of the NAS device through this path. Therefore, for the storage and change of the configured description file, the embodiments of the present application do not need to rely on Nebula Apollo and are decoupled from Nebula Apollo; instead, it is implemented based on the NAS device, ensuring that the system can still maintain service continuity and performance stability after the Nebula Apollo interface is closed.

[0041] In the interface file processing method provided by the embodiments of the present application, before the microservice center stores the configured description file in the NAS device, it also includes the process of the microservice center updating the configured description file. The schematic diagram of the processing flow for the microservice center to update the configured description file provided by the embodiments of the present application is as Figure 3 shown, and at least includes the following steps: Step S201, the microservice center receives an API service management request, and the API service management request includes at least one of API publishing, API migration, API archiving, and API hot editing.

[0042] In some embodiments, the API service management request can be triggered and sent by a user.

[0043] Step S202, the microservice center performs API service management based on the API service management request.

[0044] In some embodiments, if the API service management request is API publishing, the microservice center performs API publishing according to the service management request. If the API service management request is API migration, the microservice center performs API migration according to the service management request. If the API service management request is API archiving, the microservice center performs API archiving according to the service management request. If the API service management request is API hot editing, the microservice center performs API hot editing according to the service management request.

[0045] Step S203, the microservice center updates the configured description file based on the result of the API service management.

[0046] In some embodiments, the microservice center newly updates the configured description file in the NAS device.

[0047] In some embodiments, after performing step S203, the method may further include: Step S204, the microservice center writes an asynchronous notification message for updating the configured description file into the message table of the database.

[0048] Step S205, the microservice center scans the configured description file stored in the NAS device at a second period.

[0049] In some embodiments, the second period can be flexibly set according to actual needs, such as being set to 6 seconds or 10 seconds.

[0050] Step S206, if the configured description file stored in the NAS device is different from the configured description file stored in the memory of the API microservice, the API microservice reloads the configured description file stored in the NAS device.

[0051] In some embodiments, the API microservice can compare whether the MD5 value of the configured description file in its own memory is the same as the MD5 value in the NAS device; if they are the same, it indicates that the configured description file in the memory of the API microservice is the same as the configured description file stored in the NAS device; if they are different, it indicates that the configured description file in the memory of the API microservice is different from the configured description file stored in the NAS device.

[0052] In the embodiments of the present application, the directories in the NAS may include, from largest to smallest in terms of directory level, the NAS mount point, the API configured description file, and, based on naming by microservices, each interface corresponds to a file.

[0053] A schematic structural diagram of a composition of an interface file processing device for implementing the interface file processing method provided by the embodiments of the present application is as Figure 4 shown, and at least includes: A microservice center 300; the microservice center 300 includes: A storage module 301, configured to store the configured description file into the NAS device, and the configured description file is mounted to each instance of the API microservice through the NAS device; A receiving module 302, configured to receive an interface file listening request sent by the API microservice; A scanning module 303, configured to suspend the listening request, scan the message table in the database of the microservice center at a first period, and obtain a scanning result; the scanning result is used for the API microservice to determine whether it is necessary to reload the configured description file; A sending module 305, configured to, if the scanning result indicates that there is an asynchronous notification message in the message table of the database, the microservice center sends a listening response message to the API microservice; the listening response message is used to indicate that there is an asynchronous notification message in the message table of the database, so that the API microservice loads the changed configured description file from the NAS device.

[0054] In some embodiments, if the scanning result indicates that there is no asynchronous notification message in the message table of the database, after a preset duration from receiving the listening request, the sending module 305 is further configured to send a response code to the API microservice; the response code is used to indicate that the configured description file has not changed.

[0055] In some embodiments, before the microservice center stores the configured description file into a Network Attached Storage (NAS) device, the receiving module 302 is further configured to receive an API service management request, where the API service management request includes at least one of API publishing, API migration, API archiving, and API hot editing; The microservice center further includes a processing module ( Figure 4 not shown in the figure) configured to perform API service management based on the API service management request; update the configured description file based on the result of the API service management.

[0056] In some embodiments, the storage module 301 is further configured to write the asynchronous notification message for updating the configured description file into the message table of the database.

[0057] In some embodiments, the microservice center has the permission to modify the configured description file; Microservices other than the microservice center only have the permission to read the configured description file.

[0058] An embodiment of this application further provides an electronic device, including: a processor and a memory communicatively connected to the processor; where the processor memory stores instructions executable by the processor, and when the instructions are executed by the processor, the processor is enabled to execute the interface file processing method provided in the above embodiments of this application.

[0059] A schematic diagram of the hardware composition structure of the electronic device provided in the embodiment of this application is shown in Figure 5As shown, it includes: a processor 410 and a memory 450 communicatively connected to the processor 410. The electronic device further includes a bus 440; each module of the electronic device is coupled together through the bus 440. Among them, there can be multiple processors. It can be understood that the bus 440 is used to realize the connection and communication between these modules. In addition to the data bus, the bus 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 5 all kinds of buses are labeled as bus 440.

[0060] The processor 410 has the ability to process signals, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware models, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0061] The memory 450 stores executable instructions for implementing the interface file processing method provided by the embodiments of the present application; the memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memories, hard disk drives, optical disc drives, etc. Optionally, the memory 450 includes one or more storage devices that are physically located away from the processor core 410.

[0062] In some embodiments, the memory 450 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof.

[0063] In some embodiments, the electronic device may further include: An operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks; A network communication module 452, for reaching other computing devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wi-Fi (Wireless Fidelity), and USB (Universal Serial Bus), etc.

[0064] The embodiments of the present application provide a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by the processor, the processor will be triggered to execute the interface file processing method provided by the embodiments of the present application. For example, as Figures 1 to 3 shown in the interface file processing method.

[0065] In some embodiments, the computer-readable storage medium may be a ferroelectric random access memory (FRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disc, or a memory such as a CD-ROM; it may also be various devices including one or any combination of the above memories.

[0066] In some embodiments, the executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or being deployed as a module, model, subroutine, or other unit suitable for use in a computing environment.

[0067] As an example, the executable instructions may be deployed to execute on one computing device, or on multiple computing devices located at one location, or alternatively, on multiple computing devices distributed at multiple locations and interconnected by a communication network.

[0068] The embodiments of the present application provide a computer program product, where the computer program product includes a computer program / instructions, and when the computer program / instructions are executed by a processor, the interface file processing method described in the present application is implemented.

[0069] The above are only the embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An interface file processing method, characterized in that, The method includes: The microservice center stores the configured description file in a Network Attached Storage (NAS) device, and the configured description file is mounted to each instance of the Application Programming Interface (API) microservice through the NAS device; The microservice center receives an interface file listening request sent by the API microservice; The microservice center suspends the listening request, scans the message table in the database of the microservice center at a first period, and obtains a scan result; the scan result is used for the API microservice to determine whether to reload the configured description file; If the scan result indicates that there is an asynchronous notification message in the message table of the database, the microservice center sends a listening response message to the API microservice; The listening response message is used to indicate that there is an asynchronous notification message in the message table of the database, so that the API microservice loads the changed configured description file from the NAS device.

2. The method according to claim 1, wherein The method further includes: If the scan result indicates that there is no asynchronous notification message in the message table of the database, the microservice center sends a response code to the API microservice after a preset duration from receiving the listening request; the response code is used to indicate that the configured description file has not changed.

3. The method according to claim 1, wherein Before the microservice center stores the configured description file in the Network Attached Storage (NAS) device, the method further includes: The microservice center receives an API service management request, which includes at least one of API publishing, API migration, API archiving, and API hot editing; The microservice center performs API service management based on the API service management request; The microservice center updates the configured description file based on the result of the API service management.

4. The method according to claim 1, characterized in that, Before the microservice center stores the configured description file in the Network Attached Storage (NAS) device, the method further includes: The microservice center writes an asynchronous notification message for updating the configured description file into the message table of the database.

5. The method according to claim 1, wherein The microservice center has the permission to modify the configured description file; Microservices other than the microservice center only have the permission to read the configured description file.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: the microservice center scans the configured description file stored in the NAS device at a second period; If the configured description file stored in the NAS device is different from the configured description file stored in the memory of the API microservice, the API microservice reloads the configured description file stored in the NAS device.

7. An interface file processing device, characterized in that, The interface file processing device at least includes a microservice center; the microservice center includes: A storage module, which is used to store the configured description file in a Network Attached Storage (NAS) device, and the configured description file is mounted to each instance of the Application Programming Interface (API) microservice through the NAS device; A receiving module, which is used to receive an interface file listening request sent by the API microservice; A scanning module, configured to suspend the listening request, scan a message table in a database of the microservice center according to a first period, and obtain a scanning result; the scanning result is used for the API microservice to determine whether to reload the configured description file; A sending module, configured to, if the scanning result indicates that there is an asynchronous notification message in the message table of the database, the microservice center sends a listening response message to the API microservice; the listening response message is used to indicate that there is an asynchronous notification message in the message table of the database, so that the API microservice loads the changed configured description file from the NAS device.

8. An electronic device, characterized in that, The electronic device includes: A processor and a memory communicatively connected to the processor; wherein, The processor memory stores instructions executable by the processor, and when the instructions are executed by the processor, the processor is enabled to execute the interface file processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Stores executable instructions, which when executed by a processor, implement the interface file processing method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, the interface file processing method according to any one of claims 1 to 6 is implemented.

Images