Fault injection service management method and device, equipment and storage medium
By receiving deployment instructions to obtain the fault injection service identifier and namespace, and automatically installing service components and tool components, the problem of low deployment efficiency of fault injection services is solved, efficient fault injection service management is achieved, and network bandwidth consumption is reduced.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TENCENT TECHNOLOGY (SHENZHEN) CO LTD
- Filing Date
- 2021-05-12
- Publication Date
- 2026-05-19
AI Technical Summary
In existing technologies, fault injection services have low deployment efficiency and require a large amount of network bandwidth consumption during the upgrade process.
By receiving deployment instructions, the fault injection service identifier and namespace are obtained, the deployment configuration is acquired, and service components and tool components are automatically installed based on the configuration, thereby realizing the automatic deployment of the fault injection service. A modular design is adopted for independent storage and upgrade of components.
It improves the deployment efficiency of fault injection services, reduces network bandwidth consumption, supports dynamic loading of deployment configurations, and allows for individual upgrades of each component.
Smart Images
Figure CN115344274B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of Internet technology, and in particular to a fault injection service management method, apparatus, device, and storage medium. Background Technology
[0002] With the development of Internet technology, various fault injection services have emerged. Users can easily inject faults into devices to verify the corresponding performance of the devices.
[0003] In related technologies, the resources required for fault injection services need to be manually uploaded and installed on designated devices by users through a transfer tool, which seriously affects the deployment efficiency of fault injection services. Furthermore, when upgrading the fault injection service itself or the fault injection tool used by the fault injection service, all resources need to be uploaded again using the transfer tool, resulting in additional network bandwidth consumption. Summary of the Invention
[0004] This application provides a fault injection service management method, apparatus, device, and storage medium, which can improve the deployment efficiency of fault injection services and reduce additional network bandwidth consumption.
[0005] On the one hand, this application provides a fault injection service management method, the method comprising:
[0006] Upon receiving a deployment instruction for a fault injection service, the fault injection service identifier and namespace are obtained from the deployment instruction. The fault injection service represents the service that performs fault injection behavior, and the namespace is used to indicate the deployment environment of the fault injection service.
[0007] Based on the fault injection service identifier and the namespace, obtain the deployment configuration;
[0008] Based on the deployment configuration, service components and tool components are obtained. The service components are used to support the operation of the fault injection service, and the tool components represent the fault injection tools used by the fault injection service.
[0009] Install the deployment configuration, the service components, and the tool components locally to trigger the startup of the fault injection service.
[0010] On the other hand, a fault injection service management device is provided, the device comprising:
[0011] The deployment instruction receiving module is used to receive a deployment instruction for the fault injection service, and to obtain the fault injection service identifier and namespace from the deployment instruction. The fault injection service represents the service that performs fault injection behavior, and the namespace is used to indicate the deployment environment of the fault injection service.
[0012] The deployment configuration acquisition module is used to acquire the deployment configuration based on the fault injection service identifier and the namespace;
[0013] The component acquisition module is used to acquire service components and tool components based on the deployment configuration. The service components are used to support the operation of the fault injection service, and the tool components represent the fault injection tools used by the fault injection service.
[0014] The deployment and implementation module is used to install the deployment configuration, the service components, and the tool components locally, and to trigger the startup of the fault injection service.
[0015] On the other hand, an apparatus is provided, the apparatus including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or at least one program being loaded by the processor and executed as described above for the fault injection service management method.
[0016] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the fault injection service management method as described above.
[0017] This application obtains the deployment configuration by using the fault injection service identifier and namespace in the deployment command, and then obtains service components and tool components based on the deployment configuration. This enables automatic deployment of the fault injection service without the need for manual uploading and installation using any transfer tools, thus improving the deployment efficiency of the fault injection service. Decoupling the configuration from the function allows for dynamic loading of the deployment configuration. Through modular design, the deployment configuration, service components, and tool components are stored independently, which not only realizes the functional and configuration structure, but also allows for individual upgrades of each component, reducing unnecessary network bandwidth consumption. Attached Figure Description
[0018] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application.
[0020] Figure 2 This is a schematic diagram of the architecture of an implementation environment provided in an embodiment of this application.
[0021] Figure 3 This is a flowchart illustrating a fault injection service management method provided in an embodiment of this application.
[0022] Figure 4 This is a flowchart illustrating another fault injection service management method provided in this application embodiment.
[0023] Figure 5 This is an example diagram of deployment configuration management provided in the embodiments of this application.
[0024] Figure 6 This is a flowchart illustrating another fault injection service management method provided in this application embodiment.
[0025] Figure 7 This is a flowchart illustrating another fault injection service management method provided in this application embodiment.
[0026] Figure 8 This is an example diagram of the service file structure after installation, provided in an embodiment of this application.
[0027] Figure 9 This is a flowchart illustrating another fault injection service management method provided in this application embodiment.
[0028] Figure 10 This is a flowchart illustrating another fault injection service management method provided in this application embodiment.
[0029] Figure 11 This is a flowchart illustrating another fault injection service management method provided in this application embodiment.
[0030] Figure 12 This is an example diagram illustrating the installation and deployment of a fault injection service provided in an embodiment of this application.
[0031] Figure 13 This is an example diagram illustrating the installation and deployment of another fault injection service provided in this application embodiment.
[0032] Figure 14 This is a structural block diagram of a fault injection service management device provided in an embodiment of this application.
[0033] Figure 15 This is a schematic diagram of the hardware structure of a device provided in an embodiment of this application. Detailed Implementation
[0034] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.
[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.
[0036] First, the relevant terms used in the embodiments of this application are explained as follows:
[0037] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to enable data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, allowing for flexible and convenient on-demand resource utilization.
[0038] Cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, the resources in the "cloud" are infinitely scalable, readily available, on-demand, expandable, and pay-as-you-go.
[0039] Fault injection: a reliability verification technique that involves deliberately introducing faults into a system through controlled experiments and observing the system's behavior when faults are present.
[0040] Fault injection service: also known as fault injection agent or agent, is a service that performs fault injection behavior, and it is usually provided to users in the form of microservice.
[0041] Remote configuration: The configuration required by the Agent is uniformly triggered by a remote configuration management platform.
[0042] Chaos engineering is a discipline that experiments on distributed systems to improve their fault tolerance. Examples include assessing system disaster recovery limits, verifying the dynamic scaling capabilities of cloud services / resources, and validating the effectiveness of monitoring / alarms and the completeness of problem-solving processes.
[0043] ChaosBlade is a chaos engineering tool that follows the principles of chaos engineering experiments, providing rich implementations of fault scenarios to help distributed systems improve fault tolerance and recoverability. The chaos experiments supported by ChaosBlade not only cover basic resources such as CPU full load, high disk I / O, and network latency, but also application experiments running on the JVM, such as call exceptions, delayed or exception-throwing specified methods, and returning specific values. It also includes container-related experiments, such as killing containers.
[0044] Dockerfile: A text file used to build an image. The text contains instructions and explanations required to build the image.
[0045] Blue Shield: A tool for continuous builds that enables automated compilation, packaging, distribution, and deployment of projects.
[0046] Please see Figure 1 This diagram illustrates an implementation environment provided by an embodiment of this application, which can implement the fault injection service management method of this application. Figure 1 As shown, the implementation environment may include one or more devices 01 (represented as 01a, 01b, ..., 01n in the figure), a file server 02, a service management platform 03, and a configuration management platform 04.
[0047] Device 01 can be a standalone server, a server cluster or distributed system composed of multiple physical servers, a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms, or a container, such as a cloud container in a cloud platform or a non-cloud container such as a personal host. No specific restrictions are imposed here. The cloud platform can be an STKE platform, PCG123 platform, Sumeru platform, or Taf framework, etc., without specific limitations.
[0048] In this embodiment, the configuration management platform 04 is used to manage the versions of each component in each service, such as the service's own version, the version of the fault injection tool used by the service, etc.; the service management platform 03 is used to manage each service, such as detecting the health status of each service, storing service information of each service on the corresponding device, etc.; the file server 02 is used to store the resources required by each service, such as the deployment configuration required by each service, the service components running on the service itself, and storing the fault injection tools used by each service, etc.
[0049] In one application scenario, a user can send a deployment instruction for the fault injection service to device 01 by entering a command on the device. Device 01 obtains the fault injection service identifier and namespace from the deployment instruction, pulls the deployment configuration from file server 02 based on the fault injection service identifier and namespace, and pulls the service components and tool components from file server 02 based on the deployment configuration. Then, it installs the deployment configuration, service components and tool components locally, triggering the start of the fault injection service.
[0050] After the fault injection service is successfully started, device 01 can register with the service management platform 03 and report the service information related to the fault injection service. The target service will then use the service management platform 03 to determine whether to send an upgrade instruction for the corresponding component to device 01 based on the service information, so that device 01 can upgrade the corresponding component.
[0051] Indicative Figure 2 This diagram illustrates the architecture of an implementation environment provided in an embodiment of this application. Figure 2 As shown, the fault injection service deployed on each device 01 and the target service 05 both contain a Software Development Kit (SDK) provided by the service management platform. After each successful startup, the fault injection service can register and report its service information to the service management platform 03 through this SDK. This service information may include the IP address of the deployed device, the port number running on that device, the version of the service components, the version of the tool components, and the version of the deployment configuration.
[0052] Service management platform 03 can use the aforementioned service information to perform health checks on the fault injection services deployed on each device 01 to detect whether the fault injection services are operating normally. Target service 05 can also use the software development kit to detect from service management platform 03 whether the fault injection services on each device need updating, such as service component updates or tool component updates. If an update is required, it sends a component upgrade command to the fault injection service on the device requiring the update through the update service in the target service, causing the fault injection service to pull the corresponding component from file server 02 again to achieve the component upgrade.
[0053] In another application scenario, when it is necessary to update the version of the fault injection tool or the version of the fault injection service itself, the user can configure it through the configuration management platform 04, and then the configuration management platform 04 will send the corresponding configuration update command to the device 01 so that the device 01 can upgrade the deployment configuration.
[0054] Optionally, after obtaining the deployment configuration from the file server 02 based on the fault injection service identifier and namespace, device 01 can also obtain the latest version control file and configuration file from the configuration management platform 04, and then replace the old version control file and configuration file in the deployment configuration to ensure consistency with the configuration in the configuration management platform 04.
[0055] As can be seen from the above application scenarios, users can deploy the fault injection service with just a simple command, without the need for manual uploading and installation using any transfer tools, thus improving the deployment efficiency of the fault injection service; decoupling configuration from function, combined with remote configuration, enables dynamic loading and deployment of configurations, and all configurations are centralized in the configuration management platform, allowing for unified management of configurations scattered across various devices; through modular design, each component can be upgraded individually, such as specifying an upgrade for the fault injection tool or the fault injection service, reducing unnecessary network bandwidth consumption.
[0056] For ease of description, the following will be referred to as Figure 1 The fault injection service management method of this application is introduced using the implementation environment shown as an example. This method can be applied to the equipment in this implementation environment.
[0057] Figure 3This is a flowchart illustrating a fault injection service management method provided in an embodiment of this application. This application provides the operational steps of the method described in the embodiment or flowchart, but based on conventional or non-inventive methods, more or fewer operational steps may be included. The order of steps listed in the embodiment is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiment or drawings. Specifically, as shown... Figure 3 As shown, the method may include:
[0058] S301, Receive deployment instructions for fault injection service, obtain fault injection service identifier and namespace from deployment instructions. Fault injection service represents the service that performs fault injection behavior, and namespace is used to indicate the deployment environment of fault injection service.
[0059] In this embodiment, the fault injection services deployed on different devices may differ, and the purpose of each deployed fault injection service may also vary, such as for development or testing. Therefore, when a user issues a deployment command to a device, the device needs to know which fault injection service needs to be deployed and in which deployment environment the fault injection service runs. The fault injection service identifier can be a unique identifier such as a service name, and the deployment environment can be a development environment or a testing environment, etc. Optionally, both the fault injection service identifier and the deployment environment can be pre-registered in a service management platform.
[0060] In one application scenario, a user can send deployment instructions to a device via a script file or a single command. For example, a user can send a deployment instruction to a device by executing the shell command `curl * / |bash -s ${service+namespace}`, where "curl * / |bash -s" is the deployment command to be executed by the device, `${service+namespace}` are the parameters required by the deployment command, `service` is the identifier of the fault injection service, and `namespace` is the namespace.
[0061] In another application scenario, the download and startup service script (start.sh) can be configured in the device startup file or the service image. When the device starts up or the service is running, a deployment command for the fault injection service is triggered. Since the script is in the service image, changes to the service image and the fault injection service do not affect the service itself; they only pull files from the specified server.
[0062] S303 obtains the deployment configuration based on the fault injection service identifier and namespace.
[0063] In this embodiment, the device can obtain a deployment configuration from a file server based on the fault injection service identifier and namespace. This deployment configuration is stored in the file server as a compressed package, such as a compressed package ending with .tar.gz. Specifically, the device can query the file server for resources whose names contain the fault injection service identifier and namespace as deployment configurations. For example, the resource name is named sequentially with the fault injection service identifier, namespace, underscore, and configuration identifier (e.g., conf), meaning the compressed package corresponding to the deployment configuration is named ${service+namespace}_conf.
[0064] In some embodiments, the deployment configuration includes configuration files (such as conf.toml), version control files (such as version.toml), download and start service scripts (such as start.sh), restart service scripts (such as restart.sh), and multiple resource download scripts.
[0065] The configuration file is used for basic configurations related to the storage service, such as the fault injection service identifier, namespace, etc.; the version control file is used for the versions of resources required by the storage service, such as the version of the deployment configuration, the version of the service components, and the version of the tool components, etc.; the download and start service script is used to download the deployment configuration, service components, and tool components and start the fault injection service, and the restart service script is used to restart the fault injection service; the resource download script is used to obtain the resources required by the service, such as the configuration download script for obtaining the deployment configuration, the service download script for obtaining the service components, and the tool download script for obtaining the tool components, etc.
[0066] In other embodiments, to ensure that the obtained service components and tool components are the latest versions, the deployment configuration can be updated according to the configuration in the configuration management platform after obtaining the deployment configuration from the file server.
[0067] In one possible implementation, the format of the configuration stored in the configuration management platform can be consistent with the format of the configuration in the deployment configuration. Therefore, files obtained from the configuration management platform can be directly used to update files in the deployment configuration. For example, version control files in the deployment configuration can be updated directly from version control files obtained from the configuration management platform.
[0068] In another possible implementation, the configuration format in the configuration management platform may differ from that in the deployment configuration. For example, the version control file may be in TOML format, while the configuration management platform stores the corresponding configuration of the version control file in JSON format. Therefore, after retrieving the corresponding configuration from the configuration management platform, a format conversion is required. Specifically, as follows... Figure 4 As shown, step S303 may further include the following after its implementation:
[0069] S3041, based on the fault injection service identifier and namespace, obtain version control file information and configuration file information.
[0070] The device can obtain version control file information and configuration file information from the configuration management platform. The version control file information is used to indicate the information of the version control file, and the configuration file information is used to indicate the information of the configuration file.
[0071] S3043 updates the version control files in the deployment configuration using version control file information, and updates the configuration files in the deployment configuration using configuration file information.
[0072] When updating the version control file in the deployment configuration using version control file information, you can iterate through each record in the version control file information and convert it into statements corresponding to the language used by the version control file; alternatively, you can use a conversion tool to directly convert the version control file information into the format corresponding to the version control file. For example, if the version control file information is a file named config.json and the version control file is config.toml, you can convert each key and value in config.json into statements in config.toml accordingly, or you can directly use a conversion tool to convert config.json into config.toml, and then replace the config.toml in the deployment configuration with the converted config.toml.
[0073] Similarly, when updating configuration files in the deployment configuration using configuration file information, you can iterate through each record in the configuration file information and convert it into statements corresponding to the language used in the configuration file; alternatively, you can use a conversion tool to directly convert the configuration file information into the format corresponding to the configuration file. For example, if the configuration file information is a file named version.json and the configuration file is version.toml, you can convert each key and value in version.json into statements in version.toml accordingly, or you can directly use a conversion tool to convert version.json into version.toml, and then replace the version.toml in the deployment configuration with the converted version.toml.
[0074] The deployment configuration in the file server is automatically packaged and stored by the Continuous Integration (CI) pipeline, and its availability depends on the build time of Continuous Integration. After retrieving the deployment configuration from the file server, updating the version control files and configuration files in the deployment configuration ensures that the relevant information of the deployed fault injection service is consistent with the configuration management platform, avoiding the need to update the configuration or upgrade components after deployment.
[0075] Version control files and configuration files in the configuration management platform can be written by the user during service registration or set directly through the platform's interface. The service registration process can be triggered by the user through the platform's interface or distributed via the upper-layer northbound Chaos Engineering Experiment Platform. The Chaos Engineering Experiment Platform can register all services for each project.
[0076] Indicatively, Figure 5 This shows an example diagram of deployment configuration management. (For example...) Figure 5 As shown, when registering service 505 through the Chaos Engineering Experiment Platform, project information (such as project identifier) is registered in project management 506, and service information (such as fault injection service identifier, namespace, and version information of each component) is registered in service management platform 03. Then, configuration information is registered through the configuration registration interface provided by configuration management platform 04. This configuration information includes project information and service information, thus forming version control file information (or version control file) and configuration file information (or configuration file).
[0077] The continuous integration environment 501 constructs 502 the resources corresponding to each fault injection service in the configuration management platform 04 to obtain the deployment configuration 503 corresponding to each fault injection service, and then archives 503 of the obtained deployment configuration to the file server 02. The device first obtains the deployment configuration 503 from the file server 02, and then obtains the version control file and configuration file based on the version control file information and configuration file information obtained from the configuration management platform 04, and updates the deployment configuration 503 obtained from the file server 02.
[0078] S305, based on the deployment configuration, obtains service components and tool components. Service components are used to support the operation of the fault injection service, and tool components represent the fault injection tools used by the fault injection service.
[0079] In this embodiment, the deployment configuration, service components, and tool components are all stored in the same file server. The device can directly obtain the service components and tool components from the file server. The service components and tool components are stored in the form of compressed packages in the file server.
[0080] In one possible implementation, to facilitate separate management of configuration and functionality, service components and tool components can be stored separately from the deployment configuration. For example... Figure 6 As shown, step S305 may include the following in a specific implementation:
[0081] S3051 reads the version control file in the deployment configuration to determine the service version information and tool version information.
[0082] The following is an example of a version control file, illustrated in the diagram. The version control file includes service version information (serviceVersion), tool version information (bladeVersion), and configuration version information (confVersion). The service version information represents the version information of the service components, the tool version information represents the version information of the fault injection tool, and the configuration version information represents the version information of the deployment configuration.
[0083] [Version]
[0084] serviceVersion="1.0.3" # This represents the version information of the service component.
[0085] bladeVersion="0.5.0" # This represents the version information of the tool component (chaosblade-linux);
[0086] confVersion="0.0.1" # This represents the version information of the deployment configuration (conf).
[0087] S3053, determine service components based on service version information, and determine tool components based on tool version information.
[0088] Specifically, the device can obtain service components that match the service version information from the service component library, and tool components that match the tool version information from the tool component library.
[0089] In another possible implementation, the device can directly execute the service download script in the deployment configuration to automatically obtain service components, and execute the tool download script in the deployment configuration to automatically obtain tool components. The service download script contains execution statements that read service version information from the version control file and execute statements that pull service components from the service component library; similarly, the tool download script contains execution statements that read tool version information from the version control file and execute statements that pull tool components from the tool component library. It should be noted that the service download script and tool download script can be script files in the target language, such as Python or shell, etc., without specific limitations.
[0090] S307 installs the deployment configuration, service components, and tool components locally, triggering the startup of the fault injection service.
[0091] In this embodiment, deployment configurations, service components, and tool components can be installed in a modular manner in different storage locations, meaning each component has its own storage space. This modular storage allows for specifying which module to update when updating the fault injection service, without affecting other modules. This enables hot updates of the entire or partial module, reducing unnecessary network bandwidth consumption. Furthermore, since all deployment configurations, service components, and tool components are installed locally, updates to the fault injection service do not require fetching all resources from a remote location, enabling hot updates of the fault injection service.
[0092] In one possible implementation, the fault injection service on each device is managed using a two- or multi-level directory structure, with the fault injection service serving as the root directory, and the deployment configuration and each component corresponding to subdirectories of the root directory. For example... Figure 7 As shown, step S307 may include the following in a specific implementation:
[0093] S3071, Create the root directory corresponding to the fault injection service.
[0094] The root directory can be named using the fault injection service identifier (such as the service name), a connector (such as an underscore or hyphen), and the fault injection tool name. For example, if the fault injection service identifier is the service name (service) and the fault injection tool name is chaosBlade, then the root directory name could be service-chaosBlade. By using this naming convention, the service and the fault injection tool used by the service can be quickly identified.
[0095] S3073 specifies that deployment configurations, service components, and tool components are installed in the configuration directory, service directory, and tool directory, respectively. The configuration directory, service directory, and tool directory are all subdirectories of the root directory.
[0096] Indicatively, Figure 8 The image shown is an example of the service file structure after installation. Figure 8 In this directory, conf / is the configuration directory, bin / is the service directory, chaosblade-linux is the tool directory, conf.tar.gz contains the deployment configuration, service.tar.gz contains the service components, and chaosblade-linux.tar.gz contains the tool components.
[0097] S3075 checks the configuration directory, service directory, and tool directory to determine whether the service startup conditions are met.
[0098] In this embodiment, the service startup condition refers to whether all files in each directory are complete. If all files are complete, the service startup condition is deemed met. If any files are missing, i.e., the service startup condition is not met, an alarm message can be output and the deployment of the fault injection service can be stopped.
[0099] In some embodiments, service startup conditions include essential files that are indispensable for starting the service. The device can determine whether the essential files exist in each directory, and if they do, the service startup conditions are met.
[0100] In other embodiments, to ensure the security of the acquired files, the completeness of files in each directory can be further checked based on the hash values of each file. The hash value of each file is generated when it is stored on the file server. Optionally, this hash value is automatically generated by the CI pipeline. Therefore, if it is determined that the necessary file exists in each directory, the hash value of the necessary file can be compared with the hash value stored on the file server. If they match, the service startup conditions are met.
[0101] S3077: If the service startup conditions are met, the fault injection service is started.
[0102] If the service startup conditions are met, the device can start the fault injection service by executing the service restart script (restart.sh) in the configuration directory.
[0103] In summary, users can deploy fault injection services with just a simple naming convention, without the need for manual uploading and installation using any transfer tools, thus improving deployment efficiency. Furthermore, decoupling configuration from functionality and implementing a modular design allows for individual upgrades of each component, reducing unnecessary network bandwidth consumption.
[0104] In one possible implementation, such as Figure 9 As shown, after step S307 is implemented, the fault injection service management method provided in this application embodiment may further include:
[0105] S309 After the fault injection service is successfully started, a service reporting message is sent to the service management platform. The service reporting message carries the service information of the fault injection service so that the target service can detect whether the target component needs to be upgraded based on the service information, and send a component upgrade instruction to the device where the fault injection service is deployed if the target component needs to be upgraded.
[0106] Devices can register with the service management platform using the software development kit (SDK) provided by the fault injection service, enabling the sending of service reporting messages. These service reporting messages can include the fault injection service identifier, namespace, IP address of the deployed device, port number used on that device, deployment configuration version information, service component version information, tool component version information, and service build time.
[0107] After each successful startup, the fault injection service reports to the service management platform. The target service used for service discovery can then determine whether an upgrade is needed based on the reported information. If an upgrade is required, it sends a component upgrade command to the fault injection service. Intuitively, users can also proactively trigger the target service to send a component upgrade command to the device by sending a component upgrade request. The device primarily uses configuration bash script files stored locally in its configuration directory to update and upgrade component versions.
[0108] In one specific implementation, such as Figure 10 As shown, step S307 may further include the following after its implementation:
[0109] S311, Receive the component upgrade instruction sent by the target service, and obtain the upgrade identifier in the component upgrade instruction. The upgrade identifier is used to indicate the target component to be upgraded.
[0110] The target component can be a service component and / or a tool component, meaning that it is possible to choose to upgrade either the service component or the tool component, or to upgrade both the service component and the tool component. The upgrade identifier can be in string form, for example, "1" indicates an upgrade of the service component, "2" indicates an upgrade of the tool component, and "3" indicates an upgrade of both the service component and the tool component; this application does not impose specific limitations on this.
[0111] S313, determine the target download script corresponding to the target component from the configuration directory, which is used to mark the installation location of the deployment configuration.
[0112] To illustrate, the configuration directory is as follows: Figure 8 The conf / directory shown has the following target download scripts: get_service.sh when the target component is a service component; get_chaosblade.sh when the target component is a utility component; and get_service.sh and get_chaosblade.sh when the target component is both a service component and a utility component.
[0113] S315, execute the target download script to obtain the target component.
[0114] Corresponding to the aforementioned step S305, the target component can be obtained from the file server or component library by executing the target download script.
[0115] S317, perform an upgrade operation on the target component and restart the fault injection service.
[0116] Because of the modular storage approach of the fault injection service, any component can be upgraded on demand without transmitting the entire fault injection service data packet. For example, assuming the service component is 400MB and the tool component is 300MB, using current upgrade methods, the data packet for the service and tool components as a whole would be at least 700MB, even without considering the size of the deployment configuration. However, the fault injection service is used for fault injection operations. If the device is currently detecting network latency, transmitting a larger data packet not only increases network bandwidth consumption but also has a more severe impact on network latency performance verification.
[0117] As described in step S303 above, the configuration file (conf.toml) included in the deployment configuration stores basic configurations such as the fault injection service identifier (service), namespace, authentication information (Token), and project information (ProjectId). As shown below, this basic configuration is consistent with the information used when the fault injection service registers with the service management platform. In other words, once the fault injection service is registered, the configuration file is rarely modified; therefore, the configuration file can be updated only when a deployment command is received.
[0118] [Polaris]
[0119] [Polaris.Service-Chaosblade_Test]
[0120] namespace = "Test" # The namespace registered in the service management platform;
[0121] service = "chaosblade_service_agent_v0" # The fault injection service identifier registered in the service management platform;
[0122] Token = "XXXXXXX" # Authentication token in the service management platform;
[0123] ProjectId = "YYYYY" # Project information registered in the service management platform.
[0124] In one possible implementation, since the configuration file contains authentication information and project information, when sending a service reporting message to the service management platform, the authentication information and project information can be encapsulated in the service reporting message. This allows the service management platform to authenticate the service based on the authentication information and project information. Only if the authentication is successful will the service reporting message be processed, thus ensuring the legality of the service reporting.
[0125] In some implementations, the device can also upgrade the configuration file (conf.toml) after receiving a component upgrade instruction. Therefore, before step S313, the method may further include: determining a configuration download script from the configuration directory; executing the configuration download script to obtain a new configuration file; and replacing the configuration file in the configuration directory with the new configuration file. Illustratively, the configuration directory is as follows: Figure 8 The conf / directory shown contains the get_conf.sh script, which is the configuration download script.
[0126] In summary, this application separates configuration from functionality, allowing for independent configuration updates; through a unified fault configuration platform, it standardizes the writing of subsequent fault injection behaviors, enabling fault injection services, and providing a unified, compliant, and highly operable protocol standard for open-source collaboration, thereby improving the universality and utilization of fault injection services and avoiding reinventing the wheel.
[0127] In one possible implementation, after the configuration management platform updates the version of a relevant component in a fault injection service, it can send a configuration update command to the device where the fault injection service is deployed, so that the device updates the version control file in its deployment configuration. For example... Figure 11 As shown, step S307 may further include the following after its implementation:
[0128] S319 receives a configuration update instruction from the configuration management platform, determines the version download script from the configuration directory, and uses the configuration directory to mark the installation location of the deployment configuration.
[0129] Indicative, such as Figure 8 As shown, conf / is the configuration directory, and get_version.sh in conf / can be identified as the version download script.
[0130] S321, execute the version download script to obtain the new version control file.
[0131] Alternatively, the version download script can obtain the new version control file from the configuration management platform.
[0132] S323, replace the version control file in the configuration directory with the new version control file, and restart the fault injection service.
[0133] After restarting the fault injection service, the device can send a service reporting message to the service management platform. Since the service reporting message contains the version information of each component, the target service can compare the reported version information with the version information currently used by the fault injection service. If an inconsistency is found, it can be determined that the device deployed by the fault injection service sends a component upgrade instruction so that the device can reacquire the corresponding component and upgrade the component.
[0134] In summary, by unifying the version control files of all fault injection services under the configuration management platform, the problem of inconsistent configurations scattered across various devices can be solved. By setting target services for service discovery, component upgrades can be automatically performed when needed, eliminating the need for manual login to the device again to replace files using transfer tools, thus improving the efficiency of component upgrades.
[0135] In a specific application environment, we will take the deployment of fault injection service for cloud containers in the STKE cloud platform when the business starts as an example.
[0136] Please see Figure 12 The diagram illustrates an example of the installation and deployment of a fault injection service provided in an embodiment of this application. Figure 12 As shown, the download and startup script `start.sh` contains executable statements, including the fault injection service identifier `service` and its namespace. When the business code is compiled via the Blue Shield pipeline, `start.sh` is written into the `kickStart.d` executable file of the business service process through the Dockerfile in the container startup file. Since `start.sh` is already copied into `kickStart.d`, the `start.sh` script can be executed synchronously when the business service process is started via `kickStart.d`, thus allowing the fault injection service and the business image to be deployed directly in the same cloud container. Figure 12 In this context, conf is the configuration directory corresponding to the deployment configuration, service is the service directory corresponding to the service component, chaosblade-linux is the tool directory corresponding to the tool component, and service_chaosblade is the root directory corresponding to the fault injection service.
[0137] In another specific application environment, we will take the example of a user actively sending a deployment command to the device to illustrate the deployment of a fault injection service.
[0138] Please see Figure 13 This diagram illustrates an example of the installation and deployment of another fault injection service provided in an embodiment of this application. Figure 13 As shown, the Blue Shield pipeline pre-stores deployment configurations, service components, and tool components in a file server, a service component library, and a tool component library, respectively. Users execute shell commands on the device to send deployment instructions. The device retrieves the deployment configuration from the file server using the fault injection service identifier and namespace in the deployment instructions. Then, based on the version information in the deployment configuration, it retrieves the service components from the service component library and the tool components from the tool component library, thereby achieving automatic installation and deployment of the fault injection service.
[0139] As can be seen from the technical solutions provided by the above embodiments, this application obtains the deployment configuration through the fault injection service identifier and namespace in the deployment instruction, and then obtains service components and tool components based on the deployment configuration. This enables automatic deployment of the fault injection service without the need for manual uploading and installation using any transmission tools, thereby improving the deployment efficiency of the fault injection service. Decoupling the configuration from the function enables dynamic loading of the deployment configuration. Through modular design, the deployment configuration, service components, and tool components are stored independently, which not only realizes the function and configuration structure, but also allows for individual upgrades of each component, reducing unnecessary network bandwidth consumption.
[0140] Based on the same inventive concept as the above-described method embodiments, this application also provides a fault injection service management device, which can implement the functions of the above-described device-side method embodiments. For example... Figure 14 As shown, the device 1400 may include:
[0141] The deployment instruction receiving module 1410 is used to receive a deployment instruction for the fault injection service, and to obtain the fault injection service identifier and namespace from the deployment instruction. The fault injection service represents the service that performs fault injection behavior, and the namespace is used to indicate the deployment environment of the fault injection service.
[0142] The deployment configuration acquisition module 1420 is used to obtain the deployment configuration based on the fault injection service identifier and namespace;
[0143] The component acquisition module 1430 is used to acquire service components and tool components based on the deployment configuration. The service components are used to support the operation of the fault injection service, and the tool components represent the fault injection tools used by the fault injection service.
[0144] The deployment implementation module 1440 is used to install deployment configurations, service components, and tool components locally and trigger the startup of the fault injection service.
[0145] In one possible implementation, the device 1400 may further include:
[0146] The configuration update instruction receiving module is used to receive configuration update instructions sent by the configuration management platform, determine the version download script from the configuration directory, and mark the installation location of the deployment configuration.
[0147] Configure the script execution module to execute the version download script to obtain the new version control file;
[0148] The configuration upgrade module is used to replace the version control files in the configuration directory with new version control files and restart the fault injection service.
[0149] In one possible implementation, the device 1400 may further include:
[0150] The component upgrade instruction receiving module is used to receive the component upgrade instruction sent by the target service, and to obtain the upgrade identifier in the component upgrade instruction. The upgrade identifier is used to indicate the target component to be upgraded.
[0151] The component script determination module is used to determine the target download script corresponding to the target component from the configuration directory, which is used to mark the installation location of the deployment configuration;
[0152] The component script execution module is used to execute the target download script to obtain the target component;
[0153] The component upgrade module is used to perform upgrade operations on the target component and restart the fault injection service.
[0154] In one possible implementation, the device 1400 may further include:
[0155] The service registration module is used to send a service reporting message to the service management platform after the fault injection service is successfully started. The service reporting message carries the service information of the fault injection service, so that the target service can detect whether the target component needs to be upgraded based on the service information, and send a component upgrade instruction to the device where the fault injection service is deployed if the target component needs to be upgraded.
[0156] In one possible implementation, the deployment configuration acquisition module 1420 may include:
[0157] The information acquisition unit is used to acquire version control file information and configuration file information based on the fault injection service identifier and namespace;
[0158] The file update unit is used to update the version control files in the deployment configuration using version control file information, and to update the configuration files in the deployment configuration using configuration file information.
[0159] In one possible implementation, the component acquisition module 1430 may include:
[0160] The version information reading unit is used to read the version control file in the deployment configuration to determine the service version information and tool version information.
[0161] The service component acquisition unit is used to determine the service component based on the service version information.
[0162] The tool component acquisition unit is used to determine tool components based on tool version information.
[0163] In one possible implementation, the deployment implementation module 1440 may include:
[0164] The root directory creation unit is used to create the root directory corresponding to the fault injection service;
[0165] The installation unit is used to install the deployment configuration, service components and tool components in the configuration directory, service directory and tool directory respectively. The configuration directory, service directory and tool directory are all subdirectories of the root directory.
[0166] The startup condition detection unit is used to detect the configuration directory, service directory, and tool directory to determine whether the service startup conditions are met.
[0167] The service startup unit is used to trigger the startup of the fault injection service when the service startup conditions are met.
[0168] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.
[0169] This application also provides an apparatus including a processor and a memory, wherein the memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the fault injection service management method provided in the above method embodiments.
[0170] Furthermore, Figure 15 A schematic diagram of the hardware structure of a device is shown, which can participate in or include the apparatus or system provided in the embodiments of this application. Figure 15 As shown, device 15 may include one or more processors 1502 (shown as 1502a, 1502b, ..., 1502n in the figure) (processor 1502 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 1504 for storing data, and a transmission device 1506 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 15 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, device 15 may also include a... Figure 15 The more or fewer components shown, or having the same Figure 15 The different configurations shown.
[0171] It should be noted that the aforementioned one or more processors 1502 and / or other data processing circuitry are generally referred to herein as "data processing circuitry". This data processing circuitry may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within device 15 (or mobile device). As involved in the embodiments of this application, this data processing circuitry serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).
[0172] The memory 1504 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of this application. The processor 1502 executes various functional applications and data processing by running the software programs and modules stored in the memory 1504, thereby realizing the above-mentioned fault injection service management method. The memory 1504 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1504 may further include memory remotely located relative to the processor 1502, and these remote memories can be connected to the device 15 via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0173] The transmission device 1506 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of device 15. In one example, the transmission device 1506 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 1506 may be a radio frequency (RF) module used for wireless communication with the Internet.
[0174] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows a user to interact with the user interface of device 15 (or a mobile device).
[0175] This application also provides a computer-readable storage medium storing at least one instruction or at least one program, which is loaded and executed by a processor to implement the fault injection service management method provided in the above method embodiments.
[0176] Optionally, in this embodiment, the computer-readable storage medium may be located at at least one of a plurality of network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.
[0177] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. The device's processor reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the device to perform the fault injection service management method provided in the above-described method embodiments.
[0178] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.
[0179] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and electronic device embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0180] The foregoing description has fully disclosed the specific embodiments of this application. It should be noted that any modifications made by those skilled in the art to the specific embodiments of this application do not depart from the scope of the claims. Accordingly, the scope of the claims of this application is not limited to the foregoing specific embodiments.
Claims
1. A fault injection service management method, characterized in that, The method includes: Upon receiving a deployment instruction for a fault injection service, the fault injection service identifier and namespace are obtained from the deployment instruction. The fault injection service represents the service that performs fault injection behavior, and the namespace is used to indicate the deployment environment of the fault injection service. Based on the fault injection service identifier and the namespace, obtain the deployment configuration; Based on the deployment configuration, service components and tool components are obtained. The service components are used to support the operation of the fault injection service, and the tool components represent the fault injection tools used by the fault injection service. Install the deployment configuration, the service components, and the tool components locally to trigger the startup of the fault injection service; The step of installing the deployment configuration, the service components, and the tool components locally, and triggering the startup of the fault injection service, includes: Create a root directory corresponding to the fault injection service; The deployment configuration, the service component, and the tool component are respectively installed in the configuration directory, service directory, and tool directory, where the configuration directory, service directory, and tool directory are all sub-directories of the root directory; The configuration directory, the service directory, and the tool directory are checked to determine whether the service startup conditions are met. If the service startup conditions are met, the fault injection service is started.
2. The method according to claim 1, characterized in that, After installing the deployment configuration, the service components, and the tool components locally and triggering the startup of the fault injection service, the method further includes: Upon receiving a configuration update instruction from the configuration management platform, the system determines the version download script from the configuration directory, where the configuration directory is used to mark the installation location of the deployment configuration; Execute the version download script to obtain the new version control file; Replace the version control file in the configuration directory with the new version control file, and restart the fault injection service.
3. The method according to claim 1, characterized in that, After installing the deployment configuration, the service components, and the tool components locally and triggering the startup of the fault injection service, the method further includes: Upon receiving a component upgrade instruction sent by the target service, the upgrade identifier in the component upgrade instruction is obtained, and the upgrade identifier is used to indicate the target component to be upgraded; The target download script corresponding to the target component is determined from the configuration directory, which is used to mark the installation location of the deployment configuration; Execute the target download script to obtain the target component; Perform an upgrade operation on the target component and restart the fault injection service.
4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: After the fault injection service is successfully started, a service reporting message is sent to the service management platform. The service reporting message carries the service information of the fault injection service, so that the target service can detect whether the target component needs to be upgraded based on the service information, and send a component upgrade instruction to the device where the fault injection service is deployed if the target component needs to be upgraded.
5. The method according to claim 1, characterized in that, After obtaining the deployment configuration based on the fault injection service identifier and the namespace, the method further includes: Based on the fault injection service identifier and the namespace, obtain version control file information and configuration file information; The version control file information is used to update the version control file in the deployment configuration, and the configuration file information is used to update the configuration file in the deployment configuration.
6. The method according to claim 1 or 5, characterized in that, The process of obtaining service components and tool components based on the deployment configuration includes: Read the version control file in the deployment configuration to determine the service version information and tool version information; The service component is determined based on the service version information, and the tool component is determined based on the tool version information.
7. The method according to claim 1 or 2, characterized in that, The deployment configuration includes configuration files, version control files, download and start service scripts, restart service scripts, and multiple resource download scripts. The download and start service scripts are used to download the deployment configuration, service components, and tool components and start the fault injection service. The restart service scripts are used to restart the fault injection service.
8. A fault injection service management device, the device comprising: The deployment instruction receiving module is used to receive a deployment instruction for the fault injection service, and to obtain the fault injection service identifier and namespace from the deployment instruction. The fault injection service represents the service that performs fault injection behavior, and the namespace is used to indicate the deployment environment of the fault injection service. The deployment configuration acquisition module is used to acquire the deployment configuration based on the fault injection service identifier and the namespace; The component acquisition module is used to acquire service components and tool components based on the deployment configuration. The service components are used to support the operation of the fault injection service, and the tool components represent the fault injection tools used by the fault injection service. The deployment and implementation module is used to install the deployment configuration, the service components, and the tool components locally, and to trigger the startup of the fault injection service. The deployment and implementation module includes: The root directory creation unit is used to create the root directory corresponding to the fault injection service; The installation unit is used to install the deployment configuration, service components and tool components in the configuration directory, service directory and tool directory respectively. The configuration directory, service directory and tool directory are all subdirectories of the root directory. The startup condition detection unit is used to detect the configuration directory, service directory, and tool directory to determine whether the service startup conditions are met. The service startup unit is used to trigger the startup of the fault injection service when the service startup conditions are met.
9. A device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the fault injection service management method as described in any one of claims 1-7.