A test method and device of a distributed system, a storage medium and an electronic device

Abstract

This application discloses a testing method, apparatus, storage medium, and electronic device for a distributed system. The method includes: performing feature alignment on acquired multi-source heterogeneous data to obtain aligned feature tuples; identifying anomaly features in the aligned feature tuples that have changed relative to a historical baseline constructed from historical running data and / or preset application service constraint rules; generating exploratory test cases based on the multidimensional differential feature vectors of the anomaly features; triggering simulation testing based on the exploratory test cases, and performing multidimensional full-link testing in the shadow environment of the simulation test to determine the type and / or root cause of the anomaly features. This significantly improves the quality assurance efficiency and intelligent operation and maintenance level of the software system.

Description

Technical Field

[0001] This application relates to the fields of computer software testing and artificial intelligence application technology, specifically to a testing method, apparatus, and system for a distributed system. This application also relates to a computer storage medium and an electronic device. Background Technology

[0002] With the widespread adoption of cloud computing and microservice architectures, the depth and breadth of enterprise digital transformation are constantly expanding. Against this backdrop, software systems exhibit two significant characteristics: First, extreme decomposition and decoupling of the architecture, for example, business logic is broken down into numerous independent microservice units, with complex inter-service call relationships. Second, the diversification of business data formats, for example, business processing in industries such as finance and logistics involves massive amounts of unstructured data such as invoices and waybills. While this technological evolution improves efficiency, it also raises profound challenges: Challenge 1: The "Black Box" Dilemma of Unstructured Data Recognition. In industries such as finance and logistics, OCR (Optical Character Recognition) is widely used to process invoices and waybills. However, traditional OCR only provides confidence probabilities and cannot perceive business logic. For example, OCR might identify "100.00 yuan" as "10,000 yuan" with high confidence. This "visually correct but logically incorrect" defect is extremely difficult to detect in traditional monitoring and often leads to significant financial losses.

[0003] Challenge 2: Fault "Noise" and "Isolation" in Microservice Architecture. In distributed systems, an OutOfMemoryError (OOM) or configuration change in one node can trigger a chain reaction throughout the entire system. While existing monitoring tools (such as Prometheus and Zabbix) can issue alerts, testers often need to spend hours performing log reviews, severely impacting the time to repair (MTTR).

[0004] Challenge 3: The "Gap" Between Testing and Production Environments. Traditional testing processes are linear: development, testing, deployment. Live bugs often cannot be automatically fed back into the testing system after deployment. Summary of the Invention

[0005] This application provides a testing method for distributed systems to solve the problem that existing technologies cannot effectively construct fault causal chains or implement front-end and back-end logic consistency verification, resulting in difficulty in capturing hidden defects and low efficiency in fault location.

[0006] This application provides a testing method for a distributed system, including: The acquired multi-source heterogeneous data is feature aligned to obtain aligned feature tuples. Identify the anomalies in the alignment feature tuples that have changed relative to the historical baseline constructed from historical running data and / or the preset application service constraint rules. Based on the multidimensional difference feature vector of the aforementioned anomaly characteristics, exploratory test cases are generated; The simulation test is triggered based on the exploratory test cases, and multi-dimensional full-link tests are performed in the shadow environment of the simulation test to determine the type and / or root cause of the abnormal features.

[0007] In some embodiments, the step of performing feature alignment on the acquired multi-source heterogeneous data to obtain aligned feature tuples includes: Obtain unstructured images from the front end and structured interface data from the back end; Extract the first data features of the unstructured image and the second data features of the structured interface data; The first data feature and the second data feature are aligned according to the semantic mapping to obtain the aligned feature tuple.

[0008] In some embodiments, identifying anomalies in the alignment feature tuples that have changed relative to a historical baseline constructed from historical operational data and / or preset application service constraint rules includes: The first data feature and the second data feature in the alignment feature data are compared to construct a difference feature vector; Based on the distance between the differential feature vector and the historical statistical distribution in the historical baseline, determine whether a statistical anomaly has occurred; and / or, based on the matching correspondence between the differential feature vector and the preset application service constraint rules, determine whether a constraint anomaly has occurred. Clustering is performed on the identified statistical anomalies and / or constraint anomalies to obtain the anomaly characteristics.

[0009] In some embodiments, generating exploratory test cases based on the multidimensional difference feature vector of the anomaly features includes: The test synthesis engine is invoked based on the multidimensional difference feature vector to generate exploratory test cases for the anomaly features.

[0010] In some embodiments, triggering simulation testing based on the exploratory test cases and performing multi-dimensional end-to-end testing in the shadow environment of the simulation test to determine the type and / or root cause of the anomaly features includes: Simulation tests are triggered based on the aforementioned exploratory test cases; In the shadow environment of the simulation test, a multi-dimensional, full-link simulation test is performed on the anomaly characteristics. Based on the response data of the multi-dimensional full-link simulation test and the constraint rules in the knowledge base of the preset application service constraint rules, an assertion comparison is performed. If the assertion result indicates that the multi-dimensional full-link simulation test has passed, the abnormal feature is determined to be a new entity type, and the root cause is a configuration misconfiguration. If the assertion result indicates that the multi-dimensional full-link simulation test has a conflict, the abnormal feature is determined to be a defect type, and the root cause is a program logic break.

[0011] In some embodiments, it also includes: When the anomaly feature belongs to a new entity type, the exploratory test case is decomposed to obtain the atomic feature points of the exploratory test case; The similarity between the new entity type and the existing entity type is determined based on the atomic feature points. If the similarity meets the preset requirements, the new entity type is attached to the node corresponding to the existing entity type. If the similarity does not meet the preset requirements, an entity prototype node is added to the knowledge base.

[0012] In some embodiments, it also includes: Generate differentiated service constraint rules for newly added entity prototype nodes stored in the knowledge base, and assign initial confidence labels; The newly added entity prototype node was simulated and backtested to obtain the backtesting feedback results; Based on the backtest feedback results, the confidence level of the differentiated service constraint rules is adjusted by weight decay or gain. When the confidence level of the differentiated service constraint rule meets the preset confidence level requirement, the differentiated service constraint rule is merged into the corresponding application service constraint rule; when the confidence level of the differentiated service constraint rule does not meet the preset confidence level requirement, the differentiated service constraint rule is frozen or removed from the knowledge base.

[0013] In some embodiments, it also includes: When the confidence level of the differentiated service constraint rule meets the preset confidence level requirement, determine whether there is a conflict between the differentiated service constraint rule and the application service constraint rule of the existing entity type corresponding node; If not, then a parameterized template is constructed for the abnormal feature according to the differentiated service constraint rules and stored in the knowledge base.

[0014] This application also provides a distributed system testing apparatus, including: The acquisition unit is used to perform feature alignment on the acquired multi-source heterogeneous data to obtain aligned feature tuples. The identification unit is used to identify the abnormal features in the alignment feature tuple that have changed relative to the historical baseline constructed from historical running data and / or the preset application service constraint rules. The generation unit is used to generate exploratory test cases based on the extracted multidimensional feature vector of the anomaly features; The determination unit is used to trigger simulation testing based on the exploratory test cases, and to perform multi-dimensional full-link testing in the shadow environment of the simulation test to determine the type and / or root cause of the abnormal features.

[0015] This application also provides a testing system, including: a data acquisition and access layer, a processing engine layer, an automated test verification layer, and a knowledge feedback layer; The data acquisition and access layer includes: a multi-source heterogeneous data acquisition module, used to acquire multi-source heterogeneous data; The processing engine layer includes an identification module, a semantic analysis module, and a feature extraction module. The identification module is used to identify the unstructured data acquired by the multi-source heterogeneous data acquisition module. The semantic analysis module is used to perform feature alignment on the multi-source heterogeneous data to obtain aligned feature tuples. The feature extraction module is used to extract features from the aligned feature tuples and identify the abnormal features in the aligned feature tuples that have changed relative to the historical baseline constructed from historical running data and / or the preset application service constraint rules. The automated test verification layer includes: a test case generation module, a simulation backtesting module, and an assertion analysis module; the test case generation module is used to generate exploratory test cases based on the extracted multi-dimensional feature vectors of the anomaly features; the simulation backtesting module is used to trigger simulation backtesting based on the exploratory test cases, and perform multi-dimensional full-link testing in the shadow environment of the simulation backtesting to obtain the response data of the simulation backtesting; the assertion analysis module is used to determine the type and / or root cause of the anomaly features.

[0016] This application also provides a computer storage medium including a computer program that, when run on an electronic device, causes the electronic device to perform the steps in the above-described distributed system testing method.

[0017] This application also provides an electronic device, including: processor; The memory is used to store programs that process data generated by the electronic device. When the program is read and executed by the processor, it performs the steps in the test method for the distributed system described above.

[0018] Compared with the prior art, this application has the following advantages: The distributed system testing method provided in this application breaks down the barriers between front-end and back-end data verification by aligning features of acquired multi-source heterogeneous data to obtain aligned feature tuples. Based on the aligned feature tuples, it identifies anomalies that have changed relative to historical baselines and / or preset application service constraint rules constructed from historical operating data. Based on the multidimensional differential feature vectors of the anomalies, it generates exploratory test cases. By triggering simulation tests based on the exploratory test cases and executing multidimensional full-link tests in the shadow environment of the simulation tests, it determines the type and / or root cause of the anomalies, thereby achieving automatic characterization of fault root causes and constructing a knowledge base with self-evolution capabilities. This significantly improves the quality assurance efficiency and intelligent operation and maintenance level of the software system. Attached Figure Description

[0019] Figure 1 This is a flowchart of a testing method for a distributed system provided in this application.

[0020] Figure 2 This is a schematic diagram of an embodiment of feature alignment in a distributed system testing method provided in this application.

[0021] Figure 3 This is a schematic diagram of an embodiment of knowledge graph evolution in a distributed system testing method provided in this application.

[0022] Figure 4 This is a schematic diagram of the structure of a test device for a distributed system provided in this application.

[0023] Figure 5 This is a schematic diagram of the structure of a test system for a distributed system provided in this application.

[0024] Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0025] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0026] It should be noted that the terms "first," "second," "third," etc., in the claims, specification, and drawings of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. Such data are interchangeable where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown or described in this application. Furthermore, the terms "comprising," "having," and their variations are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0027] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship. "Contains A, B and / or C" means containing any one, two, or three of A, B, and C.

[0028] It should be understood that in the embodiments of this application, "B corresponding to A", "B corresponding to A", "A corresponds to B", or "B corresponds to A" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information.

[0029] Based on the above background description, it is clear that the inventive concept of this application arises from the technical challenges faced by enterprises in the evolution of software system architecture and the complexity of physical entities involved in digital transformation. Regarding the technical challenges mentioned above, existing technologies offer the following solutions: Option 1: Rule-based monitoring system based on fixed thresholds. This option primarily monitors the system's health status by pre-configuring static thresholds (such as CPU utilization and response time limits), representing the most traditional operational monitoring method. However, this approach lacks semantic correlation and struggles to handle complex fault scenarios. Fixed threshold monitoring can only judge "numerical values" and cannot understand the "business semantics" of the data. When a microservice experiences an OOM (Out of Memory) or other underlying fault triggering cascading alarms (i.e., fault "noise"), this option cannot automatically construct the causal chain of "configuration change - performance anomaly - business reconciliation failure." Faced with massive amounts of fragmented alarms, operations personnel still need to spend a significant amount of time manually reviewing logs, severely impacting the time to repair (MTTR).

[0030] Option 2: A source tracing solution based on distributed tracing (APM). Technologies such as Skywalking and Zipkin assist developers in troubleshooting and performance analysis by recording the propagation path and time distribution of requests in the microservice call chain. However, this solution cannot achieve end-to-end logical consistency verification. Distributed tracing technology primarily focuses on the call chain and performance metrics on the server side, ignoring the logical consistency between the actual display state on the client (frontend) and the final entity data. This results in the frontend UI and backend API being technically isolated, making it impossible to detect the hidden defect of "inconsistency between frontend visual display and backend logical data."

[0031] Option 3: Conventional OCR Post-Processing Verification Solution. This solution typically uses dictionary matching or regular expressions to perform format verification and basic filtering on the recognized text. However, this approach remains at a shallow level and cannot detect business logic. Conventional regular expression or dictionary verification can only verify format compliance and cannot perform in-depth business logic checks. For example, existing technology lacks effective means to detect and intercept errors such as misidentifying "100.00 yuan" as "10000 yuan," which conforms to the format but violates the business logic.

[0032] Furthermore, all three solutions mentioned above focus on "discovery" or "recording," lacking the closed-loop capability to automatically transform negative samples from the production environment into test cases. This prevents the testing system from evolving on its own, making it difficult to eliminate similar defects at their root.

[0033] Based on the above, this application proposes a testing method for distributed systems, such as... Figure 1 As shown, the test method includes: Step S101: Perform feature alignment on the acquired multi-source heterogeneous data to obtain aligned feature tuples; Step S102: Identify the anomaly features in the alignment feature tuples that have changed relative to the historical baseline constructed from historical running data and / or the preset application service constraint rules; Step S103: Generate exploratory test cases based on the extracted multidimensional feature vectors of the anomaly features; Step S104: Trigger simulation backtesting based on the exploratory test cases, and perform multi-dimensional full-link testing in the shadow environment of the simulation backtesting to determine the type and / or root cause of the abnormal features.

[0034] Before describing the above steps in detail, the technical terms used in this application will be explained.

[0035] Fault fingerprint: A unique vector representation generated by extracting features from multi-dimensional operational metrics (such as CPU utilization, memory growth rate, stack traces, and exception codes) of a system when a fault occurs. It is used to quickly locate historically similar problems in a fault database through similarity calculations.

[0036] Semantic Consistency Verification: A data verification mechanism that uses business logic rules (such as arithmetic relations, data type constraints, and contextual associations) to determine whether the structured data output by OCR or the interface conforms to logical common sense, rather than relying solely on the confidence score of the algorithm itself.

[0037] Automated Test Validation Vector: This refers to a set of targeted test cases dynamically extracted from an automated test case library based on identified fault characteristics. It includes not only scripts but also the specific environment variables and input parameters required to trigger the fault.

[0038] Multi-source heterogeneous data refers to data sets in distributed systems (such as microservice architectures) where the data sources and structural forms differ significantly. "Multi-source" can be understood as data originating from different system layers or nodes. In the embodiments of this application, this can be understood as two completely different physical locations: the front-end presentation layer (UI images, screenshots, etc.) and the back-end logic layer (API protocol streams, database logs, etc.). For example, a front-end UI image is a screenshot of a user's mobile phone screen, in the form of an unstructured pixel matrix (such as a PNG image displaying "Payment Successful"), whose text meaning cannot be directly read by a computer. Back-end API messages: These can originate from a server network interface, in the form of structured JSON data (such as {"status":"success","code":200}), whose field names and values ​​can be directly parsed by a computer. "Heterogeneous" refers to data with different representations and / or structures. In the embodiments, this refers to the difference in form between unstructured data and structured data, where unstructured data (such as UI images, pixel matrices that machines cannot understand), and structured data (such as API messages, JSON / XML fields that machines can directly parse).

[0039] Feature tuples: These are ordered data structures formed by binding semantically related front-end visual features with back-end data features through feature alignment. The structure can be represented as T=(F ui F api ), where F ui F represents front-end visual features (such as "displaying amount 100 yuan") apiThis represents backend data features (such as "amount field is 100"). Its function is to serve as the basic unit for multimodal difference calculation. In the algorithm, frontend visual features and backend data features are treated as data with a binding relationship, thereby determining whether the two are semantically matched.

[0040] Anomaly features refer to: In multimodal logical consistency verification, a set of features that are found to have significant deviations from historical norms or expected logic. It includes: Defect features of incorrect types (such as numerical inconsistencies, status contradictions) and / or newly emerging entity (which can also be understood as business) pattern features (such as a new function entry, a new field type). It belongs to the data that needs to be further simulated and backtested and / or verified. Error-type anomalies can occur when the system is supposed to be consistent but there are contradictions or violations of expectations. Such as UI and API inconsistencies, state machine contradictions, numerical out-of-bounds, etc. For example: The UI shows an inventory of 0, while the API returns 100; or the UI order status is "cancelled", but the API returns a status of "awaiting shipment". Another example: OCR recognizes the capital amount on the reimbursement form UI as "Ten thousand yuan exactly", and the lowercase amount as "¥10000.00", but the API returns the reimbursement_amount field as 100000 (one zero more), or the UI shows "payable 100", and the API returns "paid 90". Another example: OCR recognizes that the invoice UI has a red "void" stamp (or the recognized status label is "voided"), but the API returns the status of this invoice as still valid and allows it to continue to be used for deduction. Another example: OCR extracts "unit price 100, quantity 5, tax amount 13" from the detail list, but the total amount shown on the UI or returned by the API is not 513; or the tax rate changes from 13% to 6% without triggering any change approval. Another example: OCR recognizes the payee account name on the payment interface as "Beijing A Company", but the payee_account_name returned by the API actually points to "Shanghai B Company". New entity (business) anomalies are, for example: It is found that a "face recognition payment" button appears on the UI, and a face_id field appears in the API, etc. No more examples are given here, and the above examples are only used to illustrate that the defect features of the error type are not restrictive. Those that have not appeared in historical operations and belong to new business pattern features can also be new buttons / new pages on the UI and new endpoints / new fields in the API, without historical mappings. For example: A "face recognition payment" button appears on the UI, and a face_id field appears in the API. Another example: OCR discovers a new button "Cross-border RMB settlement" on the UI, and at the same time a cross_border_cny_id field appears in the API traffic, while historically this system only supported domestic RMB settlement. Another example: OCR recognizes a completely new bill format - full e-invoice (new version of electronic invoice) in the reimbursement upload area, and a new electronic_invoice_no is added to the API (originally there was only paper_invoice_no), etc.

[0041] Of course, this can also include: some abnormal features in logic and behavior patterns, such as: no new entities themselves are added, and no business errors are reported, but the relationships, interaction sequences, or weight ratios between entities have deviated. It can also include: misaligned field mappings, reversed process order, and distorted data distribution, i.e., some abnormal features in modal behavior. For example: previously, the UI's "A button" always triggered the API's / api / v1 / a, but now it triggers / api / v1 / b (A and B are old interfaces, but the mapping is wrong). Another example: OCR identifies the department attributing expenses on the UI as "R&D Department," but the underlying API's cost_center_id routes the expenses to "Marketing Department" (misaligned old department mappings may lead to incorrect budget application). Yet another example: historically, the UI would first display the "password input box" and then call the API to obtain the token; now, the token is obtained first, and then the UI renders the password box (reversed sequence logic may lead to security vulnerabilities or rendering flickering). For example, historically, in the `payment_method` field returned by the API, application A accounted for 70% and application B for 30%; in this verification, application B accounted for 99% and application A for 1% (not reaching the error threshold, but an anomaly deviating from the norm, possibly indicating a silent failure in a certain link). Another example: OCR identified the currently logged-in user's role as "cashier," but the UI rendered a "funds refund" button (which is historically invisible to cashiers), and the API had access to the refund interface. Therefore, anomaly characteristics can include not only defect types and new entity (business) types, but also behavioral pattern changes, compliance and environmental shifts, etc.

[0042] Historical baseline refers to the statistical regularity or characteristic distribution model exhibited by the system under normal operating conditions within a preset time window. It is constructed by statistically analyzing massive amounts of historical operational data. Examples include the historical numerical distribution range (mean, variance) of a field, the historical position coordinate range of UI elements, and the historical trigger frequency of specific business scenarios. Its function is to act as a "dynamic benchmark" to determine whether current characteristics are "newly emerging" or "unseen" (i.e., identifying new business models). For example, if the system's statistics for the "order amount" field have consistently ranged from "0 yuan to 5000 yuan" over the past 30 days, and a "100,000 yuan" order suddenly appears, although it doesn't violate business rules (the amount itself is legal), it deviates significantly from the historical baseline. The system will mark it as a statistical anomaly (potentially a large new transaction or data tampering).

[0043] Application service constraint rules refer to logical validation criteria predefined according to business logic, interface contracts, or system design documents, used to constrain data relationships. Specifically, they typically exist in the form of logical expressions or assertion functions. Examples include: "Inventory can only be deducted when the payment status is 'successful'," "The amount displayed on the front end must equal the amount transmitted on the back end," and "The mobile phone number must be 11 digits." Their function is to act as an "absolute benchmark" to determine whether features violate business logic (i.e., identify errors, defects, and bugs). They focus on "correctness," not "change." For example: Consistency rule: "The 'payment status' displayed on the front end must be consistent with the status_code in the backend API (e.g., the UI displays 'success' corresponding to code: 200)." Logical rule: "If the backend returns inventory < 0 (negative inventory), the frontend UI must not display the 'buy now' button." Formatting rule: "The mobile phone number field must satisfy an 11-digit regular expression."

[0044] Regarding step S101: Perform feature alignment on the acquired multi-source heterogeneous data to obtain aligned feature tuples.

[0045] Based on the above explanation of technical terminology, the purpose of this step is to obtain aligned feature tuples. In this embodiment, it can refer to an ordered data structure formed by binding semantically related front-end visual features with back-end data features. The multi-source heterogeneous data can be data with significant differences in source and structure. In this embodiment, the multi-source heterogeneous data can be collected front-end UI image data and back-end protocol streams (or underlying protocol streams), where the back-end protocol stream (or underlying protocol stream) refers to the sequence of network data packets generated during communication and interaction between the client (front-end) and the server (back-end) during the operation of the software system. For example, in the process of distributed system interaction, the structured data sequence transmitted between the front-end application and the back-end service, following a specific communication protocol (such as HTTP / HTTPS, RPC), can also be understood as a logical script running a service application. The request and response messages during the business interaction process can be captured in real time by the traffic acquisition module through network proxy interception, network card traffic mirroring, or server-side bytecode injection, and parsed into structured data features (such as JSON objects) for subsequent processing by the processing engine layer. In a product transaction scenario, there may be multiple types of coupons. Therefore, the feature tuple can include identifying the discount amount in the UI image and the underlying discount logic script in the backend. For example, identifying information such as 100 or coupon in the UI image and aligning it with unstructured data related to the 100 coupon in the backend (such as API messages) yields an aligned feature tuple.

[0046] Therefore, before obtaining the aligned feature tuple data, feature alignment needs to be performed on the acquired multi-source heterogeneous data. This involves acquiring unstructured image data from the front end and structured interface data from the back end, extracting the first data feature from the unstructured image data and the second data feature from the structured interface data, and then aligning the first and second data features. For example... Figure 2 As shown, the specific implementation includes: Step S101-1: Obtain the unstructured image from the front end and the structured interface data from the back end.

[0047] Step S101-2: Extract the first data feature of the unstructured image and the second data feature of the structured interface data; in this embodiment, data features can be extracted by recognizing unstructured images, for example: recognizing data features in UI images through OCR.

[0048] Step S101-3: Align the first data feature and the second data feature according to the semantic mapping to obtain the aligned feature tuple.

[0049] In application service testing, the text displayed on the front end and the code returned by the back end may differ literally, even though they are semantically identical. For example: The system simultaneously captured two sets of data from completely different sources: Source A: Front-end visual data (unstructured), data format: a screenshot of an app interface (pixel matrix). Content: The image contains a coupon with the text "¥20 off for purchases over ¥100, valid only at member stores".

[0050] Source B: Backend protocol stream data (structured), data format: API response message in JSON format. The content can be JSON code.

[0051] OCR recognition was performed on the UI image, and features were extracted: The text "Spend 100 and get 20 off, for members only" was recognized using OCR, and the keyword features "spend", "discount", and "member" were extracted. A visual feature vector was constructed: F ui {Business type keyword: discount for purchases over a certain amount, scope keyword: membership}. Parse API data, parse JSON fields, and extract key data features: field type: "THRESHOLD_DISCOUNT", field scope: "MEMBER_STORE", construct data feature vector F. api{Business type enumeration: Discount for full load, Applicable scope enumeration: Membership store}. Semantically match the front-end keywords "full load" and "discount" with the back-end enumeration values ​​"THRESHOLD_DISCOUNT" and "MEMBER_STORE", establishing a connection between the front-end "discount for full load" and the back-end "THRESHOLD_DISCOUNT"; semantically match the front-end keyword "membership" with the back-end enumeration value "MEMBER_STORE", and establish a connection between them. The obtained aligned feature tuples can include multiple pairs, for example: First tuple: T1 = (Front-end feature "discount for full load", Back-end feature THRESHOLD_DISCOUNT), explanation: binds the business type identified by the front-end with the business type defined by the back-end as a feature pair. Second tuple: T2 = (Front-end feature "membership", Back-end feature "MEMBER_STORE"), explanation: binds the applicable scope identified by the front-end with the applicable scope defined by the back-end as a feature pair.

[0052] Step S101 utilizes feature alignment technology to construct a unified "aligned feature tuple" by integrating front-end visual features with back-end data features, and then builds a "multidimensional differential feature vector" containing multiple dimensions such as numerical, semantic, and structural features. This not only breaks down the barriers between data modalities but also accurately captures hidden defects such as "visually correct but logically incorrect" (e.g., correct OCR recognition but conflicting business meaning), significantly improving the depth and accuracy of testing.

[0053] Regarding step S102: Identify the anomaly features in the alignment feature tuples that have changed relative to the historical baseline constructed from historical running data and / or the preset application service constraint rules.

[0054] Following the example in step S101 above, step S102 identifies tuples T1 and T2. The comparison reference standard for identification can be a historical baseline constructed from historical operating data and / or preset application service constraint rules. The historical baseline and constraint rules are explained below.

[0055] Regarding historical baselines: The scenario could be: the data itself is logically correct, but there are problems compared to historical operating patterns.

[0056] Suppose we compare tuple T2 with the statistical distribution in the historical baseline database (this can be done through the processing engine layer): Tuple T2 data consists of: front-end "Member" and back-end "MEMBER_STORE". Historical baseline: Querying the past 30 days of operational data reveals that the "MEMBER_STORE" (Member Store) field appears only 0.1% of the time (extremely rare), and has never appeared in combinations where the "Coupon Type" is "Discount". The historical mainstream baseline is "ALL_STORE" (All Stores). The difference calculation result is the distance between the current feature vector and the historical distribution center. A significant deviation was found, thus indicating a deviation from historical statistical patterns and identifying anomalies such as the emergence of a new entity (or new business).

[0057] The scenario could be: an error occurs in the backend data, or there is an error in the frontend display logic, resulting in a breakdown in business logic.

[0058] Suppose tuple T2 is input into the processing engine layer for validation: the front end is "full reduction", the back end is "THRESHOLD_DISCOUNT", and the preset constraint rule is: the full reduction coupon has a threshold amount. The "threshold amount field" associated with the coupon is 0 in the backend data. This is judged as: inconsistent with the business logic rule, indicating an anomaly that contradicts the preset constraint rule, such as a defect or bug.

[0059] Based on the above example, the specific implementation process of step S102 may include: Step S102-1: Compare the first data feature and the second data feature in the aligned feature data to construct a difference feature vector; wherein, the first data feature and the second data feature may include multiple data features. This step may be performed after obtaining the aligned feature tuple T = (first data feature F) ui Second data feature F api After that, the first data feature F within the tuple is calculated by calling the difference calculation method. ui Second data feature F api Perform deep comparison to generate differential feature vectors V representing the differences. diff Assume the first data feature F ui Second data feature F api For numerical data types (such as amount, quantity, etc.), calculate the absolute difference or relative error rate between the two. For example: F ui To identify the value of 100 yuan in the UI image, F api To obtain the value 10000 returned by the API, the numerical difference vector component is |F. ui -F api | = |100 - 10000 | = 9900. If the first data feature F ui Second data feature F apiFor text or enumeration types (such as status, category, etc.), calculate the matching degree between two objects using semantic similarity, and encode the calculated matching degree result into vector components. For example: F ui For the UI text "Discount for full", F api For the API enumeration "THRESHOLD", if a match is determined, the component is encoded as 1 (same or identical); if F ui For "direct reduction" and F api If the two are determined to be mismatched, the component is encoded as 0 (conflict). Verify the first data feature F. ui Second data feature F api Are there any missing fields? For example, if a field matching F cannot be found... ui Corresponding F api This generates "missing" vector components. Therefore, the difference eigenvector V diff =[V 数值 V 语义 V 结构 ] represents the degree of deviation between the front end and the back end, and can be a multi-dimensional difference feature vector including numerical, semantic, and structural features.

[0060] Step S102-2: Determine whether statistical anomalies have occurred based on the distance between the differential feature vector and the historical statistical distribution in the historical baseline; and / or, determine whether constraint anomalies have occurred based on the matching correspondence between the differential feature vector and the preset application service constraint rules. This step can be verified through the historical baseline dimension, the constraint rule dimension, or a dual-dimensional verification using both the historical baseline and constraint rules. Dimension one: Statistical anomaly determination based on the historical baseline can be achieved by extracting historical statistical distribution data corresponding to the aligned feature tuples from the knowledge base, such as historical mean, standard deviation, etc. Calculate the differential feature vector V. diff The distance from the historical distribution center. If the distance exceeds a preset threshold, then the first data feature F is determined. ui Second data feature F api Deviation, or statistical anomaly, can be understood as the appearance of new entities or abnormal traffic flows that have never occurred before. Dimension Two: Constraint anomaly determination based on constraint rules can be achieved by analyzing the differential feature vector V. diff The input is fed into the processing engine layer for logical assertions, such as: making assertions based on application business logic rules (or application entity logic rules), like requiring equal amounts or state mappings to conform to a dictionary. If the difference vector originates from the first data feature F... ui Second data feature F api The semantic consistency of the comparison results is encoded as a conflict (e.g., F). ui Successfully displayed (corresponding to F)api If the status is "failed" or a hard constraint condition is violated, then a constraint change is determined to have occurred.

[0061] Step S102-3: Cluster the identified statistical anomalies and / or constraint anomalies to obtain the anomaly features. Specifically, this can be done by mapping each sample determined to have an anomaly to a point in the feature space, where the point is represented by a multidimensional difference feature vector V. diff Definition. Based on the calculated feature distances (such as Euclidean distance or cosine similarity) between different sample points, samples with feature distances less than a preset distance threshold are aggregated into anomaly feature clusters. For example: if multiple anomaly samples are obtained within a certain period, clustering can be used to discover the V of these samples. diff If samples show high similarity in UI location features (such as the top of the page) and amount ratio features (discount ratio), they can be determined to belong to the same new business model (such as "brand collaboration tiered coupons"). It is understood that in this embodiment, the anomaly features include multi-dimensional differential feature vectors, such as the aforementioned V... diff =[V 数值 V 语义 V 结构 The anomaly features may include defect features of error types and / or newly emerging entity pattern features (i.e., new business model features). The anomaly features may include differential feature vectors of multiple dimensions, such as: anomaly features related to text, anomaly features related to UI position, anomaly features related to amount, etc.

[0062] It is understood that in this embodiment, the anomaly features may include defect features of error type and / or newly emerging entity pattern features (i.e., new business model features). Of course, it is not limited to the feature types exemplified in the examples. The anomaly features may include multi-dimensional differential feature vectors, such as: anomaly features related to text, anomaly features related to UI position, anomaly features related to amount, etc. The above examples are for illustrative purposes only and are not intended to limit the content or type of anomaly features. When the identification result of step S102 indicates the existence of a changed anomaly feature, exploratory test cases are generated in step S103 to facilitate determining the attribution of the anomaly feature.

[0063] Regarding step S103: Generate exploratory test cases based on the multidimensional differential feature vector of the anomaly features.

[0064] In this embodiment, the test synthesis engine can be invoked based on the multidimensional difference feature vector to generate exploratory test cases targeting the anomaly features. Of course, the difference feature vector of the anomaly features can be one-dimensional or multi-dimensional. Key feature parameters of the anomaly are extracted from the multidimensional difference feature vector; for example, parameters related to V... 数值 V语义 V 结构 These are related feature parameters. Based on these feature parameters, the test input set and verification points are automatically constructed. For example, assuming the differential feature vector indicates "new type of coupon," simulated business inputs (such as constructing different product combinations, various settlement scenarios, etc.) are automatically generated as driving data for the test. Verification scripts are generated based on the driving data, such as monitoring scripts for the settlement interface, to verify whether incomplete or unknown coupon data can be correctly processed.

[0065] Regarding step S104: Trigger simulation backtesting based on the exploratory test cases, and perform multi-dimensional full-link testing in the shadow environment of the simulation backtesting to determine the type and / or root cause of the abnormal features.

[0066] The specific implementation process of step S104 may include: Step S104-1: Trigger simulation backtesting based on the exploratory test cases; specifically, the system may inject the exploratory test cases generated in step S103 into the simulation backtesting scheduler. The scheduler parses the execution instructions in the test cases, initializes the backtesting task context, and prepares the corresponding test dataset (such as simulated product combinations, coupon parameters, etc.) to prepare for subsequent offline or real-time verification.

[0067] Step S104-2: Perform multi-dimensional full-link simulation testing on the abnormal features in the shadow environment of the simulation backtesting. Specifically, the system can mirror and copy the real-time traffic of the production environment and import it into a shadow environment with a configuration completely consistent with the production environment. Data changes in this shadow environment will not be reflected in the real production database, ensuring the security of the test. The system replays the copied traffic in the shadow environment and injects simulated inputs (such as abnormal coupon features) from the exploratory test cases into the request link. The test execution engine drives the request to pass through the gateway layer, business logic layer, and data persistence layer in sequence, completely simulating the user's link process from "placing an order" to "settling," and capturing the response data, logs, and status codes of each node in real time.

[0068] Step S104-3: Based on the response data of the multi-dimensional full-link simulation test and the constraint rules in the knowledge base of the preset application service constraint rules, an assertion comparison is performed. If the assertion result indicates that the multi-dimensional full-link simulation test has passed, the abnormal feature is determined to be a new entity type, and the root cause is a configuration misconfiguration. If the assertion result indicates that the multi-dimensional full-link simulation test has a conflict, the abnormal feature is determined to be a defect type, and the root cause is a program logic break. Specifically, the system can collect the response data after the shadow environment has been executed and input it into the assertion analysis module for judgment. For example, if the response data shows that the business process is normally closed (e.g., order creation is successful, amount calculation is correct) and conforms to the preset application service constraint rules (e.g., status transition is correct), the assertion result is passed, and the system determines that the abnormal feature belongs to a new entity type (e.g., normal new business). The root cause is determined to be a configuration misconfiguration (i.e., the system capability supports it, but it is misjudged as an abnormality only because the rules are not configured). For example, if the response data contains error status codes, throws exceptions (such as null pointer exceptions or type mismatches), or violates application service constraints (such as incorrect amount calculations), the assertion result is a conflict, and the system determines that the anomaly is a defect type (Bug). The root cause is identified as a break in program logic (i.e., the system code has a logical defect that prevents it from handling the specific business scenario).

[0069] In step S104, the multi-dimensional full-link simulation test execution end-to-end closed-loop link, from simulating user operation behavior to the final output result of the full-link API request process, realizes the leap from static writing to dynamic generation of test cases, improves the coverage of unknown entity (business) forms, and at the same time, the automatic determination of the root cause of front-end rendering or back-end logic through bidirectional comparison also improves the efficiency of end-to-end quality assurance.

[0070] Based on the above, this embodiment can have dual-path discrimination: new entities (requirements) and defects (bugs). For new entities / misconfigurations, the above method clarifies that even if there is no corresponding unknown entity in the knowledge base (e.g., an unknown coupon type), if the entire chain is successfully closed-loop during multi-dimensional full-link simulation testing and the execution result is logically consistent, then an automatic completion mechanism can be triggered to store the new entity in the knowledge base. For defects, i.e., those determined to be program logic breaks (logic conflicts), such as chain interruptions during multi-dimensional full-link simulation testing (e.g., interface errors), logical contradictions (e.g., coupons displaying a 50 discount, but the actual discount at checkout is 10), or UI anomalies (text overlap), automatic capture of chain logs, stack traces, and front-end / back-end difference data can be triggered to automatically generate a bug ticket.

[0071] To reduce reliance on manual maintenance, such as Figure 3 As shown, this embodiment may further include: Step S105-1: When the abnormal feature belongs to a new entity type, the exploratory test case is decomposed to obtain the atomic feature points of the exploratory test case.

[0072] Step S105-2: Determine the similarity between the new entity type and the existing entity type based on the atomic feature points; if the similarity meets the preset requirements, then attach the new entity type to the node corresponding to the existing entity type; if the similarity does not meet the preset requirements, then add an entity prototype node in the knowledge base.

[0073] The purpose of step S105-1 is to reverse engineer the exploratory test cases used to verify the new entity type, abstracting them into reusable knowledge units. The specific process may include: Step S105-11: Parse the input parameters of the exploratory test case, extract the set of preconditions required to activate the new entity type, such as: extract the effective time range of the coupon, the applicable product category ID, the user membership level requirements, etc., and obtain the atomic feature points of the triggering conditions.

[0074] Step S105-12: Analyze the calculation formulas or processing rules in the execution process of the exploratory test cases. For example, extract the "full discount formula: Amount = Original Price - Floor(Original Price / Threshold)" from the calculation logic of the settlement amount. "Reduction value", obtain logical atom feature points.

[0075] Step S105-13: Analyze the state change sequence on the execution path of the exploratory test cases. For example, extract state transition paths such as "pending payment → partial refund → closed" to obtain state atomic feature points.

[0076] The above three atomic feature points are only one example in this embodiment and are not intended to limit the implementation of step S105. In other application scenarios, atomic feature points related to exploratory test cases may be included.

[0077] Step S105-2 can determine the topological position of a new entity type in the knowledge graph through similarity, thereby enabling the dynamic evolution of the knowledge base. Specifically, it may include: Step S105-21: Map the atomic feature points extracted in step S105-1 to vectors in a high-dimensional feature space. Simultaneously, retrieve the feature vectors corresponding to all existing entity type nodes from the knowledge base. Calculate the cosine similarity or Euclidean distance between the new entity vector and each existing entity vector. If the calculated similarity value exceeds a preset attachment threshold (e.g., similarity > 0.7), the new entity is determined to be a special variant of an existing entity. Create a new child node in the knowledge graph and attach it to the existing entity node with the highest similarity, establishing a parent-child inheritance relationship. The new child node automatically inherits the parent node's general rules (such as general monetary calculation logic) and only needs to maintain its unique differentiated features (such as brand restriction rules), achieving compressed knowledge storage. If the similarity with all existing entities is below the preset threshold, the new entity is determined to belong to a completely new and independent business category. In the root directory or corresponding level of the knowledge graph, create a brand new independent node and mark it as the new entity (business) prototype node. This new entity prototype node becomes the starting point of the new knowledge cluster, and similar entities that appear later can be attached to it to expand the knowledge base categories.

[0078] Based on the above, it may further include: Step S106-1: Generate differentiated service constraint rules for the newly added entity prototype nodes stored in the knowledge base, and assign initial confidence labels; Step S106-2: Perform simulation backtesting on the newly added entity prototype node and obtain backtesting feedback results; Step S106-3: Based on the backtest feedback results, adjust the confidence level of the differentiated service constraint rules by weight decay or gain; Step S106-4: When the confidence level of the differentiated service constraint rule meets the preset confidence level requirement, the differentiated service constraint rule is merged into the corresponding application service constraint rule; when the confidence level of the differentiated service constraint rule does not meet the preset confidence level requirement, the differentiated service constraint rule is frozen or removed from the knowledge base.

[0079] Based on the above, it also includes: Step S107-1: When the confidence level of the differentiated service constraint rule meets the preset confidence level requirement, determine whether there is a conflict between the differentiated service constraint rule and the application service constraint rule of the existing entity type corresponding node; Step S107-2: If not, construct a parameterized template for the abnormal feature according to the differentiated service constraint rules, and store the parameterized template in the corresponding node of the knowledge base.

[0080] Step S107-3: If so, update or overwrite the conflict rules according to the preset conflict arbitration strategy to achieve iterative correction of the knowledge base.

[0081] For example: The above is a description of an embodiment of a distributed system testing method provided in this application. This method has the following advantages compared with the prior art: First, it can construct a unified "aligned feature tuple" by integrating front-end visual features and back-end data features through feature alignment technology, and build a "multi-dimensional difference feature vector" containing multiple dimensions such as numerical, semantic, and structural features. This not only breaks down the barriers between data modalities, but also accurately captures the hidden defects of "visually correct but logically incorrect" (such as correct OCR recognition but conflicting business meanings), significantly improving the depth and accuracy of testing. It overcomes the problem in existing technologies where front-end UI images (unstructured data) and back-end API protocol streams (structured data) are separated, making joint verification difficult.

[0082] Second, a dual-dimensional verification method can be used: using "application service constraint rules" to identify logical errors (defects) and using "historical baselines" to identify statistical anomalies (new services or new entities). Further qualitative analysis through simulation testing can automatically determine whether the anomaly characteristics belong to "program logic breakage (bug)" or "new entity type (new service)," thus achieving precise fault location and classification, greatly reducing the troubleshooting costs for operations and maintenance personnel. Furthermore, dual-dimensional verification can be selected according to different scenarios or needs; application service constraint rules can be chosen to identify logical errors, or historical baselines can be chosen to identify statistical anomalies. Additional dimension verification items can also be added based on requirements or scenarios.

[0083] Third, this application changes the traditional testing model of "testing without regard to consequences, stopping once testing is complete," automatically converting identified anomalies into exploratory test cases, thus achieving automated verification. By introducing a "confidence decay" mechanism, new rules are continuously verified. High-confidence rules are merged and made valid, while low-confidence rules are automatically frozen or removed, achieving automatic slimming and iteration of the knowledge base. By calculating the similarity between new entities and existing entities, "mounting" or "creating" operations are automatically executed, realizing the dynamic evolution of the knowledge graph topology. This gives the system the ability to learn and correct itself, reducing manual maintenance costs.

[0084] Fourth, by introducing a "shadow environment" for simulation backtesting, high-fidelity end-to-end verification of unknown anomalies can be performed without affecting real production data and user business. This ensures both the authenticity of the test (real traffic, real system architecture) and the absolute security of the production system, solving the pain point that traditional testing cannot cover complex production environment scenarios. Unlike traditional fixed script testing, this invention utilizes multi-dimensional differential feature vectors and automatically maps test strategies through a test synthesis engine to generate targeted "exploratory test cases." Simultaneously, by breaking down complex business logic into atomic feature points such as "trigger conditions, operational logic, and state machine transitions," parameterized and templated business logic is achieved, resulting in highly reusable and generalizable test cases.

[0085] Fifth, the technical solution of this application differs from traditional fixed script testing. It utilizes multi-dimensional differential feature vectors and automatically maps test strategies through a test synthesis engine to generate targeted "exploratory test cases." Simultaneously, by breaking down complex business logic into atomic feature points such as "trigger conditions, operational logic, and state machine transitions," it achieves parameter templating of business logic, enabling the generated test cases to possess high reusability and generalization capabilities.

[0086] In specific application scenarios, such as e-commerce application service platforms, the introduction of fault fingerprinting and automated testing significantly improves the mean time to repair (MTTR) and test regression efficiency of distributed systems under complex changing environments, effectively ensuring the stability of the core transaction chain of e-commerce. For example, it deeply couples the reconciliation logic of e-commerce application service platforms with multi-source heterogeneous data of system monitoring indicators, solving the problems of fragmented alarms (e.g., an OOM fault may cause 100 different alarms, such as high CPU, slow response, log errors, etc., pointing 100 alarms to the same multi-dimensional differential feature vector, i.e., aggregating them into a fault fingerprint and reporting it once) and the difficulty of root cause localization in traditional monitoring.

[0087] The above is a detailed description of an embodiment of a distributed system testing method provided in this application. Corresponding to the aforementioned embodiment of a distributed system testing method, this application also discloses an embodiment of a distributed system testing apparatus. Please refer to [link / reference]. Figure 4 Since the device embodiments are basically similar to the method embodiments, the description is relatively simple, and relevant parts can be referred to in the description of the method embodiments. The device embodiments described below are merely illustrative.

[0088] like Figure 4 As shown, the test apparatus for the distributed system provided in this application includes: The acquisition unit 401 is used to perform feature alignment on the acquired multi-source heterogeneous data to obtain aligned feature tuples; The identification unit 402 is used to identify the abnormal features in the alignment feature tuple that have changed relative to the historical baseline constructed from the historical running data and / or the preset application service constraint rules. The generation unit 403 is used to generate exploratory test cases based on the extracted multidimensional feature vector of the anomaly features; The determination unit 404 is used to trigger simulation testing based on the exploratory test cases, and perform multi-dimensional full-link testing in the shadow environment of the simulation test to determine the type and / or root cause of the abnormal features.

[0089] The specific implementation process of the obtaining unit 401 includes: an obtaining subunit, an extraction subunit, and an obtaining subunit; the obtaining subunit is used to obtain the unstructured image from the front end and the structured interface data from the back end; the extraction subunit is used to extract the first data feature of the unstructured image and the second data feature of the structured interface data; the obtaining subunit is used to perform feature alignment on the first data feature and the second data feature according to the semantic mapping to obtain the aligned feature tuple.

[0090] The specific implementation process of the identification unit 402 includes: a construction subunit, a first determination subunit, and a second determination subunit; the construction subunit is used to compare the first data feature and the second data feature in the alignment feature data to construct a differential feature vector; the first determination subunit is used to determine whether a statistical anomaly has occurred based on the distance between the differential feature vector and the historical statistical distribution in the historical baseline; and / or, to determine whether a constraint anomaly has occurred based on the matching correspondence between the differential feature vector and the preset application service constraint rules; the second determination subunit is used to determine the changed anomaly features based on the occurred statistical anomaly and / or constraint anomaly.

[0091] The specific implementation process of the generation unit 403 includes: calling the test synthesis engine to generate exploratory test cases for the abnormal features based on the multidimensional difference feature vector.

[0092] The specific implementation process of the determining unit 404 includes: a triggering subunit, a simulation testing subunit, and a determination subunit; the triggering subunit is used to trigger simulation testing according to the exploratory test cases; the simulation testing subunit is used to perform multi-dimensional full-link simulation testing on the abnormal feature in the shadow environment of the simulation test; the determination subunit is used to perform assertion comparison based on the response data of the multi-dimensional full-link simulation testing and the constraint rules in the knowledge base of the preset application service constraint rules. If the assertion result is that the multi-dimensional full-link simulation testing passes, then the abnormal feature is determined to be a new entity type, and the root cause is a configuration misconfiguration; if the assertion result is that the multi-dimensional full-link simulation testing conflicts, then the abnormal feature is determined to be a defect type, and the root cause is a program logic break.

[0093] Further, it may include: a decomposition unit, used to decompose the exploratory test case to obtain the atomic feature points of the exploratory test case when the abnormal feature belongs to a new entity type; and an attachment unit, used to determine the similarity between the new entity type and an existing entity type based on the atomic feature points. If the similarity meets a preset requirement, the new entity type is attached to the node corresponding to the existing entity type. If the similarity does not meet the preset requirement, it includes an addition unit, used to add an entity prototype node to the knowledge base.

[0094] Further, it may include: a rule generation unit, for generating differentiated service constraint rules for newly added entity prototype nodes stored in the knowledge base, and assigning initial confidence labels; a backtesting acquisition unit, for performing simulation backtesting on the newly added entity prototype nodes and obtaining backtesting feedback results; an adjustment unit, for adjusting the confidence of the differentiated service constraint rules by weight decay or gain based on the backtesting feedback results obtained by the backtesting acquisition unit; a merging unit, for merging the differentiated service constraint rules into the corresponding application service constraint rules when the confidence of the differentiated service constraint rules meets the preset confidence requirements; and a freezing or removal unit, for freezing or removing the differentiated service constraint rules from the knowledge base when the confidence of the differentiated service constraint rules does not meet the preset confidence requirements.

[0095] It may further include: a conflict determination unit, used to determine whether there is a conflict between the differentiated service constraint rule and the application service constraint rule of the existing entity type corresponding node when the confidence level of the differentiated service constraint rule meets the preset confidence level requirement; and a storage unit, used to construct a parameterized template for the abnormal feature according to the differentiated service constraint rule and store it in the knowledge base when the determination result of the conflict determination unit is negative.

[0096] The above is a description of an embodiment of a testing device for a distributed system provided in this application. The specific implementation process of the device can be referred to the above-described embodiment of the testing method, and will not be described in detail here.

[0097] Based on the above, this application also provides a testing system for distributed systems, such as... Figure 5 As shown, the system includes: a data acquisition and access layer 501, a processing engine layer 502, an automated testing and verification layer 503, and a knowledge feedback layer 504.

[0098] The data acquisition and access layer 501 includes: a multi-source heterogeneous data acquisition module, used to acquire multi-source heterogeneous data; The processing engine layer 502 includes an identification module, a semantic analysis module, and a feature extraction module. The identification module is used to identify the unstructured data acquired by the multi-source heterogeneous data acquisition module. The semantic analysis module is used to perform feature alignment on the multi-source heterogeneous data to obtain aligned feature tuples. The feature extraction module is used to extract features from the aligned feature tuples and identify the abnormal features in the aligned feature tuples that have changed relative to the historical baseline constructed from historical running data and / or the preset application service constraint rules. The automated test verification layer 503 includes: a test case generation module, a simulation test module, and an assertion analysis module; the test case generation module is used to generate exploratory test cases based on the extracted multi-dimensional feature vectors of the anomaly features; the simulation test module is used to trigger simulation tests based on the exploratory test cases, and perform multi-dimensional full-link tests in the shadow environment of the simulation tests to obtain the response data of the simulation tests; the assertion analysis module is used to determine the type and / or root cause of the anomaly features.

[0099] The knowledge feedback layer 504 is used to generate differentiated service constraint rules for newly added entity prototype nodes stored in the knowledge base and assign initial confidence labels; the feedback is sent to the automated testing and verification layer 503 to perform simulation backtesting on the newly added entity prototype nodes and obtain backtesting feedback results; based on the backtesting feedback results, the confidence of the differentiated service constraint rules is adjusted by weight decay or gain; when the confidence of the differentiated service constraint rules meets the preset confidence requirements, the differentiated service constraint rules are merged into the corresponding application service constraint rules; when the confidence of the differentiated service constraint rules does not meet the preset confidence requirements, the differentiated service constraint rules are frozen or removed from the knowledge base.

[0100] The specific implementation process of the above-mentioned testing system can also be found in the content of the above-mentioned testing method embodiments, and will not be described in detail here.

[0101] Based on the above, this application also provides a computer storage medium, including a computer program, which, when run on an electronic device, causes the electronic device to perform the steps in the test method described above.

[0102] Based on the above, this application also provides an electronic device, such as... Figure 6 As shown, it includes: Processor 601; The memory 602 is used to store a program for processing data generated by the electronic device. When the program is read and executed by the processor, it performs the steps in the test method described above.

[0103] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0104] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0105] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0106] 1. Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0107] 2. Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

Claims

1. A method of testing a distributed system, characterized by, include: The acquired multi-source heterogeneous data is feature aligned to obtain aligned feature tuples. Identify the anomalies in the alignment feature tuples that have changed relative to the historical baseline constructed from historical running data and / or the preset application service constraint rules. Based on the multidimensional difference feature vector of the aforementioned anomaly characteristics, exploratory test cases are generated; The simulation test is triggered based on the exploratory test cases, and multi-dimensional full-link tests are performed in the shadow environment of the simulation test to determine the type and / or root cause of the abnormal features.

2. The method of claim 1, wherein, The step of aligning the acquired multi-source heterogeneous data to obtain aligned feature tuples includes: Obtain unstructured images from the front end and structured interface data from the back end; Extract the first data features of the unstructured image and the second data features of the structured interface data; The first data feature and the second data feature are aligned according to the semantic mapping to obtain the aligned feature tuple.

3. The method of claim 1, wherein, The identification of anomalies in the alignment feature tuples that have changed relative to the historical baseline constructed from historical operational data and / or preset application service constraint rules includes: The first data feature and the second data feature in the alignment feature data are compared to construct a difference feature vector; Based on the distance between the differential feature vector and the historical statistical distribution in the historical baseline, determine whether a statistical anomaly has occurred; and / or, based on the matching correspondence between the differential feature vector and the preset application service constraint rules, determine whether a constraint anomaly has occurred. Clustering is performed on the identified statistical anomalies and / or constraint anomalies to obtain the anomaly characteristics.

4. The method of claim 1, wherein, The step of generating exploratory test cases based on the multidimensional difference feature vector of the anomaly features includes: The test synthesis engine is invoked based on the multidimensional difference feature vector to generate exploratory test cases for the anomaly features.

5. The method of claim 1, wherein, The step of triggering simulation testing based on the exploratory test cases and performing multi-dimensional end-to-end testing in the shadow environment of the simulation test to determine the type and / or root cause of the anomaly features includes: Simulation tests are triggered based on the aforementioned exploratory test cases; In the shadow environment of the simulation test, a multi-dimensional, full-link simulation test is performed on the anomaly characteristics. Based on the response data of the multi-dimensional full-link simulation test and the constraint rules in the knowledge base of the preset application service constraint rules, an assertion comparison is performed. If the assertion result indicates that the multi-dimensional full-link simulation test has passed, the abnormal feature is determined to be a new entity type, and the root cause is a configuration misconfiguration. If the assertion result indicates that the multi-dimensional full-link simulation test has a conflict, the abnormal feature is determined to be a defect type, and the root cause is a program logic break.

6. The method of claim 5, wherein, Also includes: When the anomaly feature belongs to a new entity type, the exploratory test case is decomposed to obtain the atomic feature points of the exploratory test case; The similarity between the new entity type and the existing entity type is determined based on the atomic feature points. If the similarity meets the preset requirements, the new entity type is mounted onto the node corresponding to the existing entity type. If the similarity does not meet the preset requirements, a new entity prototype node will be added to the knowledge base.

7. The method according to claim 6, characterized in that, Also includes: Generate differentiated service constraint rules for newly added entity prototype nodes stored in the knowledge base, and assign initial confidence labels; The newly added entity prototype node was simulated and backtested to obtain the backtesting feedback results; Based on the backtest feedback results, the confidence level of the differentiated service constraint rules is adjusted by weight decay or gain. When the confidence level of the differentiated service constraint rule meets the preset confidence level requirement, the differentiated service constraint rule is merged into the corresponding application service constraint rule. If the confidence level of the differentiated service constraint rule does not meet the preset confidence level requirement, the differentiated service constraint rule will be frozen or removed from the knowledge base.

8. The method of claim 6, wherein, Also includes: When the confidence level of the differentiated service constraint rule meets the preset confidence level requirement, determine whether there is a conflict between the differentiated service constraint rule and the application service constraint rule of the existing entity type corresponding node; If not, then a parameterized template is constructed for the abnormal feature according to the differentiated service constraint rules and stored in the knowledge base.

9. A distributed system testing apparatus, characterized by, include: The acquisition unit is used to perform feature alignment on the acquired multi-source heterogeneous data to obtain aligned feature tuples. The identification unit is used to identify the abnormal features in the alignment feature tuple that have changed relative to the historical baseline constructed from historical running data and / or the preset application service constraint rules. The generation unit is used to generate exploratory test cases based on the extracted multidimensional feature vector of the anomaly features; The determination unit is used to trigger simulation testing based on the exploratory test cases, and to perform multi-dimensional full-link testing in the shadow environment of the simulation test to determine the type and / or root cause of the abnormal features.

10. A test system, characterized by include: Data acquisition and access layer, processing engine layer, automated testing and verification layer, and knowledge feedback layer; The data acquisition and access layer includes: a multi-source heterogeneous data acquisition module, used to acquire multi-source heterogeneous data; The processing engine layer includes an identification module, a semantic analysis module, and a feature extraction module. The identification module is used to identify the unstructured data acquired by the multi-source heterogeneous data acquisition module. The semantic analysis module is used to perform feature alignment on the multi-source heterogeneous data to obtain aligned feature tuples. The feature extraction module is used to extract features from the aligned feature tuples and identify the abnormal features in the aligned feature tuples that have changed relative to the historical baseline constructed from historical running data and / or the preset application service constraint rules. The automated test verification layer includes: a test case generation module, a simulation backtesting module, and an assertion analysis module; the test case generation module is used to generate exploratory test cases based on the extracted multi-dimensional feature vectors of the anomaly features; the simulation backtesting module is used to trigger simulation backtesting based on the exploratory test cases, and perform multi-dimensional full-link testing in the shadow environment of the simulation backtesting to obtain the response data of the simulation backtesting; the assertion analysis module is used to determine the type and / or root cause of the anomaly features.

11. A computer storage medium, characterized in that The method includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps of the method as described in any one of claims 1 to 8.

12. An electronic device, comprising: include: processor; A memory for storing a program for processing data generated by an electronic device, wherein when the program is read and executed by the processor, it performs the steps of the method as described in any one of claims 1 to 8.

Images