Data retrieval method, apparatus, device, and medium

Abstract

The present application relates to the technical field of data processing, and provides a data retrieval method, device, equipment and medium, the method comprising: performing analysis processing on a received retrieval request to obtain a target orchestration configuration; performing graph construction processing on the target orchestration configuration based on a graph construction strategy to obtain a target runtime dependency graph; performing node selection and parallel execution processing on the target runtime dependency graph to obtain a batch output result; and performing aggregation generation processing on the batch output result using an aggregation generation strategy to obtain a target retrieval result. The present application can be applied to data interaction scenarios in the fields of finance and medicine, and can improve the flexibility of orchestration topology changes and parallel scheduling capability in request-level retrieval scenarios by implementing the embodiments of the present application, thereby forming a technical closed loop between configuration and execution and improving flexibility and retrieval efficiency.

Description

Technical Field

[0001] This invention relates to the field of data processing technology and can be applied to the financial and medical fields. In particular, it relates to a data retrieval method, apparatus, device, and medium. Background Technology

[0002] In existing technologies, common orchestration schemes typically employ predefined workflow models, such as static task orchestration based on directed acyclic graphs, process orchestration based on state machines, or persistent workflow execution based on code embedding. These schemes usually predefine task nodes, dependencies, and execution paths before execution, making them suitable for relatively stable batch processing or long transaction processes.

[0003] However, in request-level dynamic data retrieval scenarios, existing technologies have at least the following shortcomings: Insufficient flexibility in orchestration topology changes: Topology changes require modification of definitions or redeployment, resulting in poor flexibility; Insufficient utilization of request-level parallelism: For multiple independent upstream calls, if static sequential execution is still used or execution plans are not built based on real-time dependencies, latency budget will be wasted, increasing overall response time; Lack of a unified technical closed loop between configuration and execution: Configuration and execution are disconnected, making it impossible to achieve runtime topology reconstruction solely through configuration, and making it difficult to adapt to the integration of services that evolve at high frequencies.

[0004] Therefore, existing retrieval technologies suffer from insufficient flexibility in orchestration topology changes, inadequate parallel scheduling capabilities in request-level retrieval scenarios, and a disconnect between configuration and execution, leading to problems of low flexibility and low retrieval processing efficiency. Summary of the Invention

[0005] This invention provides a data retrieval method, apparatus, device, and medium, aiming to solve the problems of insufficient flexibility in orchestration topology changes, insufficient parallel scheduling capability in request-level retrieval scenarios, and disconnect between configuration and execution in existing retrieval technologies, which lead to low flexibility and retrieval processing efficiency.

[0006] To address the aforementioned problems, in a first aspect, embodiments of the present invention provide a data retrieval method, the data retrieval method comprising: The received search request is parsed and processed to obtain the target orchestration configuration; Based on the graph construction strategy, the target orchestration configuration is processed to obtain the target runtime dependency graph; The target runtime dependency graph is subjected to node selection and parallel execution processing to obtain batch output results; The batch output results are aggregated using an aggregation generation strategy to obtain the target retrieval results.

[0007] Secondly, embodiments of this application provide a data retrieval device, the data retrieval device comprising: The parsing unit is used to parse and process the received retrieval request to obtain the target orchestration configuration; The graph construction unit is used to perform graph construction processing on the target orchestration configuration based on the graph construction strategy to obtain the target runtime dependency graph; A parallel execution unit is used to select nodes and perform parallel execution processing on the target runtime dependency graph to obtain batch output results. The aggregation generation unit is used to aggregate and generate the batch output results using an aggregation generation strategy to obtain the target retrieval results.

[0008] Thirdly, embodiments of this application provide a computer device, the computer device including a memory and a processor connected to the memory; the memory is used to store a computer program, and the processor is used to run the computer program stored in the memory to perform the method described in the first aspect above.

[0009] Fourthly, embodiments of this application provide a storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, implement the method described in the first aspect above.

[0010] This invention provides a data retrieval method, apparatus, device, and medium. The method includes parsing a received retrieval request to obtain a target orchestration configuration; performing graph construction processing on the target orchestration configuration based on a graph construction strategy to obtain a target runtime dependency graph; performing node selection and parallel execution processing on the target runtime dependency graph to obtain batch output results; and using an aggregation generation strategy to aggregate the batch output results to obtain target retrieval results. This invention can be applied to data interaction scenarios in the financial and medical fields. In its embodiments, the invention parses and processes received retrieval requests to obtain target orchestration configurations. It can flexibly parse according to retrieval requests to quickly and accurately obtain the corresponding target orchestration configurations, adapting to frequently changing integrated topologies. Based on a graph construction strategy, the invention performs graph construction processing on the target orchestration configurations to obtain a target runtime dependency graph. This allows for the instant construction of the target runtime dependency graph based on the target orchestration configuration upon each request arrival, realizing the configurable evolution of the orchestration topology and ensuring the accuracy of subsequent processing. Node selection and parallel execution processing are performed on the target runtime dependency graph to obtain batch output results, achieving parallel scheduling capabilities in request-level retrieval scenarios. This reduces unnecessary waiting between independent data retrieval nodes, forming a unified technical closed loop between configuration and execution, thereby reducing request-level composite response latency. Finally, an aggregation generation strategy is used to aggregate the batch output results to obtain target retrieval results. This allows for field fusion during execution, rather than simply stacking post-processing node outputs, effectively avoiding inconsistencies in multi-source data aggregation and improving the quality and controllability of the final response. Therefore, the embodiments of the present invention can improve the flexibility of orchestration topology changes and the parallel scheduling capability in request-level retrieval scenarios, forming a technical closed loop between configuration and execution, thereby improving flexibility and retrieval efficiency. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating the data retrieval method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a sub-process of the data retrieval method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a sub-process of the data retrieval method provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a sub-process of the data retrieval method provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a sub-process of the data retrieval method provided in an embodiment of the present invention; Figure 6 A schematic block diagram of a data retrieval device provided in an embodiment of the present invention; Figure 7 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0015] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0016] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0017] It should be noted that if any AI models, software tools, or components not belonging to the applicant appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The user personal information involved in the embodiments of this application is obtained by an entity authorized (knowing and consenting) by the relevant parties or fully authorized by all parties through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0018] Please see Figures 1 to 5 , Figure 1 A flowchart illustrating the data retrieval method provided in an embodiment of the present invention; Figure 2This is a schematic diagram of a sub-process of the data retrieval method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a sub-process of the data retrieval method provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a sub-process of the data retrieval method provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a sub-process of the data retrieval method provided in an embodiment of the present invention. For example... Figure 1 As shown, this embodiment of the invention provides a data retrieval method, which includes the following steps S110-S140.

[0019] S110. Parse and process the received search request to obtain the target orchestration configuration.

[0020] In this embodiment, the application scenario of this solution can be in data interaction scenarios in the financial and medical fields. For example, it can be applied to data interaction scenarios such as unified customer view generation, risk signal aggregation, and compliance audit data assembly in financial services, or to data interaction scenarios such as customer health status view generation in medical services. Taking the unified customer view retrieval in a fintech scenario as an example, after the customer service workbench initiates the retrieval request, customer-related data needs to be retrieved from account services, transaction services, fraud signal services, and risk profile warehouses to generate a unified customer view, which is used to assist in risk identification, compliance review, and customer service.

[0021] The retrieval request is a raw HTTP / RPC / API request sent by the client, business system, or upper-layer service, which may include data such as request path, request body, request header, and query parameters.

[0022] In one embodiment, such as Figure 2 As shown, the step of parsing the received search request to obtain the target orchestration configuration includes: S111. The search request is transformed to obtain the arrangement context; S112. Perform configuration matching processing based on the preset configuration repository and the orchestration context to obtain the matching result; S113. Determine the target orchestration configuration based on the matching result.

[0023] In this embodiment, the step of converting the retrieval request to obtain the orchestration context includes: extracting the retrieval request to obtain request parameters; performing parameter validation on the request parameters using validation rules to obtain validation results; and performing structural conversion on the request parameters according to the validation results and the conversion format to obtain the orchestration context.

[0024] The process of extracting and processing the retrieval request to obtain the request parameters involves directly extracting and processing the request path, request body, request header, and query parameters from the retrieval request. The request parameters include fields such as operation name, customer identifier, function flag, and request serial number.

[0025] The verification results are obtained by performing parameter verification processing on the request parameters using verification rules. Specifically, the verification rules mainly verify necessary parameters, which can be pre-configured by the customer. If the necessary parameters exist in the request parameters, the verification result is successful; if the necessary parameters do not exist in the request parameters, the verification result is unsuccessful.

[0026] The orchestration context is obtained by performing structural transformation on the request parameters based on the verification result and the transformation format. Specifically, the transformation format is a JSON structure description. When the verification result is successful, the orchestration context is obtained by directly performing structural transformation on the request parameters using the transformation format. When the verification result is unsuccessful, a parameter error message is returned.

[0027] The step of performing configuration matching processing based on a preset configuration repository and the orchestration context to obtain a matching result includes: reading the orchestration context to obtain read data; and using the read data as a matching condition to perform a search and matching process in the configuration repository to obtain a matching result.

[0028] The process of reading the orchestration context to obtain read data specifically involves reading information such as operation name, tenant identifier, and function flag from the orchestration context as read data. That is, the read data includes information such as operation name, tenant identifier, and function flag.

[0029] The process involves using the read data as a matching condition to perform a search and matching process in the configuration repository to obtain a matching result. Specifically, the process uses the operation name, tenant identifier, or function flag as a matching condition to perform a search and matching process in the configuration repository to obtain a matching configuration version, which is then used as the matching result.

[0030] The step of determining the target orchestration configuration based on the matching result specifically involves: if the matching result indicates that a configuration version exists, then the corresponding configuration version is loaded from the configuration repository as the target orchestration configuration; if the matching result indicates that a configuration version does not exist, then a general version is loaded from the configuration repository as the target orchestration configuration.

[0031] The target orchestration configuration refers to the orchestration data related to several steps, including at least step data, timeout budget, retry strategy, aggregation strategy, and node failure semantics. The step data includes step definitions, invocation targets, input mappings, and dependency constraints. A step definition is the metadata description of each retrieval / transform step, including step name, step type (e.g., api / warehouse / transform), and whether it is optional. The invocation target is the actual resource address invoked by the step, such as the endpoint of an HTTP interface (e.g., / accounts / {customer_id}) or the SQL query statement of a data warehouse. The input mapping is the rule for binding orchestration context parameters or upstream node outputs to the current node's input parameters, such as {customer_id} being from request.cus. The `tomer_id` value is used to reference the output of node A, and `${steps.A.output.account_number}` refers to the output of node A. Dependency constraints: the set of preceding steps that the current step depends on (`depends_on`), which determines when the node can be executed. Timeout budget: the maximum allowed execution time for a single node (`timeout_ms`), triggering failure handling if the timeout occurs. Retry strategy: the number of retries after failure and the retry method (e.g., exponential backoff and random jitter). Aggregation strategy: the node output merging method, the list of required steps, field priorities, and missing step handling rules (e.g., `on_missing_required:fail`). Node failure semantics: the handling method after a node failure, divided into three categories: required nodes (blocking), optional nodes (skipping), and backup nodes (alternative calls).

[0032] As can be seen from the above embodiments, parsing the received retrieval request to obtain the target orchestration configuration can flexibly parse according to the retrieval request to quickly and accurately obtain the corresponding target orchestration configuration, which can adapt to the frequently changing integration topology, thereby improving retrieval efficiency.

[0033] S120. Based on the graph construction strategy, the target orchestration configuration is processed to obtain the target runtime dependency graph.

[0034] In this embodiment, after obtaining the target orchestration configuration, a target runtime dependency graph can be obtained by performing graph construction processing on the target orchestration configuration based on a graph construction strategy.

[0035] In one embodiment, such as Figure 3 As shown, the graph construction process for the target orchestration configuration based on the graph construction strategy to obtain the target runtime dependency graph includes: S121. Traverse the step data in the target orchestration configuration to obtain the node description structure for each step; S122. Perform relation parsing on the node description structure to obtain node dependencies; S123. Based on the node dependency relationship, construct edges for each node to obtain an initial runtime dependency graph; S124. Use a preset retrieval algorithm to perform dependency retrieval processing on the initial runtime dependency graph to obtain retrieval results; S125. If the search result is a normal dependency, then the initial runtime dependency graph is used as the target runtime dependency graph.

[0036] In this embodiment, the step data in the target orchestration configuration is traversed to obtain a node description structure for each step. Specifically, the corresponding node description structure is obtained by traversing the step data for each step in the target orchestration configuration. The node description structure includes a node identifier, node type, in-degree count, set of dependent nodes, set of successor nodes, execution parameters, status fields, and output cache key, etc. Each step is treated as a node, meaning each step corresponds to a node description structure.

[0037] The node dependency relationships are obtained by parsing the node description structure. Based on these dependencies, edges are constructed for each node to obtain an initial runtime dependency graph. Specifically, the node dependency relationships are obtained by parsing the dependent node set and the successor node set. Directed edges are then established for each node based on these dependencies, and the corresponding in-degree count and successor node set are updated to obtain the initial runtime dependency graph. For example, if step A depends on step B, a directed edge from B to A is established in the graph, the in-degree count of A is incremented by 1, and A is added to the successor node set of B.

[0038] The initial runtime dependency graph is processed using a preset retrieval algorithm to obtain retrieval results. Specifically, the preset retrieval algorithm is the Kahn topological sorting algorithm, which is used to check whether there is a circular dependency in the initial runtime dependency graph. If a circular dependency exists, the retrieval result is a circular dependency; if there is no circular dependency, the retrieval result is a normal dependency.

[0039] If the search result is a normal dependency, the initial runtime dependency graph is used as the target runtime dependency graph; if the search result is a circular dependency, a configuration error response is returned, and error verification is performed on the initial runtime dependency graph to obtain error data.

[0040] As can be seen from the above embodiments, by performing graph construction processing on the target orchestration configuration based on the graph construction strategy to obtain the target runtime dependency graph, the target runtime dependency graph can be constructed on the target orchestration configuration in real time when each request arrives, realizing the configurable evolution of the orchestration topology, ensuring the accuracy of subsequent processing, and thus improving retrieval efficiency.

[0041] S130. Perform node selection and parallel execution processing on the target runtime dependency graph to obtain batch output results.

[0042] In this embodiment, after obtaining the target runtime dependency graph, node selection and parallel execution processing can be performed on the target runtime dependency graph to obtain batch output results.

[0043] In one embodiment, such as Figure 4 As shown, the step of performing node selection and parallel execution processing on the target runtime dependency graph to obtain batch output results includes: S131. Perform parameter judgment processing on each node object in the target runtime dependency graph to obtain an initial ready queue; S132. The scheduling calculation strategy is used to perform scheduling calculation on each node object in the initial ready queue to obtain a scheduling score; S133. Based on the scheduling score, sort all node objects in the initial ready queue to obtain an ordered queue. S134. Perform node selection processing on the ordered queue according to the concurrency restriction rules to obtain several node batches; S135. Perform parallel execution on each of the plurality of node batches to obtain the batch output result.

[0044] In this embodiment, the parameter judgment processing of each node object in the target runtime dependency graph to obtain the initial ready queue is specifically as follows: for each step, a corresponding node object is generated, and the in-degree count of each node object is judged. If the in-degree count is 0, the corresponding node is added to the initial ready queue; if the in-degree count is not 0, the corresponding node is removed.

[0045] The scheduling calculation strategy is used to perform scheduling calculations on each node object in the initial ready queue to obtain a scheduling score; specifically, the formula corresponding to the scheduling calculation is as follows: ;P i U represents the scheduling score of the i-th node; i Indicates the urgency of a node; the priority value can be explicitly specified by configuration, or dynamically calculated based on whether the node is on the critical path; W iThis represents the weight of a node as a prerequisite for its successor; for example, it could be the number of all its successor nodes, especially the number of required successor nodes. i This represents the estimated node latency, which can be proportionally calculated based on the `timeout_ms` parameter in the configuration or predicted based on historical execution statistics. α, β, and γ are non-negative weight parameters. Specifically, α, β, and γ can be set to 0.5, 0.3, and 0.2, respectively. If priority scheduling is not enabled, the execution engine selects nodes according to a first-in, first-out (FIFO) rule.

[0046] The node sorting process is performed on all node objects in the initial ready queue according to the scheduling score to obtain an ordered queue. Specifically, the nodes in the initial ready queue are sorted in descending order according to the scheduling score to obtain the ordered queue.

[0047] The process of selecting nodes from the ordered queue according to concurrency limits yields several node batches. Specifically, a preset global maximum concurrency is obtained, which can be set according to node type. For example, API call nodes can have a maximum of 6, data warehouse query nodes a maximum of 2, and local transformation nodes a maximum of a certain number. The selected nodes from the ordered queue are then processed according to the global maximum concurrency to obtain the aforementioned node batches.

[0048] The process of parallel execution of each node batch in the plurality of node batches to obtain the batch output result specifically involves parallel execution of multiple nodes in the first node batch in the plurality of node annotations to obtain the first batch output result. If the first batch output result is successful, the in-degree count of the corresponding dependent node is decremented by 1. At this time, parameter judgment processing is performed on each node object in the target runtime dependency graph again to obtain an initial ready queue. Scheduling calculation processing is performed on each node object in the initial ready queue using a scheduling calculation strategy to obtain a scheduling score. Based on the scheduling score, node sorting processing is performed on all node objects in the initial ready queue to obtain an ordered queue. Node selection processing is performed on the ordered queue according to concurrency restriction rules to obtain a plurality of node batches. Each node batch in the plurality of node batches is then executed in parallel again to obtain the batch output result. For example, the configuration has four steps: A (no dependencies), B (no dependencies), C (depends_on:[A]), and D (depends_on:[A,B]); 1. Initially: the in-degree counts of A and B are 0, and they are added to the ready queue; the in-degree count of C is 1, and the in-degree count of D is 2; 2. When A executes successfully, the in-degree count of C is decremented by 1. At this time, the in-degree count of C becomes 0, satisfying the dependency condition, and C is enqueued; at the same time, the in-degree count of D is decremented by 1, and the in-degree count of D becomes 1, still not satisfying the dependency condition; 3. When B executes successfully, the in-degree count of D is decremented by 1. At this time, the in-degree count of D becomes 0, satisfying the dependency condition, and D is enqueued. If the output result of the first batch is an execution failure, the reason for the failure will be recorded, and the execution of the second batch of the plurality of node batches will continue until each of the plurality of node batches is completed.

[0049] As can be seen from the above embodiments, by selecting nodes and performing parallel execution on the target runtime dependency graph to obtain batch output results, the parallel scheduling capability in the request-level retrieval scenario is realized, unnecessary waiting between independent data retrieval nodes is reduced, and a unified technical closed loop between configuration and execution is formed, thereby reducing the request-level composite response latency and improving retrieval efficiency.

[0050] S140. The batch output results are aggregated and generated using an aggregation generation strategy to obtain the target retrieval results.

[0051] In this embodiment, after determining the batch output results, the batch output results can be aggregated and generated using an aggregation generation strategy to obtain the target retrieval results.

[0052] In one embodiment, such as Figure 5 As shown, the process of aggregating and generating the batch output results using an aggregation generation strategy to obtain the target retrieval results includes: S141. Perform a merging and mapping process on the batch output results to obtain the initial search results; S142. Perform field scoring calculation on each node field in the initial search results to obtain a field score; S143. The target search result is obtained by filtering the initial search results according to the field score.

[0053] In this embodiment, the initial search result is obtained by merging and mapping the batch output results. Specifically, the initial search result is obtained by performing field mapping processing on the batch output results according to a preset field mapping function. The field mapping function is... X represents the batch output result, Y represents the converted standard output structure, and f1 to f k This represents the corresponding field transformation function. Each node in the initial search results includes several fields.

[0054] The process involves calculating a field score for each node field in the initial search results. Specifically, this is done using a preset field credibility scoring formula. The field credibility scoring formula is as follows: C j S represents the field score of the j-th node field. j T represents the source priority score. j Q represents the time freshness rating. j Let λ1, λ2, and λ3 represent the data quality score, and let λ1 + λ2 + λ3 = 1.

[0055] The target search result is obtained by filtering the initial search results according to the field score. Specifically, the highest field score is selected from the initial search results as the target search result.

[0056] As can be seen from the above embodiments, by using the aggregation generation strategy to aggregate and generate the batch output results to obtain the target retrieval results, progressive integrity verification and field fusion can be performed during the execution process, rather than simply stacking the output of post-processing nodes. This effectively avoids inconsistency problems in the process of multi-source data aggregation, improves the quality and controllability of the final response, and thus improves retrieval efficiency.

[0057] In summary, the embodiments of the present invention obtain target orchestration configurations by parsing and processing received retrieval requests. This allows for flexible parsing based on retrieval requests to quickly and accurately obtain the corresponding target orchestration configurations, adapting to frequently changing integrated topologies. By constructing a target runtime dependency graph based on the target orchestration configuration using a graph construction strategy, the target runtime dependency graph can be built in real-time upon each request arrival, realizing the configurable evolution of the orchestration topology and ensuring the accuracy of subsequent processing. Node selection and parallel execution processing of the target runtime dependency graph to obtain batch output results enables parallel scheduling capabilities in request-level retrieval scenarios, reducing unnecessary waiting between independent data retrieval nodes and forming a unified technical closed loop between configuration and execution, thereby reducing request-level composite response latency. The aggregation generation strategy is used to aggregate and generate target retrieval results from the batch output results. This allows for field fusion during execution, rather than simply stacking post-processing node outputs, effectively avoiding inconsistencies in multi-source data aggregation and improving the quality and controllability of the final response. Therefore, the embodiments of the present invention can improve the flexibility of orchestration topology changes and the parallel scheduling capability in request-level retrieval scenarios, forming a technical closed loop between configuration and execution, thereby improving flexibility and retrieval efficiency.

[0058] Figure 6 This is a schematic block diagram of a data retrieval device provided in an embodiment of the present invention. Figure 6 As shown, an embodiment of the present invention provides a data retrieval device 700 that implements the method described above. The data retrieval device 700 includes: The parsing unit 701 is used to parse the received retrieval request to obtain the target orchestration configuration; Graph construction unit 702 is used to perform graph construction processing on the target orchestration configuration based on a graph construction strategy to obtain a target runtime dependency graph; Parallel execution unit 703 is used to perform node selection and parallel execution processing on the target runtime dependency graph to obtain batch output results; The aggregation generation unit 704 is used to perform aggregation generation processing on the batch output results using an aggregation generation strategy to obtain the target retrieval results.

[0059] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned device can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0060] The above-described device can be implemented as a computer program, and the computer program can be implemented in, for example... Figure 7 It runs on the computer device shown.

[0061] Please see Figure 7 , Figure 7 This is a schematic block diagram of an electronic device provided in an embodiment of the present invention. The electronic device 800 can be a terminal or a server. The terminal can be an electronic device with communication functions. The server can be a standalone server or a server cluster composed of multiple servers.

[0062] See Figure 7 The electronic device 800 includes a processor 802, a memory, and a network interface 805 connected via a system bus 801. The memory may include a non-volatile storage medium 803 and internal memory 804.

[0063] The non-volatile storage medium 803 may store an operating system 8031 ​​and a computer program 8032. The computer program 8032 includes program instructions that, when executed, cause the processor 802 to perform a data retrieval method.

[0064] The processor 802 provides computing and control capabilities to support the operation of the entire electronic device 800.

[0065] The internal memory 804 provides an environment for the execution of the computer program 8032 in the non-volatile storage medium 803. When the computer program 8032 is executed by the processor 802, the processor 802 can execute a data retrieval method.

[0066] This network interface 805 is used for network communication with other devices. Those skilled in the art will understand that... Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device 800 to which the present invention is applied. The specific electronic device 800 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0067] The processor 802 is used to run a computer program 8032 stored in the memory to perform the following steps: The received search request is parsed and processed to obtain the target orchestration configuration; Based on the graph construction strategy, the target orchestration configuration is processed to obtain the target runtime dependency graph; The target runtime dependency graph is subjected to node selection and parallel execution processing to obtain batch output results; The batch output results are aggregated using an aggregation generation strategy to obtain the target retrieval results.

[0068] It should be understood that, in this embodiment of the invention, the processor 802 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0069] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0070] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the following steps: The received search request is parsed and processed to obtain the target orchestration configuration; Based on the graph construction strategy, the target orchestration configuration is processed to obtain the target runtime dependency graph; The target runtime dependency graph is subjected to node selection and parallel execution processing to obtain batch output results; The batch output results are aggregated using an aggregation generation strategy to obtain the target retrieval results.

[0071] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0072] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0073] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0074] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0075] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0076] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The software tools, models, or components appearing in the embodiments of the present invention are merely illustrative examples and do not represent actual use.

Claims

1. A data retrieval method, characterized in that, The data retrieval method includes: The received search request is parsed and processed to obtain the target orchestration configuration; Based on the graph construction strategy, the target orchestration configuration is processed to obtain the target runtime dependency graph; The target runtime dependency graph is subjected to node selection and parallel execution processing to obtain batch output results; The batch output results are aggregated using an aggregation generation strategy to obtain the target retrieval results.

2. The method according to claim 1, characterized in that, The graph-based strategy is used to perform graph construction processing on the target orchestration configuration to obtain the target runtime dependency graph, including: The step data in the target orchestration configuration is traversed to obtain the node description structure for each step; The node dependency relationships are obtained by parsing the node description structure. Based on the node dependencies, an initial runtime dependency graph is obtained by constructing edges for each node. The initial runtime dependency graph is processed using a preset retrieval algorithm to obtain retrieval results. If the search result is a normal dependency, then the initial runtime dependency graph is used as the target runtime dependency graph.

3. The method according to claim 1, characterized in that, The process of selecting nodes and performing parallel execution on the target runtime dependency graph to obtain batch output results includes: The initial ready queue is obtained by performing parameter judgment processing on each node object in the target runtime dependency graph; The scheduling score is obtained by performing scheduling calculations on each node object in the initial ready queue using a scheduling calculation strategy. Based on the scheduling score, all node objects in the initial ready queue are sorted to obtain an ordered queue; The ordered queue is processed according to the concurrency restriction rules to obtain several node batches; The batch output results are obtained by executing each of the several node batches in parallel.

4. The method according to claim 1, characterized in that, The process of aggregating and generating the batch output results using an aggregation generation strategy to obtain the target retrieval results includes: The batch output results are merged and mapped to obtain the initial search results; The field score is obtained by performing field score calculation on each node field in the initial search results; The target search results are obtained by filtering the initial search results according to the field scores.

5. The method according to claim 1, characterized in that, The step of parsing and processing the received search request to obtain the target orchestration configuration includes: The search request is transformed to obtain the orchestration context; The matching result is obtained by performing configuration matching processing based on the preset configuration repository and the orchestration context; The target orchestration configuration is determined based on the matching results.

6. The method according to claim 5, characterized in that, The process of transforming the retrieval request to obtain the orchestration context includes: The search request is processed to obtain the request parameters; The request parameters are validated using validation rules to obtain validation results. The orchestration context is obtained by performing structural transformation on the request parameters based on the verification results and conversion format.

7. The method according to claim 5, characterized in that, The step of performing configuration matching processing based on a preset configuration repository and the orchestration context to obtain a matching result includes: The orchestration context is read and processed to obtain the read data; The read data is used as a matching condition to perform a search and matching process in the configuration repository to obtain the matching result.

8. A data retrieval device, characterized in that, The data retrieval device includes: The parsing unit is used to parse and process the received retrieval request to obtain the target orchestration configuration; The graph construction unit is used to perform graph construction processing on the target orchestration configuration based on the graph construction strategy to obtain the target runtime dependency graph; A parallel execution unit is used to select nodes and perform parallel execution processing on the target runtime dependency graph to obtain batch output results. The aggregation generation unit is used to aggregate and generate the batch output results using an aggregation generation strategy to obtain the target retrieval results.

9. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A storage medium, characterized in that, The storage medium stores a computer program, which includes program instructions that, when executed by a processor, can implement the method as described in any one of claims 1-7.

Images