An intelligent retrieval system for high-concurrency credit lease business data

Abstract

The present application relates to the technical field of information retrieval and credit leasing, and particularly relates to an intelligent retrieval system for high-concurrency credit leasing business data. The present application acquires multi-source data through a data acquisition unit, and after encapsulation by a data processing unit, an asynchronous queue dispatching vector generation unit calls an AI model to generate a feature vector. During retrieval, a retrieval recognition unit analyzes a retrieval request and determines a mixed retrieval weight, and performs a query by fusing semantic and keyword strategies. A management unit adaptively switches index types, adjusts retrieval nodes and index parameters based on the number of retrieval requests and concurrent pressure within a preset time. A calculation unit determines a retrieval representation value based on the response time of a standard query, an analysis unit determines a retrieval state according to the retrieval representation value, and dynamically adjusts the number of retrieval nodes based on the retrieval state, optimizes the preset number of retrieval requests and related parameters, and forms a closed loop of data-driven and autonomous optimization. The present application improves retrieval efficiency.

Description

Technical Field

[0001] This invention relates to the fields of information retrieval and credit leasing technology, and in particular to an intelligent retrieval system for high-concurrency credit leasing business data. Background Technology

[0002] Credit leasing businesses rely on multi-source heterogeneous data for core operations. This data encompasses structured user credit scores, transaction records, and asset and liability information; semi-structured leasing agreements and reports; and unstructured public opinion texts, contract terms, and user reviews. This type of business has three main pain points: First, significant peak concurrency occurs. Leasing platforms often face thousands to tens of thousands of search requests per second during promotional activities and holidays, making traditional search systems prone to response delays and service degradation. Second, extremely high data credibility is required. Search results must meet financial compliance, data authenticity, and timeliness requirements to avoid risk decisions due to "factual illusions" or data tampering. Third, insufficient business adaptability. General search systems struggle to balance semantic understanding accuracy with credit-specific rules such as risk control thresholds and leasing qualification matching, and multi-source data integration suffers from semantic fragmentation and weak correlation mining capabilities.

[0003] While existing technologies such as vector databases can achieve high-concurrency, low-latency retrieval through indexes like HNSW, they lack reliable verification and dynamic adaptation mechanisms specifically for credit leasing businesses. Credit AI Agent technology can handle task decomposition and tool invocation, but it is not deeply integrated with high-concurrency retrieval scenarios and struggles to cope with resource scheduling demands under peak traffic. Retrieval enhancement generation technology, while improving semantic understanding, has shortcomings in the governance of multi-source heterogeneous credit data and the optimization of caching strategies in high-concurrency scenarios. Furthermore, traditional retrieval systems employ fixed indexing strategies, unable to dynamically adjust based on data volume and concurrency pressure, and lack specific compliance review and result correction mechanisms for credit businesses, making it difficult to meet the triple requirements of "efficient retrieval, reliable assurance, and business adaptation" for credit leasing businesses.

[0004] Chinese Patent Publication No. CN119963301A discloses a blockchain-based rental management system, including a property quality assessment module, a tenant credit module, and a rental management module. The property quality assessment module stores historical rental data and calculates property quality coefficients to filter qualified rental properties. The tenant credit module records tenant rental history information, collects tenant credit scores, number of late payment payments, and number of contract terminations based on this information, calculates tenant risk scores, and assigns risk ratings to tenants. The rental management module prioritizes searches based on property quality coefficients, calculates recommended rental terms and tenant target rental terms, and calculates the final rental deposit.

[0005] Therefore, the existing technology has the following problems: the response delay and poor business adaptability in high-concurrency credit leasing business data retrieval result in low retrieval efficiency. Summary of the Invention

[0006] To address this issue, the present invention provides an intelligent retrieval system for high-concurrency credit leasing business data, which overcomes the problems of response delay and poor business adaptability in existing high-concurrency credit leasing business data retrieval, resulting in low retrieval efficiency.

[0007] To achieve the above objectives, the present invention provides an intelligent retrieval system for high-concurrency credit leasing business data, comprising: The data acquisition unit is used to acquire multi-source data using a distributed multi-source access architecture; A data processing unit, connected to the data acquisition unit, is used to encapsulate multi-source data entries to be generated into vectorized tasks, and to send the vectorized tasks to an asynchronous task queue. A vector generation unit, connected to the data processing unit, is used to call the corresponding feature extraction model to generate feature vectors based on the data category in the vectorized task obtained from the asynchronous task queue. The retrieval and identification unit, which is connected to the vector generation unit, is used to parse the user's retrieval request and determine the weight parameters of the retrieval strategy based on the core elements generated after parsing the user's retrieval request, so as to call multiple feature vectors for retrieval. The retrieval strategy includes semantic retrieval and keyword retrieval. The management unit, which is connected to the retrieval and identification unit, is used to adaptively switch the index type, adjust the retrieval node and adjust the index parameters based on the number of retrieval requests and the concurrency pressure within a preset time period and the comparison results with the corresponding preset number of retrieval requests and the preset concurrency pressure. The index parameters include at least the number of cluster centers. A calculation unit, which is connected to the management unit, is used to determine the retrieval characterization value based on the response time of retrieval queries whose recall rate is greater than or equal to the preset recall rate within a preset time. An analysis unit, connected to the computing unit, is used to determine the search status based on the search characterization value, adjust the number of search nodes based on the search status, and adjust the preset number of search requests based on the search status after adjusting the number of search nodes.

[0008] Furthermore, the system also includes a storage unit, which is used to store the vector features and the multi-source data in several index categories based on a hash algorithm, and each segment corresponds to an independent retrieval node. The index categories include at least rental category, user region and data update time.

[0009] Furthermore, the system also includes a prediction unit, which is used to predict peak traffic based on historical concurrent data and time-series prediction models, so as to automatically trigger node expansion and resource addition.

[0010] Furthermore, the system also includes a data verification unit, which is used to periodically verify the vector features and the multi-source data based on the full lifecycle trajectory of the data in the data traceability chain, so as to mark and remove tampered data.

[0011] Furthermore, the calculation unit is also used to filter eligible queries with a recall rate greater than or equal to a preset recall rate within a preset time period; the calculation unit is also used to sort the response times corresponding to the eligible queries in ascending order of numerical values ​​to generate a response time sequence; the calculation unit is also used to determine the retrieval characterization value based on the numerical value corresponding to the preset quantile of the response time sequence.

[0012] Furthermore, the analysis unit is also used to obtain the abnormal response rate corresponding to multiple preset periods when the retrieval status is unqualified, wherein the abnormal response rate is the ratio of the number of retrieval requests with a response time greater than a preset time to the total number of retrieval requests; the analysis unit is also used to calculate the growth slope of the abnormal response rate for each preset period, and when the growth slope of the abnormal response rate corresponding to multiple consecutive preset periods is greater than a preset slope, adjust the number of retrieval nodes based on the average of the growth slope of the abnormal response rate corresponding to multiple preset periods; wherein, when the retrieval characteristic value is greater than a preset characteristic value, the retrieval status is determined to be unqualified.

[0013] Furthermore, the analysis unit is also used to increase the number of retrieval nodes based on the ratio of the average growth slope of the abnormal response rate corresponding to multiple preset periods to the first preset average value, and the increase in the number of retrieval nodes is proportional to the ratio.

[0014] Furthermore, the analysis unit is also used to calculate the difference between the search characteristic value and the preset characteristic value when the search status is unqualified after adjusting the number of search nodes; the analysis unit is also used to reduce the preset number of search requests based on the ratio of the difference to the preset difference when the difference is greater than the preset difference, and the reduction of the preset number of search requests is proportional to the ratio.

[0015] Furthermore, the analysis unit is also used to obtain the request throughput when the retrieval status is unqualified after adjusting the preset number of retrieval requests; the analysis unit is also used to increase the number of cluster centers based on the ratio of request throughput to preset request throughput when the request throughput is greater than the preset request throughput, and the increase in the number of cluster centers is proportional to the ratio.

[0016] Furthermore, the analysis unit is also used to repeatedly adjust the number of cluster centers at least once if the retrieval status is unqualified after adjusting the number of cluster centers, until the number of adjustments is less than a preset number and the retrieval status is qualified, or the number of adjustments is equal to the preset number, and then stop adjusting; the analysis unit is also used to obtain multiple historical retrieval characteristic values ​​within a preset time period if the retrieval status is unqualified after stopping the adjustment; the analysis unit is also used to calculate the variance and average value of the multiple retrieval characteristic values ​​respectively, and if the variance is less than a preset variance and the average value is greater than a second preset average value, the data verification period is reduced based on the ratio of the variance to the preset variance, and the reduction in the data verification period is inversely proportional to the ratio.

[0017] Compared with existing technologies, the advantages of this invention are as follows: This invention acquires multi-source data through a data acquisition unit, encapsulates it through a data processing unit, and then uses an asynchronous queue scheduling vector generation unit to call an AI model to generate feature vectors. During retrieval, the retrieval identification unit parses the retrieval request and determines the hybrid retrieval weight, integrating semantic and keyword strategies for querying. The management unit adaptively switches index types and adjusts retrieval nodes and index parameters based on the number of retrieval requests and concurrency pressure within a preset time period. The calculation unit determines the retrieval representation value based on the response time of the qualified query, and the analysis unit determines the retrieval status based on the retrieval representation value, dynamically adjusts the number of retrieval nodes based on the retrieval status, and optimizes the preset number of retrieval requests and related parameters, forming a data-driven, self-optimizing closed loop. This invention improves retrieval efficiency.

[0018] Furthermore, this invention uses a hash algorithm in the storage unit to divide the data into shards, and each shard corresponds to an independent retrieval node, which can avoid single points of failure and data skew, thereby enabling more accurate data tracing and further improving retrieval efficiency.

[0019] Furthermore, this invention predicts peak traffic based on historical concurrent data and a time-series prediction model, enabling timely expansion of nodes and additional resources, thereby further reducing operation and maintenance costs and further improving retrieval efficiency.

[0020] Furthermore, this invention determines the retrieval characteristic value based on the response time series corresponding to the qualified query, which can combine business effect (recall rate), technical performance (latency), and system adaptive logic (dynamic index) to more accurately determine the retrieval status, thereby enabling more precise adjustments in the future and further improving retrieval efficiency.

[0021] Furthermore, this invention determines the reason for the unqualified retrieval status based on the comparison result of the growth slope of the abnormal response rate corresponding to multiple consecutive preset periods and the preset slope. This can more accurately determine whether the unqualified retrieval status is caused by the number of retrieval requests that flood in within a short period of time exceeding the design capacity of the current computing resources and index architecture. This allows for more effective adjustments in the future, thereby further improving retrieval efficiency.

[0022] Furthermore, the present invention increases the number of retrieval nodes based on the ratio of the average growth slope of the abnormal response rate corresponding to multiple preset periods to the first preset average value, which can more effectively process retrieval requests, thereby further reducing response time and further improving retrieval efficiency.

[0023] Furthermore, the present invention reduces the preset number of search requests based on the ratio of the difference between the search characteristic value and the preset characteristic value to the preset difference. This enables more effective switching of index types when the data volume increases, thereby allowing for more effective searching through index types suitable for large data volumes, and further improving search efficiency.

[0024] Furthermore, the present invention adjusts the number of cluster centers based on the ratio of request throughput to preset request throughput, which can index more effectively under high concurrency pressure, thereby further reducing response time and further improving retrieval efficiency.

[0025] Furthermore, the present invention determines the reasons for unqualified search status based on the variance and average value of search characterization values ​​over multiple historical preset time periods. In addition, by reducing the data verification cycle based on the ratio of variance to preset variance, the present invention can more effectively avoid the situation where the discriminative power of the generated vector features decreases due to upstream data pollution such as image blurring and text noise, thereby further reducing response time and further improving search efficiency. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the structure of an intelligent retrieval system for high-concurrency credit leasing business data according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the steps of an intelligent retrieval method for high-concurrency credit leasing business data according to an embodiment of the present invention. Figure 3 This is a flowchart illustrating the steps of determining the comparison result between the retrieved characterization value and the preset characterization value in an embodiment of the present invention. Figure 4 This is a flowchart illustrating the steps of determining the retrieval status based on adjusting the number of cluster centers in an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0028] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0029] It should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0030] Please see Figure 1 The diagram shown is a structural schematic of an intelligent retrieval system for high-concurrency credit leasing business data according to an embodiment of the present invention. The present invention provides an intelligent retrieval system for high-concurrency credit leasing business data, comprising a data acquisition unit, a data processing unit, a vector generation unit, a retrieval and identification unit, a management unit, a calculation unit, and an analysis unit.

[0031] The data acquisition unit is used to acquire multi-source data using a distributed multi-source access architecture; The data processing unit is connected to the data acquisition unit and is used to encapsulate the multi-source data entries to be generated into vectorized tasks, and to send the vectorized tasks to the asynchronous task queue. The vector generation unit is connected to the data processing unit and is used to call the corresponding feature extraction model to generate feature vectors based on the data category in the vectorized task obtained from the asynchronous task queue. The retrieval and identification unit is connected to the vector generation unit. It is used to parse the user's retrieval request and determine the weight parameters of the retrieval strategy based on the core elements generated after parsing the user's retrieval request, so as to call multiple feature vectors for retrieval. The retrieval strategy includes semantic retrieval and keyword retrieval. The management unit is connected to the retrieval and identification unit. It is used to adaptively switch the index type, adjust the retrieval node and adjust the index parameters based on the number of retrieval requests and the concurrency pressure within a preset time period and the comparison results with the corresponding preset number of retrieval requests and the preset concurrency pressure. The index parameters include at least the number of cluster centers. The computing unit is connected to the management unit and is used to determine the retrieval characterization value based on the response time of retrieval queries with a recall rate greater than or equal to a preset recall rate within a preset time. The analysis unit is connected to the computing unit and is used to determine the search status based on the search characterization value, adjust the number of search nodes based on the search status, and adjust the preset number of search requests based on the search status after adjusting the number of search nodes.

[0032] Specifically, the data acquisition unit adopts a distributed multi-source access architecture to obtain data sources such as direct connections to credit reporting agencies, the core database of the leasing platform, public opinion platforms, and regulatory systems. It automatically adapts structured, semi-structured, and unstructured data through NL2API and NL2SQL interfaces, reducing intermediate data transfer links and lowering latency.

[0033] Specifically, the vector generation unit generates feature vectors based on the data category by calling the corresponding feature extraction model. This includes: extracting multi-dimensional feature vectors from structured data through feature engineering; generating 128-dimensional floating-point vectors from unstructured text through the OpenAI Embedding model; and extracting feature vectors from image-type data such as rental material inspection reports through the MobileNet model.

[0034] Specifically, when a user submits a search request containing natural language descriptions and keyword combinations, the credit AI Agent first activates its planning capabilities to analyze the intent. The process involves using a thought chain strategy to progressively reason through the request, transforming vague descriptions into structured query conditions. Simultaneously, relying on a thought tree strategy, multiple possible interpretation branches are generated for complex or fuzzy queries. For example, "short-term rental of high-end equipment" is broken down into multiple sub-conditions such as "rental period less than or equal to 3 months" and "equipment level is flagship," exploring the optimal parsing path in parallel. Through the synergy of these hybrid strategies, structured core business elements are systematically extracted from the original request, such as the exact type of leased item, the user's credit score range, the rental period, and risk control thresholds. Furthermore, dense retrieval (semantic similarity matching) and sparse retrieval (keyword matching) are integrated, dynamically adjusting weights based on the search intent. For instance, semantic retrieval accounts for 70% of the weight in ordinary scenarios, while keyword precision matching increases to 60% in risk control scenarios.

[0035] Specifically, the management unit has a built-in multi-index adaptive strategy that automatically switches index types based on data volume and concurrency pressure. For example, when the data volume is less than or equal to 1 million records, the HNSW index is used; when the data volume is between 1 million and 10 million records, the IVF_PQ index is switched to, and at least the number of cluster centers (nlist) in the index parameters is adjusted to balance retrieval speed and memory usage; when the data volume is greater than 10 million records, GPU acceleration is enabled.

[0036] Specifically, the prediction unit continuously collects historical concurrency data from the system, such as QPS, response time, and resource utilization, and inputs this data into an LSTM time-series prediction model for training and analysis to identify periodic patterns and trends in traffic changes. When the model predicts that the concurrent request volume during a specific future period, such as after a promotional event begins, will exceed the current resource capacity threshold, the prediction unit immediately generates a scaling-up command. This command automatically triggers the resource management process, adding retrieval nodes and computing resources in real time, and automatically scaling down after the traffic peak subsides.

[0037] Specifically, the data verification unit periodically initiates a verification process. By retrieving the entire lifecycle record in the data traceability chain, including complete metadata such as data source, collection time, processing procedure, vector generation version, storage location, and access logs, it performs bidirectional comparison and consistency verification on the vector characteristics already in the database and their corresponding multi-source original data. Specifically, by verifying hash values, digital signatures, timestamp continuity, and business logic relevance, it identifies data entries that have been tampered with, damaged, or have abnormal sources. These entries are automatically marked as risky and moved to an isolation area, while simultaneously triggering alarms and notifying related modules to update the index or regenerate the vector.

[0038] Please see Figure 2 The diagram shows a flowchart of the intelligent retrieval method for high-concurrency credit leasing business data according to an embodiment of the present invention. The specific steps of the intelligent retrieval method for high-concurrency credit leasing business data according to an embodiment of the present invention are as follows: S1, acquires multi-source data through a distributed multi-source access architecture via a data acquisition unit; S2, the multi-source data entries to be generated into vectorized tasks are encapsulated into vectorized tasks by the data processing unit connected to the data acquisition unit, and the vectorized tasks are sent to the asynchronous task queue. S3, the vector generation unit connected to the data processing unit calls the corresponding feature extraction model to generate feature vectors based on the data categories in the vectorized tasks obtained from the asynchronous task queue; S4, the user's search request is parsed by the search and recognition unit connected to the vector generation unit, and the weight parameters of the search strategy are determined based on the core elements generated after parsing the user's search request, so as to call multiple feature vectors for retrieval, wherein the search strategy includes semantic search and keyword search. S5, the management unit connected to the retrieval and identification unit adaptively switches the index type, adjusts the retrieval node and adjusts the index parameters based on the number of retrieval requests and the concurrency pressure within a preset time period and the comparison results with the corresponding preset number of retrieval requests and the preset concurrency pressure, wherein the index parameters include at least the number of cluster centers; S6, the calculation unit connected to the management unit determines the retrieval characterization value based on the response time of the retrieval query with a recall rate greater than or equal to the preset recall rate within a preset time. In this process, qualified queries with a recall rate greater than or equal to the preset recall rate are filtered within the preset time. The response times of the qualified queries are sorted in ascending order of numerical values ​​to generate a response time sequence. The retrieval characterization value is determined based on the numerical value corresponding to the preset percentile of the response time sequence. S7, the analysis unit connected to the computing unit determines the search status based on the search characterization value, adjusts the number of search nodes based on the search status, and adjusts the preset number of search requests based on the search status after adjusting the number of search nodes.

[0039] Please see Figure 3 The diagram shown is a flowchart illustrating the steps of determining the comparison results between the retrieved characterization value and the preset characterization value in an embodiment of the present invention.

[0040] Specifically, the system is based on the physical performance limits of underlying hardware such as GPU memory and CPU thread count, as well as the actual fault tolerance requirements of the business. It also combines query-interception sample data obtained from the statistical analysis of historical risk case databases to set the values ​​of subsequent preset or critical parameters, so as to achieve the optimal balance between risk identification accuracy and system throughput within the hardware performance boundary.

[0041] Specifically, the process of setting the preset characterization value involves first establishing an initial latency target range based on system design specifications and the business's tolerance for response speed. Then, under typical datasets and production-level loads, stress tests and performance profiling are conducted, gradually adjusting the hybrid retrieval parameters until a 95% recall rate is achieved. Simultaneously, the TP95 distribution of the response time at this point is recorded, and this measured value is compared with the initial target, with approximately 20%-30% safety redundancy reserved. Finally, the preset characterization value is determined. Therefore, based on the above experimental results, this embodiment of the invention sets the preset characterization value L0 = 15ms. The comparison process between the retrieval characterization value L and the preset characterization value L0 is as follows: If the retrieval characteristic value L is less than or equal to the preset characteristic value L0, then the retrieval status is determined to be qualified; If the search characteristic value L is greater than the preset characteristic value L0, then the search status is determined to be unqualified.

[0042] Specifically, when the retrieval status is unqualified, the abnormal response rate corresponding to multiple preset periods is obtained. The abnormal response rate is the ratio of the number of retrieval requests with a response time longer than a preset time to the total number of retrieval requests. The growth slope of the abnormal response rate for each preset period is calculated. If the growth slope of the abnormal response rate corresponding to multiple consecutive preset periods is greater than the preset slope, it indicates that the number of retrieval requests that have flooded in in a short period of time exceeds the design capacity of the current computing resources and index architecture such as HNSW. Then, the number of retrieval nodes is adjusted based on the average growth slope of the abnormal response rate corresponding to multiple preset periods.

[0043] Specifically, setting the preset slope is an engineered process based on historical baseline analysis and stress testing: First, during the system's steady-state operation, such as during normal periods without promotions, the natural fluctuations of the abnormal response rate are observed and statistically analyzed over a long period, and the maximum slope of its normal fluctuations is calculated as a baseline reference. Then, through simulated stress testing, the load is gradually increased until the system reaches a performance inflection point, such as a sharp increase in response time and a surge in error rate. The typical growth slope of the abnormal response rate before the inflection point is recorded as a critical value. Combining the baseline slope and the critical slope, and reserving a certain safety margin, such as between 20% and 30%, the preset slope is finally set to a conservative value that is slightly higher than the daily fluctuations but significantly lower than the stress test critical value. Therefore, based on experimental results, this embodiment of the invention sets the first preset average value to 0.2.

[0044] Specifically, in this embodiment of the invention, a preset ratio P0 = 1.25 is set between the average growth slope of the abnormal response rate corresponding to multiple preset periods and the first preset average value. The comparison process between the ratio P of the average growth slope of the abnormal response rate corresponding to multiple preset periods and the first preset average value and the preset ratio P0 is as follows: If the ratio P of the average growth slope of the abnormal response rate corresponding to multiple preset periods to the first preset average value is less than or equal to the preset ratio P0, then the number of search nodes will be adjusted to 1.58 times the original number of search nodes, wherein the adjusted number of search nodes will be rounded up. If the ratio P of the average growth slope of the abnormal response rate corresponding to multiple preset periods to the first preset average value is greater than the preset ratio P0, then the number of search nodes will be adjusted to 2.09 times the original number of search nodes, wherein the adjusted number of search nodes is rounded up. Specifically, the above multiples are determined based on a comprehensive analysis of historical experience and experimental data to identify the corresponding values ​​that yielded the best results.

[0045] Specifically, if the retrieval status is unsatisfactory after adjusting the number of retrieval nodes, the difference between the retrieval characteristic value and the preset characteristic value is calculated. If the difference is greater than the preset difference, it indicates that the data volume has increased, but the index type remains HNSW, which is suitable for small data volumes, resulting in a sharp drop in retrieval efficiency. Therefore, the preset number of retrieval requests is adjusted based on the ratio of the difference to the preset difference. In this embodiment of the invention, the preset difference is set to 1ms, and the preset ratio Q0 is set to 1.16. The comparison process based on the ratio Q0 of the difference to the preset difference is as follows: If the ratio Q of the difference to the preset difference is less than or equal to the preset ratio Q0, the preset number of search requests will be adjusted to 0.92 times the original preset number of search requests, where the adjusted preset number of search requests will be rounded up. If the ratio Q of the difference to the preset difference is greater than the preset ratio Q0, the preset number of search requests will be adjusted to 0.83 times the original preset number of search requests, where the adjusted preset number of search requests will be rounded up. Specifically, the above multiples are determined based on a comprehensive analysis of historical experience and experimental data to identify the corresponding values ​​that yielded the best results.

[0046] Specifically, if the retrieval status is unsatisfactory after adjusting the preset number of retrieval requests, the request throughput is obtained. If the request throughput is greater than the preset request throughput, it indicates that the current concurrency pressure is too high, making the index of large data volumes unsatisfactory. Therefore, the number of cluster centers is adjusted based on the ratio of the obtained request throughput to the preset request throughput. The preset request throughput is set based on benchmark performance tests of the system under specific data scale and index architecture: First, under typical business load and stable resource conditions, stress tests are conducted on the system, gradually increasing concurrent requests until the response time reaches the unsatisfactory threshold, and the request throughput at this point is recorded as the performance inflection point value. Then, combining peak business traffic prediction and system stability requirements, a safety factor of 0.6 is multiplied by the performance inflection point value to obtain the preset request throughput. Therefore, based on the above experimental results, this embodiment of the invention sets the preset ratio of request throughput to preset request throughput R0 = 1.19. The comparison process between the ratio R of request throughput to preset request throughput and the preset ratio R0 is as follows: If the ratio R of the requested throughput to the preset requested throughput is less than or equal to the preset ratio R0, then the number of cluster centers will be adjusted to twice the original number of cluster centers. If the ratio R of the requested throughput to the preset requested throughput is greater than the preset ratio R0, then the number of cluster centers will be adjusted to 4 times the original number of cluster centers. Specifically, the above multiples are determined based on a comprehensive analysis of historical experience and experimental data to identify the corresponding values ​​that yielded the best results.

[0047] Please see Figure 4 The diagram shown is a flowchart illustrating the steps of determining the retrieval status based on adjusting the number of cluster centers in an embodiment of the present invention.

[0048] If the retrieval status is unsatisfactory after adjusting the number of cluster centers, the number of cluster centers is adjusted at least once until the number of adjustments is less than the preset number and the retrieval status is satisfactory, or the number of adjustments is equal to the preset number. If the retrieval status is still unsatisfactory after stopping the adjustment, multiple historical retrieval characteristic values ​​within a preset time period are obtained. The variance and average of the multiple retrieval characteristic values ​​are calculated respectively. If the variance is less than the preset variance and the average is greater than the second preset average, it indicates that the discriminative power of the generated vector features has decreased due to upstream data pollution such as image blurring or text noise. In order to achieve the target recall rate, the system needs to scan more candidate data to improve the response time. Therefore, the data verification period is adjusted based on the ratio of the variance to the preset variance.

[0049] Specifically, the preset variance is used to determine whether the fluctuation of the retrieval representation value is abnormally stable. It is determined based on historical data from a healthy operating period of the system without data contamination: First, under steady-state conditions, retrieval representation values ​​for multiple time windows are collected and calculated over a long period, and the variance distribution of these sequences is statistically analyzed. The high percentile value of this distribution is taken as a reference upper limit to exclude variance caused by normal business fluctuations such as traffic cycle changes. Therefore, based on the above experimental results, this embodiment of the invention sets the preset variance to 0.3.

[0050] Specifically, the second preset average value is used to determine whether the retrieval characterization value is at an abnormally high level. The specific determination process is as follows: First, during the same healthy operation period, the normal baseline average value of the retrieval characterization value is determined; then, by injecting different levels of data noise, such as image blurring and text noise, experiments are conducted to observe the growth trend of the characterization value, and to identify the critical pollution level at which the system performance begins to deteriorate significantly and the index parameter adjustment is ineffective. The average value of the retrieval characterization value at this time is recorded as the second preset average value.

[0051] Specifically, in this embodiment of the invention, a preset ratio T0 = 0.86 is set between the variance and the preset variance. The comparison process between the ratio T0 and the preset variance is as follows: If the ratio T of the variance to the preset variance is less than or equal to the preset ratio T0, the data verification period will be adjusted to 0.71 times the original data verification period, where the adjusted data verification period is rounded up. If the ratio T of the variance to the preset variance is greater than the preset ratio T0, the data verification period will be adjusted to 0.87 times the original data verification period, where the adjusted data verification period is rounded up. Specifically, the above multiples are determined based on a comprehensive analysis of historical experience and experimental data to identify the corresponding values ​​that yielded the best results.

[0052] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0053] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An intelligent retrieval system for high-concurrency credit leasing business data, characterized in that, include: The data acquisition unit is used to acquire multi-source data using a distributed multi-source access architecture; A data processing unit, connected to the data acquisition unit, is used to encapsulate multi-source data entries to be generated into vectorized tasks, and to send the vectorized tasks to an asynchronous task queue. A vector generation unit, connected to the data processing unit, is used to call the corresponding feature extraction model to generate feature vectors based on the data category in the vectorized task obtained from the asynchronous task queue. The retrieval and identification unit, which is connected to the vector generation unit, is used to parse the user's retrieval request and determine the weight parameters of the retrieval strategy based on the core elements generated after parsing the user's retrieval request, so as to call multiple feature vectors for retrieval. The retrieval strategy includes semantic retrieval and keyword retrieval. The management unit, which is connected to the retrieval and identification unit, is used to adaptively switch the index type, adjust the retrieval node and adjust the index parameters based on the number of retrieval requests and the concurrency pressure within a preset time period and the comparison results with the corresponding preset number of retrieval requests and the preset concurrency pressure. The index parameters include at least the number of cluster centers. A calculation unit, which is connected to the management unit, is used to determine the retrieval characterization value based on the response time of retrieval queries whose recall rate is greater than or equal to the preset recall rate within a preset time. An analysis unit, connected to the computing unit, is used to determine the search status based on the search characterization value, adjust the number of search nodes based on the search status, and adjust the preset number of search requests based on the search status after adjusting the number of search nodes.

2. The intelligent retrieval system for high-concurrency credit leasing business data according to claim 1, characterized in that, The system also includes a storage unit, which is used to store the vector features and the multi-source data in several index categories based on a hash algorithm, and each segment corresponds to an independent retrieval node. The index categories include at least rental category, user region and data update time.

3. The intelligent retrieval system for high-concurrency credit leasing business data according to claim 2, characterized in that, The system also includes a prediction unit, which is used to predict peak traffic based on historical concurrent data and time-series prediction models, so as to automatically trigger node expansion and resource addition.

4. The intelligent retrieval system for high-concurrency credit leasing business data according to claim 3, characterized in that, The system also includes a data verification unit, which is used to periodically verify the vector features and the multi-source data based on the full lifecycle trajectory of the data in the data traceability chain, so as to mark and remove tampered data.

5. The intelligent retrieval system for high-concurrency credit leasing business data according to claim 4, characterized in that, The calculation unit is also used to filter eligible queries with a recall rate greater than or equal to a preset recall rate within a preset time period; The calculation unit is also used to sort the response times corresponding to the compliance query in ascending order of numerical values ​​to generate a response time sequence; The calculation unit is also used to determine the retrieval representation value based on the numerical value corresponding to the preset quantile of the response time sequence.

6. The intelligent retrieval system for high-concurrency credit leasing business data according to claim 5, characterized in that, The analysis unit is also used to obtain the abnormal response rate corresponding to multiple preset periods when the retrieval status is unqualified. The abnormal response rate is the ratio of the number of retrieval requests with a response time greater than a preset time to the total number of retrieval requests. The analysis unit is also used to calculate the growth slope of the abnormal response rate for each preset period, and when the growth slope of the abnormal response rate for multiple consecutive preset periods is greater than the preset slope, the number of retrieval nodes is adjusted based on the average value of the growth slope of the abnormal response rate for multiple preset periods. If the search characteristic value is greater than the preset characteristic value, the search status is determined to be unqualified.

7. The intelligent retrieval system for high-concurrency credit leasing business data according to claim 6, characterized in that, The analysis unit is also used to increase the number of retrieval nodes based on the ratio of the average growth slope of the abnormal response rate corresponding to multiple preset periods to the first preset average value, and the increase in the number of retrieval nodes is proportional to the ratio.

8. The intelligent retrieval system for high-concurrency credit leasing business data according to claim 7, characterized in that, The analysis unit is also used to calculate the difference between the search characteristic value and the preset characteristic value when the search status is unqualified after adjusting the number of search nodes. The analysis unit is also used to reduce the preset number of search requests based on the ratio of the difference to the preset difference when the difference is greater than the preset difference, and the reduction in the preset number of search requests is proportional to the ratio.

9. The intelligent retrieval system for high-concurrency credit leasing business data according to claim 8, characterized in that, The analysis unit is also used to obtain the request throughput when the retrieval status is unqualified after adjusting the preset number of retrieval requests. The analysis unit is also used to increase the number of cluster centers based on the ratio of request throughput to preset request throughput when the request throughput is greater than the preset request throughput, and the increase in the number of cluster centers is proportional to the ratio.

10. The intelligent retrieval system for high-concurrency credit leasing business data according to claim 9, characterized in that, The analysis unit is also used to repeatedly adjust the number of cluster centers at least once if the retrieval status is not satisfactory after adjusting the number of cluster centers, until the number of adjustments is less than the preset number and the retrieval status is satisfactory or the number of adjustments is equal to the preset number and then the adjustment stops. The analysis unit is also used to obtain multiple historical preset time period search characterization values ​​when the search status is unqualified after the adjustment is stopped. The analysis unit is also used to calculate the variance and average of multiple retrieval characterization values ​​respectively. When the variance is less than the preset variance and the average is greater than the second preset average, the data verification period is reduced based on the ratio of the variance to the preset variance, and the reduction in the data verification period is inversely proportional to the ratio.

Images