A data acquisition method and device, electronic equipment and computer readable medium

Abstract

The application discloses a data acquisition method and device, electronic equipment and a computer readable medium, and relates to the technical field of computers. The method comprises the following steps: receiving a data acquisition request, acquiring corresponding input parameter attributes and a method signature; determining a cache primary key value based on the input parameter data and the method signature; calling a whitelist to verify the cache primary key value; in response to the verification being passed, determining a target interface identifier based on the data acquisition request, and then calling a corresponding target interface to acquire attribute values corresponding to the input parameter attributes; and in response to the verification failing, calling a metadata cache container to acquire corresponding annotation configuration information from the metadata container based on the method signature, and then acquiring attribute values corresponding to the input parameter attributes in the distributed cache based on the annotation configuration information. The whole process has very low code invasiveness, and a degradation backup solution and performance optimization are performed, so that the availability and performance of the original interface are guaranteed, and the maintenance difficulty and cost are reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data acquisition method, device, electronic device, and computer-readable medium. Background Art

[0002] Retrieving data through a distributed cache is complex and difficult to maintain. This makes it difficult to quickly establish a development advantage within the flexible iteration requirements, slowing down development progress. Multi-version logic is prone to low-level errors. As the business grows, the code required for maintenance increases exponentially, significantly increasing costs.

[0003] During the implementation of this application, the inventors discovered that the prior art has at least the following problems:

[0004] When obtaining data based on distributed cache, the implementation is complex, difficult to maintain, and costly. Summary of the Invention

[0005] In view of this, the embodiments of the present application provide a data acquisition method, device, electronic device and computer-readable medium, which can solve the problems of complex implementation, difficult maintenance and high cost when acquiring data based on distributed cache.

[0006] To achieve the above objectives, according to one aspect of an embodiment of the present application, a data acquisition method is provided, comprising:

[0007] Receive data acquisition request and obtain corresponding input parameter attributes and method signature;

[0008] Determine the cache primary key value based on the input data and method signature;

[0009] Call the whitelist to verify the cache primary key value;

[0010] In response to the verification being passed, the target interface identifier is determined based on the data acquisition request, and then the corresponding target interface is called to obtain the attribute value corresponding to the input parameter attribute;

[0011] In response to the verification failure, the metadata cache container is called to obtain the corresponding annotation configuration information from the metadata container based on the method signature, and then obtain the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information.

[0012] Optionally, call the whitelist to validate the cache key value, including:

[0013] Based on the fast failure mechanism, the matching thread between the whitelist and the cache primary key value is detected. In response to no fast failure being detected, the whitelist is parsed to obtain the customized whitelist attribute value.

[0014] Get dynamically updated whitelist configuration information and verify the customized whitelist attribute values.

[0015] Optionally, calling a whitelist to verify the cache primary key value also includes:

[0016] In response to detecting the fast fail, it is determined that validation of the cache primary key value has failed.

[0017] Optionally, before calling the metadata cache container, the method further includes:

[0018] Obtain each data acquisition request received in the cache processing logic, and then obtain metadata corresponding to each data acquisition request;

[0019] The dimension of the method signature is determined, the metadata is converted into data of the dimension of the method signature, and then stored in the initialized cache container to generate a metadata cache container.

[0020] Optionally, after calling the whitelist to verify the cache primary key value, the method further includes:

[0021] In response to the verification result being empty, repeated verification is prevented within a preset time.

[0022] Optionally, determine the cache primary key value, including:

[0023] Get the prefix of the method signature;

[0024] Combine the prefix with the input data to generate combined data;

[0025] Generates cache primary key value based on combined data.

[0026] Optionally, based on the combined data, a cache primary key value is generated, including:

[0027] Encrypt and compress the combined data to generate a cache primary key value.

[0028] Optionally, after obtaining the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information, the method further includes:

[0029] The output form of the attribute value is determined based on the annotation configuration information, and the attribute value is output in the output form.

[0030] In addition, the present application also provides a data acquisition device, including:

[0031] A receiving unit is configured to receive a data acquisition request and obtain corresponding input parameter attributes and method signatures;

[0032] A cache primary key value determining unit is configured to determine a cache primary key value based on input parameter data and a method signature;

[0033] A verification unit, configured to call a whitelist to verify the cache primary key value;

[0034] The interface calling unit is configured to, in response to the verification being passed, determine the target interface identifier based on the data acquisition request, and then call the corresponding target interface to obtain the attribute value corresponding to the input parameter attribute;

[0035] The cache calling unit is configured to call the metadata cache container in response to a verification failure to obtain corresponding annotation configuration information from the metadata container based on the method signature, and then obtain the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information.

[0036] Optionally, the verification unit is further configured to:

[0037] Based on the fast failure mechanism, the matching thread between the whitelist and the cache primary key value is detected. In response to no fast failure being detected, the whitelist is parsed to obtain the customized whitelist attribute value.

[0038] Get dynamically updated whitelist configuration information and verify the customized whitelist attribute values.

[0039] Optionally, the verification unit is further configured to:

[0040] In response to detecting the fast fail, it is determined that validation of the cache primary key value has failed.

[0041] Optionally, the data acquisition device further includes a metadata cache container generation unit configured to:

[0042] Obtain each data acquisition request received in the cache processing logic, and then obtain metadata corresponding to each data acquisition request;

[0043] The dimension of the method signature is determined, the metadata is converted into data of the dimension of the method signature, and then stored in the initialized cache container to generate a metadata cache container.

[0044] Optionally, the data acquisition device further includes an anti-brushing control unit configured to:

[0045] After calling the whitelist to verify the cache primary key value, in response to the verification result being empty, repeated verification is prevented within a preset time.

[0046] Optionally, the cache primary key value determining unit is further configured to:

[0047] Get the prefix of the method signature;

[0048] Combine the prefix with the input data to generate combined data;

[0049] Generates cache primary key value based on combined data.

[0050] Optionally, the cache primary key value determining unit is further configured to:

[0051] Encrypt and compress the combined data to generate a cache primary key value.

[0052] Optionally, the data acquisition device further includes an output unit configured to:

[0053] The output form of the attribute value is determined based on the annotation configuration information, and the attribute value is output in the output form.

[0054] In addition, the present application also provides a data acquisition electronic device, including: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the one or more processors implement the data acquisition method as described above.

[0055] In addition, the present application also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the data acquisition method as described above is implemented.

[0056] One embodiment of the above invention has the following advantages or beneficial effects: the present application obtains the corresponding input parameter attributes and method signatures by receiving a data acquisition request; determines the cache primary key value based on the input parameter data and method signature; calls the whitelist to verify the cache primary key value; in response to the verification being passed, determines the target interface identifier based on the data acquisition request, and then calls the corresponding target interface to obtain the attribute value corresponding to the input parameter attribute; in response to the verification failing, calls the metadata cache container to obtain the corresponding annotation configuration information from the metadata container based on the method signature, and then obtains the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information. Through the metadata cache container, the corresponding annotation configuration information is pulled based on the method signature, and the cached data in the distributed cache is quickly pulled through the annotation configuration information. The code is extremely low intrusive throughout the process, and a downgrade backup plan and performance optimization are implemented to ensure the availability and performance of the original interface and reduce maintenance difficulty and cost.

[0057] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings are provided to facilitate a better understanding of the present application and do not constitute an undue limitation on the present application.

[0059] Figure 1 is a schematic diagram of the main process of the data acquisition method according to the first embodiment of the present application;

[0060] Figure 2 is a schematic diagram of the main process of the data acquisition method according to the second embodiment of the present application;

[0061] Figure 3 is a schematic diagram of an application scenario of the data acquisition method according to the third embodiment of the present application;

[0062] Figure 4 Schematic diagram of metadata containerization design of a data acquisition method according to an embodiment of the present application;

[0063] Figure 5 This is a schematic diagram of a whitelist customization configuration and parsing process according to a data acquisition method according to an embodiment of the present application;

[0064] Figure 6 is a schematic diagram of the main units of a data acquisition device according to an embodiment of the present application;

[0065] Figure 7 is an exemplary system architecture diagram to which embodiments of the present application may be applied;

[0066] Figure 8 It is a structural diagram of a computer system of a terminal device or server suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0067] The following describes exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. The acquisition, storage, use, processing, etc. of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0068] Figure 1 is a schematic diagram of the main process of the data acquisition method according to the first embodiment of the present application, such as Figure 1 As shown, the data acquisition method includes:

[0069] Step S101: Receive a data acquisition request and obtain the corresponding input parameter attributes and method signature.

[0070] In this embodiment, the execution subject of the data acquisition method (for example, it can be a server) can receive the data acquisition request through a wired connection or a wireless connection. Get the input parameter attributes in the data acquisition request, such as userID, appID, etc. The method signature obtained from the data acquisition request is the method signature corresponding to the interface that receives the data acquisition request. Specifically, the method signature is used to distinguish the identifiers of different methods. In the embodiment of the present application, the method is a function. The method is composed of a method name, a parameter list, a return value, and a method body. The method signature is composed of a method name + a parameter list.

[0071] Step S102: Determine the cache primary key value based on the input parameter data and method signature.

[0072] The cache primary key value refers to the cache key, which can be used to identify the location of the requested cache data.

[0073] Specifically, determining the cache primary key value includes:

[0074] Get the prefix of the method signature; combine the prefix with the input parameter data to generate combined data; generate the cache primary key value based on the combined data.

[0075] For example, the prefix of the method signature, such as ab, is combined with the input parameter data, such as userID. The combination can be sequentially combined to obtain abuserID, which is the cache primary key value, or interspersed combinations can be performed, such as abIDuser, which is the cache primary key value. The embodiments of the present application do not limit the specific combination of the prefix of the method signature and the input parameter data.

[0076] Specifically, based on the combined data, a cache primary key value is generated, including:

[0077] Encrypt and compress the combined data to generate a cache primary key value.

[0078] The combined data needs to be compressed because the key formed by the prefix + the interface input parameter is used as the cache key value. Without compression, the key will be very large. If compression is not performed, the cost of a single request will increase. Therefore, compression is performed to reduce the size of the cache key.

[0079] The execution entity can serialize the combined data and then perform MD5 encryption and compression to generate a cache primary key value. Data serialization refers to the process of converting in-memory data format into binary (or text) format. Data serialization can be used for data compression and multi-language compatibility.

[0080] Of course, the execution entity can also perform MD5 encryption compression on the interface input parameter and then combine it with the signature prefix to generate a cache primary key value. This application does not specifically limit the timing of encryption and compression.

[0081] Step S103: calling the whitelist to verify the cache primary key value.

[0082] Specifically, the whitelist is called to verify the cache primary key value, including:

[0083] Based on the fast failure mechanism, the matching threads of the whitelist and the cache primary key value are detected. In response to no fast failure being detected, the whitelist is parsed to obtain the customized whitelist attribute value; the dynamically updated whitelist configuration information is obtained, and then the customized whitelist attribute value is verified.

[0084] Specifically, calling the whitelist to verify the cache primary key value also includes:

[0085] In response to detecting the fast fail, it is determined that validation of the cache primary key value has failed.

[0086] Specifically, the whitelist can be dynamically configured based on the fields that the user configures to be parsed. For example, if a user wants to use the patientID field in an API as a whitelist field, after configuring it, subsequent requests to the API can quickly skip the cache and request the real data if the configured field is found in the whitelist.

[0087] For example, Figure 5 As shown, according to the whitelist configured by the configuration center (for example, DUCC), whitelist-related parsing can be performed. After fast failure and whitelist parsing (developers can customize the whitelist standby fields), it is determined whether data needs to be obtained from the cache. DUCC is a configuration center that can uniformly manage cluster configuration files and dynamically update configuration parameters. Running projects can dynamically update runtime parameters through the configured DUCC configuration items. For example, if a whitelist is dynamically added in the configuration center, the system can hear it in time, make corresponding modifications, and take effect.

[0088] Specifically, the whitelist verification requires the use of an algorithm for parsing the interface input parameters. A fast failure mechanism is introduced here, so that it does not need to parse the input parameters every time. Under certain conditions, it can fail quickly and return quickly, further improving efficiency. During the iteration process, if a thread changes the collection structure, fail-fast will occur. Since the data synchronization between the original interface and the corresponding cache may be delayed, it will cause data inconsistency problems between the original interface and the corresponding cache. For example Figure 5As shown, when the whitelist matching logic is enabled, the execution entity can first call the fast-fail mechanism to determine whether the current data acquisition request meets the fast-fail requirement. For example, the execution entity can call the fast-fail mechanism to obtain the data consistency requirement identifier corresponding to the current data acquisition request, such as 10%. Based on this data consistency requirement identifier, the execution entity determines whether the current data acquisition request has high consistency requirements for the acquired data. 10% corresponds to lower consistency requirements for the requested data. If the data consistency requirement identifiers of all historical data acquisition requests are between 50% and 100%, this indicates that the historical data acquisition requests had high consistency requirements and must obtain data from the original interface corresponding to the request. Data acquisition requests with lower consistency requirements can obtain the corresponding data from the cache. When the fast-fail mechanism detects a change in the data consistency requirement identifier in the current data acquisition request, it immediately responds with a fast-fail, intercepting the process of obtaining the requested data from the original interface and switching to executing the logic related to obtaining the requested data from the cache. In response to the current data acquisition request not having a fast failure, the metadata cache container is called to obtain the interface whitelist configuration / parse the whitelist paradigm nested type, and then parse to obtain the introspection one whitelist attribute value, for example, userID is A Insurance Company (here userID is not in the historical whitelist, it is a user-defined whitelist attribute). Then, the execution subject can pull the updated whitelist configuration information of the configuration center (DUCC) to determine whether the user-defined whitelist attribute value matches the updated whitelist. If it matches, the data corresponding to the request is obtained from the original interface. If it does not match, the relevant logic of obtaining the data corresponding to the request from the cache is executed. Among them, the fast failure FastFail is to save CPU resources. By judging the current whitelist configuration and the DUCC whitelist switch of the configuration center, a fast failure is achieved, and the whitelist verification logic is directly completed. In special cases, the whitelist nested metadata container may always be empty (whitelist is not configured or the whitelist switch is always closed). Users can flexibly configure multiple nested types of whitelist standby fields. The embodiment of the present application can automatically parse and obtain the value of the field used as the whitelist on the interface input parameter, and use it as the user-defined whitelist attribute value to verify the updated whitelist.

[0089] Specifically, after calling the whitelist to verify the cache primary key value, the method further includes:

[0090] In response to the verification result being empty, repeated verification is prevented within a preset time.

[0091] When the execution subject determines that the verification result is null, the anti-brushing control process is executed. Anti-brushing control is mainly used when the interface returned by the interface does not meet expectations, such as the return value is Null. If anti-brushing is turned on, when the same request has a null value, access will be restricted for 3 seconds (the specific time can be configured), thereby protecting the database or downstream interface.

[0092] Step S104: determine whether the verification is successful.

[0093] Step S105 , in response to the verification being passed, the target interface identifier is determined based on the data acquisition request, and then the corresponding target interface is called to obtain the attribute value corresponding to the input parameter attribute.

[0094] Input attribute, such as userID, and corresponding attribute value, such as the company name or user name corresponding to userID.

[0095] Step S106, in response to verification failure, calling the metadata cache container to obtain corresponding annotation configuration information from the metadata container based on the method signature, and then obtaining the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information.

[0096] Annotation configuration information is based on current business needs. The corresponding annotations are configured on the interfaces that need to be cached. At the same time, different annotation attributes or different configuration values ​​in the DUCC configuration center are configured according to specific business needs.

[0097] In response to a verification failure, the execution subject executes the logic related to obtaining the requested data based on the cache. Specifically, it calls the metadata cache container to obtain the annotation configuration information corresponding to the method signature, and then obtains the data corresponding to the annotation configuration information, which is specifically the attribute value corresponding to the input parameter attribute.

[0098] Specifically, after obtaining the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information, the method further includes:

[0099] The output form of the attribute value is determined based on the annotation configuration information, and the attribute value is output in the output form.

[0100] This embodiment obtains the corresponding input parameter attributes and method signatures by receiving a data acquisition request; determines the cache primary key value based on the input parameter data and method signature; calls the whitelist to verify the cache primary key value; in response to successful verification, determines the target interface identifier based on the data acquisition request, and then calls the corresponding target interface to obtain the attribute value corresponding to the input parameter attribute; in response to failed verification, calls the metadata cache container to obtain the corresponding annotation configuration information from the metadata container based on the method signature, and then obtains the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information. Through the metadata cache container, the corresponding annotation configuration information is pulled based on the method signature, and the cached data in the distributed cache is quickly pulled through the annotation configuration information. The code is extremely low-invasive throughout the process, and a downgrade backup plan and performance optimization are implemented to ensure the availability and performance of the original interface and reduce maintenance difficulty and cost.

[0101] Figure 2 This is a schematic diagram of the main flow of the data acquisition method according to the second embodiment of the present application. Figure 2 As shown, the data acquisition method includes:

[0102] Step S201: Receive a data acquisition request and obtain the corresponding input parameter attributes and method signature.

[0103] Step S202: Determine the cache primary key value based on the input parameter data and method signature.

[0104] Step S203: calling the whitelist to verify the cache primary key value.

[0105] Step S204: determine whether the verification is successful.

[0106] Step S205 , in response to the verification being passed, determining the target interface identifier based on the data acquisition request, and then calling the corresponding target interface to obtain the attribute value corresponding to the input parameter attribute.

[0107] The principles of steps S201 to S205 are similar to those of steps S101 to S105 and will not be described in detail here.

[0108] Step S206 , obtaining each data acquisition request received in the cache processing logic, and then obtaining metadata corresponding to each data acquisition request.

[0109] Step S207 , determining the dimension of the method signature, converting the metadata into data of the dimension of the method signature, and then storing it into the initialized cache container to generate a metadata cache container.

[0110] The main operation process of container initialization is as follows: first read the annotations and XML configuration files, load these configuration files into memory, and these configuration files will be treated as Resource objects in memory. These objects will then be parsed into BeanDefinition instances and finally registered in the Spring container.

[0111] Before calling the metadata cache container, you must first build the metadata cache container. Figure 4 As shown, the specific construction method of the metadata cache container is as follows: the execution subject pulls metadata from all metadata information requests involved in the cache processing logic in the cache resolution core process, determines whether the cache container has been initialized, and if so, pulls metadata. If the pull is successful, the pulled metadata information is returned to the cache resolution core process; if the cache container is not initialized, the container is initialized in singleton mode to ensure thread safety while greatly improving the performance of the cache container. After the container has been successfully initialized, if the metadata fails to be pulled, the annotations, interface return values, Json deserialization, whitelist nested strings and other metadata information that are first parsed by the singleton mode initialization container are placed together in the initialized cache container based on the interface signature dimension, thereby generating a metadata cache container and returning the pulled metadata information to the cache resolution core process.

[0112] Step S208, in response to the verification failure, calls the metadata cache container to obtain the corresponding annotation configuration information from the metadata container based on the method signature, and then obtains the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information. The embodiment of the present application involves high scalability and high compatibility, so it will contain many complex calculations, such as dynamic parsing of the return value type of multi-layer nested interfaces, dynamic parsing of nested attributes of whitelists, etc. In order to further improve the performance, it can be done to the greatest extent without affecting the performance of the original interface. Therefore, a metadata containerization design based on the container idea is designed. Most of the complex calculation logic is abstracted into query operations with a complexity of O1, which greatly improves the performance.

[0113] The principle of step S208 is similar to that of steps S101 to S106 and will not be described again here.

[0114] Figure 3 Schematic diagram of the application scenario of the data acquisition method according to the third embodiment of the present application. The data acquisition method is applied to the scenario of determining whether to execute data acquisition of the original interface or to obtain the requested data from the cache for a data acquisition request. Figure 3As shown, when the execution subject detects that a business method of an interface has been called, it switches to the current interface request. Then it determines whether the cache master switch is turned on. In response to the master switch being turned off, the method of obtaining data from the original interface is executed; in response to the master control being turned on, the configuration parsing / cache result process is executed, specifically: obtaining the method signature of the interface, calling the container loading / configuration pull service to pull the annotation configuration information from the constructed metadata cache container based on the method signature, performing maintenance and updating of the local container, and returning the pulled annotation configuration information through the configuration center (DUCC), and then parsing the corresponding configuration information. Among them, the configuration maintenance of the local container includes: personalized key, expiration time, automatic refresh switch, automatic refresh interval, anti-brush switch, and anti-brush time maintenance. Dynamic configuration push is executed through DUCC, specifically including: dynamic push of configuration information of master control, whitelist and debug log. After pulling the annotation configuration information, the execution subject can perform signature prefix compression based on the method signature prefix and interface input parameter information to obtain the cache primary key value. The execution entity can then verify and match the cache primary key value with the dynamically configured whitelist. In response to successful verification, it executes the method of obtaining the requested data from the original interface. In response to failed verification, it dynamically analyzes the verification result, performs anti-brush control, and asynchronously refreshes the cache, and finally returns the dynamic adaptation result, that is, obtaining the requested data from the cache corresponding to the original interface.

[0115] This application can be applied to any scenario that requires the use of distributed cache services. Developers will no longer need to consider how to design and implement distributed caches, but only need to concentrate on core business development. At the same time, by configuring the @SimpleRedis annotation on the interface that requires the use of distributed cache (this annotation is used to indicate the conditions and cache location for the cache), the data / data set returned by this interface can be easily cached in JIMDB. The next access to the interface will automatically pull data through JIMDB. Not only that, developers can also flexibly expand the required content by passing custom configurations to the annotations, such as cache expiration time, personalized cache key, whitelist for projects, automatic refresh switch, automatic refresh interval, whether to enable interface anti-brushing, etc. At the same time, SimpleRedis also supports DUCC-based cache master control, whitelist dynamic configuration, log switch and other functions, which can more flexibly control distributed cache.

[0116] In an embodiment of the present application, when the interface is called, the annotation will cut into the method of the current interface, and go through the general control, whitelist fast failure, configuration information parsing, and at the same time perform signature parsing, key compression, caching, asynchronous refresh, anti-brushing and other related operations. It will eventually automatically adapt and pull and return the original data / data set from JIMDB to the user interface. The code intrusion is extremely low throughout the process, and a downgrade backup plan and performance optimization are implemented to ensure the availability and performance of the original interface.

[0117] Figure 6 Schematic diagram of the main units of the data acquisition device according to an embodiment of the present application. Figure 6 As shown, the data acquisition device includes a receiving unit 601 , a cache primary key value determining unit 602 , a verification unit 603 , an interface calling unit 604 and a cache calling unit 605 .

[0118] The receiving unit 601 is configured to receive a data acquisition request and obtain corresponding input parameter attributes and method signatures;

[0119] The cache primary key value determining unit 602 is configured to determine the cache primary key value based on the input parameter data and the method signature;

[0120] The verification unit 603 is configured to call the whitelist to verify the cache primary key value;

[0121] The interface calling unit 604 is configured to, in response to the verification being passed, determine the target interface identifier based on the data acquisition request, and then call the corresponding target interface to obtain the attribute value corresponding to the input parameter attribute;

[0122] The cache calling unit 605 is configured to call the metadata cache container in response to verification failure to obtain corresponding annotation configuration information from the metadata container based on the method signature, and then obtain the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information.

[0123] In some embodiments, the verification unit 603 is further configured to: based on a fast failure mechanism, detect the matching threads of the whitelist and the cache primary key value; in response to no fast failure being detected, parse the whitelist to obtain a customized whitelist attribute value; obtain dynamically updated whitelist configuration information, and then verify the customized whitelist attribute value.

[0124] In some embodiments, the verification unit 603 is further configured to: in response to detecting a fast failure, determine that verification of the cache primary key value fails.

[0125] In some embodiments, the data acquisition device further includes Figure 6The metadata cache container generation unit, not shown in the figure, is configured to: obtain each data acquisition request received in the cache processing logic, and then obtain the metadata corresponding to each data acquisition request; determine the dimension of the method signature, convert the metadata into data of the dimension of the method signature, and then store it into the initialized cache container to generate a metadata cache container.

[0126] In some embodiments, the data acquisition device further includes Figure 6 The anti-brushing control unit not shown in the figure is configured to: after calling the whitelist to verify the cache primary key value, in response to the verification result being empty, prevent repeated verification within a preset time.

[0127] In some embodiments, the cache primary key value determination unit 602 is further configured to: obtain a prefix of the method signature; combine the prefix with the input parameter data to generate combined data; and generate a cache primary key value based on the combined data.

[0128] In some embodiments, the cache primary key value determining unit 602 is further configured to: encrypt and compress the combined data to generate a cache primary key value.

[0129] In some embodiments, the data acquisition device further includes Figure 6 The output unit not shown in the figure is configured to: determine the output form of the attribute value based on the annotation configuration information, and then output the attribute value in the output form.

[0130] It should be noted that the data acquisition method and the data acquisition device of the present application have corresponding relationships in terms of specific implementation contents, so the repeated contents will not be described again.

[0131] Figure 7 An exemplary system architecture 700 is shown to which the data acquisition method or data acquisition device according to the embodiments of the present application can be applied.

[0132] like Figure 7 As shown, system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. Network 704 is used to provide a medium for communication links between terminal devices 701, 702, 703 and server 705. Network 704 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0133] Users can use terminal devices 701, 702, and 703 to interact with server 705 via network 704 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 701, 702, and 703, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0134] The terminal devices 701 , 702 , and 703 may be various electronic devices having a data acquisition processing screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0135] Server 705 can be a server that provides various services, such as a background management server that provides support for data acquisition requests submitted by users using terminal devices 701, 702, and 703 (only as an example). The background management server can receive data acquisition requests, obtain corresponding input parameter attributes and method signatures; determine the cache primary key value based on the input parameter data and method signature; call the whitelist to verify the cache primary key value; in response to the verification passing, determine the target interface identifier based on the data acquisition request, and then call the corresponding target interface to obtain the attribute value corresponding to the input parameter attribute; in response to the verification failing, call the metadata cache container to obtain the corresponding annotation configuration information from the metadata container based on the method signature, and then obtain the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information. Through the metadata cache container, the corresponding annotation configuration information is pulled based on the method signature, and the cached data in the distributed cache is quickly pulled through the annotation configuration information. The code is extremely low intrusive throughout the process, and a downgrade backup plan and performance optimization are implemented to ensure the availability and performance of the original interface and reduce maintenance difficulty and cost.

[0136] It should be noted that the data acquisition method provided in the embodiment of the present application is generally executed by the server 705 , and accordingly, the data acquisition device is generally set in the server 705 .

[0137] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0138] Reference below Figure 8 , which shows a structural diagram of a computer system 800 of a terminal device suitable for implementing an embodiment of the present application. Figure 8 The terminal device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0139] like Figure 8As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the computer system 800 are also stored in the RAM 803. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0140] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, and the like; an output section 807 including displays such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card or a modem. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. Removable media 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like, is installed in the drive 810 as needed, so that computer programs read therefrom can be installed into the storage section 808 as needed.

[0141] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 809, and / or installed from a removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present application are executed.

[0142] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can include, for example, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0144] The units involved in the embodiments described in this application can be implemented by software or hardware. The units described can also be set in a processor. For example, it can be described as follows: a processor includes a receiving unit, a cache primary key value determination unit, a verification unit, an interface calling unit, and a cache calling unit. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0145] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device receives a data acquisition request and obtains the corresponding input parameter attributes and method signatures; determines the cache primary key value based on the input parameter data and method signature; calls a whitelist to verify the cache primary key value; in response to the verification passing, determines the target interface identifier based on the data acquisition request, and then calls the corresponding target interface to obtain the attribute value corresponding to the input parameter attribute; in response to the verification failing, calls the metadata cache container to obtain the corresponding annotation configuration information from the metadata container based on the method signature, and then obtains the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information.

[0146] According to the technical solution of the embodiments of this application, the metadata cache container pulls the corresponding annotation configuration information based on the method signature, and then quickly pulls the cached data in the distributed cache using the annotation configuration information. The entire process is extremely non-invasive, and a fallback solution and performance optimization are implemented to ensure the availability and performance of the original interface, reducing maintenance difficulty and costs.

[0147] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A data acquisition method, characterized in that: include: Receive data acquisition request and obtain corresponding input parameter attributes and method signature; Determine a cache primary key value based on the input parameter attributes and the method signature; Calling a whitelist to verify the cache primary key value; In response to the verification being successful, the target interface identifier is determined based on the data acquisition request, and the corresponding target interface is called to obtain the attribute value corresponding to the input parameter attribute; when the fast failure mechanism detects that the data consistency requirement identifier in the current data acquisition request has changed, it immediately responds with a fast failure, intercepts the process of obtaining the requested data from the target interface, and switches to executing the relevant logic of obtaining the requested data from the distributed cache; Pull metadata from all metadata information requests involved in the cache processing logic in the cache resolution core process, determine whether the cache container has been initialized, and if the cache container has been initialized, pull metadata. If the metadata pulling is successful, return the pulled metadata to the cache resolution core process; if the cache container has not been initialized, initialize the cache container in singleton mode; After the cache container is successfully initialized, if the metadata pulling fails, the metadata containing annotations, interface return values, Json deserialization, and whitelist nested strings parsed for the first time by initializing the cache container based on the singleton mode, and the metadata containing annotations, interface return values, Json deserialization, and whitelist nested strings parsed are placed into the cache container that has been initialized based on the dimension of the interface method signature, thereby generating a metadata cache container; In response to a verification failure, the metadata cache container is called to obtain corresponding annotation configuration information from the metadata container based on the method signature, and then obtain the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information.

2. The method according to claim 1, characterized in that The calling of the whitelist to verify the cache primary key value includes: Based on a fast failure mechanism, detecting a matching thread between the whitelist and the cache primary key value, and in response to not detecting a fast failure, parsing the whitelist to obtain a custom whitelist attribute value; The dynamically updated whitelist configuration information is obtained, and then the customized whitelist attribute value is verified.

3. The method according to claim 2, characterized in that The calling whitelist to verify the cache primary key value also includes: In response to detecting a fast fail, it is determined that verification of the cache primary key value has failed.

4. The method according to claim 1, wherein After calling the whitelist to verify the cache primary key value, the method further includes: In response to the verification result being empty, repeated verification is prevented within a preset time.

5. The method according to claim 1, wherein Determining the cache primary key value includes: Get the prefix of the method signature; Combine the prefix with the input attribute to generate combined data; Based on the combined data, a cache primary key value is generated.

6. The method according to claim 5, characterized in that Generating a cache primary key value based on the combined data includes: The combined data is encrypted and compressed to generate a cache primary key value.

7. The method according to claim 1, characterized in that After acquiring the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information, the method further includes: An output form of the attribute value is determined based on the annotation configuration information, and the attribute value is output in the output form.

8. A data acquisition device, characterized in that: include: A receiving unit is configured to receive a data acquisition request and obtain corresponding input parameter attributes and method signatures; a cache primary key value determining unit, configured to determine a cache primary key value based on the input parameter attributes and the method signature; a verification unit, configured to call a whitelist to verify the cache primary key value; The interface calling unit is configured to, in response to successful verification, determine a target interface identifier based on the data acquisition request, and then call the corresponding target interface to obtain the attribute value corresponding to the input parameter attribute; when the fast failure mechanism detects that the data consistency requirement identifier in the current data acquisition request has changed, immediately respond with a fast failure, intercept the process of obtaining the requested data from the target interface, and instead execute the relevant logic of obtaining the requested data from the distributed cache; a cache calling unit configured to pull metadata from all metadata information requests involved in the cache processing logic in the cache resolution core process, determine whether the cache container has been initialized, and if so, pull metadata; if the metadata pulling is successful, return the pulled metadata to the cache resolution core process; if the cache container has not been initialized, initialize the cache container in singleton mode; After the cache container is successfully initialized, if the metadata pulling fails, the metadata containing annotations, interface return values, Json deserialization, and whitelist nested strings parsed for the first time by initializing the cache container based on the singleton mode, and the metadata containing annotations, interface return values, Json deserialization, and whitelist nested strings parsed are placed into the cache container that has been initialized based on the dimension of the interface method signature, thereby generating a metadata cache container; In response to a verification failure, the metadata cache container is called to obtain corresponding annotation configuration information from the metadata container based on the method signature, and then obtain the attribute value corresponding to the input parameter attribute in the distributed cache based on the annotation configuration information.

9. The device according to claim 8, characterized in that The verification unit is further configured to: Based on a fast failure mechanism, detecting a matching thread between the whitelist and the cache primary key value, and in response to not detecting a fast failure, parsing the whitelist to obtain a custom whitelist attribute value; The dynamically updated whitelist configuration information is obtained, and then the customized whitelist attribute value is verified.

10. A data acquisition electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

11. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images