Short link generation method and system

Abstract

The invention discloses a short link generation method and system, and relates to the technical field of distributed cluster processing, and the short link generation method comprises the steps: employing a random function to generate a first preset number of random numbers, carrying out the character mapping of each random number through a base64 function, and obtaining a first preset number of characters, the first preset number of characters form a short link; storing the generated short link in an array; when the array is full, stopping generating the short links, and writing the short links in the array into a short link file and a local cache; and receiving a long link sent by a first user side, allocating an unused short link to the long link, and sending the short link to the first user side. According to the method and the device, a large number of short links are generated in batches, so that a distributed ID generation service with relatively high complexity is prevented from being introduced, the complexity of a distributed system is reduced, and the requirement on system performance in a high-concurrency scene is met.

Description

Technical Field

[0001] The present invention relates to the field of distributed cluster processing technology, and can also be used in the financial field, and in particular to a short link generation method and system. Background Art

[0002] Web pages on the Internet have a one-to-one corresponding network address, namely URL (Uniform Resource Locator). Usually users need to copy a longer URL (i.e., long link) for sharing. Since long links reduce the information sharing experience and efficiency and increase the sharing and transmission costs, the long link is processed by an algorithm to obtain a newly generated short link, thus solving the problem of copying a longer link for sharing.

[0003] At present, one-way hashing algorithm, self-increasing natural number algorithm and distributed generation algorithm are usually used to generate short links, but the above short link generation algorithms have the following defects:

[0004] The disadvantage of using a one-way hash algorithm to generate short links is that there is a problem of hash collision. Since a string of a specific length is intercepted from a specific position, the short links calculated from two different long links are the same. Therefore, when generating short links, it is necessary to determine the hash collision. If there is a hash collision, the conflict needs to be processed through relevant calculations. Usually, conflict processing requires multiple calculations and judgments, resulting in excessive computation and affecting system performance.

[0005] Compared with the one-way hash algorithm, the self-increasing natural number algorithm does not have the problem of hash collision and the algorithm is simpler. However, since the generated short link is guessable, there is a security problem.

[0006] Compared with the first two algorithms, the distributed ID generation service can ensure that the generated short links have a certain degree of disorder and uniqueness to ensure security, and can also avoid the problem of hash conflicts. However, the implementation of distributed ID generation services usually requires the introduction and maintenance of a separate distributed service cluster, which not only relies on specific distributed systems and algorithms, but also needs to consider synchronization and consistency issues in a distributed environment, as well as the choice of technology stacks in distributed systems, thus increasing the complexity of distributed systems.

[0007] This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No admission is made that the description herein is prior art by inclusion in this section. Summary of the invention

[0008] The embodiment of the present invention provides a short link generation method for batch generating a large number of short links, avoiding the hash conflict problem and introducing a distributed ID generation service with higher complexity to reduce the complexity of the distributed system.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] In a first aspect, the present invention provides a short link generation method comprising:

[0011] Generate a first preset number of random numbers using a random function, and perform character mapping on each of the random numbers using a base64 function to obtain a first preset number of characters, wherein the first preset number of characters constitute a short link;

[0012] Store the generated short link in an array;

[0013] When the array is full, stop generating short links, and write the short links in the array into the short link file and the local cache;

[0014] receiving a long link sent by a first user terminal, allocating an unused short link to the long link and sending the short link to the first user terminal;

[0015] The mapping relationship between the long link and the short link corresponding to the long link is written into the database.

[0016] Furthermore, the short link generation method further includes:

[0017] Receiving a query request sent by a second user terminal;

[0018] The corresponding short link is searched according to the query request, and the long link corresponding to the found short link is sent to the second user terminal.

[0019] Furthermore, the random function is used to generate a first preset number of random numbers, and each of the random numbers is subjected to character mapping through a base64 function to obtain a first preset number of characters, and the first preset number of characters constitute a short link, including:

[0020] Step 1: Generate a random number using a random function;

[0021] Step 2: Map the random number to characters using a base64 function;

[0022] Repeat steps 1 and 2 for a preset number of times to generate a first preset number of characters to obtain a short link.

[0023] Furthermore, before storing the generated short link in the array, the short link generation method further includes:

[0024] Check the generated short links for duplication through filters;

[0025] If the duplication check fails, continue to generate the next short link;

[0026] If the duplication check passes, the generated short link is stored in the array.

[0027] Furthermore, writing the short links in the array into the short link file and the local cache includes:

[0028] Write the short links in the array into a short link file;

[0029] Load the short link in the short link file to the local cache.

[0030] Furthermore, the short link generation method further includes:

[0031] Divide into multiple sub-clusters according to preset rules;

[0032] Load the server serial number of the sub-cluster and the corresponding short link in the local cache to the cache server cluster;

[0033] If there is an abnormality in the sub-cluster, the sub-cluster is restarted, and the short link in the local cache of the sub-cluster is obtained from the cache server cluster.

[0034] Furthermore, the dividing into a plurality of subclusters according to a preset rule includes: dividing into odd-numbered subclusters and even-numbered subclusters according to the last digit of the server serial number.

[0035] Further, the loading of the short link in the short link file to the local cache includes:

[0036] Get the offset from the pre-generated offset record file with write permission;

[0037] Reading a second preset number of short links in the short link file starting from the offset through the read permission;

[0038] A second preset number of short links are loaded into the local cache.

[0039] Further, loading a second preset number of short links into the local cache includes:

[0040] Creating a ring array of preset capacity in the local cache;

[0041] Writing a second preset number of short links into the annular array;

[0042] When the number of remaining unused short links in the ring array is lower than a preset threshold, a second preset number of short links are reloaded and written into the ring array.

[0043] Further, searching for a corresponding short link according to the query request and sending a long link corresponding to the found short link to the second user terminal includes:

[0044] First, according to the query request, find out whether there is a corresponding short link in the local cache;

[0045] If the corresponding short link exists in the local cache, sending the long link corresponding to the found short link from the local cache to the second user terminal;

[0046] If the short link does not exist in the local cache, the database is searched, and the long link corresponding to the found short link is sent from the database to the second user terminal.

[0047] Further, sending the long link corresponding to the found short link to the second user terminal includes:

[0048] Constructing a redirection response between the short link in the query request and the long link corresponding to the found short link;

[0049] The redirection response is sent to the second user terminal, so that the second user terminal obtains the long link corresponding to the found short link.

[0050] Furthermore, the short link generation method further includes:

[0051] Querying the creation time of the mapping relationship in the database, and comparing the creation time with a preset storage duration;

[0052] If the preset storage time is exceeded, the short link in the mapping relationship is loaded into the short link file, and the mapping relationship in the database is deleted.

[0053] In a second aspect, an embodiment of the present invention further provides an application server cluster for batch generating short links and a short link query service applicable to a scenario financial platform, so as to realize the high performance requirements of the system and improve the query speed of short links. The application server cluster includes:

[0054] A short link generation module, used for generating a first preset number of random numbers by using a random function, and performing character mapping on each of the random numbers by using a base64 function to obtain a first preset number of characters, wherein the first preset number of characters constitute a short link;

[0055] A data storage module, used to store the generated short link into an array;

[0056] A first writing module, used for stopping generating short links when the array is full, and writing the short links in the array into a short link file and a local cache;

[0057] A first data processing module, configured to receive a long link sent by a first user terminal, allocate an unused short link to the long link, and send the short link to the first user terminal;

[0058] The second writing module is used to write the mapping relationship between the long link and the short link corresponding to the long link into a database.

[0059] In a third aspect, an embodiment of the present invention further provides a short link generation system for cloud-based deployment of the short link generation system, realizing multi-instance deployment across availability zones, and realizing high availability at the data level through cloud-based products, thereby reducing the complexity of the distributed system. The short link generation system includes a load balancer, a Web server cluster, an application server cluster, a cache server cluster, and a Hadoop cluster;

[0060] The load balancer is used to receive a long link sent by a first user terminal and receive a query request sent by a second user terminal;

[0061] The Web server cluster is used to send the received long link and the query request to the application server cluster;

[0062] The Hadoop cluster is used to store databases, short link files and offset record files;

[0063] The application server cluster includes:

[0064] A short link generation module, used for generating a first preset number of random numbers by using a random function, and performing character mapping on each of the random numbers by using a base64 function to obtain a first preset number of characters, wherein the first preset number of characters constitute a short link;

[0065] A data storage module, used to store the generated short link into an array;

[0066] A first writing module, used for stopping generating short links when the array is full, and writing the short links in the array into a short link file and a local cache;

[0067] A first data processing module, configured to receive a long link sent by a first user terminal, allocate an unused short link to the long link, and send the short link to the first user terminal;

[0068] The second writing module is used to write the mapping relationship between the long link and the short link corresponding to the long link into a database.

[0069] An embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned short link generation method when executing the computer program.

[0070] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the short link generation method described above is implemented.

[0071] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the short link generation method is implemented.

[0072] A short link generation method and system provided by an embodiment of the present invention avoids the problems of repeatability and guessability of generated short links by batch generating a large number of short links, thereby improving generation efficiency and enhancing the security of generating short links. By preloading a certain number of short links, the response speed is improved, the response time is reduced, and the requirements for system performance in high-concurrency scenarios are met. By first querying the short links in the local cache and then querying the short links in the database, a set of short link query services suitable for scenario financial platforms is provided to achieve the high performance requirements of the system. By cloud-deploying the short link generation system, multi-instance deployment across availability zones is achieved, and high availability at the data level is achieved through cloud products, thereby avoiding the introduction of a distributed ID generation service with higher complexity and reducing the complexity of the distributed system. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0074] Figure 1 This is a topological diagram of the cloud deployment of the short link generator in an embodiment of the present invention;

[0075] Figure 2 Schematic diagram of a process for generating a short link in one embodiment of the present invention;

[0076] Figure 3 Schematic diagram of a flow chart of a method for generating a short link in another embodiment of the present invention;

[0077] Figure 4 Schematic diagram of a flow chart of a method for generating a short link in another embodiment of the present invention;

[0078] Figure 5 Schematic diagram of a flow chart of a method for generating a short link in another embodiment of the present invention;

[0079] Figure 6 Schematic diagram of a flow chart of a method for generating a short link in another embodiment of the present invention;

[0080] Figure 7 Schematic diagram of a flow chart of a method for generating a short link in another embodiment of the present invention;

[0081] Figure 8 A schematic diagram of the structure of an application server cluster in an embodiment of the present invention;

[0082] Fig. 9 This is a schematic diagram of the structure of a short link generation system in one embodiment of the present invention;

[0083] Fig.10 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0084] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0085] The information collected in the technical solution of the present invention is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0086] The acquisition, transmission, storage, use, and processing of data in the technical solution of the present invention are in compliance with the relevant provisions of national laws and regulations.

[0087] It should be noted that in the embodiments of the present invention, certain software, components, models and other existing solutions in the industry may be mentioned, which should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present invention, but it does not mean that the applicant has or will necessarily use the solution.

[0088] In order to improve the stability of the system, reduce the complexity of the distributed system, and avoid introducing a more complex distributed ID generation service, the present invention provides a short link generator solution suitable for a scenario financial platform.

[0089] Figure 1 This is a topological diagram of the cloud deployment of the short link generator in an embodiment of the present invention. Figure 1 As shown, a short link generator includes a load balancer 100 , a Web server cluster 110 , an application server cluster 120 , a cache server cluster 130 , and a Hadoop cluster 140 .

[0090] The short link generator is deployed in the cloud using infrastructure such as computing, storage, network, database and big data cluster, and is divided into three areas: Internet area 11, open area 12 and big data cloud 13.

[0091] The load balancer 100 is used to receive a long link sent by a first client and receive a query request sent by a second client.

[0092] The web server cluster 110 is set in the Internet zone 11 , and the web server cluster 110 is used to send the received long links and query requests to the application server cluster 120 .

[0093] The application server cluster 120 includes a first application server cluster 1201 , a second application server cluster 1202 , and an application server 1203 pre-selected in the application server cluster 120 .

[0094] The first application server cluster 1201 is used to provide online transactions in response to user requests, including a first user terminal providing a long link to request generation of a corresponding short link and a second user terminal providing a query request to request querying a mapping long link of the short link.

[0095] The second application server cluster 1202 is used to generate all the estimated tens of billions of short links at one time through an algorithm, and store them in the short link file.

[0096] The application server 1203 is used to delete the short link mapping data that has expired, and append these short links to the file system.

[0097] The cache server cluster 130 is used to receive and store the short link in the local cache of the first application server cluster 1201 .

[0098] The Hadoop cluster 140 is set in the big data cloud 13, and the Hadoop cluster 140 is used to store short link files, databases, and offset record files.

[0099] Figure 2 FIG. 1 is a flow chart of a method for generating a short link in an embodiment of the present invention. The execution subject of the method for generating a short link is the application server cluster 120. Figure 2 As shown, the short link generation method includes steps 201 to 205.

[0100] Step 201: Generate a first preset number of random numbers using a random function, and perform character mapping on each random number using a base64 function to obtain a first preset number of characters, where the first preset number of characters constitute a short link.

[0101] Step 202: Store the generated short link into an array.

[0102] Step 203 : when the array is full, stop generating short links, and write the short links in the array into the short link file and the local cache of the application server cluster 120 .

[0103] Step 204: Receive the long link sent by the first user terminal, allocate an unused short link to the long link and return the short link to the first user terminal.

[0104] Step 205: Write the mapping relationship between the long link and the short link corresponding to the long link into the database.

[0105] from Figure 2 It can be seen from the process shown that in the embodiment of the present invention, by generating short links in batches, the efficiency of short link generation is improved, and the conflicts, low efficiency and low security caused by generating short links one by one online are avoided. By preloading a part of the short links into the local cache, when the user requests to generate a short link, the request can be responded to more quickly, reducing the number of times the database is accessed. The security of multiple servers concurrently reading short links is guaranteed by read and write permissions, and the external services of the application server cluster 120 ensure the high availability of the short link generation system 900.

[0106] like Figure 3 As shown, each step is explained in detail below.

[0107] Step 201: Generate a first preset number of random numbers using a random function, and perform character mapping on each random number using a base64 function to obtain a first preset number of characters, where the first preset number of characters constitute a short link.

[0108] In one embodiment, if Figure 3 As shown, step 201 includes step 301 and step 302 .

[0109] Step 301: Generate a random number by using a random function.

[0110] Step 302: Perform character mapping using the base64 function according to the generated random number to obtain the corresponding character.

[0111] Step 303: Determine whether the current number is less than the preset number.

[0112] Step 304: If the current number is less than the preset number, return to step 301 to continue generating the next random number.

[0113] Step 305: If the current number is greater than the preset number, a preset number of characters have been generated and a short link is obtained.

[0114] Specifically, a random function is first used to randomly return a number from 0 to 61 numbers, and then the number is character mapped through the base64 function to obtain the character corresponding to the number. Steps 301 to 304 are repeated a preset number of times to generate a 6-bit string, which is a short link.

[0115] Among them, the preset number of times is 6 times and the preset number is 6. Those skilled in the art can also set it according to actual needs, and the present invention is not limited thereto. Base64 uses 64 character codes, and only 62 characters are used in the present invention, including 0-9, az, AZ. Since the two symbols "+" and "-" will be encoded as "%2B" and "%2F" in the URL, and "%" is a reserved character of an SQL statement, it will cause a conflict when writing to the database, so the encoding of the two symbols "+" and "-" is removed.

[0116] Step 202: Store the generated short link into an array.

[0117] Specifically, after a short link is randomly generated according to step 201, the short link is stored in a pre-created array.

[0118] like Figure 3 As shown, step 202 includes steps 306 to 309 .

[0119] Step 306: Perform duplication check on the generated short link through a filter.

[0120] Step 307: Determine whether the repeatability check is passed.

[0121] Step 308: If the duplication check fails, continue to generate the next short link.

[0122] Step 309: If the repeatability check passes, the generated short link is stored in an array.

[0123] Specifically, the filter is used to determine whether the newly generated short link conflicts with the short link stored in the array. If the repeatability check fails, a short link is regenerated through the above steps. If the repeatability check passes, the generated short link is stored in the array. Among them, the present invention uses the Bloom filter of Hbase, and other filters can also be used. Since the Bloom filter can quickly determine whether an element exists in a set and supports dynamic addition of elements. Therefore, compared with other data structures, the Bloom filter has great advantages in storage and query efficiency, and has the characteristics of high efficiency and low storage space.

[0124] Exemplarily, in addition to Bloom filters, the present invention uses security functions and standardized security solutions provided by the bank's cloud infrastructure. Among them, cloud security products include website application-level intrusion prevention systems for filtering CC attacks, providing vulnerability patches, and preventing tampering and information leakage. Security group access control lists (ACLs) are used to provide network control at the subnet and server granularity. Dragon Guard is used to provide security functions such as password cracking interception, abnormal login reminders, Trojan horse detection and killing, and vulnerability scanning monitoring. Dragon Fortress is used to manage, control and audit operation and maintenance personnel and application equipment.

[0125] Step 310: Determine whether the array is full.

[0126] Step 311: If the array is not full, continue to generate the next short link.

[0127] Specifically, it is determined whether the array is full. If the array is not full, the above steps are repeated to continue generating the next short link.

[0128] Step 203: When the array is full, stop generating short links, and write the short links in the array into the short link file and the local cache.

[0129] Writing the short links in the array into the short link file and the local cache includes steps 312 to 313.

[0130] Step 312: Write the short links in the array into the short link file.

[0131] Specifically, if the array is full, the short links stored in the array are sequentially written into the short link file, and the array is cleared. The short link file is an HDFS file in the Hadoop cluster 140, which stores the generated short links persistently.

[0132] Step 313: Load the short link in the short link file to the local cache.

[0133] Step 314: Determine the cumulative number of short links that have been generated.

[0134] Step 315: If the number of generated short links does not exceed the rated number, continue to generate the next short link.

[0135] Step 316: If the number of generated short links exceeds the rated number, stop generating short links.

[0136] Specifically, the rated number in the present invention is 10.4 billion short links, each of which occupies 6 characters (ie, bytes). Since these strings are stored in the short link file without segmentation, the storage space occupied is approximately 10.4 billion*6B=62.4GB.

[0137] For example, the string can be stored without segmentation, without considering the use of separators, or stacking, that is, a certain number of characters in the previous string and the next string are overlapped and shared. Because the separator will cause an additional 15% storage overhead, that is, 1 character is required to separate every 6 characters. Although stacking can reduce the size of the initial short link file, for example, stacking 3 characters reduces the storage space by 50%, it is easy to have an infinite loop during the repetitive check, that is, all the short links determined by the prefix characters already exist, making it difficult to recycle expired short links in the future.

[0138] In the embodiment of the present invention, by generating short links in batches, conflicts and low efficiency caused by real-time online generation of short links are avoided, security is improved, and repeatability checks are performed through Bloom filters to avoid conflicts in the generation of short links and improve query efficiency.

[0139] like Figure 4 As shown, in one embodiment, step 313 includes steps 41 to 48 .

[0140] Obtaining the offset from the pre-generated offset record file through write permission includes steps 41 and 42.

[0141] Step 41: Open the pre-generated offset record file with write permission.

[0142] Specifically, when requesting to preload a short link, an exclusive file lock must first be obtained, that is, when one of the servers in the application server cluster 120 successfully opens the offset record file with write permission, the other servers will be blocked, waiting to obtain the exclusive file lock (i.e., write file lock), so step 41 is a mutually exclusive operation. The pre-generated offset record file is stored in the HDFS file system of the Hadoop cluster 140. In addition to storing the short link file, the HDFS file system also generates a file in advance that records the read offset, i.e., the offset record file.

[0143] Step 42: Read the offset recorded in the offset record file.

[0144] Reading a second preset number of short links in the short link file starting from the offset by using the read permission includes steps 43 to 48 .

[0145] Step 43: Open the short link file with read permission.

[0146] Step 44: Read a fixed number of strings starting from the offset.

[0147] Among them, the second preset number = a fixed number of strings / the length of a short link. Since the length of a short link is composed of a preset number of characters, the length of a short link in the present invention is 6. The fixed number of strings can be set according to the needs of those skilled in the art, and this embodiment does not make any specific restrictions.

[0148] Step 45: Close the short link file.

[0149] Step 46: Update the offset record file.

[0150] Specifically, after one of the servers in the application server cluster 120 reads a fixed number of character strings starting from an offset, the offset after reading is recorded, and the offset is recorded in an offset record file to update the offset record file.

[0151] Step 47: Close the offset record file.

[0152] Step 48: Load a second preset number of short links into the local cache.

[0153] Specifically, when the next server in the application server cluster 120 needs to load the short link to the local cache, it only needs to read a fixed number of strings starting from the updated offset. By repeating the above steps, multiple servers in the application server cluster 120 sequentially preload the second preset number of short links in the short link file, so that when a user requests to generate a short link, the request can be responded to more quickly.

[0154] In an embodiment of the present invention, due to the huge number of short links generated, if the short links are only stored in the database, the database will be frequently accessed when the short links are subsequently queried, thereby affecting the database performance. Therefore, in an embodiment of the present invention, the application server cluster 120 preloads a certain number of short links into the local cache, so that when a user requests to generate a short link, the request can be responded to more quickly, reducing the number of times the database is accessed. In order to avoid the situation where multiple servers load the same short link at the same time, the mutual exclusion of access by multiple servers is achieved by adopting the mutual exclusion of the write file lock in the HDFS file system, and the range of reading short links is determined by the offset record file, thereby ensuring that the short links read by multiple servers in the application server cluster 120 are not repeated.

[0155] like Figure 5 As shown, in one embodiment, step 48 includes steps 51 to 53 .

[0156] Step 51: Create a circular array of preset capacity in the local cache.

[0157] Specifically, after a fixed number of strings are read from the short link file, a circular array of a corresponding size is created according to the number of strings.

[0158] Step 52: Write a second preset number of short links into the ring array.

[0159] Specifically, the second preset number of short links is written into the circular array, that is, the second preset number of short links is loaded into the local cache of the server. By repeating steps 51 to 52, the application server cluster 120 preloads a certain number of short links into the local cache.

[0160] Step 53: When the number of unused short links remaining in the ring array is lower than a preset threshold, a second preset number of short links is reloaded and written into the ring array.

[0161] Specifically, the local cache update of the server in the application server cluster 120 adopts a pre-update mechanism, and when the number of unused short links remaining in the ring array is less than 20% of the capacity of the ring array, the short link update operation is performed. The short links in the ring array are updated by preloading the short links to the local cache in the above steps 41 to 48.

[0162] In the embodiment of the present invention, the application server cluster 120 caches short links in a ring buffer. Compared with the traditional linked list storage method, the ring buffer is more suitable for local caching of short links. First, since the number of short links in the local cache is fixed, the ring buffer can recycle the memory by pre-allocating a fixed size of memory space, reducing the memory application and garbage collection of the linked list storage method, and running more stably. Secondly, since each node of the linked list needs to store pointers in addition to storing short links. The pointer of the linked list data structure also occupies a considerable amount of memory compared to the short link itself. Therefore, in order to save memory, the linked list data structure is not used. By adopting the ring array structure, the space utilization rate is high, and there is no need to store pointers, which effectively improves the memory usage efficiency, improves the memory utilization rate, and reduces the memory occupancy. In addition, the ring array makes full use of the locality principle and CPU cache, and has better performance. Finally, the short links in the ring array are updated by the pre-update mechanism, thereby ensuring the stability of the service.

[0163] Step 204: Receive the long link sent by the first user terminal, allocate an unused short link to the long link and send the short link to the first user terminal.

[0164] Specifically, after the application server cluster 120 preloads a certain number of short links into the local cache, it receives the long link sent by the first user terminal, allocates a short link that is not used in the local cache to the long link, and sends the short link to the first user terminal. The first user terminal is a scene financial platform user, also known as a platform user.

[0165] Step 205: Write the mapping relationship between the long link and the short link corresponding to the long link into the database.

[0166] Specifically, the mapping relationship between the long link and the short link assigned to the long link is stored in the HBase database in the Hadoop cluster 140. When each mapping from a long link to a short link is generated, the mapping relationship is written into the HBase database.

[0167] like Figure 6 As shown, in one embodiment, the short link generation method further includes steps 61 to 63:

[0168] Step 61: Divide into multiple sub-clusters according to preset rules.

[0169] Step 61 includes dividing the server serial numbers into odd subclusters and even subclusters according to the last digits of the server serial numbers.

[0170] Specifically, according to the parity of the last digit of the server serial number, the servers are divided into two subclusters, namely, an odd subclusters and an even subclusters. The odd subclusters correspond to servers with the last digits 1 / 3 / 5 / 7 / 9, and the even subclusters correspond to servers with the last digits 2 / 4 / 6 / 8.

[0171] Step 62 : Load the server serial number of the sub-cluster and the corresponding short link in the local cache to the cache server cluster 130 .

[0172] Specifically, the odd-numbered sub-cluster loads the server serial number and the short link in the local cache of the server corresponding to the server serial number into the cache server cluster 130 for persistent storage.

[0173] Step 63: If there is an abnormality in the sub-cluster, restart the sub-cluster and obtain the short link in the local cache of the sub-cluster from the cache server cluster 130 .

[0174] Specifically, if the odd sub-cluster has a shutdown exception, after the odd sub-cluster is restarted, the cache server cluster 130 obtains the short link in the local cache of the corresponding sub-cluster, thereby realizing the short link recovery in the local cache of the odd sub-cluster. The even sub-cluster repeats steps 62 and 63 to realize the local cache recovery of the even sub-cluster. During the local cache recovery process of the sub-cluster, the cache recovery of the sub-cluster is monitored in real time until the external service of the application server cluster 120 is restored to normal.

[0175] In the embodiment of the present invention, since online transactions are directly directed to Internet users outside the scenario finance platform, in the case of an abnormal restart of the application server cluster 120, the short link in the local cache of the application server cluster 120 is restored to ensure that the external services of the application server cluster 120 are normal after the restart. In addition to abnormal restart situations, when the application server cluster 120 needs to be restarted, such as when the application software of the scenario finance platform is changed or an emergency drill is conducted, the cache is restored through the above steps to ensure that the external services are consistent after the restart.

[0176] like Figure 7 As shown, in one embodiment, the short link generation method further includes:

[0177] Step 71: Receive a query request sent by a second user terminal.

[0178] Specifically, when an Internet user obtains a mapped short link, the Internet user clicks on the short link through the browser client to view the content information in the short link. At this time, the browser generates a query request and sends it to the short link generator. Therefore, the short link generator needs to receive the query request and return the original long link corresponding to the query request to the Internet user. Among them, the second user terminal is the browser client of the Internet user.

[0179] Searching for the corresponding short link according to the query request, and sending the long link corresponding to the found short link to the second user terminal, includes steps 72 to 74.

[0180] Step 72: First search whether there is a corresponding short link in the local cache according to the query request.

[0181] Specifically, according to the short link in the query request, first search whether the short link exists in the local cache, wherein the query request includes the short link and other data information.

[0182] Step 73: If the corresponding short link exists in the local cache, the long link corresponding to the found short link is sent from the local cache to the second user terminal.

[0183] Step 74: If the short link does not exist in the local cache, search the database, and send the long link corresponding to the found short link from the database to the second user terminal.

[0184] Specifically, the database is an Hbase database, the application server cluster 120 caches some short links and the mapping relationship between short links and long links, and the Hbase database stores all the mapping relationships between short links and long links for querying the long link address corresponding to the short link.

[0185] In an embodiment of the present invention, the long link corresponding to the query request is obtained by querying the local cache of the server. Since the most recently generated short link and the mapping relationship between the short link and the long link are cached, the storage efficiency of the short link is improved, the pressure on the application server cluster 120 is reduced, the response speed is improved, and the frequency of database access is reduced.

[0186] In one embodiment, sending the long link corresponding to the found short link to the second user terminal in step 73 and step 74 includes:

[0187] Construct a redirect response between the short link in the query request and the long link corresponding to the found short link.

[0188] Specifically, after obtaining the long link address, the HTTP redirection mechanism is used to redirect the short link in the query request obtained from the second user end to the long link found. The HTTP response status code corresponding to the short link in the query request is set to 302, and then the long link corresponding to the found short link is set to the address field of the response header.

[0189] Exemplarily, there are two types of HTTP redirection response codes: 301 and 302. Among them, 301 indicates permanent redirection, that is, once the browser has visited the short link, the redirected long link will be cached locally, and subsequent visits will directly access the long link based on the local cache. 302 indicates temporary redirection, that is, each time the browser visits, it needs to first request the short link generator, obtain the long link returned by the short link generator, and then access the long link. Since the present invention needs to count the number of clicks of Internet users outside the scene financial platform, the present invention uses 302 as the redirection response code.

[0190] A redirection response is sent to the second user terminal so that the second user terminal obtains the long link corresponding to the found short link.

[0191] Specifically, the query request is redirected to the long link address corresponding to the found short link by constructing a redirect response, so as to send the redirect response to the browser client of the Internet user. When the user accesses the short link, the HTTP redirect response will redirect the query request to the corresponding original long link.

[0192] In an embodiment of the present invention, by constructing a redirect response, the original long link is redirected to the short link in the query request, thereby ensuring that the user and the browser client can successfully access the web page content corresponding to the original long link, while avoiding 404 errors caused by structural adjustments, simplifying user operations, and thus improving user experience.

[0193] In one embodiment, the short link generation method includes:

[0194] Query the creation time of the mapping relationship in the database and compare the creation time with the preset storage duration.

[0195] Specifically, the creation time of the mapping relationship between the long link and the short link in the HBase database is queried, and the creation time is compared with the preset storage duration to filter out expired mapping records.

[0196] If the preset storage time is exceeded, the short link in the mapping relationship is loaded into the short link file, and the mapping relationship in the database is deleted.

[0197] Specifically, if the preset storage time is exceeded, the short link file is opened with write permission, and the short link in the expired mapping relationship is loaded into the short link file. The short link file is closed, and the expired mapping relationship in the HBase database is deleted. The preset storage time is 13 months, which can also be set according to the needs of technicians in this field.

[0198] In the embodiment of the present invention, the short link data is recycled on a regular basis every day, thereby realizing the recycling of short links. The recycling task is generally performed during the low-peak business period in the early morning to reduce the impact on system performance.

[0199] Exemplarily, the following estimates are made for the performance indicators of the short link generation system 900.

[0200] Since the scene finance platform currently has tens of billions of registered users, tens of millions of monthly active users, and millions of life content, it is estimated that 800 million new short links are generated every month through applications, WeChat applets, and off-site advertisements of the scene finance platform. And since the validity period of the short link is 13 months, the number of short links generated is 800 million * 13 = 10.4 billion. If each short link record occupies 1KB of storage capacity, without considering backup, the total storage space is 10.4 billion * 1KB = 10.4TB. Therefore, the storage capacity of the short link generation system 900 is at least 11TB.

[0201] There are two types of transactions in the short link generation system 900. One transaction is to generate a short link corresponding to a long link, and the other transaction is to query a long link corresponding to a short link. Considering that the transaction frequency of generating a short link is much lower than the transaction frequency of querying a short link, the overall transaction volume of the short link generation system 900 is estimated and analyzed by querying the transaction volume.

[0202] For example, if 90% of the query transactions are concentrated on short links generated in the last month, and each short link is queried at least 5 times on average, the average transaction volume is about 100,000, that is, TPM = (800 million * 5) / 0.9 / (30 * 24 * 60). If the transaction proportion of the peak 6 hours during the day is estimated to be 80%, the peak transaction volume is about 300,000, that is, TPM = (800 million * 5 * 0.8) / 0.9 / (30 * 6 * 60).

[0203] Therefore, the above estimation is used to predict the network bandwidth requirement of the short link generation system 900. If the peak transaction is 300,000 and each response is 1KB, the peak network bandwidth is about 43Mb, that is, 300,000 / 60*1KB*8bit=43Mb.

[0204] Figure 8 FIG. 1 is a schematic diagram of the structure of the application server cluster 120 in an embodiment of the present invention. Figure 8 As shown, the application server cluster 120 includes a short link generation module 801 , a data storage module 802 , a first writing module 803 , a first data processing module 804 , and a second writing module 805 .

[0205] The short link generation module 801 is used to generate a first preset number of random numbers using a random function, and perform character mapping on each random number through a base64 function to obtain a first preset number of characters, and the first preset number of characters constitute a short link.

[0206] The data storage module 802 is used to store the generated short link in an array.

[0207] The first writing module 803 is used to stop generating short links when the array is full, and write the short links in the array into the short link file and the local cache.

[0208] The first data processing module 804 is used to receive the long link sent by the first user terminal, allocate an unused short link to the long link, and send the short link to the first user terminal.

[0209] The second writing module 805 is used to write the mapping relationship between the long link and the short link corresponding to the long link into the database.

[0210] In one embodiment, the application server cluster 120 further includes a data receiving module and a second data processing module.

[0211] The data receiving module is used to receive a query request sent by the second user terminal.

[0212] The second data processing module is used to search for the corresponding short link according to the query request, and send the long link corresponding to the found short link to the second user terminal.

[0213] In one embodiment, the short link generation module 801 includes a random number generation unit, a character mapping unit and a short link generation unit.

[0214] The random number generating unit is used to generate a random number by using a random function.

[0215] The character mapping unit is used to obtain characters by performing character mapping on random numbers through a base64 function.

[0216] The short link generating unit is used to generate a first preset number of characters to obtain a short link.

[0217] Specifically, after the random number generation unit and the character mapping unit execute a preset number of times, the short link generation unit generates a first preset number of characters to obtain a short link.

[0218] In one embodiment, the application server cluster 120 further includes a data verification module.

[0219] The data validation module is used to check the duplication of the generated short links through filters.

[0220] If the repeatability check fails, the short link generation module 801 is used to continue to generate the next short link. If the repeatability check passes, the data storage module 802 is used to store the generated short link in an array.

[0221] In one embodiment, the first writing module 803 includes a data writing unit and a cache loading unit.

[0222] The array writing unit is used to write the short links in the array into the short link file.

[0223] The cache loading unit is used to load the short links in the short link file into the local cache.

[0224] In one embodiment, the application server cluster 120 further includes a cluster partitioning module, a cluster loading module and a cluster processing module.

[0225] The cluster division module is used to divide into multiple sub-clusters according to preset rules.

[0226] The cluster loading module is used to load the server serial number of the sub-cluster and the corresponding short link in the local cache to the cache server cluster 130 .

[0227] The cluster processing module is used to restart the sub-cluster if an abnormality occurs in the sub-cluster, and obtain the short link in the local cache of the sub-cluster from the cache server cluster 130 .

[0228] In one embodiment, the cluster division module is specifically configured to divide the servers into odd sub-clusters and even sub-clusters according to the last digits of the server serial numbers.

[0229] In one embodiment, the cache loading unit includes an offset acquisition subunit, a reading subunit, and a short link loading subunit.

[0230] The offset acquisition subunit is used to obtain the offset from the pre-generated offset record file through write permission.

[0231] The reading subunit is used to read a second preset number of short links in the short link file starting from the offset through the read permission.

[0232] The short link loading subunit is specifically configured to load a second preset number of short links into the local cache.

[0233] In one embodiment, the short link loading subunit includes a ring array generating subunit, a ring array writing subunit and a ring array updating subunit.

[0234] The ring array generation subunit is used to create a ring array of preset capacity in the local cache.

[0235] The ring array writing subunit is used to write a second preset number of short links into the ring array.

[0236] The ring array updating subunit is used for reloading a second preset number of short links into the ring array when the number of remaining unused short links in the ring array is lower than a preset threshold.

[0237] In one embodiment, the second data processing module includes a query unit, a first feedback unit and a second feedback unit.

[0238] The query unit is used to first search whether a corresponding short link exists in the local cache according to the query request.

[0239] The first feedback unit is configured to send the long link corresponding to the found short link from the local cache to the second user terminal if the corresponding short link exists in the local cache.

[0240] The second feedback unit is used to search the database if the short link does not exist in the local cache, and send the long link corresponding to the found short link from the database to the second user terminal.

[0241] In one embodiment, the first feedback unit and the second feedback unit include a redirection subunit and a feedback subunit.

[0242] The redirection subunit is used to construct a redirection response between the short link in the query request and the long link corresponding to the found short link.

[0243] The feedback subunit is used to send a redirection response to the second user terminal, so that the second user terminal obtains the long link corresponding to the found short link.

[0244] In one embodiment, the application server cluster 120 further includes an expired link query module and an expired link deletion module.

[0245] The expired link query module is used to query the creation time of the mapping relationship in the database and compare the creation time with the preset storage duration.

[0246] The expired link deletion module is used to load the short link in the mapping relationship into the short link file and delete the mapping relationship in the database if the preset storage time is exceeded.

[0247] The present invention also provides a short link generation system 900, as described in the following embodiments. Since the principle of the short link generation system 900 to solve the problem is similar to that of the short link generation method, the implementation of the short link generation system 900 can refer to the implementation of the short link generation method, and the repeated parts will not be repeated.

[0248] like Fig. 9 As shown, the short link generation system 900 includes a load balancer 100 , a Web server cluster 110 , an application server cluster 120 , a cache server cluster 130 , and a Hadoop cluster 140 .

[0249] The load balancer 100 is used to receive a long link sent by a first client and receive a query request sent by a second client.

[0250] The web server cluster 110 is used to send the received long link and query request to the application server cluster 120 .

[0251] The Hadoop cluster 140 is used to store databases, short link files, and offset record files.

[0252] The application server cluster 120 includes a short link generation module 801 , a data storage module 802 , a first writing module 803 , a first data processing module 804 , and a second writing module 805 .

[0253] The short link generation module 801 is used to generate a first preset number of random numbers using a random function, and perform character mapping on each random number through a base64 function to obtain a first preset number of characters, and the first preset number of characters constitute a short link.

[0254] The data storage module 802 is used to store the generated short link in an array.

[0255] The first writing module 803 is used to stop generating short links when the array is full, and write the short links in the array into the short link file and the local cache.

[0256] The first data processing module 804 is used to receive a long link sent by a first user terminal, allocate an unused short link to the long link, and send the short link to the first user terminal.

[0257] The second writing module 805 is used to write the mapping relationship between the long link and the short link corresponding to the long link into the database.

[0258] It should be noted that the short link generation method provided in the embodiment of the present invention can be used in the financial field, and can also be used in any technical field other than the financial field. The embodiment of the present invention does not limit the application field of the short link generation method.

[0259] Fig.10 A schematic diagram of the physical structure of an electronic device 1000 provided in an embodiment of the present invention is shown in FIG. Fig.10 As shown, the electronic device 1000 includes a processor 1001 , a memory 1002 and a bus 1003 .

[0260] The processor 1001 and the memory 1002 communicate with each other via the bus 1003 .

[0261] The processor 1001 is used to call the program instructions in the memory 1002 to execute the short link generation method provided by the above method embodiments.

[0262] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the short link generation method is implemented.

[0263] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the short link generation method is implemented.

[0264] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0265] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0266] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0267] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0268] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A short link generation method, characterized in that: include: Generate a first preset number of random numbers using a random function, and perform character mapping on each of the random numbers using a base64 function to obtain a first preset number of characters, wherein the first preset number of characters constitute a short link; Store the generated short link in an array; When the array is full, stop generating short links, and write the short links in the array into the short link file and the local cache; receiving a long link sent by a first user terminal, allocating an unused short link to the long link and sending the short link to the first user terminal; The mapping relationship between the long link and the short link corresponding to the long link is written into the database.

2. The short link generation method according to claim 1, characterized in that: The short link generation method further includes: Receiving a query request sent by a second user terminal; The corresponding short link is searched according to the query request, and the long link corresponding to the found short link is sent to the second user terminal.

3. The short link generation method according to claim 1, characterized in that: The method of using a random function to generate a first preset number of random numbers, and performing character mapping on each of the random numbers through a base64 function to obtain a first preset number of characters, wherein the first preset number of characters constitute a short link, includes: Step 1: Generate a random number using a random function; Step 2: Map the random number to characters using a base64 function; Repeat steps 1 and 2 for a preset number of times to generate a first preset number of characters to obtain a short link.

4. The short link generation method according to claim 3, characterized in that: Before storing the generated short link in the array, the short link generation method further includes: Check the generated short links for duplication through filters; If the duplication check fails, continue to generate the next short link; If the duplication check passes, the generated short link is stored in the array.

5. The short link generation method according to claim 1, characterized in that: Writing the short links in the array into the short link file and the local cache comprises: Write the short links in the array into a short link file; Load the short link in the short link file to the local cache.

6. The short link generation method according to claim 5, characterized in that: The short link generation method further includes: Divide into multiple sub-clusters according to preset rules; Load the server serial number of the sub-cluster and the corresponding short link in the local cache to the cache server cluster; If there is an abnormality in the sub-cluster, the sub-cluster is restarted, and the short link in the local cache of the sub-cluster is obtained from the cache server cluster.

7. The short link generation method according to claim 6, characterized in that: The dividing into a plurality of sub-clusters according to a preset rule includes: dividing into odd-numbered sub-clusters and even-numbered sub-clusters according to the last digit of the server serial number.

8. The short link generation method according to claim 5, characterized in that: The loading of the short link in the short link file to the local cache comprises: Get the offset from the pre-generated offset record file with write permission; Reading a second preset number of short links in the short link file starting from the offset through the read permission; A second preset number of short links are loaded into the local cache.

9. The short link generation method according to claim 8, characterized in that: The loading of a second preset number of short links into the local cache comprises: Creating a ring array of preset capacity in the local cache; Writing a second preset number of short links into the annular array; When the number of remaining unused short links in the ring array is lower than a preset threshold, a second preset number of short links are reloaded and written into the ring array.

10. The short link generation method according to claim 2, characterized in that: The searching for the corresponding short link according to the query request and sending the long link corresponding to the found short link to the second user terminal includes: First, according to the query request, find out whether there is a corresponding short link in the local cache; If the corresponding short link exists in the local cache, sending the long link corresponding to the found short link from the local cache to the second user terminal; If the short link does not exist in the local cache, the database is searched, and the long link corresponding to the found short link is sent from the database to the second user terminal.

11. The short link generation method according to claim 10, characterized in that: Sending the long link corresponding to the found short link to the second user terminal includes: Constructing a redirection response between the short link in the query request and the long link corresponding to the found short link; The redirection response is sent to the second user terminal, so that the second user terminal obtains the long link corresponding to the found short link.

12. The short link generation method according to claim 1, characterized in that: The short link generation method further includes: Querying the creation time of the mapping relationship in the database, and comparing the creation time with a preset storage duration; If the preset storage time is exceeded, the short link in the mapping relationship is loaded into the short link file, and the mapping relationship in the database is deleted.

13. An application server cluster, characterized in that: include: A short link generation module, used for generating a first preset number of random numbers by using a random function, and performing character mapping on each of the random numbers by using a base64 function to obtain a first preset number of characters, wherein the first preset number of characters constitute a short link; A data storage module, used to store the generated short link into an array; A first writing module, used for stopping generating short links when the array is full, and writing the short links in the array into a short link file and a local cache; A first data processing module, configured to receive a long link sent by a first user terminal, allocate an unused short link to the long link, and send the short link to the first user terminal; The second writing module is used to write the mapping relationship between the long link and the short link corresponding to the long link into a database.

14. A short link generation system, characterized in that: Including load balancer, Web server cluster, application server cluster, cache server cluster and Hadoop cluster; The load balancer is used to receive a long link sent by a first user terminal and receive a query request sent by a second user terminal; The Web server cluster is used to send the received long link and the query request to the application server cluster; The Hadoop cluster is used to store databases, short link files and offset record files; The application server cluster includes: A short link generation module, used for generating a first preset number of random numbers by using a random function, and performing character mapping on each of the random numbers by using a base64 function to obtain a first preset number of characters, wherein the first preset number of characters constitute a short link; A data storage module, used to store the generated short link into an array; A first writing module, used for stopping generating short links when the array is full, and writing the short links in the array into a short link file and a local cache; A first data processing module, configured to receive a long link sent by a first user terminal, allocate an unused short link to the long link, and send the short link to the first user terminal; The second writing module is used to write the mapping relationship between the long link and the short link corresponding to the long link into a database.

15. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the short link generation method according to any one of claims 1 to 12 is implemented.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the short link generation method according to any one of claims 1 to 12 is implemented.

17. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the short link generation method according to any one of claims 1 to 12 is implemented.

Images