Feed flow mixed diffusion method and device based on railway CMS derivative application scenarios

Abstract

The application relates to a Feed flow mixed diffusion method and device based on a railway CMS derivative application scene, and the method comprises the following steps: acquiring a multi-level authorization information storage structure, wherein the multi-level authorization information storage structure comprises a user table, a unit table, a Feed table and a column table; generating a diffusion mode judgment rule based on the column table and a user activity model; performing composite authorization calculation on a Feed flow source according to the multi-level authorization information storage structure and the diffusion mode judgment rule, and performing aggregation calculation on the result of the first authorization calculation to generate a Feed flow weighting table of authorized source users; and based on the request information of the Feed flow, sequentially passing through the dynamic and static request separation of a dynamic and static separation gateway, the read-write diffusion separation of a user authorized Feed flow calling module and then entering a write diffusion acquisition path to obtain Json data of the Feed flow. The application distinguishes the read-write diffusion scenes in the mixed diffusion mode, simplifies the authorization mode and the authentication calculation amount of the read diffusion, and facilitates the requirement of actual use.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of railway CMS derivative application scenarios, and particularly relates to a Feed flow mixed diffusion method and device based on a railway CMS derivative application scenario. BACKGROUND

[0002] In the CMS (content publishing) system derivative application scenarios of the common railway industry (such as a website, a training platform, a knowledge base, etc.), the published content is often provided in a service manner of authorized access. However, in the case of millions of classified authorized users and a large amount of content, the classified and graded authorized retrieval using a traditional relational database cannot meet the requirements of actual use. Therefore, it is necessary to improve it. SUMMARY

[0003] Therefore, it is necessary to provide a Feed flow mixed diffusion method and device based on a railway CMS derivative application scenario in order to solve the problem that the classified and graded authorized retrieval using a traditional relational database cannot meet the requirements of actual use.

[0004] The present application provides a Feed flow mixed diffusion method based on a railway CMS derivative application scenario, which comprises the following steps:

[0005] obtaining a multi-level authorized information storage structure, wherein the multi-level authorized information storage structure comprises a user table, a unit table, a Feed table and a column table;

[0006] generating a diffusion mode determination rule based on the column table and a user activity model;

[0007] performing a composite authorized calculation on a Feed flow source according to the multi-level authorized information storage structure and the diffusion mode determination rule, and performing an aggregation calculation on the result of the first authorized calculation to generate a Feed flow weighting table of authorized source users;

[0008] based on the request information of the Feed flow, sequentially passing through the dynamic and static request separation of the dynamic and static separation gateway, the read-write diffusion separation of the user authorized Feed flow calling module, and entering the write diffusion acquisition path to obtain the Json data of the Feed flow.

[0009] In one embodiment, the generating of the diffusion mode determination rule based on the column table and the user activity model comprises the following steps:

[0010] performing a composite calculation on the sensitivity of the resource and the activity of the user to generate a priority determination function, and the priority determination function is as follows:

[0011]

[0012] wherein, a user activity, a resource sensitivity, a user activity weight, a resource sensitivity weight, a total weight;

[0013] In response to the determination of the priority of the Feed stream source, the Feed stream source is subjected to write diffusion processing, and other Feed stream sources are subjected to read diffusion processing, and the specific expression is as follows:

[0014] .

[0015] In one embodiment, the generation of the diffusion mode determination rule based on the channel table and the user activity model comprises:

[0016] Based on the channel differentiation method and the user activity, a priority determination function is generated, and the priority determination function is as follows:

[0017]

[0018] wherein, a user activity;

[0019] In response to the determination of the priority of the Feed stream source, the Feed stream source is subjected to write diffusion processing, and other Feed stream sources are subjected to read diffusion processing, and the specific expression is as follows:

[0020] .

[0021] In one embodiment, the aggregation calculation of the results of the first authorization calculation to generate the Feed stream weighting table of the authorized source user comprises:

[0022] According to the authorized Feed stream prediction category, the full-amount Feed stream calculation list of the corresponding round is respectively started according to the publishing time, and the Feed stream prediction category includes popular selected and important prompt;

[0023] Determine the Feed and user involved in the Feed stream change of the current round;

[0024] The Feed stream table is persisted in the FastDFS distributed file system by write diffusion, and the Feed stream weighting table of the authorized source user is generated.

[0025] In one embodiment, the determination of the Feed and user involved in the Feed stream change of the current round comprises:

[0026] According to the scheduled publishing time, the ID list of all Feeds involved in the current round is obtained;

[0027] The real-time calculation user access frequency, session duration and content interaction depth to obtain the Feed and user involved in the current round of Feed stream change.

[0028] In one embodiment, the Feed stream table is persisted to the FastDFS distributed file system by write diffusion, and the Feed stream weighting table of the authorized source user is generated, comprising:

[0029] Obtain all involved Feed and user information;

[0030] Based on the aggregation calculation method, the Feed stream file list of each user is read in turn, and the Json sequence in the list file is appended and saved to generate the Feed stream weighting table of the authorized source user.

[0031] In one embodiment, the results of the first authorized calculation are aggregated to generate the Feed stream weighting table of the authorized source user, further comprising:

[0032] Call Purge to update the cache.

[0033] The application also provides a Feed stream hybrid diffusion device based on a railway CMS derivative application scenario, comprising:

[0034] The acquisition module is used to acquire a multi-level authorization information storage structure, and the multi-level authorization information storage structure comprises a user table, a unit table, a Feed table and a column table;

[0035] The generation module is used to generate a diffusion mode determination rule based on the column table and a user activity model;

[0036] The calculation module is used to perform composite authorized calculation on the Feed stream source according to the multi-level authorization information storage structure and the diffusion mode determination rule, and perform aggregation calculation on the results of the first authorized calculation to generate the Feed stream weighting table of the authorized source user;

[0037] The separation module is used to sequentially pass through the dynamic and static request separation of the dynamic and static separation gateway, the read-write diffusion separation of the user authorized Feed stream calling module, and then enter the write diffusion acquisition path to obtain the Json data of the Feed stream based on the request information of the Feed stream.

[0038] The application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the Feed stream hybrid diffusion method based on the railway CMS derivative application scenario according to any one of the above when executing the computer program.

[0039] The application further provides a computer readable storage medium storing a computer program, which, when executed by a processor, implements the Feed flow mixed diffusion method based on a railway CMS derived application scenario as any of the above.

[0040] The Feed flow mixed diffusion method and device based on a railway CMS derived application scenario simplifies the authorization mode and authentication calculation amount of read diffusion, facilitates meeting the requirements of actual use, eliminates the IO bottleneck of read diffusion mode by applying the write diffusion mode in the complex authorization scenario, and solves the resource competition problem in the high concurrency situation. Meanwhile, the write diffusion in the Feed business prepositions the corresponding calculation to the write operation link, prepositions the complex authentication, weighting, and group flow calculation process to directly form a result set, and the acquisition party directly pulls the source Feed flow list in the result set, so that the business process is simple and faster. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0042] Figure 1 A Feed flow mixed diffusion method based on a railway CMS derived application scenario in an embodiment;

[0043] Figure 2 A first flowchart for generating diffusion mode determination rules in an embodiment;

[0044] Figure 3 A second flowchart for generating diffusion mode determination rules in an embodiment;

[0045] Figure 4 A flowchart for performing aggregation calculation on the results of the first authorization calculation in an embodiment;

[0046] Figure 5 A flowchart for determining the Feed and user involved in the Feed flow change in the current round in an embodiment;

[0047] Figure 6 A flowchart for generating a Feed flow weighting table of the authorized source user in an embodiment;

[0048] Figure 7Figure 1 is a schematic diagram of a feed flow mixing and diffusion device structure based on a railway CMS derived application scenario in one embodiment.

[0049] Figure 8 Figure 1 is a schematic diagram of a feed flow mixing and diffusion device structure based on a railway CMS derived application scenario in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0051] Railway industry companies at all levels have the industry characteristics of clear hierarchical management and strict authority management. In the process of building related content publishing systems, there is a problem that the content published by users cannot be efficiently and accurately locked to the target audience in the company website group of the multi-level management system. In this scenario, the corresponding information needs to be served in an authorized access manner. However, when facing millions of users and a large amount of classified information, relying on traditional relational databases for classified and authorized retrieval cannot meet the requirements of actual use.

[0052] Feed refers to a network summary (or content summary), which is a data format for providing frequently updated content to users. Content publishers form a network summary together, allowing users to subscribe to a channel to obtain content distributed by multiple content publishers. That is, Feed is a combination of several message sources subscribed by users to form a content aggregator, helping users to continuously obtain the latest source content. Common Internet Feed flow applications include common content platforms such as NetEase News, Weibo, TikTok, and Bilibili.

[0053] Early Internet segment content service providers usually provide corresponding subscription sources. Typical application scenarios such as RSS channels allow users to subscribe to corresponding RSS channels, which will update and push the channel information subscribed by users in time sequence, and then display it. With the further expansion of application scenarios and needs, the corresponding Feed flow sorting mode has also developed. Currently, there are mainly three kinds.

[0054] 1. Simple timeline mode: display the list according to the time sequence of Feed flow content update. Common application scenarios include WeChat Moments.

[0055] 2. Permission weight calculation mode: that is, according to the corresponding user identity and weight factor to calculate the permission and weight of the content, so as to decide which content to show and the order of its production. Common application scenarios such as B station subscription channel.

[0056] 3. Intelligent recommendation mode: the later stage of the permission weight calculation mode, the commercial value demand of Internet application is highlighted, such as user stickiness and advertising effect demand. At this time, the intelligent recommendation algorithm mode based on user portrait appears. Common application scenarios such as Douyin and Toutiao.

[0057] The biggest feature of feed flow business is that the network content summary obtained by the information acquisition party (audience) is provided and authorized by the publisher, and the publisher feed flow set is obtained. From a technical point of view, there are mainly two kinds: read diffusion (pull) and write diffusion (push).

[0058] In a complex multi-source feed flow system, there are multiple feed flows respectively authorized to user groups, and in the authorization process, not only the acquisition relationship between user entities and feed entities is authorized, but also the related attributes of user entities (such as: user's gender, age), the associated entities of user entities (such as: user's belonging unit, user's professional engaged in) and the related attributes of these associated entities (such as: user's belonging unit level), as well as the related attributes of feed entities (such as: feed publishing time, content length), the associated entities of feed entities (such as: feed's belonging column, feed source website or system) and the related attributes of these associated entities (such as: feed's belonging column classification) are further authorized. Such authorization information will be stored as a cascade relationship between multiple entity relationships.

[0059] At this time, the authorization information of a single user is output to the feed flow, which usually adopts the following two ways.

[0060] 1. Read diffusion

[0061] Working principle: only the corresponding rule storage is performed during authorization, and real-time permission aggregation calculation is performed when the user access request is triggered. The total time consumption formula of its permission aggregation calculation is as follows:

[0062]

[0063] Where n is the number of associated permission sources, is the query time consumption of a single rule, is the aggregation calculation time consumption, and k is the proportion of total rule query in total time consumption .

[0064] The storage space occupancy rate calculation formula of its authorization information is as follows:

[0065]

[0066] Wherein M, N are the number of authorized objects and authorized objects respectively, O represents the spatial complexity. In a complex multi-source Feed flow system, the authorized object refers to the relevant attributes of the user entity, the associated entity and its related attributes. The authorized object refers to the relevant attributes of the Feed, the associated entity and its related attributes.

[0067] Typical features:

[0068] In the read mode case, in order to obtain the authorized Feed flow of the user, the user will re-calculate the access permission according to the authorization rules every time the acquisition operation is performed, and aggregate the Feed flow results returned after different rule calculations, and finally form the results returned to the user. This will result in a high compression rate of the storage space of the authorization information, but in the user authentication process, especially in the high concurrency scenario of the complex authorization system, the consumption of computing resources and computing time will surge.

[0069] 2, write diffusion

[0070] Working principle: The permission aggregation calculation is performed at the time of authorization, and the corresponding user authorization information copy is stored according to different users. The total time formula of the calculation is consistent with that of read diffusion.

[0071] The storage space occupancy rate of the authorization information is calculated as follows:

[0072]

[0073] Wherein M, N are the number of authorized objects and authorized objects respectively. The storage capacity increases with the product of the scale of the authorized objects and the authorized objects.

[0074] Typical features:

[0075] In the write mode case, it is necessary to perform pre-permission aggregation calculation on the Feed flow of each user of the acquisition party when the Feed appears, and form the authorized Feed flow result data of each user for storage. Therefore, when the acquisition party accesses the Feed flow, it will directly access the stored Feed flow. This will result in strong immediacy of permission acquisition (only single key-value type query is required), but the storage cost is high.

[0076] The retrieval of the database in the read expansion mode is mainly based on the operation of the binary tree in the index mode, and the time complexity of the binary tree depends on the structure of the tree. In the balanced case, the time complexity of the lookup, insertion and deletion is O(log n). If the tree is unbalanced, the time complexity can degenerate to O(n). Therefore, the optimal time complexity of the database retrieval is O(log n). That is, the calculation time complexity of the read expansion mode is O(log 2 N). And the calculation time of the write expansion mode is O(log n) which has been completed in advance, so the calculation time complexity obtained at this time is O(l). The data measured in the model development verification environment shows that in the case of 2 million users, 50 sources and a total of 50,000 Feed random authorization, the read expansion of the Feed stream acquisition is about 80 seconds, and the write expansion is about 60 milliseconds.

[0077] To comprehensively balance the contradiction between storage occupancy and user authorization efficiency, the following will be described in combination with Figures 1-8 The application discloses a Feed stream mixed diffusion method and device based on a railway CMS derivative application scenario.

[0078] As Figure 1 shown, in one embodiment, a Feed stream mixed diffusion method based on a railway CMS derivative application scenario comprises the following steps:

[0079] Step S100, acquiring a multi-level authorization information storage structure, the multi-level authorization information storage structure comprising a user table, a unit table, a Feed table and a column table.

[0080] The user table is an authorized object table, the unit table is an authorized object associated entity table, the Feed table is an authorized object table, and the column table is an authorized object associated entity table.

[0081] Step S200, generating a diffusion mode judgment rule based on the column table and a user activity model.

[0082] Whether the user authorization list of any Feed stream source is subjected to read or write diffusion, the priority of the Feed stream of the source needs to be judged, only the source subjected to the priority judgment is subjected to write diffusion processing, and other sources still adopt read diffusion.

[0083] Step S300, according to the multi-level authorization information storage structure and the diffusion mode judgment rule, performing a composite authorization calculation on the Feed stream source, and performing an aggregation calculation on the result of the first authorization calculation to generate a Feed stream weighted table of the authorized source user.

[0084] 1. Multi-source permission fuse mechanism

[0085] Adopt the permission calculation topology based on DAG (Directed Acyclic Graph), and carry out parallel calculation of the permission verification of three dimensions of user attributes (gender / profession), organizational level (China Railway Group - Road Bureau - Station segment three levels) and content characteristics (column classification / security level).

[0086] Each computing node sets a fuse threshold (such as triggering fuse when single node response time > 200ms), which guarantees system-level service availability.

[0087] 2、Dynamic weight allocation model

[0088] Construct the weight matrix of permission factors: W = [w_org, w_attr, w_cont]^T, where: w_org = 0.6 (organizational structure weight), w_attr = 0.3 (user attribute weight), w_cont = 0.1 (content feature weight)

[0089] Real-time calculation of user-content matching degree: Score = Σ (w_i * match_level_i)

[0090] 3、Multi-level cache acceleration strategy

[0091] Establish a three-level cache architecture: L1 cache: Redis hot data, L2 cache: local memory queue, L3 cache: SSD persistent cache.

[0092] Adopt LRU-K algorithm to optimize cache eviction strategy, and improve cache hit rate to 92%.

[0093] 4、Feed stream weighted table core value

[0094] Through the pre-generated weighted table, the real-time authentication time complexity is reduced from O (n log n) to O (1).

[0095] Support four weight update strategies: timed full refresh, incremental real-time update (delay < 500ms), emergency strategy override (highest priority), and gray batch release.

[0096] Through the establishment of a complete technical system of pre-computation, dynamic adjustment and hierarchical storage, the authentication efficiency bottleneck faced by traditional railway CMS systems in the scene of millions of users is effectively solved, and a new architecture paradigm is provided for industry-level content distribution systems.

[0097] Step S400, based on the request information of the Feed stream, through the dynamic and static request separation of the dynamic and static separation gateway, the read-write diffusion separation of the user access Feed stream calling module, and then entering the write diffusion access path to obtain the Json data of the Feed stream.

[0098] By classifying dynamic requests and static requests, the back-end computing pressure is conveniently reduced, and by separating read diffusion and write diffusion, the decision efficiency and accuracy are conveniently improved.

[0099] The Feed flow mixed diffusion method based on the railway CMS derived application scenario distinguishes the read diffusion and write diffusion scenarios in the mixed diffusion mode, simplifies the authorization mode and authentication calculation amount of read diffusion, and eliminates the IO bottleneck of the read diffusion mode in the write diffusion mode of complex authorization scenarios. In the write diffusion mode, each user only reads data belonging to himself, and these data can also be distributed, thereby solving the resource competition problem under high concurrency. Meanwhile, in the Feed business, the write diffusion prepositions corresponding calculation to the write operation link, prepositions the complex authentication, weighting and group flow calculation process to directly form a result set, and the acquisition party directly pulls the source Feed flow list in the result set, so that the business process is simple and faster.

[0100] In the embodiment, referring to Figure 2 , based on the column table and the user activity model, a diffusion mode determination rule is generated, including the following steps:

[0101] In step S211, the sensitivity of the resource and the activity of the user are combined to calculate a priority determination function, and the priority determination function is as follows:

[0102]

[0103] Among them, is the user activity, is the resource sensitivity, is the user activity weight, is the resource sensitivity weight, is the total weight.

[0104] The priority determination obtained by combining the sensitivity of the resource and the activity of the user is convenient to determine whether the user authorization list of any Feed flow source is read or write diffusion.

[0105] In step S212, in response to the priority determination of the Feed flow source, the Feed flow source is processed by write diffusion, and other Feed flow sources are processed by read diffusion, and the specific expression is as follows:

[0106] .

[0107] In the embodiment, referring to Figure 3 , based on the column table and the user activity model, a diffusion mode determination rule is generated, including the following steps:

[0108] Step S221, based on the column differentiation method and user activity, a priority determination function is generated, and the priority determination function is as follows:

[0109]

[0110] wherein, is the user activity.

[0111] Since the calculation of sensitivity to resources is difficult to unify the standard, and when a single Feed changes to a high sensitivity resource, it will induce the calculation storm of re-authentication and write diffusion of all users, which will cause serious resource occupation risk. Therefore, the mixed diffusion model used in the homogeneous multi-source Feed stream usually only matches the user activity identification.

[0112] Step S222, in response to the Feed stream source passing the priority determination, the Feed stream source is processed by write diffusion, and other Feed stream sources are processed by read diffusion, and the specific expression is as follows:

[0113] 。

[0114] After expert brainstorming and analysis and selection by ANP method, it is decided to use the column differentiation method, and the business experts confirm which columns are suitable for write diffusion authorization. The columns using read diffusion authorization only perform column-level authorization, further compressing the database pressure.

[0115] At this time, the priority determination algorithm is further simplified to key-value query.

[0116] In this embodiment, referring to Figure 4 , the results of the first authorization calculation are aggregated to generate a Feed stream weighted table of authorized source users, including the following steps:

[0117] Step S310, according to the authorized Feed stream prediction category, respectively start the corresponding round of full Feed stream calculation list according to the release time, and the Feed stream prediction category includes hot selection and important prompt.

[0118] The release time is automatically released only at the whole hour, and the special task needs to be manually released.

[0119] Step S320, determine the Feed and user involved in the Feed stream change in this round.

[0120] By accurately defining the change boundary, establishing an efficient retrieval system, and implementing intelligent prediction scheduling, the system can maintain an accuracy rate of 99.998% in permission calculation, an automatic recovery rate of 92% in abnormal conditions, and a resource utilization rate fluctuation within ±3% when processing 2000+ content updates per day and serving 2 million+ users.

[0121] This fine change management becomes the key technology cornerstone supporting the mixed diffusion mode in the ultra-large scale scene.

[0122] Step S330, the Feed flow table is persisted in the FastDFS distributed file system by the write diffusion mode, and the Feed flow weighting table of the authorized source user is generated.

[0123] By precomputing the user-level Feed flow and persisting, it is convenient to eliminate the real-time authentication overhead, guarantee real-time response, shorten the reading delay, separate the computing layer and the storage layer through FastDFS, facilitate to reduce the influence range of single node failure, and through the composite calculation of time weight and business weight in the file storage stage, it is convenient to realize dynamic weight integration and shorten the weight calculation time.

[0124] In the embodiment, referring to Figure 5 , the Feed and user involved in the current Feed flow change are determined, including the following steps:

[0125] Step S321, according to the scheduled publishing time, the ID list of all Feeds involved in the current round is obtained.

[0126] Taking "hot selection" as an example, because the database calculation time is longer, the system aggregation calculation module will asynchronously call the corresponding stored procedure of the database, according to the scheduled publishing time, and input the feed id list of all Feeds involved in the current round.

[0127] Step S322, the user access frequency, session duration and content interaction depth are calculated in real time to obtain the Feed and user involved in the current Feed flow change.

[0128] The stored procedure first queries the feed role for user to confirm the user ID list authorized, and saves it into the session temporary table temp user table session.

[0129] Then according to the corresponding feed id and the corresponding condition in the feed role for user attribute and the feed role for unit, the corresponding user id and user name are obtained, and then the corresponding records are inserted into the temporary table again.

[0130] Finally, the temporary table temp user table session contains all the authorized users and the Feeds involved in this time.

[0131] In the embodiment, referring toFigure 6 The Feed stream table is persisted into the FastDFS distributed file system by means of write diffusion, and a Feed stream weighting table of authorized source users is generated, including the following steps:

[0132] In step S331, information of all involved Feed and users is acquired.

[0133] In step S332, based on an aggregation calculation method, a Feed stream file list of each user is read in sequence, and a Json sequence in the list file is appended and saved to generate the Feed stream weighting table of authorized source users.

[0134] At this time, the user first acquires from the proxy cache, and if not hit, then acquires the file from the FastDFS. Finally, the Json data of the Feed stream is obtained, and the first 100 records are preloaded, and after being displayed via the front-end processing, the traditional subsequent data is continuously transmitted.

[0135] In the embodiment, the result of the first authorized calculation is aggregated to generate the Feed stream weighting table of authorized source users, and the following steps are further included:

[0136] In step S340, a Purge update cache is called.

[0137] A specific application example is as follows:

[0138] Taking a railway mobile information collaboration platform (three-level independent authorized content publishing station group) as an example, the access of content is divided into public source (a column in a website is completely public) and authorized source (the content in a column in a website needs to be authorized one by one, and only the user who obtains the authorization can access). Each employee (user) will accept the authorized content push of at least one first-level station (a national railway group-level station) and at most three authorized content pushes of three stations (a national railway group-level station, a subordinate railway bureau-level station, and a subordinate station and section-level station). Taking four websites of the national railway group website, the Beijing bureau website, the Beijing station website, and the Beijing locomotive depot website as an example, the data implementation and experimental data in the experimental environment are as follows.

[0139] The related properties and table data of the authorized object (authorized user) table are as follows.

[0140]

[0141] Table 1: Authorized object table (user table)

[0142] The related properties and table data of one of the entities (units) associated with the authorized object are as follows.

[0143]

[0144] Table 2: Authorized object associated entity example (unit table)

[0145] The relevant attributes and table data of the authorized object (authorized Feed) table are as follows.

[0146]

[0147] Table 3: Authorized object table (Feed table)

[0148] The relevant attributes and table data of one of the entities (column) associated with the authorized object are as follows.

[0149]

[0150] Table 4: Authorized object associated entity example (column table)

[0151] The specific news content (Feed) is authorized and calculated according to the above data.

[0152] For example, the authorization of GT_001 this Feed is filtered for gender "male", and since the website to which the GT_001 Feed belongs is "China Railway Group website", it is necessary to perform relational database filtering search for all railway audiences. Finally, the final authorized users can be obtained as "Zhang San, Wang Wu, Zhou Qi".

[0153] The authorization of GT_002 this Feed is filtered for units "Beijing Bureau" (including subordinate units) and unit professional category "passenger transport", and since the website to which the GT_002 Feed belongs is "China Railway Group website", it is necessary to perform relational database filtering search for all railway audiences. Finally, the final authorized users can be obtained as "Wang Wu, Zhao Liu".

[0154]

[0155] Table 5: Feed content authorized calculation table

[0156] Then, the results of the first authorization calculation are aggregated and calculated.

[0157] The Feed obtained by all authorized users is sorted by news time (here, the time order is consistent with the news name order), and the Feed flow table of each user can be obtained. When a new Feed appears in the system, it can be sequentially added to the last row of the Feed flow table of the corresponding authorized user. At this time, the Feed flow table of user "Wang Wu" needs to be further adjusted according to the weight, and here, the final weighted table is obtained by rearranging according to whether the top condition is met, as shown in Table 6. The weighting method is not limited to this mode, and can also be calculated and sorted according to the user access number and other conditions. This calculation can occur in the backend or be dispatched to the front end.

[0158]

[0159] Table 6 Weighted table of user "Wang Wu" Feed flow

[0160] At this point, the Feed flow calculation of the authorized source user has been completed, and further write spreading storage is required. The Feed flow table can be directly saved in json text form to the database or local file, and written to the cache. Other public source Feed flow tables are directly saved to the database or local file in source units, and written to the cache, and users directly obtain them in the cache.

[0161] The read spreading permission is authorized according to the entire column, and the query pressure on the database is extremely small. Therefore, the experimental setup only tests the performance of the write spreading path.

[0162] In the test environment simulating 2 million users, 50 sources, 50,000 Feed contents, and corresponding permission records, the upper limit of the write spreading storage amount should be:

[0163]

[0164] The actual statistical result in the production environment is about 40th of the upper limit. According to this approximate production Record, the number of files is the same as the number of users * source number.

[0165]

[0166] The total number of files is about Each file contains about 250 records, and the average file size is about 37KB.

[0167] In the performance test, the above data set is used, 5 standard virtual machines are used for application load clustering, and the backend is connected to 5 standard virtual machine agent service cache load clustering and distributed file storage system experimental environment, and the test is carried out according to the 1000 concurrent standard requirements.

[0168] Write spreading performance:

[0169] In the case of using cache preloading, the acquisition time of the first 100 data of the actual measured Feed stream is about 60 to 70 milliseconds. In the case of not using cache preloading, the acquisition time is about 3 seconds and gradually falls back.

[0170] The mixed diffusion mode distinguishes the read-write diffusion scenarios, simplifies the authorization mode and authentication calculation amount of read diffusion, and eliminates the IO bottleneck of read diffusion mode in the complex authorization scenario. In write diffusion, each user only reads the data belonging to himself, and these data can also be distributed, thereby solving the resource competition problem in the high concurrency situation. The Feed business is a typical read-write business scenario, and the read-write ratio is even higher than 100:1, that is, on average, there are at least 100 times of reading for 1 message published. Based on the specific application characteristics, write diffusion prepositions the corresponding calculation to the write operation link, and directly forms a result set for the complex authentication, weighting, and group flow calculation process. The acquisition party directly pulls the source Feed stream list in the result set, the business process is simple, and the speed is faster.

[0171] However, the write diffusion mode also has certain disadvantages. First, this mode will greatly consume storage resources, and the Feed data will be stored in multiple copies according to the number of users, and in the extreme case, the storage number of a single Feed is equal to the number of users. Second, the calculation of adding a Feed is complex, and in the extreme case, the Feed stream list of all users needs to be reconstructed.

[0172] The Feed stream mixed diffusion device based on the railway CMS derivative application scenario provided by the application will be described below. The Feed stream mixed diffusion device based on the railway CMS derivative application scenario described below can be correspondingly referred to the Feed stream mixed diffusion method based on the railway CMS derivative application scenario described above.

[0173] As shown in Figure 7 In one embodiment, a Feed stream mixed diffusion device based on a railway CMS derivative application scenario includes an acquisition module 710, a generation module 720, a calculation module 730, and a separation module 740.

[0174] The acquisition module 710 is used to acquire a multi-level authorization information storage structure, and the multi-level authorization information storage structure includes a user table, a unit table, a Feed table, and a column table.

[0175] The generation module 720 is used to generate a diffusion mode determination rule based on the column table and a user activity model.

[0176] The computing module 730 is configured to perform composite authorization calculation on the Feed stream source according to the multi-level authorization information storage structure and the diffusion mode determination rule, and perform aggregation calculation on the result of the first authorization calculation to generate a Feed stream weighting table of the authorized source user.

[0177] The separating module 740 is configured to, based on the request information of the Feed stream, sequentially pass through the dynamic and static request separation of the dynamic and static separation gateway, and the read-write diffusion separation of the user authorized Feed stream calling module, and then enter a write diffusion acquisition path to obtain the Json data of the Feed stream.

[0178] In the embodiment, the generating module 720 is specifically configured to:

[0179] The sensitivity of the resource and the activity of the user are combined to generate a priority determination function, and the priority determination function is as follows:

[0180]

[0181] wherein, the activity of the user, the sensitivity of the resource, a weight of the activity of the user, a weight of the sensitivity of the resource, a total weight;

[0182] In response to the priority determination of the Feed stream source, the Feed stream source is subjected to write diffusion processing, and other Feed stream sources are subjected to read diffusion processing, and the specific expression is as follows:

[0183] .

[0184] In the embodiment, the generating module 720 is specifically configured to:

[0185] Based on the column differentiation method and the activity of the user, a priority determination function is generated, and the priority determination function is as follows:

[0186]

[0187] wherein, the activity of the user;

[0188] In response to the priority determination of the Feed stream source, the Feed stream source is subjected to write diffusion processing, and other Feed stream sources are subjected to read diffusion processing, and the specific expression is as follows:

[0189] .

[0190] In the embodiment, the result of the first authorized calculation is aggregated to generate the Feed stream weighting table of the authorized source user, specifically for:

[0191] According to the authorized Feed stream prediction category, the full Feed stream calculation list of the corresponding round is started according to the publishing time, and the Feed stream prediction category includes hot selection and important prompt;

[0192] Determine the Feed and user involved in the Feed stream change of the current round;

[0193] The Feed stream table is persisted in the FastDFS distributed file system by write diffusion, and the Feed stream weighting table of the authorized source user is generated.

[0194] In the embodiment, the Feed and user involved in the Feed stream change of the current round are determined, specifically for:

[0195] According to the scheduled publishing time, the ID list of all Feeds involved in the current round is obtained;

[0196] Real-time calculation of user access frequency, session duration and content interaction depth to obtain the Feed and user involved in the Feed stream change of the current round.

[0197] In the embodiment, the Feed stream table is written to the distributed file system FastDFS by write diffusion and the cache is cleaned to generate the Feed stream weighting table of the authorized source user, specifically for:

[0198] Obtain all involved Feed and user information;

[0199] Based on the aggregation calculation method, the Feed stream file list of each user is read in turn, and the Json sequence in the list file is appended and saved to generate the Feed stream weighting table of the authorized source user.

[0200] In the embodiment, the result of the first authorized calculation is aggregated to generate the Feed stream weighting table of the authorized source user, and is also specifically for:

[0201] Call Purge to update the cache.

[0202] The Feed flow mixed diffusion device based on the railway CMS derived application scene distinguishes the read-write diffusion scene in the mixed diffusion mode, simplifies the authorization mode and the authentication calculation amount of the read diffusion, and eliminates the IO bottleneck of the read diffusion mode by applying the write diffusion mode in the complex authorization scene. In the write diffusion, each user only reads the data belonging to himself, and these data can also be distributed, thereby solving the resource competition problem under high concurrency. Meanwhile, by means of the Feed business, the write diffusion prepositions the corresponding calculation to the write operation link, prepositions the complex authentication, weighting and group flow calculation process to directly form a result set, and the acquisition party directly pulls the source Feed flow list in the result set, so that the business process is simple and faster.

[0203] Figure 8 An example of an entity structure diagram of an electronic device, which can be a smart terminal, is shown in Figure 8 The electronic device includes a processor, a memory and a network interface connected through a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the electronic device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a Feed flow mixed diffusion method based on a railway CMS derived application scene, which includes:

[0204] Obtaining a multi-level authorization information storage structure including a user table, a unit table, a Feed table and a column table;

[0205] Generating a diffusion mode determination rule based on the column table and a user activity model;

[0206] According to the multi-level authorization information storage structure and the diffusion mode determination rule, the Feed flow source is subjected to composite authorization calculation, and the result of the first authorization calculation is subjected to aggregation calculation to generate a Feed flow weighting table of the authorized source user;

[0207] Based on the request information of the Feed flow, the dynamic and static request separation of the dynamic and static separation gateway, the read-write diffusion separation of the user authorized Feed flow calling module, and the write diffusion acquisition path are sequentially entered to obtain the Json data of the Feed flow.

[0208] Those skilled in the art can understand that, Figure 8The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0209] In another aspect, the present application also provides a computer storage medium storing a computer program, which, when executed by a processor, implements a Feed flow mixed diffusion method based on a railway CMS derived application scenario, the method comprising:

[0210] obtaining a multi-level authorization information storage structure, the multi-level authorization information storage structure comprising a user table, a unit table, a Feed table, and a column table;

[0211] generating a diffusion mode determination rule based on the column table and a user activity model;

[0212] performing a composite authorization calculation on a Feed flow source according to the multi-level authorization information storage structure and the diffusion mode determination rule, and performing an aggregation calculation on the result of the first authorization calculation to generate a Feed flow weighting table of authorized source users;

[0213] Based on the request information of the Feed flow, the dynamic and static request separation of the dynamic and static separation gateway, the read-write diffusion separation of the user authorized Feed flow calling module, and the write diffusion acquisition path, the Json data of the Feed flow is obtained.

[0214] In another aspect, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer instructions from the computer readable storage medium. The processor implements a Feed flow mixed diffusion method based on a railway CMS derived application scenario when executing the computer instructions, the method comprising:

[0215] obtaining a multi-level authorization information storage structure, the multi-level authorization information storage structure comprising a user table, a unit table, a Feed table, and a column table;

[0216] generating a diffusion mode determination rule based on the column table and a user activity model;

[0217] performing a composite authorization calculation on a Feed flow source according to the multi-level authorization information storage structure and the diffusion mode determination rule, and performing an aggregation calculation on the result of the first authorization calculation to generate a Feed flow weighting table of authorized source users;

[0218] Based on the request information of the Feed stream, the request information is sequentially separated by the dynamic and static request separation of the dynamic and static separation gateway, the read-write diffusion separation of the user access Feed stream calling module, and then enters the write diffusion access path to obtain the Json data of the Feed stream.

[0219] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. Among them, any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory.

[0220] As an illustration but not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM) and the like.

[0221] Each technical feature of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of each technical feature in the above embodiments are not described, however, as long as the combination of these technical features does not exist contradictory, it should be considered as the scope of the present application.

[0222] The above-mentioned embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A feed flow hybrid diffusion method based on railway CMS derivative application scenarios, characterized in that, The method comprises: obtaining a multi-level authorization information storage structure, the multi-level authorization information storage structure comprising a user table, a unit table, a Feed table and a column table; based on the column table and the user activity model, generating a diffusion mode judgment rule; based on the multi-level authorization information storage structure and the diffusion mode judgment rule, performing composite authorization calculation on the Feed stream source, and performing aggregation calculation on the results of the first authorization calculation to generate a Feed stream weighting table of the authorized source user; based on the request information of the Feed stream, sequentially passing through the dynamic and static request separation of the dynamic and static separation gateway, the read-write diffusion separation of the user authorized Feed stream calling module, and entering the write diffusion acquisition path to obtain the Json data of the Feed stream; wherein the aggregation calculation on the results of the first authorization calculation to generate a Feed stream weighting table of the authorized source user comprises: according to the authorized Feed stream prediction category, respectively starting the full-amount Feed stream calculation list of the corresponding round according to the publishing time, the Feed stream prediction category comprising popular selected and important prompt; determining the Feed and user involved in the Feed stream change of the current round; persisting the Feed stream table to the FastDFS distributed file system through the write diffusion mode, and generating the Feed stream weighting table of the authorized source user; the determination of the Feed and user involved in the Feed stream change of the current round comprises: according to the scheduled publishing time, obtaining the ID list of all Feeds involved in the current round; real-time calculation of user access frequency, session duration and content interaction depth to obtain the Feed and user involved in the Feed stream change of the current round; the persisting of the Feed stream table to the FastDFS distributed file system through the write diffusion mode, and the generating of the Feed stream weighting table of the authorized source user comprises: obtaining all involved Feed and user information; based on the aggregation calculation method, sequentially reading the Feed stream file list of each user, and saving the Json sequence in the list file after appending to generate the Feed stream weighting table of the authorized source user. 2.The railway CMS derived application scenario based Feed flow mixed diffusion method according to claim 1, characterized in that, the generation of the diffusion mode judgment rule based on the column table and the user activity model comprises: composite calculation of the sensitivity of the resource and the activity of the user to generate a priority judgment function, the priority judgment function being as follows: wherein, is a user activity, is a resource sensitivity, is a user activity weight, is a resource sensitivity weight, is a total weight; in response to the Feed stream source passing the priority judgment, performing write diffusion processing on the Feed stream source, and performing read diffusion processing on other Feed stream sources, the specific expression being as follows: 。 3.The railway CMS derived application scenario based Feed flow mixed diffusion method of claim 1, wherein, the generation of the diffusion mode judgment rule based on the column table and the user activity model comprises: based on the column differentiation method and the user activity, generating a priority judgment function, the priority judgment function being as follows: wherein, is the user activity; in response to the Feed stream source passing the priority judgment, performing write diffusion processing on the Feed stream source, and performing read diffusion processing on other Feed stream sources, the specific expression being as follows: 。 4. The Feed flow hybrid diffusion method based on railway CMS derived application scenarios according to claim 1, characterized in that, the aggregation calculation on the results of the first authorization calculation to generate a Feed stream weighting table of the authorized source user further comprises: calling Purge to update the cache.

5. A Feed flow mixed diffusion device based on railway CMS derivative application scenarios, characterized in that, comprises: An acquisition module is configured to acquire a multi-level authorization information storage structure, which includes a user table, a unit table, a Feed table, and a column table; A generation module is configured to generate a diffusion mode determination rule based on the column table and a user activity model; A calculation module is configured to perform a composite authorization calculation on a Feed stream source according to the multi-level authorization information storage structure and the diffusion mode determination rule, and perform an aggregation calculation on a result of the first authorization calculation to generate a Feed stream weighting table of authorized source users; A separation module is configured to enter a write diffusion acquisition path after passing through dynamic and static request separation of a dynamic and static separation gateway and read-write diffusion separation of a user authorized Feed stream calling module based on request information of the Feed stream, to obtain Json data of the Feed stream. The aggregation calculation on the result of the first authorization calculation to generate the Feed stream weighting table of authorized source users includes: According to an authorized Feed stream prediction category, respectively start a full-amount Feed stream calculation list of a corresponding round according to a publishing time, and the Feed stream prediction category includes popular selected and important prompt; Determine a Feed and a user involved in a Feed stream change of the current round; Persist the Feed stream table in a FastDFS distributed file system through a write diffusion mode, and generate the Feed stream weighting table of authorized source users; The determination of the Feed and the user involved in the Feed stream change of the current round includes: According to a scheduled publishing time, acquire an ID list of all Feeds involved in the current round; Real-time calculation of a user access frequency, a session duration, and a content interaction depth is performed to obtain the Feed and the user involved in the Feed stream change of the current round; The persistence of the Feed stream table in the FastDFS distributed file system through the write diffusion mode and the generation of the Feed stream weighting table of authorized source users include: Acquire all involved Feed and user information; Based on an aggregation calculation method, read a Feed stream file list of each user in sequence, and save a Json sequence in a list file after appending, to generate the Feed stream weighting table of authorized source users. 6.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 4.

7. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 4.

Images