Methods, systems, and computer program products (content management systems) for detecting changes to external content referenced by external links.

Abstract

To provide a method, system and program for detecting changes of external content referenced by external links.SOLUTION: A method for maintaining links in a content management system 510 comprises: registering, at a first time, keywords associated with external content that is referenced by an external link embedded in an electronic document; obtaining the external content at a second time later than the first time in response to a user accessing the electronic document, where the external content is obtained by scraping a resource located at the external link to identify the external content; and determining whether the external content has changed using the obtained external content and the registered keywords.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present disclosure relates generally to the field of computing systems, and more particularly, to maintaining the integrity and accuracy of content in a content management system.

Background Art

[0002] A content management system (CMS) is a computer system (e.g., software running on a computer) for managing the creation and modification of digital content (i.e., content management). A CMS is typically used for enterprise content management and web content management. When used for web content management, the CMS may manage websites that include text, embedded graphics, photographs, videos, audio, maps, and program code that present content and interact with users.

Summary of Invention

Problem to be Solved by the Invention

[0003] An object of the present disclosure is to provide a method or the like for detecting changes to external content referenced by an external link.

Means for Solving the Problem

[0004] Embodiments of this disclosure include methods, computer program products, and systems for maintaining links in a content management system. The method includes, at a first time, registering keywords related to external content referenced by external links embedded in an electronic document. The method further includes, at a second time after the first time, retrieving the external content in response to a user accessing the electronic document. The external content may be obtained by scraping a resource located at an external link that identifies the external content. The method further includes, using the retrieved external content and the registered keywords, performing a verification step to determine whether the external content has been modified.

[0005] The above summary is not intended to describe each illustrated embodiment or all implementation examples of this disclosure. [Brief explanation of the drawing]

[0006] The drawings included in this disclosure are incorporated herein and constitute part of this specification. These drawings illustrate embodiments of this disclosure and, together with the description in the specification, illustrate the principles of this disclosure. These drawings are merely illustrative of typical embodiments and do not limit this disclosure.

[0007] [Figure 1] This is a block diagram of an example of a computing environment in which exemplary embodiments of the present disclosure can be implemented. [Figure 2] This is a flowchart illustrating an example of a method for registering keywords for external links embedded in content in a content management system according to an embodiment of this disclosure. [Figure 3] This is a flowchart of an example of a method for detecting and verifying changes to data referenced by external links, according to an embodiment of this disclosure. [Figure 4] This is a flowchart illustrating an example of a method for notifying readers and content creators of changes to data referenced by external links, according to an embodiment of this disclosure. [Figure 5] This figure shows an example of a content management system configured to validate external content referenced by embedded links, according to an embodiment of this disclosure. [Figure 6] This is an example of a system flow diagram illustrating a method for identifying and notifying changes to external content according to an embodiment of this disclosure. [Figure 7] This is a schematic block diagram of an example of a computer system that can be used to implement one or more of the methods, tools, modules, and related functions described herein, according to embodiments of the present disclosure. [Figure 8] This figure shows a cloud computing environment according to an embodiment of this disclosure. [Figure 9] This figure shows an abstraction model layer according to an embodiment of the present disclosure.

[0008] While the embodiments described herein are subject to various modifications and alternative forms, details of these embodiments are shown in the drawings as an example and are described in detail below. However, the specific embodiments described herein should not be construed as restrictive. Rather, they are intended to encompass all modifications, equivalents, and alternative forms that fall within the spirit and scope of this disclosure. [Modes for carrying out the invention]

[0009] The aspects of this disclosure generally relate to the field of computing systems, and more specifically to maintaining the integrity and accuracy of content in content management systems. While this disclosure is not necessarily limited to such uses, various aspects of this disclosure can be understood from the descriptions of various related examples.

[0010] In this specification, "content creator" refers to a user who manages, owns, publishes, generates, creates, or otherwise takes responsibility for content found within a content management system. Conversely, "reader" refers to a user who consumes content within a content management system (e.g., reads, accesses, or otherwise views it) but does not create content. For example, a content creator might create content or a webpage, or both, and a reader might read that content or webpage, or both.

[0011] Content management systems (CMS) are often used to manage websites. For example, a CMS may be deployed on a web server to host a website and its associated data (e.g., text and embedded graphics, photos, videos, audio, maps, and program code that display content and allow user interaction). Websites hosted on a web server often include external content (e.g., data from external websites, such as websites hosted by another system) or links to such external content embedded within the website. This can be important for improving convenience by linking content from different sources to a single website. For example, a website may embed data from a financial site and data from a sports site within the same view and provide this data to the user. In this way, end users can view important data without having to access multiple websites. Furthermore, using embedded external content means that when an external website is updated, the website will automatically update. Similarly, external hyperlinks can help guide users to different websites depending on the information they are looking for.

[0012] Thus, content creators using a Content Management System (CMS) specify external links to complement the content they create. Ideally, the details of the linked external content should be maintained as intended by the content creator. For example, if a content creator links to a website for a specific purpose (e.g., providing additional information on coding methods for a particular programming language), and that website changes to other content (e.g., sports highlight videos), the content creator may not want the link to remain active. However, because it links to an external source, the content creator may not be able to control what is displayed on that external source. Therefore, situations can arise where the external content becomes irrelevant to the content provided by the content creator, or becomes content unacceptable to the content creator (e.g., an advertising site instead of an information site). In either case, the failure to detect changes to external data / content can degrade the user experience. Furthermore, if a previously safe linked website becomes a vector for spyware or other security threats, the failure to detect changes to external data / content can create security risks.

[0013] To minimize the risks associated with external links, content creators can periodically review the content of linked sites and update their own websites accordingly. However, it is often difficult to immediately detect changes to the content of external links specified in a CMS, or the use of alternative content. In such cases, the only way to detect the change is for the content creator to review the linked content or for readers to inform them of the change. This creates a vulnerability where changes to external content go undetected until the next periodic review or until a user happens to alert the content creator. Between the time the change is made and the time it is detected through a periodic review or user alert, readers may be directed to external websites containing data that the content creator has not reviewed or approved, potentially leading to a poor user experience or cybersecurity risks.

[0014] Embodiments of this disclosure can address the above and other problems by automatically verifying the content of external links using, for example, keywords or other natural language processing techniques. In some embodiments, a CMS system hosting a website with external links may register keywords or other verification data, or both, for each linked external content at the time of creation. When a reader attempts to access external content, such as by clicking an external hyperlink or selecting external content to display in an applet, the content management system may perform data extraction (e.g., keyword detection) to determine what external content will be provided to the reader. The content management system may then compare the extracted data with registered data to determine whether the external content has been significantly altered. Hereinafter, the act of detecting content from an external source and the act of determining whether the external content has been altered are collectively referred to as external link verification. Based on the comparison, the content management system may decide whether to alert the content creator that the external content may have been altered. Furthermore, based on the comparison results, the content management system may decide whether to alert the reader that the external content has been altered.

[0015] By automatically validating external content when readers attempt to access it, a content management system can reduce or eliminate the number of readers who come into contact with altered external content, thereby providing a better user experience for readers. Furthermore, the content management systems disclosed herein can enable content creators to respond more quickly to altered external content by notifying them of the changes.

[0016] In some embodiments, all external links on a webpage may be validated each time a user accesses the webpage. By validating external links before a user attempts to access them, the content management system can display a warning (e.g., an icon) that changes have been made to the external content on the webpage without interfering with the user's use of the webpage.

[0017] In some embodiments, external content may be validated only after a predetermined number of readers have attempted to access it. This can be particularly advantageous for websites with high web traffic where performing checks every time a user accesses a link would be computationally expensive. These embodiments may further include validation date / time checks. That is, even if the number of readers is below the threshold requiring revalidation of the external link, revalidation of the external link may be required if the amount of time since the last validation exceeds the threshold (e.g., if the validity period of the last validation has expired).

[0018] In some embodiments, determining whether external content has been changed may utilize generating keywords from the external content by natural language processing or other known techniques. Next, the detected keywords may be compared with registered keywords to determine a change score. The change score may indicate whether there is a change to the external content (e.g., indicated as binary yes or no), the amount of change (e.g., by detecting the amount of overlap between the detected keywords and the registered keywords), or both. Next, the change score may be used (e.g., by comparing the change score against one or more thresholds) to determine whether to notify a content creator, a reader, or both of the change, and the method of such notification. For example, if the change score indicates a small change, notification may not be required. Conversely, if the change score indicates a very large change, immediate notification to the content creator, the reader, or both may be required.

[0019] In some embodiments, after verifying the content of an external link destination, the content management system may update a verification date / time. As described herein, the verification date / time may be used when determining whether re-verification of the external link is necessary. In this way, resetting the timer until re-verification of the external content can be achieved by updating the verification date / time when it is determined that no change has been made to the external content.

[0020] In some embodiments, the content management system may be configured to alert readers to the change in a plurality of ways. For example, the content management system may add a warning icon to an external link. The warning icon may notify a user that the external content may have been changed since it was last verified. Alternatively or additionally, the content management system may display a warning dialog regarding the external link. The warning dialog may be displayed on the web page that includes the external link itself, or may be presented as a pop-up when a user selects the external link. Furthermore, the content management system may completely hide the external link if it determines that the external content associated with the external link has been changed.

[0021] In some embodiments, the specific notification / alert method may be based on a change score. For example, the content management system may add a warning icon when the change score indicates a small amount of change to the external content (e.g., when the change score is less than a first threshold but greater than a score indicating no change at all). On the other hand, the content management system may add a warning dialog when the change score indicates a moderate amount of change to the external content (e.g., when the change score is greater than the first threshold and less than a second threshold). Then, the content management system may completely remove the external link when the change score indicates a large change to the external content (e.g., when the change score is greater than the second threshold).

[0022] It should be noted that the above-described effects are merely an example and should not be construed as limiting the present disclosure. Embodiments of the present disclosure may include all or part of the above-described effects, or may include none of them, within the spirit and scope of the present disclosure.

[0023] Referring next to the drawings, Figure 1 is a block diagram of a computing environment 100 as an example of an exemplary embodiment of the present disclosure. In some embodiments, the computing environment 100 may include two remote devices 102 and 112, a host device 122, and two external data sources 160.

[0024] Depending on the various embodiments, the host device 122 and the remote devices 102 and 112 may be computer systems. The remote devices 102 and 112 and the host device 122 may each include one or more processors 106, 116, and 126, and one or more memories 108, 118, and 128. The remote devices 102 and 112 and the host device 122 may be configured to communicate with each other via internal or external network interfaces 104, 114, and 124. The network interfaces 104, 114, and 124 may be, for example, modems or network interface cards. The remote devices 102 and 112 or the host device 122 or both may include a display or monitor. Furthermore, the remote devices 102 and 112 or the host device 122, or both, may include an input device in any configuration (e.g., a keyboard, mouse, scanner, or other input device), or commercial or custom software (e.g., browser software, communication software, server software, natural language processing software, search engine or web crawling software, or both, a filter module that filters content based on predefined parameters, etc.), or both. In some embodiments, the remote devices 102 and 112 or the host device 122, or both, may be a server, desktop, laptop, or a portable device such as a smartphone or tablet.

[0025] The remote devices 102 and 112 and the host device 122 may be located remotely from each other and communicate via the network 150. In some embodiments, the host device 122 may be a central hub from which the remote devices 102 and 112 can establish a communication connection, as in a client-server network model. Alternatively, the host device 122 and the remote devices 102 and 112 may be configured in any other suitable network relationship (e.g., a peer-to-peer configuration or any other network topology).

[0026] In some embodiments, the network 150 may be implemented using any number of suitable communication media. For example, the network 150 may be a wide area network (WAN), a local area network (LAN), the Internet, or an intranet. In some embodiments, the remote devices 102 and 112 and the host device 122 may be local to each other and communicate via any suitable local communication media. For example, the remote devices 102 and 112 and the host device 122 may communicate using a local area network (LAN), one or more wired connections, a wireless link or router, or an intranet. In some embodiments, the remote devices 102 and 112 and the host device 122 may be coupled in a way that allows them to communicate using one or more networks, one or more local connections, or a combination of both. For example, the first remote device 102 may be wired to the host device 122 (e.g., via an Ethernet cable), and the second remote device 112 may communicate with the host device using the network 150 (e.g., over the Internet).

[0027] In some embodiments, the network 150 can be implemented within a cloud computing environment or using one or more cloud computing services. Depending on the various embodiments, the cloud computing environment may include a network-based distributed data processing system that provides one or more cloud computing services. Furthermore, the cloud computing environment may include a large number of computers (e.g., hundreds or thousands or more) located in one or more data centers and configured to share resources over the network 150.

[0028] In some embodiments, remote devices 102 and 112 may be used to access operational data stored on the host device for the user. For example, remote devices 102 and 112 may include a user interface (UI) and one or more applications (not shown) that access data (e.g., a web page 132) stored on the web server 130 in memory 128. The data access application may be in the form of a web browser or any other suitable software module. The UI may be any type of interface (e.g., a command line prompt, a menu screen, a graphical user interface). The UI may allow the user to interact with remote devices 102 and 112 and retrieve data stored on the host device 122. If the user is a content creator rather than a reader, the UI may allow the user to create, modify, save, or otherwise publish the web page 132.

[0029] In some embodiments, the host device 122 may include a web server 130 and a content management system 136 stored in memory 128. The web server 130 may be software that enables the host device 122 to host one or more web pages 132. The web pages 132 may further include external links 134. The external links 134 may be hyperlinks to other web pages or web resources (e.g., documents) stored outside the host device 122 (e.g., an external data source 160). The web pages 132 and associated external links 134 may be generated and maintained by content creators who access the host device 122 using a remote device.

[0030] The content management system 136 may be software that maintains the web page 132. Therefore, although not shown in Figure 1, the content management system 136 may be a component of the web server 130. The content management system 136 may be configured to analyze and verify (e.g., verify the legitimacy of) external content referenced by external links 134. In some embodiments, the content management system 136 may be configured to verify external content when a reader accesses the web page 132 or clicks an external link 134 associated with the external content. For example, the content management system 136 may determine that a reader has selected (e.g., clicked) an external link 134 (e.g., using a remote device) and thereby requested external content from one of the external data sources 160. In response, the content management system 136 may analyze the external content using a data analysis unit 138. Analyzing the external content may include, for example, extracting the external content by website scraping and identifying keywords within the extracted external content. Keyword identification may be performed using any known method of extracting keywords through data analysis, such as natural language processing.

[0031] Next, the content management system 136 may use a verification unit 140 (also referred to herein as a verification engine) to verify the external content (for example, to determine whether the external content has been changed). In some embodiments, the verification unit 140 may compare keywords identified by the data analysis unit 138 with registered keywords 142 to determine whether the external content has been changed. For example, if registered keywords 142, which were registered by the content creator or automatically generated when the external content was deemed acceptable, are identified by the data analysis unit 138, the verification unit 140 may determine that the external content has not been changed. In some embodiments, the verification unit 140 may use the registered keywords 142 and the identified keywords to generate a change score indicating the degree or amount of change in the external content.

[0032] If the verification process is performed, the content management system 136 may determine whether to alert the content creator, the reader, or both. If the content management system 136 determines that the external content has been changed (or, in some embodiments, substantially changed), the content management system 136 may alert the content creator. This allows the content creator to update the web page 132 as desired. The content creator may also review the external content, determine that the changes are acceptable, and re-verify the external content. Similarly, the content management system 136 may alert the reader that the external content has been changed. Specific embodiments for alerting the reader are described herein and include, but are not limited to, adding an icon or dialog box to the external link 134 to indicate the change.

[0033] Figure 1 illustrates a computing environment 100 having one host device 122 and two remote devices 102 and 112, but a suitable computing environment for implementing embodiments of the present disclosure may include any number of remote and host devices. The various models, modules, systems, and components illustrated in Figure 1 may exist across multiple host and remote devices, if present. For example, in some embodiments, two host devices may be included. These two host devices may be connected in a communicative manner using any suitable communication connection (e.g., WAN, LAN, wired connection, intranet, or internet). For example, these host devices 122 may be part of a content delivery network (CDN), and these host devices 122 may contain copies of the same data (e.g., web pages).

[0034] Figure 1 is intended to illustrate the typical main components of a computing environment 100 as an example. However, in some embodiments, individual components may be more complex or simpler than those shown in Figure 1. Furthermore, there may be components other than those shown in Figure 1, or other components in addition to those shown in Figure 1. The number, types, and configurations of such components may vary.

[0035] Next, Figure 2 is a flowchart of Method 200 as an example of registering keywords for external links embedded in content in a content management system according to an embodiment of the present disclosure. Method 200 can be performed by hardware, firmware, software running on a processor, or any combination thereof. For example, Method 200 can be performed by a processor (e.g., in a host device 122). Method 200 may first perform step 202. In step 202, an external link is added to the web page.

[0036] As described herein, web pages may be hosted by or otherwise managed by a content management system. External links may be added to the anchor text. External links may also point to resources outside the content management system (whether they reside on the same system (or cluster of systems) or on a different system). Resources referenced by external links may be, for example, another web page, a document (such as a PDF document), or a data object. External links may be added to a web page by the owner of the web page (e.g., the content creator or someone who otherwise has control over the content). For example, a content creator may create or update a web page managed by a content management system and, in the process, add or otherwise embed external links into the web page.

[0037] In decision block 204, the content management system determines whether to register keywords automatically or manually. In some embodiments, anchor text may also be registered. The content management system may determine whether to register keywords automatically or manually when an external link is added to the web page (for example, according to step 202), when the content creator decides to register keywords, or at any other appropriate time.

[0038] In this specification, manual keyword registration includes a content creator (or any other user) submitting to the system, or otherwise inputting into the system, a list of keywords (e.g., one or more) relating to external content referenced by external links. The list of keywords consists of words contained in the external content, or words that otherwise describe the external content. In contrast, automatic keyword registration includes the system automatically identifying keywords, as described herein. This may be done with or without user input. For example, a content management system may generate keyword candidates and present them to the content creator for selection. Since the candidates are determined by the content management system, this is also considered automatic registration in this specification.

[0039] The decision of whether to register keywords manually or automatically may be made using user input. For example, a content management system may detect that a web page managed by the content management system contains one or more new external links. In response, the content management system may prompt or ask the user whether to register keywords automatically or manually (or, in some embodiments, whether to register any keywords related to external links at all). For example, the prompt may ask whether the user will provide a list of keywords related to external links or whether the content management system will automatically generate the keywords. If the user indicates that they will not register keywords or will register keywords manually, or if the user provides a list of keywords, the content management system may decide not to register keywords automatically (No in decision block 204). Otherwise, the content management system may decide to register keywords automatically (Yes in decision block 204).

[0040] If the content management system determines that automatic registration will not occur (No in decision block 204), the content management system may determine in decision block 210 whether or not to verify the keywords provided by the user. In other words, in decision block 210, the content management system verifies the manually provided keywords. If no keywords are provided by the content creator, or if the provided keywords cannot be verified (for example, if they do not match keywords discovered through analysis of external content), method 200 returns to step 202.

[0041] On the other hand, if the content management system can verify the manually provided keywords in the decision block 210, in step 212, the verified keywords are registered in the web cache for the web page (for example, in the keyword list for the website) and the verification date / time of the external links is updated. Registering in the web cache involves registering the verified keywords and associating the verified keywords with the external links (for example, in the form of a relational table or database). For example, a web page may have multiple external links, and each external link may contain its own set or list of verified keywords in the content management system. After registering in the web cache in step 212, method 200 may be terminated.

[0042] Returning to decision block 204, if the content management system determines that automatic keyword registration is appropriate (Yes in decision block 204), the content management system may automatically generate keyword candidates in step 206. Keyword candidates can be generated by analyzing external content using artificial intelligence (AI), natural language processing, or any other known technology for generating keywords from content (e.g., image analysis techniques for detecting objects in images or videos), or all of these.

[0043] For example, in some embodiments, the content management system can perform website scraping to extract data from websites associated with external links. The extracted data may then be analyzed using any appropriate technology (e.g., AI technology) to identify keyword candidates. Once keyword candidates are identified in step 206, keywords are selected in step 208.

[0044] In some embodiments, keyword selection includes presenting the content creator with a list of keyword candidates and accepting the selection of one or more keywords. In other embodiments, keywords may be automatically selected by a content management system. For example, the keyword candidates generated in step 206 may describe external content separately (i.e., without considering internal content such as a web page created by the content creator). Therefore, keyword selection in step 208 may include comparing the keyword candidates with the internal content of the web page to determine which keyword candidates apply to both external and internal content. For example, the content management system may determine a set of keywords relevant to the web page and then compare that set of keywords with the keyword candidates. Matching keyword candidates may be selected in step 208. In some embodiments, fuzzy matching (e.g., matching similar keywords such as synonyms or different tenses) may be used instead of, or in addition to, strict matching.

[0045] After selecting a keyword in step 208, you may register the keyword in step 212 and then end method 200.

[0046] In some embodiments, keywords may be weighted based on whether they are optional or essential. For example, a user may provide the content management system with a list of keywords and their associated weights during manual registration. Similarly, the content management system may automatically determine the weight of a particular keyword based on its perceived importance during the keyword candidate generation stage. Alternatively, or in addition to the above, a user may assign weights to keywords generated by the content management system, for example, in selection step 208.

[0047] The weights can be binary (e.g., optional or required), ternary, a score between two endpoints (e.g., 1-100), or any other appropriate format. The weights can be used to validate keywords both when registering them in Method 200, and when verifying that external content has not been changed (or substantially changed) when a user accesses a web page (for example, in Method 300, described later with reference to Figure 3). Keyword weights can also be used to determine what notifications (if any) are needed when external content changes. Keyword weights can be stored as notification attributes for keywords in the keyword list, as detailed below.

[0048] Next, Figure 3 is a flowchart of Method 300 as an example of a method for detecting and verifying changes to data referenced by an external link, according to an embodiment of the present disclosure. Method 300 can be performed by hardware, firmware, software running on a processor, or any combination thereof. For example, Method 300 can be performed by a processor (e.g., in a host device 122). Method 300 may be performed each time a reader accesses a web page, or each time a content creator creates or edits internal content.

[0049] Method 300 may first perform step 302, in which content managed by the content management system is read. As described herein, content managed by the content management system (also called internal content) can be created by a content creator using the content management system.

[0050] In decision block 304, the content management system determines whether the content contains external links. If the content does not contain external links, method 300 proceeds to step 324, and the content is created (for example, published). After creating the content in step 324, method 300 may be terminated.

[0051] If the content contains external links, the content management system determines in decision block 306 whether the link validation is valid. If the link was recently validated, the external link validation may be considered valid. Therefore, the date / time of the link validation for the external link may be compared with a rule (e.g., setting an expiration period) to determine whether the validation is still valid. If the link validation is valid, method 300 proceeds to decision block 318. Decision block 318 will be described in more detail later.

[0052] If link validation is not enabled, the content management system retrieves details of the external link in step 308. As described herein, details of the external link may be obtained, for example, by analyzing the external link using web cache data. For example, the content management system may obtain details by sending an HTTP request with a modification query, such as an If-Modified-Since query or an Entity Tag (ETag) query or both. Cache information may be embedded in the web cache HTTP request.

[0053] In decision block 310, the content management system determines whether the external link exists as a web cache (i.e., is cached). If the content management system determines that the link exists as a web cache, that is, if it has confirmed that there have been no changes in response to an If-Modified-Since query or an Entity Tag (ETag) query or both, as described herein, the process proceeds to decision block 318 described below.

[0054] If the link does not exist as a web cache, the content management system detects that the external content linked to has been modified, retrieves the external content (for example, by scraping), and determines in judgment block 312 whether keywords related to the link exist. This may be done by comparing keywords extracted from the external content with keywords included in the keyword list related to the external link. If the keywords match, the content management system may determine that the keyword (i.e., the keyword included in the keyword list) exists in the external content. If the content management system determines that the keyword exists, method 300 proceeds to step 316 described below.

[0055] If the system determines that the keyword does not exist, it may perform synonym processing in step 314. This may determine whether the external content has been slightly modified but still contains similar keywords and is therefore likely to still describe the same concept. Synonym processing may be performed using known techniques such as known AI or natural language processing techniques.

[0056] In step 316, the content management system updates the keyword presence / absence result and the verification date / time. That is, the content management system may decide whether to re-verify external links (for example, if the keyword or its synonym is present in the external content) or not (for example, if the keyword or its synonym is not present in the external content). The content management system may also update the verification date / time to indicate whether and when the external content was re-verified.

[0057] In decision block 318, the content management system determines whether it is necessary to notify the content creator, the reader, or both of the results described in step 316. If the external content is changed, or in some cases substantially changed, as described herein, notification may be given to the content creator, the reader, or both. Whether or not to notify the content creator and the reader, and how to notify them, may be determined based, for example, on whether or not keywords are present in the external content, whether or not synonyms are present in the external content, the number or proportion of keywords present in the external content, or the attributes of the keywords (e.g., importance weight).

[0058] If notification of the results is not required, the content management system will embed (fill) the registered external links in step 320, and create content using the embedded external links in step 324. Then, method 300 will end.

[0059] If notification of the results is required, the content management system performs the notification process in step 322 and creates the content using the embedded external link in step 324. Then, method 300 is completed. Method 400, which will be explained with reference to Figure 4, shows an example of the notification process.

[0060] Next, Figure 4 is a flowchart of Method 400 as an example of notifying readers and content creators of changes to data referenced by external links. Method 400 can be implemented by hardware, firmware, software running on a processor, or any combination thereof. For example, Method 400 can be implemented by a processor (e.g., in host device 122). Method 400 can be implemented as part of step 322 of Method 300 shown in Figure 3 if changes to external content referenced by external links are detected (e.g., during the execution of steps 312-316 of Method 300).

[0061] Method 400 may first perform step 402. In step 402, the content management system notifies the content creator of the changes. The content management system may notify the content creator of the changes based on one or more notification attributes. Notification attributes may include, for example, keyword importance, the date / time of backlink verification, or upper / lower thresholds for change scores that require notification. These attributes may be used to determine whether or not to notify the content creator.

[0062] For example, if the content management system determines that required keywords (i.e., keywords with a notification attribute indicating they are required) cannot be found in the external content, it may notify the content creator of the change. On the other hand, if all required keywords are found in the external content but no optional keywords are found, it is not necessary to notify the content creator in step 402. In other embodiments, a change score may be generated using whether or not a keyword exists (or whether or not a synonym exists, or both). Then, a notification may be made based on whether or not the change score meets a threshold.

[0063] In decision block 404, the content management system determines how to notify the reader. In the example shown in Figure 4, the content management system selects from three options 405, 407, and 409. Furthermore, the content management system uses a threshold set to make this decision. However, this is merely an example, and embodiments using more or fewer options, or using different triggers in option selection, or both, are also conceivable.

[0064] If the content management system determines that external content has been modified but the change score is below the lower threshold (for example, it does not contain one or more arbitrary keywords), the content management system selects the first path 405 and, in step 406, places a warning icon next to the registered external link on the webpage. This option may be preferable for minor changes, as it is likely that the external content is still not problematic and a relatively inconspicuous warning icon is sufficient.

[0065] If the content management system determines that the change score is greater than the lower threshold but less than the upper threshold (for example, required keywords exist only as synonyms), the content management system selects a second path 407 and, in step 408, generates a warning dialog for registered external links on the webpage. This option may be preferable for moderate changes because it is unclear whether the external content is still acceptable. The warning dialog may appear next to the external link, or in some embodiments, it may act as a pop-up when the user clicks the external link.

[0066] If the content management system determines that the change score is higher than the upper threshold (for example, if there are no required keywords at all), the content management system selects a third path 409 and hides the external links in step 410. This option may be preferable for major changes where the external content is no longer trustworthy and could pose a risk to the reader. By hiding the external links, readers will not be able to retrieve or view the external content from the webpage.

[0067] Method 400 may be terminated after performing one of steps 406 to 410. If Method 400 is performed as a sub-step of Method 300, Method 400 may be terminated by proceeding to step 324 of Method 300, for example, as shown in Figure 3.

[0068] Next, Figure 5 shows an example of a computing environment 500 having a content management system 510 configured to verify external content referenced by embedded links, according to an embodiment of the present disclosure. The computing environment 500 may further include external data sources 518, 520, and 522 referenced by external links.

[0069] Content creator 502 can create a web page 512 within the content management system 510. This is indicated by arrow 550. Content creator 502 can also register a keyword list 514 and associated attributes (Attr. in the diagram) for each external link within the web page 512. This is indicated by arrow 552. For example, keyword list 1 with a first set of attributes can be associated with external link 1, and keyword list 2 with a second set of attributes can be associated with external link 2.

[0070] Readers 504 and 506 can access webpage 512, indicated by arrow 554. Depending on when the readers access the webpage, the validation engine 516 may obtain a keyword list 514 about the external link, indicated by arrow 556. The validation engine 516 may also scrape external content 518-522 associated with the external link, indicated by arrow 558.

[0071] If the validation engine 516 detects a change to the first external content 518, it adds a notification to the webpage indicating that the external content 518 associated with external link 1 has been changed. This is indicated by arrow 560. In the example shown in Figure 5, this notification takes the form of a warning icon that readers 504-506 see when they access webpage 512. The specific content of the notification can be determined using the attributes of the keyword list 1 associated with external link 1, as described herein. Furthermore, the validation engine 516 notifies the content creator 502 that the first external content 518 has been changed. This is indicated by arrow 562.

[0072] Next, Figure 6 is a system flow diagram 600 as an example illustrating a method for identifying and notifying changes to external content according to an embodiment of the present disclosure. Flow diagram 600 shows the main entities, components, and flows in an embodiment of the present disclosure. For example, flow diagram 600 shows the actions performed by a content creator 602, readers 604, 606, a content management system 608, and an external data source (e.g., an external server 610). Flow diagram 600 shows these actions over time, with time progressing from left to right in flow diagram 600.

[0073] First, content creator 602 creates content 650, such as a web page, in step 620. Content 650 includes one or more external links to external content 624 hosted by external server 610. The created content 650 is stored in the content management system 608 in step 1. Once content 650 is stored in the content management system 608, the verification engine 622 of the content management system 608 detects external links within the content 650 in step 2. Then, in step 3, the verification engine 622 scrapes the external content 624, verifies keywords, registers external links, and adds external links in step 4.

[0074] After a certain period of time has passed since content 650 was published, the first reader 604 views content 650 in step 626. This includes the first reader 604 accessing content 650 in step 5. This triggers the validation engine 622 to operate in step 6. The validation engine 622 scrapes the external content 624 and determines in step 7 that the external content 624 has not been modified. Optionally in step 8, the validation engine 622 returns a notification to the first reader 604 that the external content has been modified. In this example, since the external content 624 has not been modified, no notification is returned to the user in step 8. Instead, the user accesses the external content 624 in step 9.

[0075] The first reader 604 re-views the content 650 in step 628. This involves the first reader 604 accessing the content 650 in step 10. This triggers the validation engine 622 to operate in step 11. In step 12, the validation engine 622 scrapes the content on the external server 610 again. However, this time, the validation engine 622 discovers partially modified content 630 instead of the original external content 624. In this case, because the content has only been partially modified, the validation engine 622 returns a notification to the first reader 604 in step 13. This notification indicates that the external content linked by the web page has been changed. However, since the external content has only been partially changed, the first reader 604 can still access the partially modified content 630 in step 14.

[0076] Subsequently, the second reader 606 views the content 650 in step 632. This includes the second reader 606 accessing the content 650 in step 15. This triggers the verification engine 622 to operate in step 16. As before, the verification engine 622 scrapes the content on the external server 610 in step 17. However, at this point, the verification engine determines that the external content has been completely modified and that the external link now refers to the alternative content 634. Therefore, in step 18, the verification engine 622 notifies the second reader 606 that the content has been modified. In some embodiments, this notification includes hiding the external link from the second reader 606, thereby preventing the second reader 606 from accessing the alternative content 634.

[0077] Furthermore, in step 19, the verification engine 622 notifies the content creator 602 that the content has been substantially changed. The content creator 602 receives the notification in step 636 and updates the content 650 in step 638 by removing the external links. The updated content is then stored as content 650 in step 20.

[0078] As described herein, in some embodiments, when registering an external link in a content management system, keywords contained in the web page are registered (anchor text can also be registered). The content management system then scrapes one or more external links on behalf of a browser (for example, using a crawler) to verify whether the keywords are contained in the external content. If the scraping results in a determination that the keywords are not contained in the external content, the content management system notifies the content creator, and new keywords are registered again.

[0079] Then, changes to external content can be detected using registered keywords. Content held in the CMS is accessed (for example, when a reader accesses a web page). If the content contains external links, the external links are accessed via scraping and the keywords are validated. If the external content does not contain the registered keywords, the user is notified that the desired information may not be included in the external link. Furthermore, the content creator is notified that the external content linked to does not match the registered keywords. This may indicate that the external content has been changed.

[0080] This process has the advantage of enabling the detection of changes in external links at an early stage (for example, before any reader accesses them). This enhances the continued reliability of the content. Furthermore, embodiments of this disclosure can handle content with embedded text (such as PDFs or DOCXs). This allows the disclosure to be used even when external links refer to documents rather than just external web pages. In addition, these embodiments help verify that the linked destination (for example, the resource referenced by the external link) is secure.

[0081] Next, Figure 7 is a schematic block diagram of a computer system 701 as an example of a computer system 701 that can be used to implement one or more of the methods, tools, modules, and any related functions described herein (for example, using one or more processor circuits or computer processors of a computer) according to embodiments of the present disclosure. In some embodiments, the main components of the computer system 701 may include one or more CPUs 702, a memory subsystem 704, a terminal interface 712, a storage interface 716, an I / O (Input / Output) device interface 714, and a network interface 718. These can all be directly or indirectly coupled for communication between components via a memory bus 703, an I / O bus 708, and an I / O bus interface unit 710.

[0082] The computer system 701 may include one or more general-purpose programmable central processing units (CPUs) 702A, 702B, 702C, and 702D (collectively referred to here as CPU 702). In some embodiments, the computer system 701 may include multiple processors, as is common in relatively large systems, but in other embodiments, the computer system 701 may instead be a system with a single CPU. Each CPU 702 can execute instructions stored in the memory subsystem 704. Each CPU 702 may also include one or more levels of onboard cache.

[0083] The system memory 704 may include computer system readable media as volatile memory, such as random access memory (RAM) 722 and cache memory 724. The computer system 701 may further include other removable / non-removable volatile / non-volatile computer system storage media. For illustrative purposes only, the storage system 726 may be provided for reading and writing to non-removable non-volatile magnetic media such as a “hard drive”. Although not shown in the figures, a magnetic disk drive for reading and writing to removable non-volatile magnetic disks (e.g., floppy disks) and an optical disk drive for reading and writing to removable non-volatile optical disks (such as CD-ROMs, DVD-ROMs, or other optical media) may be provided. Furthermore, the memory 704 may include flash memory (e.g., a flash memory stick drive or flash drive). The memory device may be connected to the memory bus 703 by one or more data medium interfaces. The memory 704 may include at least one program product having a set of program modules (e.g., at least one) configured to perform the functions of various embodiments.

[0084] One or more programs / utilities 728, each having at least one set of program modules 730, can be stored in memory 704. A program / utility 728 may include a hypervisor (also called a virtual machine monitor), one or more operating systems, one or more application programs, other program modules, and program data. Each of these, or any combination thereof, may include an implementation of a network environment. The program modules 730 generally perform functions or methods of various embodiments.

[0085] In Figure 7, the memory bus 703 is illustrated as a single bus structure providing a direct communication path between the CPU 702, the memory subsystem 704, and the I / O bus interface 710. However, in some embodiments, the memory bus 703 may include multiple different buses or communication paths. These buses or communication paths can be arranged in any of various forms, such as point-to-point links in a hierarchical star or web configuration, multiple hierarchical buses, parallel and redundant paths, or any other appropriate type of configuration. Furthermore, although the I / O bus interface 710 and the I / O bus 708 are each illustrated as single units, the computer system 701 may, in some embodiments, include multiple I / O bus interface units 710, multiple I / O buses 708, or both. Additionally, while multiple I / O interface units are illustrated, they isolate the I / O bus 708 from various communication paths leading to various I / O devices. However, in other embodiments, some or all of the I / O devices may be directly connected to one or more system I / O buses.

[0086] In some embodiments, the computer system 701 may be a multi-user mainframe computer system, a single-user system, or a server computer or similar device that has little or no direct user interface but receives requests from other computer systems (clients). Furthermore, in some embodiments, the computer system 701 may be implemented as a desktop computer, a portable computer, a laptop or notebook computer, a tablet computer, a pocket computer, a telephone, a smartphone, a network switch or router, or any other suitable type of electronic device.

[0087] Figure 7 is intended to illustrate typical main components of a computer system 701 as an example. In some embodiments, individual components may be more complex or simpler than those shown in Figure 7. Furthermore, there may be components other than those shown in Figure 7, or other components in addition to those shown in Figure 7. The number, types, and configurations of such components may vary. Moreover, the above modules are listed and described illustratively according to one embodiment and are not intended to indicate the necessity of any particular module or the exclusivity of other potential modules (or functions / purposes applicable to a particular module).

[0088] While this disclosure includes a detailed description of cloud computing, it should be understood that the implementations of the teachings described herein are not limited to cloud computing environments. Rather, embodiments of the present invention can be implemented in combination with any other type of computing environment that is currently known or may be developed in the future.

[0089] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with service providers. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0090] The characteristics are as follows:

[0091] On-demand self-service: Cloud consumers can unilaterally prepare computing power, such as server time and network storage, automatically as needed, without requiring human interaction with service providers.

[0092] Broad network access: Computing power is available over the network and accessible through standard mechanisms. This facilitates utilization by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, PDAs).

[0093] Resource pooling: A provider's computing resources are pooled and delivered to multiple consumers using a multi-tenant model. Various physical and virtual resources are dynamically allocated and reallocated as needed. Generally, consumers have a sense of location independence because they do not manage or know the exact location of the resources provided. However, consumers may be able to identify the location at a higher level of abstraction (e.g., country, state, data center).

[0094] Rapid Elasticity: Computing power can be prepared quickly and flexibly, allowing it to scale out automatically and immediately, and to be quickly released and scale in immediately. To consumers, the computing power available for preparation often appears unlimited and can be purchased in any quantity at any time.

[0095] Measured Services: Cloud systems leverage metric capabilities at a certain level of abstraction, appropriate for the type of service (e.g., storage, processing, bandwidth, active user accounts), to automatically control and optimize resource usage. Resource usage can be monitored, controlled, and reported, providing transparency to both service providers and consumers.

[0096] The service model is as follows:

[0097] Software as a Service (SaaS): The functionality offered to consumers is the ability to use the provider's applications running on a cloud infrastructure. These applications can be accessed from various client devices via thin client interfaces such as web browsers (e.g., webmail). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, except for configuring a limited number of user-specific applications.

[0098] Platform as a Service (PaaS): The functionality offered to consumers is the ability to deploy applications they have created or acquired to cloud infrastructure using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, and storage, but they can control the deployed applications and, in some cases, the configuration of their hosting environment.

[0099] Infrastructure as a Service (IaaS): The functionality provided to consumers is the provision of processors, storage, networking, and other basic computing resources that enable consumers to deploy and run any software, including operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they can control the operating system, storage, and deployed applications, and in some cases, partially control certain network components (e.g., host firewalls).

[0100] The deployment model is as follows:

[0101] Private Cloud: This cloud infrastructure is operated exclusively for a specific organization. This cloud infrastructure can be managed by that organization or a third party and can reside on-premises or off-premises.

[0102] Community Cloud: This cloud infrastructure is shared by multiple organizations to support a specific community with common interests (e.g., mission, security requirements, policies, and compliance). This cloud infrastructure can be managed by the organization or a third party and can reside on-premises or off-premises.

[0103] Public Cloud: This cloud infrastructure is provided to a large number of people or large industry groups and is owned by organizations that sell cloud services.

[0104] Hybrid Cloud: This cloud infrastructure combines two or more cloud models (private, community, or public). While maintaining the unique entities of each model, they are bound together by standards or individual technologies to achieve data and application portability (e.g., cloud bursting for load balancing across clouds).

[0105] Cloud computing environments are service-oriented environments that emphasize statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is the infrastructure, which includes a network of interconnected nodes.

[0106] Here, Figure 8 shows an exemplary cloud computing environment 850. As shown in the figure, the cloud computing environment 850 includes one or more cloud computing nodes 810. Local computer devices used by cloud consumers (e.g., PDAs or mobile phones 854A, desktop computers 854B, laptop computers 854C, or automotive computer systems 854N, or a combination thereof) can communicate with these nodes. The nodes 810 can communicate with each other. The nodes 810 can be grouped physically or virtually (not shown) in one or more networks, such as the private, community, public, or hybrid clouds or a combination thereof. This allows the cloud computing environment 850 to provide infrastructure, platforms, or software as a service, or a combination thereof, without requiring cloud consumers to maintain resources on their local computer devices. Note that the types of computer devices 854A-N shown in Figure 8 are merely examples, and it should be understood that the computing nodes 810 and the cloud computing environment 850 can communicate with any type of electronic device via any type of network or network addressable connection (e.g., using a web browser) or both.

[0107] Here, Figure 9 shows the set of functional abstraction layers provided by the cloud computing environment 850 (Figure 8). It should be understood that the components, layers, and functions shown in Figure 9 are merely illustrative, and the embodiments of the present invention are not limited to these. As illustrated, the following layers and corresponding functions are provided.

[0108] The hardware and software layer 60 includes hardware components and software components. Examples of hardware components include a mainframe 61, a reduced instruction set computer (RISC) architecture-based server 62, server 63, blade server 64, storage 65, and a network and network components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0109] The virtualization layer 70 provides an abstraction layer. From this layer, virtual entities such as virtual servers 71, virtual storage 72, virtual networks 73 including virtual private networks, virtual applications and operating systems 74, and virtual clients 75 can be provided.

[0110] As an example, the management layer 80 can provide the following functions: Resource preparation 81 enables the dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing 82 enables cost tracking as resources are used within the cloud computing environment and billing or invoicing for the consumption of these resources. As an example, these resources may include licenses for application software. Security enables not only protection of data and other resources, but also identification and verification of cloud consumers and tasks. The user portal 83 provides consumers and system administrators with access to the cloud computing environment. Service level management 84 enables the allocation and management of cloud computing resources to ensure that requested service levels are met. Service Level Agreement (SLA) planning and execution 85 enables the pre-arrangement and procurement of cloud computing resources that are expected to be needed in the future in accordance with the SLA.

[0111] Workload Layer 90 provides examples of capabilities available in a cloud computing environment. Examples of workloads and capabilities available from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom education delivery 93, data analytics processing 94, transaction processing 95, and link verification 96.

[0112] The present invention may be a system, a method, a computer program product, or a combination thereof. The computer program product may include a computer-readable storage medium storing computer-readable program instructions for causing a processor to execute an aspect of the present invention.

[0113] A computer-readable storage medium can be a tangible device capable of holding and storing instructions used by an instruction execution device. Examples of computer-readable storage media may be electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or appropriate combinations thereof. More specific examples of computer-readable storage media include portable computer diskettes, hard disks, RAM, ROM, EPROM (or flash memory), SRAM, CD-ROM, DVD, memory stick, floppy disk, punch cards, or grooved raised structures, and mechanically encoded devices on which instructions are recorded, and appropriate combinations thereof. Computer-readable storage devices as used herein should not be interpreted as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0114] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computer device / processor. Alternatively, they can be downloaded to an external computer or external storage device via a network (e.g., the Internet, LAN, WAN, or wireless network, or a combination thereof). The network may include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers or edge servers, or a combination thereof. A network adapter card or network interface within each computer device / processor receives computer-readable program instructions from the network and transfers them for storage in a computer-readable storage medium in the respective computer device / processor.

[0115] The computer-readable program instructions for performing the operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk and C++, and conventional procedural programming languages ​​such as the C programming language and similar programming languages. The computer-readable program instructions can be executed as a standalone software package, either entirely on the user's computer or partially on the user's computer. Alternatively, they can be executed partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including LANs and WANs, or it may be connected to an external computer (for example, via the Internet using an Internet Service Provider). In some embodiments, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), and programmable logic arrays (PLAs), can execute computer-readable program instructions by utilizing state information of computer-readable program instructions in order to customize the electronic circuits for the purpose of performing aspects of the present invention.

[0116] Embodiments of the present invention are described herein with reference to flowcharts or block diagrams, or both, of methods, apparatus (systems), and computer program products according to embodiments of the present invention. Each block in a flowchart or block diagram, or both, and combinations of blocks in a flowchart or block diagram, or both, are executable by computer-readable program instructions.

[0117] The above computer-readable program instructions can be provided to the processor of a general-purpose computer, a dedicated computer, or other programmable data processing device for the production of a machine. This creates a means for these instructions, executed via the processor of such computer or other programmable data processing device, to perform functions / operations identified in one or more blocks in a flowchart or block diagram, or both. The above computer-readable program instructions can further be stored in a computer-readable storage medium that can be instructed to function in a particular manner to a computer, a programmable data processing device, or other device, or a combination thereof. Thus, the computer-readable storage medium on which the instructions are stored constitutes a product containing instructions for performing functions / operations identified in one or more blocks in a flowchart or block diagram, or both.

[0118] Alternatively, a computer execution process may be generated by loading computer-readable program instructions into a computer, another programmable device, or other device, and having a series of operational steps executed on that computer, other programmable device, or other device. This ensures that the instructions executed on the computer, other programmable device, or other device perform functions / operations identified in one or more blocks in a flowchart, block diagram, or both.

[0119] The flowcharts and block diagrams in the drawings of this disclosure illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, containing one or more executable instructions for performing a particular logical function. In some other implementations, the functions shown within a block may be executed in an order different from the order shown in each figure. For example, two consecutively shown blocks may actually be achieved as a single process, executed simultaneously or substantially simultaneously, executed in a partially or entirely overlapping manner in time, or, in some cases, executed in reverse order, depending on the functions involved. Each block in a block diagram or flowchart or both, and combinations of multiple blocks in a block diagram or flowchart or both, are executable by a dedicated hardware-based system that performs a particular function or operation, or executes a combination of dedicated hardware and computer instructions.

[0120] The terms used herein are intended solely to describe specific embodiments and are not intended to limit the various embodiments. In this specification, the singular forms “a,” “an,” and “the” are intended to include the plural form unless the context makes otherwise clear. Furthermore, in this specification, the terms “includes” or “including” or both specify that a described feature, integer, step, operation, element, or component or combination thereof exists, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components, or groups or combinations thereof. In the above detailed descriptions of exemplary embodiments of the various embodiments, references have been made to the accompanying drawings (similar reference numerals represent similar elements). The accompanying drawings constitute part of this specification and illustrate specific examples of embodiments that can carry out the various embodiments. While these embodiments have been described in sufficient detail to enable those skilled in the art to carry them out, other embodiments may be used, and logical, mechanical, electrical, and other forms of modifications may be made without departing from the scope of the various embodiments. In the above descriptions, numerous specific details have been provided for the full understanding of the various embodiments. However, various embodiments may be implemented without these specific details. In other examples, detailed descriptions of well-known circuits, structures, and techniques are omitted to avoid obscuring the embodiments.

[0121] In this specification, when we refer to an item as "a number of," it means one or more items. For example, "a number of different types of networks" means one or more different types of networks.

[0122] If different reference numbers consist of a common number followed by different characters (e.g., 100a, 100b, 100c), or a common number followed by punctuation followed by different numbers (e.g., 100-1, 100-2, or 100.1, 100.2), then using only the reference number (e.g., 100) without these characters or the following numbers may refer to the entire group of elements, any subset within that group, or an exemplary specimen within that group.

[0123] Furthermore, when the phrase "at least one of" is used with a list of items, it means that different combinations of one or more of the listed items are available, and each item in the list may only be needed once. In other words, "at least one" means that any combination and any number of items in the list are available, and not all items in the list are required. An item can be a specific subject, object, or category.

[0124] For example, "at least one of item A, item B, or item C" is not limiting, but could include "item A," "item A and item B," or "item B." In this example, it could further include "item A, item B, and item C," or "item B and item C." Of course, any combination of these items is possible. To give a few examples, "at least one" is not limiting, but could include, for example, "two items A, one item B, and ten items C," "four items B, seven items C," or any other appropriate combination.

[0125] In the above description, various embodiments have been referenced. However, this disclosure is not limited to the embodiments specifically described. Any combination of the described features and elements, whether relating to a different embodiment or not, is conceivable for implementing and practicing this disclosure. Many modifications, changes, and variations may be apparent to those skilled in the art, without departing from the scope and spirit of the described embodiments. Furthermore, while embodiments of this disclosure may achieve advantages over other possible solutions or prior art, whether or not a particular advantage is achieved by a given embodiment does not limit this disclosure. Accordingly, the described aspects, features, embodiments, and advantages are illustrative only and should not be considered elements or limitations of the appended claims unless expressly stated in the claims. Furthermore, the following claims are intended to be construed as encompassing all such changes and variations that fall within the true spirit and scope of the invention.

Claims

1. A method for detecting changes to external content referenced by an external link, performed by a computer, The aforementioned computer, In the first period, register keywords related to external content referenced by external links embedded in electronic documents, In response to the user accessing the electronic document, at a second time after the first time, the system scrapes the resources located at the external link, identifies the external content, and extracts the external content from the resources. A verification process is performed to determine whether the external content has been changed by comparing the extracted external content with the registered keywords. Execute, Registering the aforementioned keywords means that the computer, Scraping the resource located at the external link that identifies the external content, Using artificial intelligence to analyze the external content and extract keyword candidates from said external content, Presenting the aforementioned keyword candidates to the owner of the electronic document, The owner of the electronic document can select one or more keyword candidates as keywords for the external content, The process includes storing the selected one or more keyword candidates as keywords in the data repository of a content management system that manages the electronic documents, method.

2. The registered keywords include optional keywords and required keywords. Performing the verification process means that the computer If the aforementioned required keywords and optional keywords are present in the external content, it is determined that the external content has not been modified. If the aforementioned required keyword is present in the external content and the aforementioned optional keyword is not present in the external content, it will be determined that the external content has been partially modified. The further includes determining that the external content has been substantially altered if the aforementioned required keywords and optional keywords are not present in the external content, The method according to claim 1.

3. The computer Based on the verification process described above, it is determined that the external content has been changed, Notifying the owner of the electronic document that the aforementioned external content has been modified, The further actions include notifying the user that the external content has been changed, The method according to claim 1.

4. Notifying the user that the external content has been changed means that the computer, Add a warning icon next to the external link in the aforementioned electronic document, Adding a dialog box, which is configured as a pop-up generated when the user attempts to select the external link, to the electronic document, The aforementioned external link is hidden from the user, This includes performing one or more actions selected from the group consisting of, The method according to claim 3.

5. Memory and A system including a processor communicatively coupled to the memory, The aforementioned processor, In the first period, register keywords related to external content referenced by external links embedded in web pages, In response to a user accessing the aforementioned webpage, at a second time after the first time, the system scrapes the resources located at the external links, identifies the external content, and extracts the external content from those resources. A verification process is performed to determine whether the external content has been changed by comparing the extracted external content with the registered keywords. It is configured to perform a method that includes, Registering the aforementioned keywords means Scraping the resource located at the external link that identifies the external content, Using artificial intelligence to analyze the external content and extract keyword candidates from said external content, To present the aforementioned keyword candidates to the owner of the web page, The owner of the aforementioned webpage can select one or more keyword candidates as keywords for the aforementioned external content, This includes storing the selected one or more keyword candidates as keywords in the data repository of a content management system that manages the web page, system.

6. The registered keywords include optional keywords and required keywords. Performing the aforementioned verification process means If the aforementioned required keywords and optional keywords are present in the external content, it is determined that the external content has not been modified. If the aforementioned required keyword is present in the external content and the aforementioned optional keyword is not present in the external content, it will be determined that the external content has been partially modified. This further includes determining that the external content has been substantially altered if the aforementioned required keywords and optional keywords are not present in the external content, The system according to claim 5.

7. Based on the verification process described above, it is determined that the external content has been changed, Notifying the owner of the webpage that the aforementioned external content has been changed, Further including notifying the user that the external content has been changed, The system according to claim 5.

8. Notifying the user that the aforementioned external content has been changed is, Add a warning icon next to the external link on the aforementioned webpage, Adding a dialog box, which is configured as a pop-up generated when the user attempts to select the external link, to the web page, The aforementioned external link is hidden from the user, Includes one or more actions selected from the group consisting of, The system according to claim 7.

9. A computer program that can be executed by a computer, In the first period, register keywords related to external content referenced by external links embedded in web pages, In response to a user accessing the aforementioned webpage, at a second time after the first time, the system scrapes the resources located at the external links, identifies the external content, and extracts the external content from those resources. A verification process is performed to determine whether the external content has been changed by comparing the extracted external content with the registered keywords. Includes, Registering the aforementioned keywords means Scraping the resource located at the external link that identifies the external content, Using artificial intelligence to analyze the external content and extract keyword candidates from said external content, To present the aforementioned keyword candidates to the owner of the web page, The owner of the aforementioned webpage can select one or more keyword candidates as keywords for the aforementioned external content, This includes storing the selected one or more keyword candidates as keywords in the data repository of a content management system that manages the web page, Computer program.

10. The registered keywords include optional keywords and required keywords. Performing the aforementioned verification process means If the aforementioned required keywords and optional keywords are present in the external content, it is determined that the external content has not been modified. If the aforementioned required keyword is present in the external content and the aforementioned optional keyword is not present in the external content, it will be determined that the external content has been partially modified. This further includes determining that the external content has been substantially altered if the aforementioned required keywords and optional keywords are not present in the external content, The computer program according to claim 9.

11. Based on the verification process described above, it is determined that the external content has been changed, Notifying the owner of the webpage that the aforementioned external content has been changed, Further including notifying the user that the external content has been changed, Notifying the user that the aforementioned external content has been changed is, Add a warning icon next to the external link on the aforementioned webpage, Adding a dialog box, which is configured as a pop-up generated when the user attempts to select the external link, to the web page, The aforementioned external link is hidden from the user, Includes one or more actions selected from the group consisting of, The computer program according to claim 9.

Images