Webpage custom cloning service method

Abstract

The invention provides a webpage custom cloning service method, which comprises the following steps of: collecting and deeply processing a webpage model by combining a webpage crawling technology with DOM (Document Object Model) analysis and dynamic data processing; according to the front-end visual design, the front-end visual targets of visual display of various data and dynamic interaction of contents are achieved by combining componentized development, dynamic data binding and advanced graphic rendering technologies; and the data driver decouples the content from the data source, so that the page display updates the content representation in real time according to the change of the data.

Description

Technical Field

[0001] The present invention relates to the technical fields of dynamic collection and management of web page models, user-defined content editing, data-driven code generation, and front-end visualization design, and in particular to a web page customized cloning service method. Background Art

[0002] At present, the deployment and configuration of most software are usually concentrated on a unified large platform. Although this centralized model guarantees information quality and system stability to a certain extent, it also brings some problems that cannot be ignored. For example, due to the relatively single production subject of content, the diversity and personalization of information are limited; at the same time, the centralized management of resources may lead to high maintenance costs and low resource utilization efficiency. While emphasizing control and standardization, this model is difficult to fully stimulate the creative potential and active participation enthusiasm of individual users, thus limiting the further development of the ecosystem.

[0003] In sharp contrast, in the video field, the mode of content production has gradually shifted from highly centralized to decentralized. Today, individual creators have become an indispensable and important part of the video ecosystem. Thanks to the popularity of social media and short video platforms, everyone can participate in content creation in a low-threshold way. This transformation has not only greatly enriched the types and forms of video content, making the video ecosystem diversified and dynamic, but also fully released the creative potential of individuals, allowing more ordinary users to create actual value in the process of sharing knowledge, recording life, and transmitting information.

[0004] The value brought by this decentralized content production model goes far beyond the diversity of the content itself. By empowering individuals, it enables more people to gain social recognition, economic benefits, and the satisfaction of self-expression in the process of producing content. At the same time, from the perspective of resource utilization, decentralized content production significantly reduces the dependence on a single platform or professional team, and disperses the originally concentrated resource consumption to each individual creator, achieving more efficient resource allocation and utilization. Therefore, this development model in the video field not only provides valuable reference for other industries, but also points out a direction for sustainable development for the future digital content ecology, that is, to fully respect individual creativity and build a diversified and symbiotic content production environment.

[0005] At present, although the decentralized creation model has achieved great success in some fields, most display platforms are still restricted by the traditional centralized architecture. Users often face the problems of high reliance on preset templates, lack of personalized setting permissions, and high creation thresholds when displaying content. For users who want to quickly display their own content, existing solutions often appear too complicated or inflexible. Summary of the invention

[0006] Consider applying decentralized models and concepts to other content display areas, and even further promote the release of individual creativity in content display. To this end, the present invention proposes and conceives a new user-driven content display solution to allow users to independently define web page content and complete the rapid display of content with extremely low technical barriers. The present invention takes "personalization, modularization, and visualization" as the core design concept, aiming to provide users with a flexible and powerful content creation tool so that they can freely build content pages according to their needs. Whether it is personal display, knowledge sharing, data analysis, or integrated display of multimedia content, it can quickly respond to user needs and become an accelerator for content creation.

[0007] In order to achieve the above object, the present invention specifically adopts the following technical solutions: A web page custom cloning service method, characterized in that: the web page model is collected and deeply processed by combining web page crawling technology with DOM parsing and dynamic data processing; the front-end visualization design realizes the front-end visualization goal of intuitively displaying multiple data and dynamically interacting with content by combining component development, dynamic data binding and advanced graphics rendering technology; data-driven decouples content from data sources so that page display updates content performance in real time according to data changes.

[0008] Furthermore, the web crawling technology combined with DOM parsing and dynamic data processing is specifically as follows: using a crawler tool to capture structured content in a web page, including: HTML code, DOM elements, CSS styles, and JavaScript embedded scripts; and then restoring the page model by parsing the HTML structure, CSS styles, and JavaScript logic.

[0009] Furthermore, in the process of collecting the web page model, the dynamic rendering of JavaScript in the web page is processed by simulating real user behavior to ensure that the final page content after JavaScript execution can be obtained; in the process of restoring the page model, the collected data is structured through HTML parsing and CSS style optimization technology to remove parts that are not related to the display; when parsing the HTML structure, the DOM elements in the web page are identified, and the content area that needs to be collected is located through the CSS selector; by optimizing and reconstructing the CSS rules, the style and content are decoupled.

[0010] Furthermore, the web crawling technology combined with DOM parsing and dynamic data processing also includes: filtering useless information, storing the collected data in a structured manner, converting the collected data into a standardized structure, formatting it after cleaning, and storing the collected web page model in JSON format.

[0011] Furthermore, the process of the front-end visual design includes: Infrastructure construction: Use the front-end framework Vue.js to achieve two-way data binding, and use the state management tool Vuex to achieve global data management; Data and graphics rendering: Generate visual content, including charts, maps, and interactive interfaces, based on collected dynamic data; Update and render the page content in real time through data-driven driving, decoupling the rendering process of data and graphics; User interaction design, using d3-zoom or interact.js to enable users to directly operate on graphics; Responsive and cross-device adaptation, use CSS Grid or Flexbox to achieve dynamic adjustment of page layout; combine media queries and breakpoint design to optimize display effects under different resolutions.

[0012] Furthermore, the implementation of the data-driven solution includes: Data collection, obtaining data from various data sources; Data processing, by cleaning, converting and storing data, and providing it to the front end in a standardized format; Data binding, dynamically binding processed data to page elements; Real-time update and visualization: Update the page in real time according to data changes, and enhance the display effect through visualization.

[0013] And, a content display system is constructed by the web page custom cloning service method as described above.

[0014] And, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the web page customization cloning service method described above when executing the program.

[0015] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the web page custom cloning service method described above.

[0016] Compared with the prior art, the present invention and its preferred solution take into account aspects such as dynamic collection and management of web page models, user-defined content editing, data-driven code generation, and front-end visualization design. By combining these technologies, it aims to build a flexible and efficient content display platform, where users can not only freely drag and configure page modules through a visual interface, but also use data-driven technology to achieve dynamic update and intelligent generation of content. In addition, the system supports accurate collection and secondary processing of web page models, making content display more personalized and practical. DETAILED DESCRIPTION

[0017] In order to make the features and advantages of this patent more obvious and easy to understand, the following embodiments are specifically described in detail as follows: It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0018] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0019] Taking a specific case, the implementation process of the embodiment of the present invention is introduced as follows: The design of the embodiments of the present invention focuses on the core areas of web page model collection and processing, data-driven technology, and front-end visualization design. Through the innovative integration of these technologies, it aims to achieve personalized display and dynamic management of user content, and provide efficient and flexible solutions for content creation and data display, including: 1. Web page model collection This embodiment uses web crawling technology combined with DOM parsing and dynamic data processing to achieve the collection and in-depth processing of web page models, including: A. Web page model collection: Use crawler tools to crawl web page content and extract the core DOM structure, including: The collection of web page models is the first step to achieve customized display of web pages, and the core lies in accurately capturing the structured content in the web pages. Through crawler technology, it is possible to automatically crawl the HTML code of the web page from the target website, obtain the DOM elements and CSS styles, as well as JavaScript embedded scripts and other information. Specifically, for dynamic web page content, this embodiment uses tools such as Puppeteer or Playwright, which can simulate real user behavior, process JavaScript dynamic rendering in web pages, and ensure that the final page content after JavaScript execution can be obtained. Compared with traditional static web page crawling tools, such as BeautifulSoup (Python) or Cheerio.js (Node.js), these tools have obvious advantages in handling dynamic content loading of modern Web applications.

[0020] For static web pages, the regular HTML structure can be parsed using tools such as BeautifulSoup or Cheerio.js. These tools can quickly extract core DOM nodes from HTML documents and further extract information such as titles, paragraphs, images, links, etc. By extracting this information, the captured web page content can be accurately restored, providing a solid foundation for subsequent content display.

[0021] B. Model parsing and optimization: Restore the page model by parsing the HTML structure, CSS style, and JavaScript logic, including: The collected web page model data usually contains a lot of redundant information and unnecessary codes. At this stage, this embodiment uses HTML parsing and CSS style optimization technology to structure the collected data and remove parts that are not related to the display. When parsing the HTML structure, it is first necessary to identify the main DOM elements in the web page, such as text, pictures, buttons, videos, etc. Through the CSS selector, the content area that needs to be collected can be accurately located, thereby avoiding unnecessary complex structures and styles.

[0022] CSS style is an important part of web page display, but there are often problems of redundancy and duplication. By optimizing and reconstructing CSS rules, style and content can be decoupled to avoid redundant style files affecting the loading speed and performance of web pages. This optimization process includes extracting common styles, removing duplicate CSS classes, and minimizing the correlation between style and content. For the JavaScript part, this embodiment analyzes its core logic and optimizes its functional modules to make the rendering and interaction of dynamic content more efficient and smooth.

[0023] In addition, unnecessary information such as advertisements, irrelevant scripts, and redundant styles in the page will also be filtered out at this stage. Preferably, this process can be completed by writing custom rules, which determine which content is valuable to display to users and which content can be ignored based on the structural characteristics of the web page. After removing these redundant information, the collected data can be purer and easier to process.

[0024] C. Data cleaning and formatting: Filter out useless information and store the collected data in a structured manner to facilitate subsequent operations, including: During the collection process, web page data often contains a lot of unstructured information and inconsistent data formats. At this stage, this embodiment uses data cleaning technology to convert the collected data into a standardized structure. The goal of data cleaning is to improve the accuracy and consistency of the data so that the content can be easily processed and displayed later.

[0025] The cleaning content mainly includes: removing redundant advertising content, invalid script tags, unused CSS classes, etc. from web pages; formatting data from different sources to conform to a unified data structure. For example, for inconsistent data such as date, price, and geographic location information, this embodiment converts them into a standard format by writing conversion rules to ensure data consistency.

[0026] Data formatting is also a key task in this stage, especially for dynamically loaded content (such as real-time news, stock data, weather forecast, etc.), which needs to be properly formatted and stored in a structured format (such as JSON or XML) to facilitate subsequent data processing and presentation. Through these processes, this embodiment can ensure that the collected web page content can be efficiently stored and called in a way that meets different display requirements.

[0027] D. Dynamic processing and export: Supports real-time editing of collected models, allowing users to generate new content after secondary processing, including: The ultimate goal of web page model collection is to provide users with dynamic and flexible content display tools. At this stage, this embodiment supports users to perform secondary processing on the collected web page models, allowing users to directly edit, update and customize web page content in the system. Through the integrated data processing engine, users can not only modify existing content, but also flexibly add or delete web page elements through custom modules to meet personalized display needs.

[0028] The collected web page models will be stored in JSON format. JSON is a lightweight data exchange format that is very suitable for the storage and transmission of web page models. The JSON format can clearly express the hierarchical structure of web page content, making subsequent data processing and interface rendering more efficient and convenient. At the same time, the JSON format supports high scalability, which is convenient for adding new functional modules or adjusting content structure in the future.

[0029] By exporting data, users can save the edited content model in different formats, which is convenient for subsequent display and application in other systems or platforms. In addition to the JSON format, this embodiment also supports exporting the collected web page content to HTML or other common web page formats, further enhancing the flexibility and scalability of the system.

[0030] 2. Front-end visual design The best solution requires building a flexible, intuitive and efficient system that supports dynamic rendering and interactive operations of complex data and content, while taking into account good user experience and cross-device adaptability. This embodiment combines component-based development, dynamic data binding and advanced graphics rendering technology. The goal of the front-end visualization design is to achieve intuitive display of multiple data and dynamic interaction of content.

[0031] A. Infrastructure construction: Select appropriate technical frameworks and rendering libraries to provide a stable development foundation for the visualization system, including: In order to achieve the rapid response and dynamic update of web page content, the present invention adopts Vue.js as the front-end development framework. Vue.js not only has the characteristics of lightweight and high performance, but also can provide a powerful data two-way binding function. The two-way data binding allows the user to keep the display of the interface and the data behind it updated synchronously when editing the web page content, which greatly improves the development efficiency and user experience. In addition, Vue.js has a high degree of scalability and can be easily integrated with other technologies, so it is very suitable for building a dynamic content display platform.

[0032] In order to manage the global data in the application, the present invention introduces the Vuex state management tool. Vuex can help developers centrally store the state of the application and ensure the state consistency between each component through a special mechanism. No matter which page the user operates on, as long as it involves data changes, Vuex can synchronize and update the relevant state in real time to ensure data consistency and timely feedback. Through this architectural design, the front-end interface can not only respond to the user's operating requirements, but also achieve efficient data management and page state maintenance at the system level.

[0033] B. Data and Graphics Rendering: Generate visualization content based on collected dynamic data, including charts, maps, and interactive interfaces, including: Data visualization is an important part of the present invention. Graphic libraries such as Echarts and D3.js are used to help users realize dynamic rendering and graphic display of data. As an open source visualization library, Echarts provides a variety of chart types, such as line charts, bar charts, pie charts, heat maps, etc., which can quickly convert background data into graphical display forms. D3.js is a more powerful data visualization tool, especially good at drawing complex custom graphics and interactive charts, which can dynamically adjust the graphic content according to the changes in data and enhance the user's interactive experience.

[0034] In the data and graphics rendering process, the present invention also introduces a data-driven design concept. The data provided by the user can drive the update and rendering of the page content in real time without manually modifying the HTML code. By decoupling the rendering process of data and graphics, users can more conveniently adjust the display content according to their own needs. At the same time, graphics rendering also supports interaction with user input, such as by clicking, selecting, dragging, etc. on the chart, the user can view detailed data or adjust the display content in real time.

[0035] C. User interaction design: Implement interactive functions such as dragging, zooming, filtering, etc. to enhance user experience, including: In order to improve the user experience, the present invention adds rich interactive functions to the front-end design. For example, users can flexibly layout web page content by dragging components and adjust the position and size of modules to achieve personalized page display. In order to enhance the interactivity of graphics, the present invention also supports zooming and filtering functions. Users can use the mouse wheel to zoom in and out, view detailed data of charts or maps, or accurately select data for display through filters.

[0036] To achieve these interactive functions, the present invention integrates tools such as d3-zoom and interact.js. d3-zoom is used to achieve the zooming and panning of graphics and maps, allowing users to perform refined operations in a larger range of data or maps. Interact.js is a powerful drag library that can handle complex user interaction actions such as dragging, zooming, and rotating, ensuring that graphics and components can accurately respond to user operations.

[0037] D. Responsiveness and cross-device adaptation: Optimize content layout to perform well on different resolutions and devices, including: With the popularity of mobile devices, responsive design has become a basic requirement in modern web development. In order to ensure that web pages can be displayed smoothly on various devices, the present invention realizes dynamic adjustment of page layout through technologies such as CSS Grid and Flexbox. These two technologies can automatically adjust the arrangement of web page elements according to the screen size, ensuring that the display effect of the web page can be optimized whether on a mobile phone, tablet or desktop computer.

[0038] CSS Grid allows developers to flexibly control the size and position of each part of the page by defining a grid layout of rows and columns; Flexbox can achieve flexible layout in one-dimensional space, allowing web page components to automatically adapt to the size of the container. In addition, responsive design further ensures the display effect of the page at different resolutions through media queries and breakpoint design. For example, on smaller screens, some components may be automatically stacked and displayed, while on large screens they are laid out side by side, thereby improving the user's visual experience.

[0039] 3. Data-driven Through data collection, processing and binding technology, content is dynamic and updated in real time. The core of data-driven is to decouple content from data sources, so that the page display is no longer static content, but updates the content in real time according to data changes. The solution is divided into the following four stages: A. Data collection: Obtain data from a variety of data sources, including APIs, databases, user input, etc., including: Data collection is the starting point of data-driven technology and the basis for all subsequent operations. By collecting multiple data sources, the comprehensiveness and timeliness of content display can be ensured. There are various types of data sources, including but not limited to external API interfaces, database queries, user input, file uploads, etc. In specific implementation, using RESTful API for data acquisition is the most common way, which can be seamlessly connected with various external services to ensure the real-time and accuracy of data. For example, by using Axios or Fetch technology, the system can obtain real-time news, social media dynamics, weather information, etc. from third-party data sources. For personalized data submitted by users, such as user input forms or uploaded files, the system can also flexibly receive and process these data to ensure the diversity and flexibility of data. In addition, database queries can provide historical data or accumulated analysis results to ensure the diversity of data sources.

[0040] B. Data processing: Cleaning, converting and storing data and providing it to the front end in a standardized format, including: The collected data is often in an unprocessed original state. Direct use of these data may result in redundancy, inconsistent formats, or problems that do not meet display requirements. Therefore, data processing is an important step to ensure data quality and consistency. At this stage, data cleaning and conversion are two main operations. In this embodiment, first, data cleaning technology will filter out invalid or duplicate data to ensure that subsequent operations only rely on accurate content. For example, invalid empty fields are cleared and duplicate records are removed to avoid affecting the loading and display efficiency of the page due to data redundancy. Secondly, the formatting and conversion of data ensures the consistency of different data source formats, so that they can be processed in a unified format. For example, the date format obtained from different interfaces is unified into the standard "YYYY-MM-DD" format, and numerical data is converted into units. The ultimate goal of data processing is to make the data standardized and standardized, so that subsequent data binding and display can be carried out smoothly.

[0041] C. Data binding: Dynamically bind processed data to page elements, including: Data binding is the core of data-driven technology. By binding data to page elements, the page content can be updated in real time according to data changes. As one of the modern front-end frameworks, Vue.js greatly simplifies the interaction between data and page elements through its two-way data binding feature. When the data changes, Vue.js will automatically trigger the corresponding view update, avoiding the complexity of manual DOM operation. For example, when displaying a real-time updated chart, data changes will be immediately reflected in the chart, and users can see the latest chart results without refreshing the page. Behind this process is Vue.js's virtual DOM technology, which achieves efficient page updates by rendering the changed parts into the virtual DOM, and then comparing and updating them to the actual DOM.

[0042] In addition, the implementation of data binding is not limited to the display of simple text or values, it also supports complex elements and component updates. For example, the data in multiple charts or dynamic tables will be updated synchronously with the backend data source to ensure the real-time and consistency of all interfaces. Through this technology, the content display of the web page is no longer a simple static display, but is adjusted in real time according to data changes, providing a flexible and dynamic user experience.

[0043] D. Real-time update and visualization: Update the page in real time according to data changes, and enhance the display effect through visualization, including: In order to further enhance the user experience, the present invention supports multiple interaction modes, such as data transmission between parent and child iframes to ensure dynamic update of page content. When a user inputs or operates data in an iframe, the system can transmit the data to other iframes in real time through a message transmission mechanism, thereby triggering content updates in other areas. For example, when a user modifies data in the form area, the chart area can automatically update its display results, providing a seamless interactive experience.

[0044] At the same time, the visualization effect of data is also a highlight of the present invention. Through front-end visualization technologies such as Echarts and D3.js, the system can convert the collected and processed data into easy-to-understand charts or images. For example, sales data is displayed in a variety of ways such as bar charts, line charts, and pie charts, so that users can intuitively understand the trends and changes of data. Data visualization not only enhances the readability of data, but also improves the interactivity of the page, allowing users to actively explore and operate data, thereby improving the interactivity and user stickiness of the page.

[0045] In addition, the system uses responsive design technology to adapt to multiple devices, ensuring that content can be displayed smoothly on different screen sizes and devices. Whether it is a widescreen display on a PC or a small screen on a mobile device, the page layout can be adaptively adjusted according to the resolution of the device to ensure the best display effect.

[0046] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

[0048] This patent is not limited to the above-mentioned best implementation mode. Anyone can derive other forms of web page custom cloning service methods under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention should be covered by this patent.

Claims

1. A web page custom cloning service method, characterized by: By combining web crawling technology with DOM parsing and dynamic data processing, the collection and in-depth processing of web page models can be achieved; the front-end visualization design achieves the front-end visualization goal of intuitive display of various data and dynamic interaction of content by combining component-based development, dynamic data binding and advanced graphics rendering technology; data-driven decouples content from data sources so that the page display can update content performance in real time according to data changes.

2. The web page custom cloning service method according to claim 1, characterized in that: The web crawling technology combined with DOM parsing and dynamic data processing is specifically as follows: using crawler tools to capture structured content in web pages, including: HTML code, DOM elements, CSS styles, and JavaScript embedded scripts; then restoring the page model by parsing the HTML structure, CSS styles and JavaScript logic.

3. The web page custom cloning service method according to claim 2, characterized in that: During the acquisition process of the web page model, the JavaScript dynamic rendering in the web page is processed by simulating real user behavior to ensure that the final page content after JavaScript execution can be obtained; In the process of restoring the page model, the collected data is structured through HTML parsing and CSS style optimization technology to remove parts that are not related to the display; when parsing the HTML structure, the DOM elements in the web page are identified, and the content area that needs to be collected is located through the CSS selector; by optimizing and reconstructing the CSS rules, the style and content are decoupled.

4. The web page custom cloning service method according to claim 2, characterized in that: The web crawling technology combined with DOM parsing and dynamic data processing also includes: filtering useless information, storing the collected data in a structured manner, converting the collected data into a standardized structure, formatting it after cleaning, and storing the collected web page model in JSON format.

5. The web page custom cloning service method according to claim 1, characterized in that: The process of front-end visual design includes: Infrastructure construction: Use the front-end framework Vue.js to achieve two-way data binding, and use the state management tool Vuex to achieve global data management; Data and graphics rendering: Generate visual content, including charts, maps, and interactive interfaces, based on collected dynamic data; Update and render the page content in real time through data-driven driving, decoupling the rendering process of data and graphics; User interaction design, using d3-zoom or interact.js to enable users to directly operate on graphics; Responsive and cross-device adaptation, use CSS Grid or Flexbox to achieve dynamic adjustment of page layout; combine media queries and breakpoint design to optimize display effects under different resolutions.

6. The web page custom cloning service method according to claim 1, characterized in that: The implementation of the data-driven solution includes: Data collection, obtaining data from various data sources; Data processing, by cleaning, converting and storing data, and providing it to the front end in a standardized format; Data binding, dynamically binding processed data to page elements; Real-time update and visualization: Update the page in real time according to data changes, and enhance the display effect through visualization.

7. A content display system, characterized in that: The method is constructed and obtained by the web page customization cloning service method as described in any one of claims 1 to 6.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the web page customized cloning service method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the web page customized cloning service method as described in any one of claims 1 to 6 are implemented.