Multi-modal webpage generation method and device, equipment and medium

Abstract

The invention provides a multi-modal webpage generation method and device, equipment and a medium, and relates to the technical field of artificial intelligence, in particular to the technical field of webpage development, deep learning and the like. The method comprises the steps that in response to a received webpage generation request, a structured outline used for a webpage to be generated is generated, and the structured outline comprises respective logic placeholders and respective content requirement information of multiple pieces of multi-modal content; based on the structured outline, calling a webpage code generation model to generate a webpage code framework, the webpage code framework comprising code placeholders corresponding to the logic placeholders; based on the content requirement information, using content acquisition modules of different modalities in parallel to acquire multiple pieces of multi-modal data corresponding to the multiple pieces of multi-modal content; based on the corresponding relation between the logic placeholders and the code placeholders, the multiple pieces of multi-modal data are filled into a webpage code framework, and a target webpage is obtained.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to the fields of web development and deep learning, specifically to a multimodal web page generation method, a multimodal web page generation device, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies mainly include natural language processing, computer vision, speech recognition, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0003] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0004] This disclosure provides a multimodal webpage generation method, a multimodal webpage generation apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of this disclosure, a method for generating a multimodal webpage is provided, comprising: in response to receiving a webpage generation request, generating a structured outline for the webpage to be generated, the structured outline including logical placeholders for each of multiple multimodal contents and their respective content requirement information; based on the structured outline, calling a webpage code generation model to generate a webpage code frame, the webpage code frame including code placeholders corresponding to the logical placeholders; based on the content requirement information, using content acquisition modules of different modalities in parallel to acquire multiple multimodal data corresponding to the multiple multimodal contents; and based on the correspondence between the logical placeholders and the code placeholders, filling the multiple multimodal data into the webpage code frame to obtain a target webpage.

[0006] According to another aspect of this disclosure, a multimodal webpage generation apparatus is provided, comprising: an outline generation unit configured to generate a structured outline for a webpage to be generated in response to receiving a webpage generation request, the structured outline including logical placeholders for each of multiple multimodal content and their respective content requirement information; a webpage code frame generation unit configured to generate a webpage code frame based on the structured outline by calling a webpage code generation model, the webpage code frame including code placeholders corresponding to the logical placeholders; a multimodal data acquisition unit configured to acquire multiple multimodal data corresponding to the multiple multimodal content by using content acquisition modules of different modalities in parallel based on the content requirement information; and a data filling unit configured to fill the multiple multimodal data into the webpage code frame based on the correspondence between the logical placeholders and the code placeholders to obtain a target webpage.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the methods described above.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the above-described method.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the above-described method when executed by a processor.

[0010] According to one or more embodiments of this disclosure, by generating a structured outline of a webpage that indicates the logical location and content requirements of each multimodal content, and using this outline to perform webpage code framework generation and acquisition of each multimodal data in parallel, this disclosure decouples content data acquisition and webpage code format conversion, eliminates the serial dependency between the two in the webpage generation process, and shortens the total time spent on webpage generation. This method can also achieve parallel acquisition of data from different modalities, further improving the efficiency of webpage generation.

[0011] Furthermore, by setting logical placeholders for multimodal content in the webpage structure outline and corresponding code placeholders in the webpage code framework, the multimodal data can be filled into the appropriate positions based on the correspondence between the logical placeholders and the code placeholders after the multimodal data is obtained. This ensures the accuracy of the final target webpage at both the code and content structure levels.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0014] Figure 1 A schematic diagram of an exemplary system in which the various methods described herein may be implemented according to embodiments of the present disclosure is shown; Figure 2 A flowchart of a multimodal webpage generation method according to an embodiment of the present disclosure is shown; Figure 3 A flowchart illustrating the generation of a structured outline for a webpage to be generated, according to an embodiment of the present disclosure, is shown. Figure 4 A flowchart illustrating content requirement information for generating map content in a map modality according to an embodiment of the present disclosure is shown; Figure 5 A structural block diagram of a multimodal web page generation apparatus according to embodiments of the present disclosure is shown; and Figure 6 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0015] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0016] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0017] The terminology used in the description of the various examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0018] In related technologies, some webpage generation methods require first preparing the content needed for the webpage to be generated, and then converting this content into webpage code format to complete the webpage generation. However, such a sequential process is time-consuming and inefficient.

[0019] To address the aforementioned issues, this disclosure generates a structured webpage outline that specifies the logical location and content requirements of each multimodal content element. This outline is then used to execute webpage code framework generation and multimodal data acquisition in parallel, decoupling content data acquisition and webpage code format conversion. This eliminates the sequential dependency between these two processes during webpage generation and shortens the overall time required for webpage generation. Furthermore, this method enables parallel acquisition of data from different modalities, further improving webpage generation efficiency.

[0020] Furthermore, by setting logical placeholders for multimodal content in the webpage structure outline and corresponding code placeholders in the webpage code framework, the multimodal data can be filled into the appropriate positions based on the correspondence between the logical placeholders and the code placeholders after the multimodal data is obtained. This ensures the accuracy of the final target webpage at both the code and content structure levels.

[0021] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0022] Figure 1 A schematic diagram of an exemplary system 100 in which the various methods and apparatus described herein can be implemented according to embodiments of this disclosure is shown. Reference Figure 1 The system 100 includes one or more client devices 101, 102, 103, 104, 105 and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105 and 106 can be configured to execute one or more applications.

[0023] In embodiments of this disclosure, server 120 may run one or more services or software applications that enable the execution of the methods of this disclosure.

[0024] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtual and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as to users of client devices 101, 102, 103, 104, 105 and / or 106 under a Software as a Service (SaaS) model.

[0025] exist Figure 1 In the configuration shown, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 can sequentially interact with server 120 using one or more client applications to utilize the services provided by these components. It should be understood that various different system configurations are possible and may differ from system 100. Therefore, Figure 1 This is an example of a system used to implement the various methods described herein, and is not intended to be limiting.

[0026] Users can use client devices 101, 102, 103, 104, 105, and / or 106 for human-computer interaction. The client devices provide interfaces that enable users to interact with them. The client devices can also output information to the user through these interfaces. Although... Figure 1 Only six client devices are described, but those skilled in the art will understand that this disclosure can support any number of client devices.

[0027] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors, or other sensing devices. These computer devices can run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as Google Chrome OS); or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. Client devices are capable of executing various applications, such as various internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and can use various communication protocols.

[0028] Network 110 can be any type of network well known to those skilled in the art, and can support data communication using any of a variety of available protocols (including but not limited to TCP / IP, SNA, IPX, etc.). By way of example only, one or more networks 110 can be a local area network (LAN), an Ethernet-based network, a token ring network, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0029] Server 120 may include one or more general-purpose computers, special-purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0030] The computing unit in server 120 can run one or more operating systems, including any of the aforementioned operating systems and any commercially available server operating system. Server 120 can also run any of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0031] In some implementations, server 120 may include one or more applications to analyze and merge data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105 and / or 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105 and / or 106.

[0032] In some implementations, server 120 can be a server for a distributed system or a server integrated with blockchain. Server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0033] System 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. Databases 130 may reside in various locations. For example, a database used by server 120 may be local to server 120, or it may be located away from server 120 and may communicate with server 120 via a network-based or dedicated connection. Databases 130 may be of different types. In some embodiments, the database used by server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data from and from the databases in response to commands.

[0034] In some embodiments, one or more of the databases 130 may also be used by an application to store application data. The databases used by the application may be of different types, such as key-value stores, object stores, or regular stores supported by a file system.

[0035] Figure 1The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0036] According to one aspect of this disclosure, a method for generating multimodal web pages is provided. For example... Figure 2 As shown, the method includes: step S201, in response to receiving a webpage generation request, generating a structured outline for the webpage to be generated, the structured outline including logical placeholders and content requirement information for each of multiple multimodal contents; step S202, based on the structured outline, calling a webpage code generation model to generate a webpage code frame, the webpage code frame including code placeholders corresponding to the logical placeholders; step S203, based on the content requirement information, using content acquisition modules of different modalities in parallel to obtain multiple multimodal data corresponding to multiple multimodal contents; and step S204, based on the correspondence between logical placeholders and code placeholders, filling the multiple multimodal data into the webpage code frame to obtain the target webpage.

[0037] Therefore, this disclosure achieves decoupling between webpage code framework generation and content data acquisition by generating a structured outline that indicates the logical location and content requirements of each multimodal content element, and by using this outline to execute webpage code framework generation and multimodal data acquisition in parallel. This eliminates the serial dependency between the two in the webpage generation process and shortens the total time spent on webpage generation. This method also enables parallel acquisition of data from different modalities, further improving the efficiency of webpage generation.

[0038] Furthermore, by setting logical placeholders for multimodal content in the webpage structure outline and corresponding code placeholders in the webpage code frame, the multimodal data can be filled into the appropriate positions based on the correspondence between the logical placeholders and the code placeholders after the multimodal data is obtained, thus ensuring the accuracy of the final target webpage.

[0039] In some embodiments, in step S201, the webpage generation request may come from user input or operation, or it may be generated in a specific way, which is not limited here.

[0040] In some embodiments, the structured outline can be based on machine-readable data structures, such as JSON (JavaScript Object Notation) and XML (Extensible Markup Language). The structured outline defines the logical structure of the target webpage to be generated, the order of content, and the required multimodal content. It also guides the webpage code generation model to generate the webpage code framework in subsequent steps and calls different modal content acquisition modules to obtain the corresponding multimodal data. The structured outline can include multiple modular multimodal content items, and different multimodal content items can be combined to form logical units.

[0041] In some embodiments, multiple multimodal content may include text, images, audio, video, maps, and other modalities.

[0042] In some embodiments, logical placeholders are used to logically mark the location of a specific multimodal content (e.g., a map component or an image) within the outline. These logical placeholders serve as indexes for data population in step S204, ensuring that the multimodal data acquired in step S203 is accurately populated into the correct position within the webpage codeframe generated in step S202.

[0043] In some embodiments, the content requirement information may be a specific instruction or parameter associated with a logical placeholder, used to guide the content acquisition module in step S203. The content requirement information may have different forms and content for different modalities. In one exemplary embodiment, for the text modality, the content requirement information may be a natural language prompt word for invoking a large language model; for the map modality, the content requirement information may be a list of points of interest and travel methods for invoking the map content acquisition module; for the image modality, the content requirement information may be a keyword description for invoking an image retrieval service or an image generation service.

[0044] According to some embodiments, step S201, in response to receiving a webpage generation request, generating a structured outline for the webpage to be generated may include: based on the webpage generation request, calling a large model to generate a structured outline.

[0045] Therefore, by invoking a large model, the natural language understanding capabilities of the large model can be leveraged to fully understand the intent expressed in the webpage generation request, thereby dynamically responding to immediate and personalized requests and generating highly customized webpage content. Compared to methods that rely on fixed preset templates, this approach enhances the flexibility of webpage generation and provides richer and more accurate generated results.

[0046] Furthermore, the structured data generation and language expression capabilities of the large model provide high-quality input for downstream web page code generation models and content acquisition modules of different modalities, improving the stability of the web page generation process while enhancing the matching degree between the web page generation results and the web page generation request.

[0047] In some embodiments, the large model can be a pre-trained language model based on the Transformer architecture. The large model can be configured to understand the natural language intent in a webpage generation request to generate a machine-readable structured outline.

[0048] In some embodiments, in step S202, a structured outline can be input into the web page code generation model to obtain a web page code framework.

[0049] In some embodiments, the webpage code generation model can be a pre-trained text-based HTML code generation model. This model transforms a text-based structured outline into standard HTML, CSS, JavaScript, and other code snippets, forming a webpage containing... container, The HTML skeleton of elements such as ..., i.e., the webpage code framework.

[0050] In some embodiments, the web page code generation model can be configured to parse the logical structure of a structured outline to identify the modality types and logical placeholders of multiple multimodal content items. The web page code generation model can iterate through each multimodal content item in the structured outline and generate corresponding HTML structure tags (e.g., ...) based on the modality type of the multimodal content (e.g., text modality). <section>(Tags). At the same time, the webpage code generation model can convert logical placeholders into specific code placeholders (e.g., empty). The element (whose ID attribute is set to the ID value of a logical placeholder). Optionally, the web page code generation model can also generate other HTML tags necessary for the web page. After all HTML tags have been generated, the web page code generation model can combine all the HTML tags to form a complete HTML skeleton, i.e., the web page code framework.

[0051] In some embodiments, a web page code frame can be an HTML document or an HTML code snippet. The web page code frame includes code (e.g., HTML tags and CSS classes) for defining the overall layout, structure, and style of the target web page, but its content area (i.e., the area to be populated with multimodal data) is occupied by code placeholders.

[0052] In some embodiments, code placeholders are one or more specific HTML elements within the webpage code frame, which act as containers for the multimodal data acquired in step S203. The code placeholders have a one-to-one correspondence with the logical placeholders in the structured outline to ensure that, in step S204, the multimodal data can be accurately populated into the correct positions within the webpage layout.

[0053] According to some embodiments, the structured outline also includes layout style requirement information. Step S202, generating a webpage code framework based on the structured outline by calling the webpage code generation model, may include: calling the webpage code generation model to generate layout style code snippets that meet the layout style requirement information, wherein the webpage code framework may include layout style code snippets.

[0054] Therefore, by generating layout style requirement information in the structured outline and generating layout style code snippets from the webpage code generation model, the layout style and specific multimodal data of the webpage can be generated independently without interference. This achieves decoupling between the webpage presentation layer and the data layer during the generation phase, improving the flexibility and maintainability of the webpage generation process. Furthermore, when it is necessary to change the visual style of a webpage, only the webpage code generation model or layout style requirement information needs to be adjusted, without changing the multimodal content acquisition process. This reduces the computational resource consumption and time cost caused by repeatedly acquiring multimodal data, improving the overall efficiency of webpage style updates.

[0055] In some embodiments, layout style requirement information may be associated with the entire webpage to require a specific layout style for the webpage, or with a specific multimodal content to require a specific layout style for that multimodal content, or with a specific logical unit to require a specific layout style for that logical unit. In an exemplary embodiment, the structured outline may include layout style requirement information indicating that text content of a text modality is highlighted, and layout style requirement information indicating that a logical unit should be laid out with an image on the left and text on the right.

[0056] In some embodiments, the web page code generation model is configured to identify and parse layout style requirement information, and retrieve or dynamically generate corresponding layout style code snippets from a pre-defined code snippet library. Layout style code snippets can be HTML tags and CSS code.

[0057] In some embodiments, in step S203, parallel use of content acquisition modules of different modalities can refer to initiating calls to multiple content acquisition modules of different modalities simultaneously or within the same time window, without waiting for the previous call to complete.

[0058] In an exemplary embodiment, the structured outline generated in step S201 can be parsed to identify all content requirement information contained within the outline. For example, the outline may contain content requirement information for map content and content requirement information for text content. Furthermore, an independent task can be generated for each content requirement information, and these tasks can be simultaneously distributed to the corresponding content acquisition module using multithreading, asynchronous I / O, or other parallel processing mechanisms.

[0059] Therefore, by utilizing content acquisition modules of different modalities in parallel, serial waiting when acquiring data of different modalities can be avoided, so that the total time for acquiring multiple multimodal data depends on the task with the longest time, rather than the sum of the time of all tasks.

[0060] In some embodiments, the content acquisition module may be a functional unit configured to acquire specific modal data. It may be a locally executed software program, algorithm code snippet, or a client interface program or encapsulated component used to call remote servers, cloud services (SaaS), third-party API interfaces (such as map API, large model API), or may be implemented in other ways, which are not limited here.

[0061] In some embodiments, in step S204, each acquired multimodal data can be traversed to identify the logical placeholder associated with the multimodal data, and a code placeholder that corresponds to the logical placeholder (e.g., has the same ID identifier) ​​can be found in the web page code frame. After finding the corresponding code placeholder, the system injects or embeds the content of the multimodal data into the HTML structure defined by the code placeholder.

[0062] In some embodiments, the specific manner of the padding operation can be adapted to the type of multimodal data. For text-modal data, the padding operation may include inserting text strings into HTML tags that serve as code placeholders (e.g., ...). or Within the content area; for image modality data, the padding operation may include assigning the link address (URL) of the image data to the image tag (e.g., a code placeholder) The attributes (e.g., src) of the map modality; for map modal data (e.g., map code snippets), the population operation can include embedding the code snippet directly into a container tag that serves as a code placeholder, or attaching it as an executable script to the appropriate location in the web page code frame.

[0063] Therefore, by filling and assembling data on the server side based on predefined placeholder correspondences, multimodal data acquired independently and in parallel can be accurately integrated into the dynamically generated web page code framework, thereby outputting a target web page with a complete structure and correct content, without requiring complex secondary assembly or logical processing on the client side.

[0064] According to some embodiments, a webpage generation request can instruct the generation of a travel guide webpage. For example... Figure 3 As shown, step S201, in response to receiving a webpage generation request, generating a structured outline for the webpage to be generated may include: step S301, determining multiple points of interest based on the webpage generation request; and step S302, generating content requirement information for map content of the map modality based on the multiple points of interest. The content requirement information of the map content indicates path planning for the multiple points of interest. The multimodal data corresponding to the map content includes map code snippets, which are used to render the path planning results for the multiple points of interest.

[0065] Therefore, by parsing the points of interest in the webpage generation request and generating content requirement information to guide route planning, the display logic of map content can be determined in the structured outline generation stage. This instructs the subsequent content acquisition module to generate map components with route display, thereby improving the practicality and information richness of travel guide webpages.

[0066] In some embodiments, a point of interest (POI) refers to an entity in geographic space that has a specific name, location, and category attributes, such as a tourist attraction, restaurant, hotel, or transportation hub. The content requirements information for the map content may be instruction data specifically designed in a structured outline to describe the needs for generating map components, and may include references to geographic entities and definitions of the spatial relationships between these entities.

[0067] In some embodiments, in step S301, the system can perform Named Entity Recognition (NER) or keyword extraction on the webpage generation request input by the user to identify one or more specific points of interest (POIs) contained therein. In other embodiments, if the webpage generation request only contains a broad destination name (e.g., a city), the system can query a preset travel knowledge graph or database to retrieve popular recommended POIs for that destination and identify these recommended POIs as multiple POIs.

[0068] In some embodiments, in step S302, the system can create a logical unit of a specific type in the structured outline and write instructions into the content requirement information of that logical unit, indicating that multiple points of interest determined in step S301 need to be connected. For example, the content requirement information may include an ordered list of points of interest and instructions to generate connections or path planning. Based on such content requirement information, the obtained map code snippet can be configured to draw path lines connecting these points of interest in sequence on the map background during rendering.

[0069] According to some embodiments, the content requirement information of the map content can indicate route planning for multiple points of interest based on at least two modes of transportation, and the map code snippet can be used to differentiate the rendering of at least two route planning results corresponding to the at least two modes of transportation information, and overlay the rendering results of at least two route planning results.

[0070] Therefore, by overlaying multiple routes planned based on different modes of transportation on the same map and rendering them with differentiated visual styles, users can intuitively compare the spatial trajectory differences of different travel options in the same view, thereby assisting users in making travel decisions, improving the efficiency of users obtaining information, and enhancing the user experience.

[0071] In some embodiments, modes of transportation may include, but are not limited to, driving, walking, public transportation, cycling, etc. Differentiated rendering can assign different visual attributes to different path objects, which may include line color, line width, line type (such as solid line, dashed line, dotted line), or transparency. Overlay display can draw multiple path layers on the same geographic coordinate system and base map.

[0072] In some embodiments, in step S302, the list of travel modes to be compared can be specified in the content requirement information of the map content according to the user's preferences or preset rules.

[0073] In some embodiments, during the stage of parallel utilization of content acquisition modules of different modalities in step S203, after receiving the above-mentioned content requirement information, the map content acquisition module can initiate a route planning request to an external map data provider to obtain the route coordinate sequence and metadata (such as time, distance, etc.) corresponding to each mode of travel.

[0074] In some embodiments, after acquiring multiple sets of path data, the map content acquisition module can execute code generation logic to generate map code snippets. In one exemplary embodiment, the module generates a map code snippet (e.g., JavaScript code) containing drawing instructions according to preset style configuration rules. This code snippet is configured to, when executed on the client side, call the front-end map library to create path objects with different style attributes and add these objects to the same map instance, thereby achieving differentiated overlay display.

[0075] In some embodiments, the map snippet can also be configured to generate an interactive information layer on the map. When a user clicks or hovers over a path, the map can display an information box containing key metrics for that path, such as travel time, distance, or estimated cost. The map snippet can also include data for generating a sidebar comparison summary to list key metrics comparisons of different modes of transportation next to the map.

[0076] According to some embodiments, such as Figure 4 As shown, step S302, generating map content requirement information for map modalities based on multiple points of interest, includes: step S401, clustering multiple points of interest to obtain point of interest clusters; step S402, generating logical travel units for point of interest clusters; and step S403, generating logical placeholders and content requirement information for the corresponding map content for each logical travel unit corresponding to a point of interest cluster. The content requirement information of the map content corresponding to the point of interest cluster can instruct path planning for the points of interest included in the cluster. Multiple multimodal data include map code fragments corresponding to the point of interest clusters.

[0077] Therefore, by clustering scattered points of interest into clusters and generating corresponding logical itinerary units, discrete travel destinations can be organized into itineraries with logical structures, realizing the reasonable breakdown of long trips. This provides users with a phased and clearer display of local route planning, while avoiding visual clutter caused by displaying too many points of interest on a single map, thus improving the user experience.

[0078] In some embodiments, a cluster of interests refers to a set of interests grouped together based on some similarity metric. A logical itinerary unit is a structured container in a structured outline used to represent a time period or logical segment in a travel itinerary, such as "Day 1 Itinerary," "Morning Itinerary," or "City Tour Section."

[0079] In some embodiments, in step S401, a clustering algorithm can be used to process all points of interest contained in the webpage generation request. The clustering algorithm may include, but is not limited to, K-Means clustering, DBSCAN algorithm, spectral clustering algorithm, or hierarchical clustering algorithm. In an exemplary embodiment, the latitude and longitude coordinates of the points of interest can be used as feature vectors input to the clustering algorithm to group geographically close points of interest into the same cluster.

[0080] In some embodiments, in step S402, a corresponding number of logical run units can be constructed in the structured outline based on the number and characteristics of the generated interest point clusters. For example, if three interest point clusters are generated in step S401, then three logical run units, "Day 1", "Day 2" and "Day 3", can be generated in the structured outline.

[0081] In some embodiments, in step S403, the system generates map content instructions specifically for each logical trip unit. For example, for the cluster of points of interest (including points of interest A, B, and C) corresponding to "Day 1", the system creates a map logical placeholder within the "Day 1" logical structure unit and sets content requirement information, indicating that path planning will only be performed for points A, B, and C. Accordingly, the subsequently obtained map code snippets will only render the local path connecting A, B, and C and will be embedded into the "Day 1" area of ​​the guide page.

[0082] According to some embodiments, step S401, clustering multiple points of interest to obtain a cluster of points of interest, may include: clustering multiple points of interest based on at least one of the following: geographical distance between points of interest, type similarity between points of interest, and historical co-visit data of points of interest.

[0083] Therefore, the above provides a variety of ways to quantify the degree of correlation between different points of interest, improving the accuracy of clustering results and thus providing travel guides that are more in line with user needs and intentions.

[0084] In some embodiments, historical co-visit data for points of interest (POIs) can be statistically derived data on the frequency of different POIs visited by the same user within the same itinerary or time period. This data reflects the strong correlation between different POIs in actual visitor behavior. The type similarity between POIs can be the degree of proximity of the category labels (e.g., "museum," "park") of different POIs in the semantic space, which can be obtained by calculating the similarity of the encoded features of the semantic labels.

[0085] In some embodiments, clustering can also be achieved by calculating the comprehensive correlation between any two points of interest (denoted as point of interest A and point of interest B). The comprehensive correlation can be based on a weighted sum of geographical distance scores, type similarity scores, and co-visit scores.

[0086] In one exemplary embodiment, geographic distance score D geo The calculation can be based on the physical distance between points of interest A and B, and the formula can be expressed as: D geo (A, B) = 1 / (1 + distance (A, B)) in, distance (A, B) represents physical distance. The formula shows that the closer the distance, the higher the score.

[0087] In one exemplary embodiment, type similarity score D topic It can be calculated based on the semantic vectors of interest point types, for example, to calculate the cosine similarity between the feature vectors of interest point A and the feature vectors of interest point B. The calculation formula can be expressed as: D topic (A, B) = cosine_similarity ( embedding (A), embedding (B)) In one exemplary embodiment, co-visit score D behavior It can be calculated based on historical co-visit data. For example, the number of co-visits (co_visit_count) can be logarithmically normalized, and the calculation formula can be expressed as: D behavior (A, B) = log(co_visit_count(A,B) + 1) / log(max_co_visit + 1) Where max_co_visit is the maximum number of covisits.

[0088] Furthermore, the three scores can be weighted and summed based on preset weights to obtain the overall correlation degree: Affinity(A, B) = w1 × D geo + w2 × D topic + w3 × D behavior Among them, w1, w2, and w3 are weighting coefficients, and the sum of the three is 1.

[0089] The aforementioned weighting coefficients can be adaptively adjusted based on user profiles. In an exemplary embodiment, if the user profile associated with the webpage generation request is "family travel," then higher weights can be assigned to comfort-related factors (such as type similarity or specific dimensions in co-visit data). If the user profile is "backpacker," then higher weights can be assigned to cost- or geographical distance-related factors. This allows for the generation of clustering results that better reflect the preferences of specific user groups.

[0090] According to some embodiments, step S401, clustering multiple points of interest to obtain a cluster of points of interest, may include: clustering the multiple points of interest based on at least one of the following constraints: maximum travel distance constraint and maximum number of points of interest within the same logical travel unit constraint.

[0091] Therefore, by introducing constraints on physical distance or quantity during the clustering process, it is possible to avoid generating logical itinerary units that result in excessively large daily itinerary spans or overly dense attractions due to only considering correlation, thereby improving the matching degree between the generated travel guide and user requests and optimizing the user experience.

[0092] In some embodiments, the maximum travel distance constraint can be an upper limit on the total path length or coverage area between all points of interest within a logical travel unit (e.g., a single-day travel unit). The maximum number of points of interest within the same logical travel unit refers to an upper limit or range of the number of points of interest contained within a logical travel unit. The total path length, number of points of interest, or range mentioned above can be set according to requirements.

[0093] In some embodiments, when the system executes the clustering algorithm, the above constraints can be used as cluster merging termination conditions, pruning conditions, or post-processing screening conditions.

[0094] In one exemplary embodiment, a maximum daily travel distance constraint can be set (e.g., max_daily_distance ≤ 50km). During clustering, when the algorithm attempts to add a new point of interest to the current cluster, the system calculates the estimated total travel distance between all points within the cluster after adding the new point. If this distance exceeds a preset threshold, the system determines that the point does not belong to the current cluster and assigns it to a new logical travel unit.

[0095] In one exemplary embodiment, the system can set a daily attraction number constraint (e.g., 3 ≤ attractions_per_day ≤ 6) to control the number of points of interest contained in each generated cluster to remain within a preset range. If the number of attractions in a cluster exceeds the upper limit, the system can use graph cutting or other splitting algorithms to split it into two smaller sub-clusters, corresponding to two different logical travel units (e.g., "first day morning" and "first day afternoon", or "first day" and "second day").

[0096] According to some embodiments, multiple multimodal content includes text content in the text modality, and the content requirement information of the text content indicates the retrieval of travel guide text for multiple points of interest.

[0097] Therefore, by clearly defining the content requirements of the text modality in the structured outline, the subsequent content acquisition module can be guided to generate detailed text descriptions for specific points of interest or itinerary units, providing users with semantic information such as background information on attractions and travel suggestions, thereby enhancing the richness of travel guide content and improving user experience.

[0098] In some embodiments, the text content may be natural language text that describes, introduces, or evaluates points of interest or itineraries. Content requirements for the text content may include prompts to drive the text generation model, such as "Please introduce the historical background of attraction A" or "Please provide sightseeing suggestions around attraction A".

[0099] In some embodiments, in step S203, after the text content acquisition module (e.g., a large language model API) receives the content requirement information, it generates corresponding strategy text paragraphs based on the prompt words therein and returns them as multimodal data.

[0100] According to some embodiments, multiple multimodal content includes image content of the image modality, and the content requirement information of the image content indicates the acquisition of travel guide images for multiple points of interest.

[0101] Therefore, by integrating image modality content requirements into the structured outline, the subsequent content acquisition module can be guided to acquire visual materials highly relevant to the points of interest, achieving a rich combination of text and images. This helps users to more intuitively establish a sensory understanding of the travel destination, thereby enhancing the richness of the travel guide content and improving the user experience.

[0102] In some embodiments, image content may be pictures showcasing the point of interest itself, its surrounding environment, or distinctive features. Content requirements for the image content may include keyword descriptions used for image retrieval or generation.

[0103] In some embodiments, in step S203, the image content acquisition module can be a functional unit supporting a text-to-image service or an image search engine. After receiving the content request information, this module generates a new image based on keywords or retrieves an existing image from an image library, and returns the image's link address (URL) or binary data as multimodal data.

[0104] According to another aspect of this disclosure, a multimodal webpage generation apparatus is provided. For example... Figure 5 As shown, the apparatus 500 includes: an outline generation unit 510, configured to generate a structured outline for the webpage to be generated in response to receiving a webpage generation request, the structured outline including logical placeholders and content requirement information for each of multiple multimodal content; a webpage code frame generation unit 520, configured to generate a webpage code frame based on the structured outline by calling a webpage code generation model, the webpage code frame including code placeholders corresponding to the logical placeholders; a multimodal data acquisition unit 530, configured to acquire multiple multimodal data corresponding to the multiple multimodal content by using content acquisition modules of different modalities in parallel based on the content requirement information; and a data filling unit 540, configured to fill the multiple multimodal data into the webpage code frame based on the correspondence between the logical placeholders and the code placeholders to obtain the target webpage.

[0105] It is understood that the operation and effects of units 510-540 in device 500 can be referred to the description of steps S201-S204 above, and will not be repeated here.

[0106] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0107] According to embodiments of this disclosure, an electronic device, a readable storage medium, and a computer program product are also provided.

[0108] refer to Figure 6 The present invention describes a structural block diagram of an electronic device 600 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0109] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0110] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, output unit 607, storage unit 608, and communication unit 609. Input unit 606 can be any type of device capable of inputting information to electronic device 600. Input unit 606 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device, and can include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 607 can be any type of device capable of presenting information, and can include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 608 can include, but is not limited to, disk and optical disk. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and can include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0111] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods, processes, and / or processes described above. For example, in some embodiments, these methods, processes, and / or processes may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the methods, processes, and / or processes described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform these methods, processes, and / or processes by any other suitable means (e.g., by means of firmware).

[0112] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0113] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0114] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0115] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0116] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0117] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0118] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0119] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of this disclosure is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure. < / section>

Claims

1. A method for generating multimodal web pages, comprising: In response to receiving a webpage generation request, a structured outline for the webpage to be generated is generated, the structured outline including logical placeholders for each of the multiple multimodal contents and their respective content requirement information; Based on the structured outline, a web page code generation model is invoked to generate a web page code framework, the web page code framework including code placeholders corresponding to the logical placeholders; Based on the content requirement information, different modal content acquisition modules are used in parallel to obtain multiple multimodal data corresponding to the multiple multimodal contents; as well as Based on the correspondence between the logical placeholders and the code placeholders, the multiple multimodal data are filled into the webpage code frame to obtain the target webpage.

2. The method according to claim 1, wherein, The webpage generation request instructs the generation of a travel guide webpage, and the response to receiving the webpage generation request, generating a structured outline for the webpage to be generated, includes: Based on the webpage generation request, multiple points of interest are determined; and Based on the multiple points of interest, content requirement information for map content of map modality is generated. The content requirement information indicates path planning for the multiple points of interest. The multimodal data corresponding to the map content includes map code snippets, which are used to render the path planning results for the multiple points of interest.

3. The method according to claim 2, wherein, The content requirement information indicates that route planning is performed on the multiple points of interest based on at least two modes of transportation. The map code snippet is used to differentiate the rendering of at least two route planning results corresponding to the at least two modes of transportation information, and to overlay the rendering results of the at least two route planning results.

4. The method according to claim 2, wherein, The content requirements information for generating map content based on the multiple points of interest include: Cluster the multiple points of interest to obtain a cluster of points of interest; Generate the logical run units corresponding to the interest point clusters; and For each logical travel unit corresponding to the point of interest cluster, a logical placeholder and content requirement information for the corresponding map content are generated, wherein the content requirement information indicates path planning for the points of interest included in the corresponding point of interest cluster, and the multiple multimodal data includes map code fragments corresponding to the point of interest cluster.

5. The method according to claim 4, wherein, The process of clustering the multiple points of interest to obtain a cluster of points of interest includes: Cluster the multiple points of interest based on at least one of the following: geographical distance between points of interest, type similarity between points of interest, and historical co-visit data of points of interest.

6. The method according to claim 4, wherein, The process of clustering the multiple points of interest to obtain a cluster of points of interest includes: The multiple points of interest are clustered based on at least one of the following constraints: maximum travel distance constraint and maximum number of points of interest within the same logical travel unit constraint.

7. The method according to claim 2, wherein, The multiple multimodal contents include text content in the text modality, and the content requirement information of the text content indicates the acquisition of travel guide text for the multiple points of interest.

8. The method according to claim 2, wherein, The multiple multimodal contents include image content of the image modality, and the content requirement information of the image content indicates the acquisition of travel guide images for the multiple points of interest.

9. The method according to any one of claims 1-8, wherein, The step of generating a structured outline for the webpage to be generated in response to receiving a webpage generation request includes: Based on the webpage generation request, the large model is invoked to generate the structured outline.

10. The method according to any one of claims 1-8, wherein, The structured outline also includes layout style requirements information, and the step of generating a webpage code framework by calling the webpage code generation model based on the structured outline includes: The webpage code generation model is invoked to generate a layout style code snippet that meets the layout style requirements, wherein the webpage code framework includes the layout style code snippet.

11. A multimodal webpage generation device, comprising: The outline generation unit is configured to generate a structured outline for the webpage to be generated in response to receiving a webpage generation request. The structured outline includes logical placeholders for each of the multiple multimodal contents and their respective content requirement information. The web page code framework generation unit is configured to generate a web page code framework by calling the web page code generation model based on the structured outline, wherein the web page code framework includes code placeholders corresponding to the logical placeholders; The multimodal data acquisition unit is configured to acquire multiple multimodal data corresponding to the multiple multimodal contents by using content acquisition modules of different modalities in parallel based on the content requirement information. as well as The data filling unit is configured to fill the multiple multimodal data into the webpage code frame based on the correspondence between the logical placeholders and the code placeholders to obtain the target webpage.

12. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-10.

13. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-10.

14. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the method of any one of claims 1-10.

Images