Picture book generation method, device and equipment, vehicle and storage medium

Abstract

The invention relates to a picture book generation method, device and equipment, a vehicle and a storage medium. The method comprises the steps of determining context information of a vehicle in response to a user instruction; wherein the context information comprises position information and user preference information; the position information comprises landmark information, interest point information and culture information of the position of the vehicle; generating a story outline according to the context information; wherein the story outline comprises chapter title information, key plot information and time distribution information; according to the story outline and the context information, generating at least one chapter text and a stylized illustration of each chapter text; and generating and outputting a picture book according to the at least one chapter text and the stylized illustration of each chapter text. The picture book story highly matched with the driving scene can be dynamically generated and output in real time, and the immersion entertainment experience of passengers is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method, apparatus, device, vehicle, and storage medium for generating picture books. Background Technology

[0002] With the rapid development of vehicle-to-everything (V2X) technology, modern vehicles can provide passengers with personalized and immersive entertainment experiences through in-vehicle entertainment systems. For example, during vehicle operation, especially in scenarios such as long-distance travel, family trips, or children riding in the car, interactive entertainment content can be provided to passengers (especially children).

[0003] Existing in-vehicle entertainment systems mainly rely on preset content to achieve simple voice interaction; for example, providing pre-stored audio, video, or interactive games with fixed templates.

[0004] However, in the above process, the preset content is fixed and cannot be dynamically adjusted, resulting in the generated content being out of touch with the actual scenario, and thus failing to meet the needs of passengers. Summary of the Invention

[0005] One objective of this invention is to provide a picture book generation method to solve the problem that the preset content is fixed and cannot be dynamically adjusted, resulting in the generated content being out of touch with the actual scene; a second objective is to provide a picture book generation device; a third objective is to provide an electronic device; a fourth objective is to provide a vehicle; a fifth objective is to provide a computer-readable storage medium; and a sixth objective is to provide a computer program product.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A method for generating picture books, comprising:

[0008] In response to a user command, the vehicle's context information is determined; wherein, the context information includes location information and user preference information; the location information includes landmark information, points of interest information, and cultural information of the vehicle's location; the user preference information is obtained based on the user command;

[0009] Based on the context information, a story outline is generated; wherein, the story outline includes chapter title information, key plot information, and time allocation information;

[0010] Based on the story outline and the context information, at least one chapter text and a stylized illustration for each chapter text are generated; wherein, the stylized illustration includes illustration images that match the story style of the chapter text; the story style includes narrative tone, emotional atmosphere, and genre characteristics;

[0011] A picture book is generated and output based on at least one chapter text and stylized illustrations for each chapter text.

[0012] Furthermore, generating a story outline based on the context information includes:

[0013] Determine the similarity between the context information and each cached record in the preset story pool;

[0014] If the similarity is determined to be less than a preset threshold, the story outline is generated based on the context information.

[0015] Furthermore, generating the story outline based on the context information includes:

[0016] Based on user preference information and location information in the context information, prompt words are generated; wherein, the prompt words include context keywords and narrative constraints;

[0017] Generate the story outline based on the given prompts.

[0018] Furthermore, the method also includes:

[0019] If the similarity is determined to be greater than or equal to the preset threshold, the picture book is generated according to the historical picture book to which the cached record corresponding to the similarity belongs in the preset story pool.

[0020] Furthermore, the step of generating at least one chapter text and a stylized illustration for each chapter text based on the story outline and the context information includes:

[0021] Based on the story outline and the context information, generate the text of at least one chapter;

[0022] Based on the context information and the chapter text, a stylized illustration of the chapter text is generated.

[0023] Furthermore, generating the at least one chapter text based on the story outline and the context information includes:

[0024] Based on the environmental data in the context information, the story outline is expanded to generate at least one chapter text.

[0025] Furthermore, generating stylized illustrations for the chapter text based on the context information and the chapter text includes:

[0026] Generate a theme illustration for the chapter text based on the chapter text;

[0027] Based on the location information, user preference information, and real-time scene images in the context information, the style transfer of the theme illustrations of the chapter text is performed to obtain stylized illustrations of the chapter text.

[0028] Furthermore, the output picture book includes:

[0029] The output mode of the picture book is determined based on the vehicle's current remaining navigation time and / or remaining navigation distance; wherein, the output mode includes the picture book content length and / or output strategy;

[0030] Output the picture book according to its output mode.

[0031] Furthermore, the process of determining the vehicle's context information in response to a user instruction includes:

[0032] In response to a user command, multi-source heterogeneous data of the vehicle is acquired; wherein, the multi-source heterogeneous data includes the user command, the location information, vehicle data, and environmental data;

[0033] The multi-source heterogeneous data is integrated to obtain the context information of the vehicle.

[0034] Based on the aforementioned technical means, by responding to user commands, the system generates the vehicle's current contextual information, including landmark information, points of interest, and cultural information about the vehicle's location. This contextual information is then processed to generate a story outline, including chapter titles, key plot points, and time allocation information. Combining this story outline with the contextual information, the system generates the text for each chapter and its accompanying illustrations to create the final picture book, which is then displayed to passengers in real time. Furthermore, by dynamically integrating contextual information such as landmarks, points of interest, and cultural information, the system achieves dynamic generation and real-time output of the picture book, significantly enhancing the passenger's immersive entertainment experience. Dynamic adjustments to the picture book content length and a cache reuse mechanism ensure that the content adapts to different driving scenarios, balancing entertainment value with driving safety.

[0035] A picture book generation device, comprising:

[0036] A determination module is used to determine the vehicle's context information in response to a user command; wherein the context information includes location information and user preference information; the location information includes landmark information, points of interest information, and cultural information of the vehicle's location; the user preference information is obtained based on the user command;

[0037] The first generation module is used to generate a story outline based on the context information; wherein, the story outline includes chapter title information, key plot information, and time allocation information;

[0038] The second generation module is used to generate at least one chapter text and a stylized illustration for each chapter text based on the story outline and the context information; wherein, the stylized illustration includes illustration images that match the story style of the chapter text; the story style includes narrative tone, emotional atmosphere and genre characteristics;

[0039] The third generation module is used to generate and output a picture book based on the at least one chapter text and the stylized illustrations of each chapter text.

[0040] Furthermore, the first generation module is specifically used to: determine the similarity between the context information and each cached record in the preset story pool; if the similarity is determined to be less than a preset threshold, generate the story outline based on the context information.

[0041] Furthermore, the first generation module is specifically used to: generate prompt words based on user preference information and location information in the context information; wherein the prompt words include context keywords and narrative constraints; and generate the story outline based on the prompt words.

[0042] Furthermore, the first generation module is also specifically used to: if it is determined that the similarity is greater than or equal to the preset threshold, generate the picture book according to the historical picture book to which the cache record corresponding to the similarity belongs in the preset story pool.

[0043] Furthermore, the second generation module is specifically used for: generating the at least one chapter text based on the story outline and the context information; and generating a stylized illustration of the chapter text based on the context information and the chapter text.

[0044] Furthermore, the second generation module is specifically used to: expand the story outline based on the environmental data in the context information to generate the at least one chapter text.

[0045] Furthermore, the second generation module is specifically used to: generate a theme illustration for the chapter text based on the chapter text; and perform style transfer on the theme illustration for the chapter text based on the location information, user preference information, and real-time scene image in the context information to obtain a stylized illustration for the chapter text.

[0046] Furthermore, the third generation module is specifically used to: determine the output mode of the picture book based on the vehicle's current remaining navigation time and / or remaining navigation distance; wherein the output mode includes the picture book content length and / or output strategy; and output the picture book according to the output mode of the picture book.

[0047] Furthermore, the determining module is specifically used to: in response to a user instruction, acquire multi-source heterogeneous data of the vehicle; wherein the multi-source heterogeneous data includes the user instruction, the location information, vehicle data, and environmental data; and integrate the multi-source heterogeneous data to obtain the context information of the vehicle.

[0048] An electronic device includes: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the picture book generation method described above.

[0049] A vehicle includes a vehicle body and electronic equipment as described above disposed in the vehicle body.

[0050] A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the above-described picture book generation method.

[0051] A computer program product comprising a computer program that, when executed by a processor, implements the above-described picture book generation method.

[0052] The beneficial effects of this invention are:

[0053] (1) This invention achieves dynamic generation and real-time output of picture books by integrating contextual information such as user preference information, landmark information, point of interest information and cultural information in real time, which significantly enhances the immersive entertainment experience of passengers.

[0054] (2) This invention ensures that the content is adapted to different driving scenarios and user situations by dynamically adjusting the length of the picture book content and using a cache reuse mechanism, thus balancing entertainment and driving safety. Attached Figure Description

[0055] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of the present invention;

[0056] Figure 2 This is a flowchart illustrating a picture book generation method according to an embodiment of the present invention;

[0057] Figure 3 This is a flowchart illustrating another picture book generation method provided in an embodiment of the present invention;

[0058] Figure 4 This is a flowchart of a call caching business process provided in an embodiment of the present invention;

[0059] Figure 5 This is a diagram illustrating the architecture of a picture book creation system according to an embodiment of the present invention.

[0060] Figure 6 This is a flowchart of a picture book generation process provided in an embodiment of the present invention;

[0061] Figure 7 This is a schematic diagram of the structure of a picture book generation device according to an embodiment of the present invention;

[0062] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0063] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0064] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0065] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, they do not violate public order and good morals, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0066] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on user rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0067] This application applies to providing interactive entertainment content for passengers (especially children) in in-vehicle scenarios such as long-distance travel, family trips, or children riding in cars. For example, Figure 1This is a schematic diagram of an application scenario provided by an embodiment of the present invention, such as... Figure 1 As shown, based on the vehicle's Global Positioning System (GPS) location, it outputs a personalized picture book corresponding to local historical information or tourist attraction recommendations.

[0068] Based on the above scenarios, it can be seen that relying on preset content or simple location association functions can only execute basic commands (such as "play music") and cannot generate personalized narrative content.

[0069] The picture book generation method provided in this application achieves dynamic generation and real-time output of picture books by integrating contextual information such as local landmark information, points of interest information, and cultural information in real time, which significantly enhances the immersive entertainment experience for passengers.

[0070] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0071] Figure 2 This is a flowchart illustrating a picture book generation method according to an embodiment of the present invention, as shown below. Figure 2 As shown, the method includes:

[0072] 201. In response to user commands, determine the vehicle's context information; wherein, the context information includes location information and user preference information; the location information includes landmark information, points of interest information, and cultural information of the vehicle's location; the user preference information is obtained based on user commands.

[0073] For example, the execution subject of this embodiment may be an electronic device. For ease of description, the electronic device will be referred to as a device. The device may be a physical device or a virtual device that executes the picture book generation method. For example, the device may be a controller in a vehicle.

[0074] The user inputs a command into the vehicle, instructing it to generate a personalized picture book. The device receives and responds to this command by obtaining the vehicle's location (e.g., latitude and longitude) through its positioning system, and acquiring location information including landmarks (e.g., historical sites), points of interest (POIs), and cultural information (e.g., historical background). Simultaneously, the device parses and recognizes the user's command using a command parsing algorithm (e.g., speech recognition), obtaining user preference information, including preferred picture book themes and / or styles, ensuring accurate understanding of user needs. The speech recognition algorithm can be implemented using conventional techniques or designed by the user based on specific circumstances; no restrictions are placed here. The device integrates the location information and user preference information to obtain contextual information (e.g., "Location: Mountains, Style: Adventure, Time: Night"), providing scene relevance for content generation and ensuring a high degree of matching between the generated content and the driving environment, enhancing narrative immersion.

[0075] In one implementation, the user can input user commands to the vehicle via voice or touch. For example, the user can input voice commands (such as "tell a space adventure story") through the vehicle's high-sensitivity microphone.

[0076] In one implementation, the device can use the vehicle's onboard Global Positioning System (GPS) for precise positioning, obtain the latitude and longitude of the vehicle's current location in real time, and call a pre-set map application programming interface (API) to parse the latitude and longitude of the current location to obtain the location information of the current location.

[0077] 202. Generate a story outline based on the context information; the story outline includes chapter title information, key plot information, and time allocation information.

[0078] For example, the device pre-introduces a multi-AI agent collaboration mechanism, that is, multiple agents are pre-set, including an outline design agent, a story writing agent, an illustration generation agent, and an editing agent. The obtained contextual information is input into the outline design agent, which processes the contextual information through its semantic understanding and natural language processing capabilities to obtain a story outline, including chapter title information, key plot information, and time allocation information.

[0079] The chapter title information may include the titles of one or more chapters. Key plot information may include opening plot points, conflict plot points, climax plot points, and ending plot points. Time allocation information may include the picture book output duration corresponding to the story outline, the remaining navigation time and / or the remaining navigation distance; wherein, the remaining navigation time and / or the remaining navigation distance is determined based on the vehicle's navigation function being activated.

[0080] In one example, based on the Large Language Model (LLM), contextual information (such as "location: Mountain A, theme: dinosaurs, time: night") is integrated into the narrative to generate a story framework including an opening, conflict, climax, and ending. This framework is then formatted and output as a story outline in JavaScript Object Notation (JSON) format, including chapter titles, key plot points, and time allocation (e.g., 2 minutes per chapter), providing guidance for story writing. An example of this story framework could be: the opening introduces the legend of dinosaur fossils in the mountains at night; the conflict describes the protagonist's encounter with a velociraptor in a cave; the climax shows the protagonist escaping by cooperating with the velociraptor; and the ending tells of the protagonist finding a fossil treasure.

[0081] In the multi-AI agent collaboration mechanism, each agent is an independently running, pre-trained AI model focused on a specific task. The specific model training algorithm for each agent can be selected by the user according to the actual situation (from conventional technologies) or designed, without any restrictions.

[0082] 203. Based on the story outline and context information, generate at least one chapter text and a stylized illustration for each chapter text; wherein, the stylized illustration includes illustration images that match the story style of the chapter text; the story style includes narrative tone, emotional atmosphere and genre characteristics.

[0083] For example, the device invokes a story writing agent, and based on this agent, performs text processing on the generated story outline and context information to generate chapter text for each chapter title in the story outline. The device then invokes an illustration generation agent, and based on this agent, performs text processing and image generation on the chapter text for each chapter title, analyzing the story style of each chapter text. The story style includes the narrative tone, emotional atmosphere, and genre characteristics of the chapter text, such as style tags like "ancient style," "science fiction," and "children's." Based on the story style of each chapter text, an illustration image matching the story style of each chapter text is generated, resulting in a stylized illustration for each chapter text. This ensures that the character images, color schemes, and other aspects of the stylized illustration correspond to the text description of each chapter text.

[0084] Specifically, leveraging the LLM language processing capabilities of the story writing agent, and combining each chapter title, the plot points in the story outline are expanded into a coherent narrative. Simultaneously, user preference information and location information from the context are incorporated, outputting multiple chapter texts in a preset format (e.g., "Chapter 1: Nighttime Adventure"). Based on the text-to-image generation model (e.g., or optionally, a Stable Diffusion model) in the illustration generation agent, the text content of each chapter is processed. For example, based on the chapter title and text content, the story style of each chapter is determined, and illustrations matching the story style of each chapter are generated to obtain stylized illustrations for each chapter.

[0085] 204. Generate and output a picture book based on at least one chapter of text and stylized illustrations for each chapter.

[0086] For example, the device invokes an editing agent and, based on this agent, integrates and processes the generated text for each chapter and its stylized illustrations to generate a digital picture book, which is the final picture book, including the generated text for each chapter and its stylized illustrations. The device then transmits the picture book to the in-vehicle screen, allowing the screen to display the text and illustrations from the picture book chapter by chapter.

[0087] In one implementation, based on the generated multimodal picture book file (i.e., the picture book) and the metadata contained in the multimodal picture book file (such as the broadcast duration: 6 minutes), the text and illustrations in the multimodal picture book file are displayed on the in-vehicle screen; and based on the metadata, a preset text-to-speech (TTS) service is called to convert the text of each chapter in the multimodal picture book file into audio for broadcast.

[0088] This embodiment provides a picture book generation method that generates picture book content with a high degree of scene matching by integrating user preference information and real-time location landmark information, point of interest information and cultural information; enhances the narrative immersion through the collaborative generation and synchronous output of text and illustrations; and further, realizes the dynamic generation and real-time output of picture books through multi-AI agent collaboration and dynamic context processing mechanism, significantly improving the immersive entertainment experience of passengers.

[0089] Figure 3 This is a flowchart illustrating another picture book generation method provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the method includes:

[0090] 301. In response to user commands, acquire multi-source heterogeneous data of the vehicle; wherein, multi-source heterogeneous data includes user commands, location information, vehicle data and environmental data.

[0091] For example, a user inputs a user command into the vehicle to instruct it to output a personalized picture book. The device receives and responds to this user command by acquiring the command and using the vehicle's positioning system to determine its location (e.g., latitude and longitude). It then obtains location information, including landmarks (e.g., historical sites), points of interest (POIs), and cultural information (e.g., historical and cultural background). Through the vehicle's on-board diagnostics (OBD) system, it acquires vehicle data and environmental data, thereby obtaining multi-source heterogeneous data about the vehicle.

[0092] Vehicle data includes, but is not limited to, vehicle speed. Environmental data includes, but is not limited to, weather data, temperature data, and light intensity data.

[0093] In one implementation, the current latitude and longitude (e.g., 31.23°N, 118.37°E) is obtained through the vehicle's GPS module; the landmarks within a 5km radius are parsed through the map API, such as identifying "geological park" as the main landmark, and then the knowledge graph database is called to extract relevant information about the landmark (e.g., "myths and legends" "geological history") and generate landmark information; if there are no significant landmarks, a general environmental description (e.g., "mountainous area") can be used by default.

[0094] In one implementation, real-time vehicle data is collected via an OBD sensor array, including vehicle speed (60 km / h, accuracy ±1 km / h) and environmental data, including external temperature (22℃, accuracy ±0.5℃), weather (light rain), and time (night). The collected data is preprocessed (using normalization and filtering algorithms) to obtain data formatted as a JSON structure (e.g., {"speed": 60, "temperature": 22, "weather": "light rain", "time": "night"}). The normalization and filtering algorithms can employ conventional techniques or be designed by the user based on specific circumstances; no restrictions are imposed here.

[0095] 302. Integrate multi-source heterogeneous data to obtain vehicle context information.

[0096] For example, keywords are extracted from location information, environmental data, user commands, and vehicle data in multi-source heterogeneous data to obtain keywords in location information, environmental data, user commands, and vehicle data. These keywords are then combined to obtain unified contextual information (e.g., "Location: Mountain A, Theme: Dinosaur, Weather: Light Rain, Time: Night").

[0097] In one implementation, user preferences are obtained by parsing and recognizing user commands, including preferred picture book themes and / or preferred picture book styles, ensuring that user needs are accurately understood. For example, the voice command can be recognized using a speech recognition algorithm to obtain user preference information (e.g., tone: enthusiastic, style: adventurous). Keywords are extracted from user preference information, location information, environmental data, user commands, and vehicle data, and the extracted keywords are combined to obtain unified contextual information. The speech recognition algorithm can employ conventional techniques or be designed by the user based on actual circumstances; no restrictions are imposed here.

[0098] For example, the theme ([Dinosaur, Adventure]), landmark ([Mount A]), and environment ([Light Rain, Night]) from user preference information are combined to generate contextual information. Among them, if the landmark is highly related to the theme (such as "Location A" and "Dinosaur"), the landmark elements are incorporated first (such as "Dinosaur Fossil Site in Location A"), and finally the contextual information is output (such as "Location: Mountain A, Theme: Dinosaur, Weather: Light Rain, Time: Night").

[0099] By utilizing real-time vehicle location, speed, and weather data to generate contextual information, which can then be used to generate story content relevant to the current environment through multi-agent collaboration, personalization and scene relevance can be further enhanced, thereby significantly improving the immersive entertainment experience for passengers.

[0100] 303. Determine the similarity between the context information and each cached record in the preset story pool.

[0101] For example, the device introduces a caching mechanism (such as the Least Recently Used (LRU) strategy) to cache each generated historical picture book in a preset story pool, and generates a cache record for each corresponding historical picture book, including multi-dimensional content such as the theme, landmarks, and environment of the historical picture book. After the device generates the current context information, it calls the preset story pool and retrieves each cached record in the preset story pool. A similarity algorithm is then used to calculate the similarity between the context information and each cached record.

[0102] Specifically, the embedding model can be used to vectorize the context information and each cached record. Then, the cosine similarity algorithm is used to calculate the similarity between the vectorized context information and each vectorized cached record.

[0103] 304. If the similarity is determined to be less than the preset threshold, a story outline is generated based on the context information.

[0104] For example, the device invokes a preset threshold (determined by the user based on actual conditions, for example, it can be set to 0.8) and compares each similarity with the preset threshold. If it is determined that the similarity is less than the preset threshold, the multi-AIAgent mechanism is triggered, and the context information is input into the outline design agent. Through the semantic understanding and natural language processing capabilities of the outline design agent, the context information is processed to obtain the story outline, including chapter title information, key plot information, and time allocation information.

[0105] In one possible implementation, step 304 includes: if the similarity is determined to be greater than or equal to a preset threshold, generating a picture book based on the historical picture book to which the cached record corresponding to the similarity in the preset story pool belongs.

[0106] Specifically, each similarity is compared with a preset threshold. If the similarity is greater than or equal to the preset threshold, the cached record corresponding to the similarity is determined from the preset story pool, and a picture book that fits the current context is generated based on the story content of the historical picture book.

[0107] For example, Figure 4 A call caching business process diagram provided in an embodiment of the present invention, such as... Figure 4As shown, the device receives a new request and generates context information (e.g., "Location: Mountain A, Theme: Dinosaur, Weather: Light Rain, Time: Night") based on the request. The device uses an embedding model to vectorize the context information, converting it into corresponding vector data (dimension: 768). Based on the cache management unit and a similarity evaluation mechanism, the device calculates the cosine similarity between the vector data corresponding to the current context information and the vector data corresponding to cached records in the preset story pool. If the similarity for the cached record "Dinosaur + Mountain A + Rainy Day" is found to be 0.87, and the preset threshold is 0.80, then the similarity is ≥0.8, indicating a cache hit. The editing agent directly calls the historical picture book corresponding to the cached record "Dinosaur + Mountain A + Rainy Day" as the picture book generated for the current scene and outputs it. If the similarity for each cached record is <0.80 (e.g., the theme changes to "Space Adventure"), then a cache miss is detected, triggering a multi-AI agent collaboration mechanism to generate a story outline based on the context information, thus generating a new picture book. After a new picture book is generated, it is stored in a preset story pool and assigned a unique identity document (ID), such as HM002. The preset story pool uses an LRU (Least Recently Used) strategy and supports offline updates. Specifically, based on a cache update and optimization strategy, the cache management unit periodically analyzes the usage frequency of each historical picture book in the preset story pool using the Least Frequently Used (LFU) algorithm, prioritizing the retention of high-frequency scenarios (such as "Mount A + Dinosaur"); and / or, if a landmark data update is detected (such as a new POI for Mount A), a cache refresh is automatically triggered, i.e., the story design agent is invoked to update the outlines of the relevant historical picture books in the preset story pool. The LFU algorithm is set by the user according to actual conditions and is not restricted here.

[0108] By employing a context-based caching mechanism and a similarity evaluation method, efficient story reuse is achieved. Specifically, context similarity evaluation reuses historical picture books with high similarity in cached records, reducing real-time generation load and significantly improving system response efficiency. The cache reuse strategy adapts to dynamic driving scenarios (such as sudden weather changes). After the vehicle-mounted perception module (such as a light sensor) identifies a change in the driving scenario, new context information corresponding to this new driving scenario is generated. Then, using a similarity evaluation method, cached records with high similarity to this new context information are identified from a pre-set story pool as target cached records. The historical picture book corresponding to the target cached record is then retrieved from the pre-set story pool and output as the final picture book, ensuring that the picture book content matches the environment while reducing computational resource consumption.

[0109] In one possible implementation, step 304, generating the story outline based on context information, includes the following steps:

[0110] The first step is to generate prompts based on user preference and location information from the context; these prompts include contextual keywords and narrative constraints.

[0111] The second step is to generate a story outline based on the prompts.

[0112] Specifically, based on the preset prompt word engineering, user preference information and location information in the context are processed. Specifically, keywords from user preference information and location information are extracted and combined to generate prompt words. These prompt words include contextual keywords (including keywords from user preference information and location information) and narrative constraints. The contextual keywords and narrative constraints from the prompt words are input into the LLM in the outline design agent for text processing, outputting a story outline including chapter titles, key plot points, and time allocations to ensure the story outline is coherent and thematically consistent.

[0113] For example, based on a large language model, user preference information, environmental data, and location information in the context are processed to generate a structured story outline, such as containing four chapters: Opening (1 minute): A rainy night on Mountain A, the protagonist discovers a legend of dinosaur fossils; Conflict (2 minutes): The protagonist enters a mysterious cave and encounters a dragon; Climax (2 minutes): In a storm, the protagonist and the dragon work together to escape; Ending (1 minute): The rain stops and the sky clears, the protagonist finds the fossil treasure. The story outline includes time allocation information (e.g., total duration 6 minutes, remaining time for matching navigation).

[0114] The structured narrative outline generation process ensures that the generated outline has logical coherence (such as chapter allocation matching navigation time), guaranteeing a high degree of matching between the subsequently generated picture book content and the driving environment (such as a light rainy night on Mount A), further enhancing the passenger's immersive experience.

[0115] 305. Generate at least one chapter text based on the story outline and context information.

[0116] For example, the device invokes the story writing agent, and based on the story writing agent, performs text processing on the generated story outline and context information to generate chapter text for each chapter title in the story outline.

[0117] Specifically, leveraging the LLM language processing capabilities of the story writing agent, and combining each chapter title, the plot points in the story outline are expanded into a coherent narrative. Simultaneously, based on the story writing agent, user preference information, location information, and vehicle data within the context are processed to determine the narrative elements corresponding to user preference information, location information, and vehicle data. For example, "nighttime" is mapped to the narrative element "mysterious atmosphere under the moonlight." These obtained narrative elements are then integrated into the coherent narrative, resulting in the chapter text for each chapter title in the story outline.

[0118] In one possible implementation, step 305 includes the following steps: expanding the story outline based on environmental data in the context information to generate at least one chapter text.

[0119] Specifically, leveraging the LLM's language processing capabilities within the story writing agent, and combining each chapter title, the plot points in the story outline are expanded into a coherent narrative. Simultaneously, based on the story writing agent, environmental data within the context is processed to determine the corresponding narrative elements. For example, "rainy day" is transformed into "a chase in a storm," and these corresponding narrative elements are integrated into the coherent narrative, resulting in the chapter text for each chapter title in the story outline. For instance, the first chapter text is: "On a rainy night in Mountain A, moonlight shines on the ancient mountains, and the young explorer, Xiao X, hears the whispers of fossil legends…".

[0120] By expanding the story outline into a coherent narrative and incorporating narrative elements corresponding to environmental data in the context, the generated content is ensured to be highly matched with the driving environment, enhancing the relevance of the picture book narrative to the user's scenario, thereby further improving the narrative immersion.

[0121] 306. Generate stylized illustrations for the chapter text based on contextual information and chapter text.

[0122] For example, the device calls the illustration to generate an Agent, and generates an Agent based on the illustration. It then performs text processing and image generation processing on the chapter text of each chapter title and context information to obtain a stylized illustration of each chapter text.

[0123] Specifically, based on the text-to-image generation model (e.g., or optionally the StableDiffusion model) in the illustration generation agent, the text content in each chapter text, as well as the location information, user preference information, environmental data, etc. in the context information, are processed to determine the story style of each chapter text, including the narrative tone, emotional atmosphere and type characteristics of each chapter text. Then, according to the story style of each chapter text, stylized illustrations for each chapter text are generated.

[0124] By extending the narrative with chapter-by-chapter text, the narrative is deeply integrated with user commands (such as "Dinosaur Adventure"). By generating chapter-by-chapter text images, the visual immersion can be enhanced (such as "Raptor Illustration"), which can further enhance the passenger's immersive experience.

[0125] In one possible implementation, step 306 includes the following steps:

[0126] The first step is to generate a theme illustration for each chapter based on the chapter text.

[0127] The second step involves performing style transfer on the theme illustrations of the chapter text based on the location information, user instructions, and real-time scene images in the context, resulting in stylized illustrations for the chapter text.

[0128] Specifically, the illustration generation agent generates image generation instructions for each chapter's text (e.g., "Generate an illustration of a velociraptor in a cave in Mountain A, in a prehistoric style"). Based on the text-to-image generation model within the illustration generation agent, the text content of these instructions is processed to obtain a corresponding themed illustration, thus matching the text theme of each chapter. The illustration generation agent also acquires location information, user preference information, and real-time scene images from the context information and performs multimodal processing on these elements to identify the illustration style. Finally, using the image-to-image technology within the illustration generation agent, the identified illustration style is used to perform style transfer on the themed illustrations for each chapter's text, for example, converting them from realistic photographs to a cartoon style, resulting in a stylized illustration for each chapter.

[0129] In one example, an illustration-based agent generates an image generation instruction for each chapter of text (e.g., "Generate an illustration of a velociraptor in a cave in Mountain A, in a prehistoric style"). Based on this illustration, the agent generates prompts in response to the image generation instructions for each chapter. It then performs multimodal processing on the text content of each chapter, along with contextual information such as location, user preferences, and real-time scene images, to generate prompts for each chapter (e.g., "You are an AI assistant proficient in text analysis and visual arts. Your task is to read a given article and create a concise and powerful English prompt to generate an illustration containing the main elements. Note that the generated illustration should be in a picture book style, warm and childlike, with soft colors, delicate illustrations, fairytale elements, and rich details, suitable for children's storybooks. Follow these steps…"). Based on the illustration-generated agent, the system generates a theme illustration for each chapter's text according to the corresponding prompts. Then, based on the illustration style in the prompts, it performs style transfer on the theme illustrations for each chapter, resulting in a stylized illustration for each chapter. For example, it generates a prehistoric-style picture book cover, and the illustration is enhanced with super-resolution (e.g., 1920x1080 resolution) to ensure clarity. The prompt engineering and image generation techniques involved in this embodiment can be pre-defined by the user according to actual needs, and are not limited here.

[0130] By generating stylized illustrations from text and images, and then combining these with the text to create subsequent picture book generation and synchronous output, we can improve the visual effects and user experience.

[0131] 307. Generate a picture book based on at least one chapter of text and stylized illustrations for each chapter.

[0132] For example, this step can be referred to as step 204, which will not be repeated here.

[0133] 308. Determine the output mode of the picture book based on the vehicle's current remaining navigation time and / or remaining navigation distance; wherein, the output mode includes the length of the picture book content and / or the output strategy.

[0134] For example, when the navigation function is activated, the device obtains the vehicle's current remaining navigation time (e.g., 5 minutes) and / or remaining navigation distance (e.g., 5 km) in real time. Based on the current remaining navigation time and / or remaining navigation distance, the device can dynamically adjust the length of the picture book content and / or the playback speed and playback duration in the output strategy to determine the output mode of the picture book, including the final length of the picture book content and / or the output strategy.

[0135] In one example, if the remaining navigation time / distance to the destination shown by the navigation is less than a corresponding preset threshold, then minor plot points of the remaining text in each chapter of the picture book are trimmed; for example, if the navigation shows a distance to the destination of <5km, the editor agent compresses the text content of the picture book (e.g., trims it to 4 minutes). And / or, based on the speech synthesis service, using a tone matching the theme (e.g., an enthusiastic tone with a pitch +10% and a speech rate 1.0x), or the user-specified voice preferences, voice broadcast content and corresponding broadcasting methods matching each chapter of the picture book are generated.

[0136] 309. Output the picture book according to the picture book output mode.

[0137] For example, the device outputs the text content and illustrations of the picture book synchronously according to the final determined output mode of the picture book. For instance, while displaying the illustrations of the picture book on the in-vehicle screen, the text of the picture book is played aloud through the audio system. Furthermore, when playing the audio through the vehicle's audio system, voice commands (such as "pause" or "turn up the volume") can be used to control the audio playback method.

[0138] For example, Figure 5 A diagram illustrating the architecture of a picture book creation system provided in an embodiment of the present invention is shown below. Figure 5 As shown, the data acquisition module collects data from the vehicle's GPS, OBD, and microphone, simultaneously acquiring real-time vehicle data. The data processing module then processes this data to obtain contextual information, including location information and images. The content generation module uses an outline design agent to process the contextual information, generating a JSON-formatted story outline. A story writing agent then processes the outline to produce Markdown-formatted story text (including multiple chapters). An illustration generation agent processes the story text to generate text and illustrations. An editing agent then edits the text and illustrations to obtain the final digital picture book. Finally, the output control module's speech synthesis unit uses a multi-emotional text-to-speech (TTS) service to output the digital picture book's content. Simultaneously, the cache management unit's LRU mechanism caches this digital picture book for subsequent new picture book generation processes, enabling cosine similarity calculations and picture book reuse, thus reducing computational resources.

[0139] For another example, Figure 6 This is a flowchart of a picture book generation process provided in an embodiment of the present invention, such as... Figure 6As shown, the device receives user voice commands, collects GPS data to obtain latitude and longitude, and parses landmarks to obtain location and landmark information. It also collects environmental and vehicle data to obtain information such as vehicle speed and weather. The device integrates the collected user voice commands with the corresponding themes, landmarks, and environment to construct contextual information. If the current picture book generation request is a new request, the contextual information is vectorized based on a cache similarity evaluation mechanism. The similarity between the vectorized contextual information and each cached record is calculated and compared with a threshold of 0.8. If the similarity is less than 0.8, it indicates a cache miss. Based on a multi-agent collaborative generation mechanism, an outline design agent processes contextual information to generate a story outline (e.g., a JSON-formatted outline with four chapters). A story writing agent processes the outline to generate multiple chapter texts (e.g., Markdown-formatted text, such as "Chapter 1: XX"). An illustration generation agent processes each chapter text to generate corresponding illustrations. An editing agent integrates the chapter text and illustrations to obtain the digital picture book. Simultaneously, the output duration of the digital picture book can be trimmed based on the remaining navigation time to generate the final picture book. If the similarity is greater than or equal to 0.8, it indicates a cache hit. The cached historical picture book corresponding to this similarity is used as the final picture book. The final picture book's text and illustrations are displayed, and the text content is simultaneously read aloud via TTS (Text-to-Speech) to complete the picture book output and playback. When a cache miss occurs (similarity less than 0.8), the generated new picture book is stored in the story pool, and a corresponding ID (e.g., HM002) is generated for storage.

[0140] By dynamically adjusting the picture book's output mode based on remaining navigation time and / or remaining navigation distance, the story is ensured to be fully narrated before reaching the destination, improving narrative integrity and user satisfaction. As the destination approaches, the narration speed can adaptively accelerate, enhancing the overall user experience. This mechanism overcomes the limitations of fixed-length content and adapts to different driving scenarios.

[0141] In this embodiment, based on the above embodiments, on the one hand, by fusing vehicle network data and user themes in real time, picture book content highly relevant to the driving environment is generated, enhancing the connection between the narrative and the environment. Combining image-to-image technology from vehicle camera photos with emotional voice output can improve user immersion. On the other hand, through a distributed multi-AI Agent architecture (story design agent, story writing agent, image generation agent, and editing agent), the picture book generation task is decomposed into parallel sub-tasks, shortening the overall generation time. Each agent focuses on a specific function (such as outline generation, text expansion, and illustration generation). The modular design reduces system coupling and facilitates individual optimization or replacement of modules. For example, the image generation agent can independently switch between image-to-image models without affecting the functions of other modules.

[0142] Figure 7 This is a schematic diagram of the structure of a picture book generation device according to an embodiment of the present invention, as shown below. Figure 7 As shown, the device includes:

[0143] The determination module 401 is used to determine the context information of the vehicle in response to a user command; wherein, the context information includes location information and user preference information; the location information includes landmark information, point of interest information, and cultural information of the vehicle's location; the user preference information is obtained based on the user command;

[0144] The first generation module 402 is used to generate a story outline based on context information; wherein, the story outline includes chapter title information, key plot information, and time allocation information;

[0145] The second generation module 403 is used to generate at least one chapter text and a stylized illustration for each chapter text based on the story outline and context information; wherein, the stylized illustration includes illustration images that match the story style of the chapter text; the story style includes narrative tone, emotional atmosphere and genre characteristics;

[0146] The third generation module 404 is used to generate and output a picture book based on at least one chapter of text and stylized illustrations for each chapter of text.

[0147] Furthermore, the first generation module 402 is specifically used to: determine the similarity between the context information and each cached record in the preset story pool; if the similarity is determined to be less than a preset threshold, generate a story outline based on the context information.

[0148] Furthermore, the first generation module 402 is specifically used to: generate prompt words based on user preference information and location information in the context information; wherein, the prompt words include context keywords and narrative constraints; and generate a story outline based on the prompt words.

[0149] Furthermore, the first generation module 402 is also specifically used to: if the similarity is determined to be greater than or equal to a preset threshold, generate a picture book based on the historical picture book to which the cached record corresponding to the similarity in the preset story pool belongs.

[0150] Furthermore, the second generation module 403 is specifically used to: generate at least one chapter text based on the story outline and context information; and generate stylized illustrations of the chapter text based on the context information and the chapter text.

[0151] Furthermore, the second generation module 403 is specifically used to: expand the story outline based on the environmental data in the context information, and generate at least one chapter text.

[0152] Furthermore, the second generation module 403 is specifically used to: generate a theme illustration for the chapter text based on the chapter text; and perform style transfer on the theme illustration for the chapter text based on the location information, user preference information, and real-time scene images in the context information to obtain a stylized illustration for the chapter text.

[0153] Furthermore, the third generation module 404 is specifically used to: determine the output mode of the picture book based on the vehicle's current remaining navigation time and / or remaining navigation distance; wherein, the output mode includes the length of the picture book content and / or the output strategy; and output the picture book according to the output mode of the picture book.

[0154] Furthermore, module 401 is specifically used to: respond to user instructions to acquire multi-source heterogeneous data of the vehicle; wherein, the multi-source heterogeneous data includes user instructions, location information, vehicle data and environmental data; and integrate the multi-source heterogeneous data to obtain the vehicle's context information.

[0155] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.

[0156] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, as shown below. Figure 8 As shown, the electronic device includes: a memory 501 and a processor 502; the memory 501 is a memory used to store instructions executable by the processor 502.

[0157] The processor 502 is configured to perform the method provided in the above embodiments.

[0158] The electronic device also includes a receiver 503 and a transmitter 504. The receiver 503 is used to receive instructions and data sent by other devices, and the transmitter 504 is used to send instructions and data to external devices.

[0159] The specific implementation process of the processor can be found in the above method embodiments, and its implementation principle and technical effect are similar, so it will not be repeated here.

[0160] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0161] This invention also provides a vehicle, which includes a vehicle body and an electronic device as described above disposed in the vehicle body. The electronic device stores computer execution instructions, which, when executed on a computer, cause the computer to execute the technical solutions described above.

[0162] This invention also provides a computer-readable storage medium storing computer-executable instructions, which, when executed on a computer, cause the computer to perform the technical solutions described above.

[0163] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0164] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. The readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in a device.

[0165] This invention also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the technical solutions described in the above embodiments.

[0166] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0167] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as magnetic disks or optical disks.

[0168] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A picture book generation method characterized by comprising: The method comprises: determining context information of the vehicle in response to a user instruction; wherein the context information comprises location information and user preference information; the location information comprises landmark information, point of interest information and cultural information of a location where the vehicle is located; and the user preference information is obtained based on the user instruction; generating a story outline based on the context information; wherein the story outline comprises chapter title information, key plot information and time allocation information; generating at least one chapter text and a stylized illustration of each chapter text based on the story outline and the context information; wherein the stylized illustration comprises an illustration image matching a story style of the chapter text; and the story style comprises a narrative tone, an emotional atmosphere and a type feature; generating and outputting a picture book based on the at least one chapter text and the stylized illustration of each chapter text.

2. The method of claim 1, wherein, The generating of the story outline based on the context information comprises: determining a similarity between the context information and each cache record in a preset story pool; if it is determined that the similarity is less than a preset threshold, generating the story outline based on the context information.

3. The method of claim 2, wherein, The generating of the story outline based on the context information comprises: generating a prompt word based on the user preference information and the location information in the context information; wherein the prompt word contains a context keyword and a narrative constraint; generating the story outline based on the prompt word.

4. The method of claim 2, wherein, The method further comprises: if it is determined that the similarity is greater than or equal to the preset threshold, generating the picture book based on a historical picture book to which the cache record corresponding to the similarity in the preset story pool belongs.

5. The method of claim 1, wherein, The generating of the at least one chapter text and the stylized illustration of each chapter text based on the story outline and the context information comprises: generating the at least one chapter text based on the story outline and the context information; generating the stylized illustration of the chapter text based on the context information and the chapter text.

6. The method of claim 5, wherein, The generating of the at least one chapter text based on the story outline and the context information comprises: extending the story outline based on environmental data in the context information to generate the at least one chapter text.

7. The method of claim 5, wherein, The generating of the stylized illustration of the chapter text based on the context information and the chapter text comprises: generating a theme illustration of the chapter text based on the chapter text; performing style transfer on the theme illustration of the chapter text based on the location information, the user preference information and a real-time scene image in the context information to obtain the stylized illustration of the chapter text.

8. The method of claim 1, wherein, The outputting of the picture book comprises: determining an output mode of the picture book based on a current navigation remaining time and / or a navigation remaining distance of the vehicle; wherein the output mode comprises a picture book content length and / or an output strategy; outputting the picture book based on the output mode of the picture book.

9. The method according to any one of claims 1-8, characterized in that, The determining of the context information of the vehicle in response to the user instruction comprises: In response to a user instruction, acquire multi-source heterogeneous data of the vehicle; wherein the multi-source heterogeneous data comprises the user instruction, the location information, vehicle data and environmental data; Integrate the multi-source heterogeneous data to obtain context information of the vehicle. 10.A picture book generating apparatus, characterized by comprising: Comprise: A determination module configured to determine context information of a vehicle in response to a user instruction; wherein the context information comprises location information and user preference information; the location information comprises landmark information, point of interest information and cultural information of a location where the vehicle is located; the user preference information is obtained based on the user instruction; A first generation module configured to generate a story outline according to the context information; wherein the story outline comprises chapter title information, key plot information and time allocation information; A second generation module configured to generate at least one chapter text and stylized illustrations of each chapter text according to the story outline and the context information; wherein the stylized illustrations comprise illustration images matching the story style of the chapter text; the story style comprises narrative tone, emotional atmosphere and type characteristics; A third generation module configured to generate and output a picture book according to the at least one chapter text and the stylized illustrations of each chapter text.

11. An electronic device, comprising: Comprise: A memory and a processor; The memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, so that the processor executes the method of any one of claims 1-9.

12. A vehicle characterized by comprising: An electronic device as claimed in claim 11 is arranged in a vehicle body.

13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method of any one of claims 1-9.

14. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-9. The computer program product comprises a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-9.

Images