AI-based personalized batch design content creation

Abstract

A data processing system implements receiving, from a sender device, a request to personalize and distribute a design content item based on content consumption needs and / or preferences of a recipient, wherein the content consumption needs and / or preferences include a data type; constructing a prompt by appending the design content item and the content consumption needs and / or preferences to an instruction string, the instruction string including instructions to a generative model to transform the design content item into a personalized content item for each recipient based on at least one of the content consumption needs or the content consumption preferences of the respective recipient; providing the prompt as input to the generative model and receiving the personalized content item as output from the generative model; providing the personalized content item to a recipient device; and causing a user interface of the recipient device to present the personalized content item.

Description

Background Technology

[0001] Artificial intelligence (AI) has the potential to automate our lives, saving time and increasing productivity. One area of ​​interest is the creation and sharing of personalized content. However, many existing AI-based personalized batch content creation platforms or applications only support sharing the same designed content items with different recipients, whether it's a thank-you note or a holiday greeting card. However, different users often have different preferences or needs. For example, blind recipients might prefer audio holiday greeting cards. Therefore, there is a need for AI-based personalized batch content creation systems and methods that support sharing designed content items adapted or modified for different recipients based on their content consumption needs or preferences. Summary of the Invention

[0002] An example data processing system according to this disclosure includes a processor and a machine-readable medium storing executable instructions. When executed, the instructions cause the processor, alone or in combination with other processors, to perform operations including: receiving from a first device associated with a sender a request to personalize and distribute a designed content item based on at least one of a plurality of recipients' content consumption needs or preferences, wherein the at least one of the content consumption needs or preferences includes a data type; constructing a first prompt word via a prompt word construction unit by appending the at least one of the content consumption needs or preferences and the designed content item to a first instruction string, the first instruction string including instructions for a generative model to convert the designed content item into a personalized content item for each recipient based on at least one of the recipient's content consumption needs or preferences; providing the first prompt word as input to the generative model and receiving the personalized content item as output from the generative model; providing the personalized content item to a second device associated with the corresponding recipient; and causing the user interface of the second device to display the personalized content item.

[0003] An example method implemented in a data processing system includes: receiving from a first device associated with a sender a request to personalize and distribute designed content items based on at least one of content consumption needs or content consumption preferences of a plurality of receivers, wherein the at least one of the content consumption needs or content consumption preferences includes a data type; constructing a first prompt word via a prompt word construction unit by appending the at least one of the content consumption needs or content consumption preferences and the designed content items to a first instruction string, the first instruction string including instructions for a generative model to convert the designed content items into personalized content items for each receiver based on at least one of the corresponding receiver's content consumption needs or content consumption preferences; providing the first prompt word as input to the generative model and receiving the personalized content items as output from the generative model; providing the personalized content items to a second device associated with the corresponding receiver; and causing a user interface of the second device to display the personalized content items.

[0004] According to the present disclosure, an exemplary non-transitory computer-readable medium data processing system has instructions stored thereon, which, when executed, cause a programmable device to perform the following functions: receiving from a first device associated with a sender a request to personalize and distribute a designed content item based on at least one of a plurality of recipients' content consumption needs or preferences, wherein the at least one of the content consumption needs or preferences includes a data type; constructing a first prompt word via a prompt word construction unit by appending the at least one of the content consumption needs or preferences and the designed content item to a first instruction string, the first instruction string including instructions for a generative model to convert the designed content item into a personalized content item for each recipient based on at least one of the recipient's content consumption needs or preferences; providing the first prompt word as input to the generative model and receiving the personalized content item as output from the generative model; providing the personalized content item to a second device associated with the corresponding recipient; and causing the user interface of the second device to present the personalized content item.

[0005] The choice of concepts to introduce a simplified form is provided in this summary, which will be further described in the following detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that address any or all of the shortcomings mentioned in any part of this disclosure. Attached Figure Description

[0006] The accompanying drawings are merely examples and not limitations, depicting one or more implementations of this teaching. In the drawings, the same reference numerals denote the same or similar elements. Furthermore, it should be understood that the drawings are not necessarily drawn to scale.

[0007] Figure 1 This is a diagram of an example computing environment in which AI-based batch design content creation based on individual consumption needs / preferences is implemented.

[0008] Figure 2 It is based on the principles described in this article. Figure 1 A conceptual diagram of an AI-based personalized batch content creation service.

[0009] Figures 3A-3C This is a diagram of an example user interface for an AI-based personalized batch content creation application that implements the technology described in this article.

[0010] Figure 4 This is a flowchart illustrating an example process for creating batch design content based on individual consumer needs / preferences using the techniques disclosed herein.

[0011] Figure 5 This is a block diagram illustrating an example software architecture, the various parts of which can be used in conjunction with the various hardware architectures described herein, which can implement any of the features described.

[0012] Figure 6 This is a block diagram illustrating the components of an example machine configured to read instructions from a machine-readable medium and execute any of the features described herein. Detailed Implementation

[0013] This paper describes systems and methods for AI-based batch design content creation based on individual consumer needs / preferences. These technologies provide a technical solution to the technical problem of a lack of fast and easy AI-based personalized batch design content creation systems and methods that use generative AI to batch modify / transform design content items based on individual recipient consumption needs / preferences. Existing AI-based personalized batch design content creation systems automate many previously manual design tasks, such as design content creation prompt generation and content item template generation. However, these systems do not support the generation of personalized design content items tailored to different recipient consumption needs / preferences.

[0014] To address these issues, the proposed technical solution improves the creation of content using one or more generative models by applying a multimodal model (LMM). This multimodal model automatically batch modifies / transforms content items based on individual recipient consumption needs / preferences. Such content consumption needs are caused by, for example, physical disabilities, language barriers, and cognitive impairments (e.g., excessive technical language, dense text, or information presented in difficult-to-understand formats). Physical disabilities include visual impairments, mobility limitations (difficulty using a mouse or keyboard), hearing impairments, learning disabilities (people with dyslexia or other learning disabilities may struggle with complex sentence structures or large blocks of text), and cognitive disabilities. Content consumption preferences can be caused by recipient physical preferences, language preferences, and / or cognitive preferences. Content consumption needs are driven by what users need to obtain from the content, while preferences reflect what users like.

[0015] The content transformation solution presented in this paper provides an improved method for distributing generative AI creations, such as content from generative AI models like GPT-4 or Dall-E, wherein the creations are bulk sent to a group of recipients by transforming the content according to each recipient's personalized consumption needs / preferences. One aspect includes (e.g., for public-facing services) a system architecture that allows for the bulk distribution of generative AI creations via transforming the creations according to each recipient's personalized consumption needs / preferences. Another aspect includes a user experience (UX) in which the bulk distribution of AI creations is accomplished by automatically transforming the AI ​​creations according to each recipient's personalized consumption choices, which provide tangible output to the system described above in the form of more personalized generative AI creations.

[0016] The system described in this paper provides a novel AI-based personalized batch design content creation service designed to simplify the creation of personalized batch design content. This service eliminates the need for manually writing design content transformation prompts and enables users to directly create personalized batch design content items for multiple recipients with different content consumption needs and / or preferences. The service autonomously executes the design content creation process behind the scenes. This service not only simplifies the workflow but also enhances the accessibility and efficiency of design content creation for recipients with different content consumption needs and / or preferences.

[0017] The system employs a large multimodal model (LMM, e.g., GPS-4V) to provide an easy-to-use user experience (UX) related to the creation of personalized batch design content that can be shared via any system, platform, or application (e.g., Google Workspace®, Copilot®, Teams®, Designer®, etc.).

[0018] The technical benefit of the solution presented in this paper is the provision of an AI-based personalized batch content creation service that, at runtime, considers one or more initial designed content items (e.g., event inventions) and the recipient's content consumption needs and / or preferences to generate transformed / personalized content items tailored to the recipient. Automated personalized batch content creation can accept any initial designed content item, thereby improving the recipient experience.

[0019] Another technical benefit of the approach presented in this paper is the use of a powerful set of tools to convert the initial designed content items into a format / data type accessible to the appropriate recipient (e.g., the blind). Therefore, the generated personalized content items meet the recipient's content consumption needs and / or preferences. This not only improves the sender's productivity but also increases the recipient's accessibility to the designed content items.

[0020] Another technical benefit of the solution presented in this article is that it significantly improves the user experience of creating personalized batch content within any platform / application when a user first registers for another platform / application, or whenever a user wishes to share the initial design content item with multiple recipients with different content consumption needs and / or preferences.

[0021] Another technical benefit of this approach is that it stores the initial / transformed design content items in the system, thereby saving users a significant amount of time and effort in creating and sharing similar design content items in the future. These and other technical benefits of the techniques disclosed herein will become apparent from the discussion of the example implementation below.

[0022] Figure 1This is a diagram of an example computing environment 100 in which the technologies described herein can be implemented. The example computing environment 100 includes a client device 105 and an application service platform 110. The application service platform 110 provides one or more cloud-based applications and / or services to support one or more web-enabled native applications on the client device 105. These applications may include, but are not limited to: AI-based personalized batch design content creation applications, demo applications, website authoring applications, collaboration platforms, communication platforms, and / or other types of applications, where users can create, view, and / or modify various types of personalized graphic designs in batches based on individual recipient content consumption needs or preferences. Figure 1 In the implementation shown, the application service platform 110 also applies generative AI to easily batch transform / modify fast and satisfactory personalized design content items based on the initial / sender-designed content items described herein. The client device 105 and the application service platform 110 communicate with each other via a network (not shown). The network may be a combination of one or more public and / or private networks, and may be at least partially provided by the Internet.

[0023] Client device 105 can be both a sender and a receiver device for users who subscribe to an AI-based personalized batch content creation service provided via application service platform 110. The service prompts the user on client device 105 to register their content consumption needs and / or preferences during service registration. The content consumption needs and / or preferences used for sending can differ from those used for receiving. Furthermore, the service can automatically update the user's content consumption needs and / or preferences based on user feedback (from the sender and / or receiver) regarding the batch content creation output.

[0024] In some implementations, the client device 105 is a computing device that can be implemented as a portable electronic device (such as a mobile phone, tablet computer, laptop computer, portable digital assistant device, portable game console, and / or other such device). In other implementations, the client device 105 can also be implemented in computing devices with other form factors, such as desktop computers, in-vehicle computing systems, information stations, point-of-sale systems, video game consoles, and / or other types of computing devices. Although in Figure 1 The example implementation illustrated includes a single client device 105, but other implementations may include a different number of client devices utilizing services provided by the application service platform 110.

[0025] The service transforms or modifies the consumption attributes of content items for individual recipients while maintaining the factual content and general design / theme of the original / sender's content item. For example, when the initial content item is a video animation card, the service automatically transforms it into an audio card for blind recipients. As another example, the service automatically transforms the video animation card into a visible color option for colorblind recipients. This transformation or modification makes the content item more accessible. Consumption attributes refer to data types (e.g., text, images, audio, video, animation, etc.), design attributes (e.g., font type, font size, color palette, texture, composition, layout, level of detail, level of abstraction, overall mood, atmosphere, etc.), recommendation channels (e.g., email, text messages, voice messages, video messages, file sharing, etc.), etc.

[0026] The term "design content item" refers to any digital design content item that can be understood by humans. Common forms of digital design content items include photographs, illustrations, charts, images, infographics, videos, animations, screenshots, memes, slideshows, pictographs, symbols, game interfaces, software application backgrounds, graphic design (e.g., publications, email marketing templates, PowerPoint presentations, menus, social media ads, banners and graphics, marketing and advertising, packaging, visual identity, art and graphic design, etc.), audiobooks, lectures and audio recordings, podcasts, interviews, news broadcasts, radio programs, audio courses, albums and songs, audio dramas and performances, soundtracks and scores, instrumental music from films, games or other media, audio guides, etc.

[0027] While various embodiments have been described with regard to visual and auditory design content items, it is envisioned that the schemes described herein can be used with other design content items involving touch, smell, and taste. For example, digital odor devices can release odors to allow a receiver to “smell” the contents of the design content item, while kinesthetic and haptic devices can allow a receiver to feel touching or holding objects in the design content item.

[0028] A visual content "theme" is a unifying concept or idea that guides the visual elements of a design project. It helps convey a specific message or atmosphere and creates a sticky and consistent look and feel for the project. Common elements of a visual content theme include color palettes, typography, images (e.g., photographs, illustrations, or icons), layouts, styles (e.g., minimalist, retro, or modern), and more.

[0029] Client device 105 includes a local application 114 and a browser application 112. In some implementations, local application 114 is a web-enabled local application that provides AI-based personalized batch design content creation. The web-enabled local application utilizes services provided by application service platform 110, including but not limited to: creating, viewing, and / or modifying various AI-based personalized batch design content creations. In some implementations, local application 114 is implemented in... Figures 3A-3C The user interface shown is illustrated. In other implementations, browser application 112 is used to access and view web-based content provided by application service platform 110. In such implementations, application service platform 110 utilizes one or more web applications, such as browser application 112, which enable users to view, create, and / or modify AI-based personalized batch design content creation using, for example, online applications. In some implementations, browser application 112 is implemented in... Figures 3A-3C The user interface shown is illustrated. In some implementations, the application service platform 110 supports both native applications 114 and browser applications 112, and users can choose which option best suits their needs.

[0030] The application service platform 110 includes a request processing unit 122, a prompt word construction unit 124, one or more generative models 126, a user database 128, a transformation engine 130, an enterprise data storage device 140, and a monitoring service (not shown).

[0031] The request processing unit 122 is configured to receive requests from the local application 114 and / or browser application 112 of the client device 105. According to the techniques provided herein, the requests may include, but are not limited to, batch personalizing initial / sender-designed content items based on individual recipient content consumption needs or preferences.

[0032] Figure 2 It is based on the principles described in this article. Figure 1 A conceptual diagram of an AI-based personalized batch design content creation service. The service utilizes advanced capabilities of LMM and LVM in AI-based personalized batch design content creation services. The service is designed to convert initial design content items 201 for a recipient list 203 (e.g., R1, R2, R3, R4) into multiple transformed design content items based on the recipients' content consumption needs or preferences, in response to a sender request received via an AI assistant 205 (e.g., Microsoft Copilot). In one embodiment, the service deploys a transformation engine 130 to perform the transformation in two phases. The initial design content items 201 can be provided by the sending user or a third-party online service (such as an online advertising service).

[0033] At stage 1, the conversion engine 130 parses the design theme and factual content (e.g., text content, such as...) Figure 2 The initial design content item 201 (thank you) on the thank-you card in the system. The conversion engine 130 also parses information from the recipient 203 (e.g., from the user database 128) to define personalized settings, including the recipient's content consumption needs or preferences (caused by the recipient's physical disability, language impairment, cognitive impairment, etc.) or content consumption preferences (caused by the recipient's physical preferences, language preferences, cognitive preferences, etc.). For recipients who have not yet subscribed to the AI-based personalized batch content creation service, or those who have subscribed but whose individual recipient content consumption needs or preferences are not provided, the service can query the recipient for this information. Alternatively, the service can extract or infer such information from existing recipient information in the system and other publicly available information about the recipient (such as news, the recipient's blog, articles, affiliations with specific non-profit organizations, social media groups, online discussion groups, etc.).

[0034] At stage 2, the conversion engine 130 works in conjunction with a generative model 126 (e.g., LMM, Dall-E, etc.) to convert the initially designed content item 201 (e.g., video to image, video to audio, image to text, etc.) at the sender device, receiver device, cloud, or a combination thereof based on the receiver’s content consumption needs or preferences.

[0035] For example, in response to the user prompt "Create a New Year's card for the global team reporting to me and send them personalized business cards in their native languages," the system retrieves team member names and location information from the company address book and identifies six team members in four different geographical locations. The system then invokes LMM 126a (e.g., GPT-4V) to perform AI-based personalized batch design content creation based on meta-prompts (such as those shown in Table 1). These meta-prompts can be adapted or extended based on different implementations, such as different generative models. Table 1

[0036] As another example, in response to the user prompt “Create colorful team morale invitations and also personalize them for people with visual impairments and color blindness (users cannot see red),” the system reuses team member names and location information and invokes LMM 126a to perform AI-based personalized batch design content creation based on meta prompts (such as those shown in Table 2). Table 2

[0037] In another embodiment, the meta-cue words in Table 1 include negative cue words that guide the LMM 126a away from generating unwanted content. Negative cue words are the opposite of positive cue words, which are used to guide the model to generate specific types of content. For example, the meta-cue words could include negative cue words to avoid generating “blurry,” “pixelated,” “low-quality,” “violent,” or “hateful” images.

[0038] In some implementations, client device 105 can deploy small generative models, which are well-suited for scenarios with limited computational resources. Example small generative models include variational autoencoders (VAEs) with low-dimensional latency, pixel RNNs and their variants, generative adversarial networks (GANs) with reduced complexity, syntax-based models, Markov chain models, and more. Additionally, generative models and GANs such as those from MobileDiffusion can be used on mobile devices.

[0039] Finally, the system incorporates a result check using LMM 126a to ensure that the final generated design content item contains key features from the initial / sender's design content item and matches the corresponding recipient's content consumption needs and / or preferences. The output, after passing the quality check, is then delivered to the recipient. Due to latency issues, alternative methods for quality checking involve parsing another recipient's information to define personalization settings while simultaneously transforming the design content item for the first recipient, or simply transforming the design content item into output for many recipients in parallel.

[0040] The system provides users with the ability to generate personalized design content items in batches from an unlimited range of design content, thereby increasing the accessibility of sender-designed content items that meet the content consumption needs and / or preferences of the corresponding recipients.

[0041] In some implementations, each generative model invocation requires passing a responsible AI test. In one embodiment, the responsible AI test is a comprehensive evaluation process that ensures the generative model adheres to ethical principles and operates safely and fairly in the real world. In another embodiment, the test not only checks whether the generative model accurately performs its intended task but also assesses its potential harm and mitigates negative impacts. For example, the meta-cues in Table 1 can be modified by a self-improving agent that modifies its own instructions based on its response to user interactions. In one embodiment, the meta-cues can include instructions guiding the agent on how to improve its own instructions based on positive, neutral, or negative user feedback on the output, such as the user's choice of a thumbs-up tab, thumbs-down tab, neutral tab, or generate more image tab, text input, etc. The system can then create another personalized content item based on the refined text cue and provide the refined output to the recipient.

[0042] In another embodiment, the system further enhances the quality of the output through a quality check to ensure that the personalized content item contains the text and theme of the initial design content item. The system is then able to send the personalized content item to the recipient. In some implementations, the service enables the personalized content item to be edited by the sender and / or (one or more) recipients, such as by adding text content to the personalized content item, thereby providing more user control over their AI-generated content (AIGC) experience.

[0043] In another embodiment, the prompt word construction unit 124 can use user data from various user data sources to generate details of the recipient's designed content consumption needs or preferences. For example, user preference data 128a can be digitized and stored in user database 128. The user data can include usernames, user organizations, user-preferred graphic design styles (e.g., minimalism, retro, Art Deco, Memphis design, Swiss style, Bauhaus, Pop Art, Punk, etc.), etc. The user data sources(s) can be online / offline databases (e.g., emails, social media posts, etc.), documents, articles, books, presentation content, and / or other types of content containing user activity information.

[0044] Figures 3A-3C This is a diagram of an example user interface for an AI-based personalized batch content creation application that implements the technology described in this article. Figures 3A-3CThe example user interface shown is a user interface for an AI-based personalized batch design content creation application within an AI-based design platform, such as, but not limited to, Microsoft Copilot®. However, the techniques disclosed herein for AI-based personalized batch design content creation are not limited to use within AI-based design platforms and can be used to generate personalized batch design content items for other types of applications, including but not limited to: presentation applications, website authoring applications, collaboration platforms, communication platforms, and / or other types of applications in which users create, view, and / or modify various personalized batch design content items. Such applications can be standalone applications, plugins, or share buttons for any application on client device 105, such as browser application 112, local application 114, etc. For example, the system can work on the web or within virtual meeting and collaboration applications (e.g., Microsoft Teams®) or email applications (e.g., Outlook®). The system can be integrated into the Microsoft Viva® platform or work within a browser (e.g., Windows® Edge®). The system can also work within social media websites / applications (e.g., Facebook®, Instagram®).

[0045] Figure 3A This demonstrates a simplified process for batch creation of personalized design content items in an AI-assisted application, employing a multimodal approach. In the user interface (UI) 302, the sender enters a request: “Create an image of a thank-you card designed in a surreal holographic glowing pastel style.” The AI ​​assistant replies: “I will attempt to create it.” The thank-you card 302b in a surreal holographic glowing pastel style is then displayed in the content pane 302a of UI 302. The content pane 302a also displays a share button 302c in the upper right corner and style buttons at the bottom for the user to select, such as pixel art, watercolor, block typography, steampunk, etc.

[0046] In UI 304, the sender enters a request: “Create a song with Suno to thank the team for a fantastic year.” The AI ​​assistant replies: “The song you created to thank the team for a fantastic year is complete! You will be able to listen to it soon.” Subsequently, the thank-you song 304b and its lyrics are displayed in the content pane 304a of UI 304. The lyrics read: “[Verse] We started small, but look where we are now, the team is so strong, it amazes everyone, through ups and downs, we have never lost our way, we did it together, we will always be here (oh-yeah)…”. The share button 304c is also shown in the lower left corner of the content pane 304a. In this step, a thank-you note containing thank-you card 302b and thank-you song 304b is created.

[0047] In step 2, the user selects the share button 304c to instruct the AI ​​assistant 205 to share the thank-you note. UI 306, displayed at the top of UI 304, shows the "Bulk Send & Personalize" option 306a. The AI ​​assistant 205 then generates personalized thank-you notes based on the individual content consumption needs and / or preferences of team members and distributes these personalized notes to them. Since the thank-you notes are work-related, the AI ​​assistant 205 distributes them via work-related channels unless any recipient has the opposite need / preference. For example, the personalized thank-you notes can be sent via work email, team collaboration applications (Teams), etc. Instead of sending actual multimedia files, the AI ​​assistant 205 creates an individual Uniform Resource Locator (URL) for each personalized thank-you note and sends that individual URL. This significantly reduces communication load and allows recipients with the same content consumption needs and / or preferences to share the same URL.

[0048] The sender selects the share button 302c to send thank-you cards to a batch of recipients at once 302b. Figure 3BThis demonstrates several ways to define bulk audience details / preferences at the sender's end. First, the system or sender (via UI 308) can retrieve a file (e.g., .docx / excel) with names and consumption formats for multiple recipients. Alternatively, the system can generate UI 310 for the sender to manually select or view the consumption formats. For example, UI 310 shows Rolly and Julie set to receive design content items via email, Susie and Alex set to receive them via SMS, and Derek set to receive them via WhatsApp. As another example, UI 312 also shows different design content item consumption formats / data types for the send button and recipients, such as audio, video, text, black & white images, etc. The system creates different URLs linking to personalized design content items for different recipients. The sender can select the "Send" button to send all thank-you design content items at once. As another example, UI 314 also shows five copy link buttons for each recipient in case the sender wants to further personalize the corresponding design content items for a specific recipient.

[0049] Figure 3C This illustrates a scenario where the sender does not define the recipient's consumption format. Instead, the sender sends the URL of an initial design content item (i.e., thank you card 302b) to each recipient via the mobile device's UI 316. The recipient device can transform the initial design content item locally based on the recipient's consumption needs / preferences. Alternatively, the system retrieves the recipient's consumption needs / preferences and transforms the initial design content item accordingly using a new URL for the recipient to view the transformed design content item. For example, in UI 318, recipient 1 is set to receive it as is, in UI 320, recipient 2 is set to receive the minimum (i.e., the text "Thank you, Alex, from Rolly"), and in UI 322, recipient 3 is set to receive audio (e.g., the narration "Thank you, Julie, from Rolly" and / or "Thank you, song 304b").

[0050] When a recipient rejects a transformed design content item, the system can directly solicit user feedback on how to adjust the transformed content item, such as font size being too small, audio being too soft, theme / background / object size being too large / too small, resolution being too high / too low, colors being too bright / too dark, theme similarity being too high / too low, style similarity being too high / too low (e.g., style retention score), etc. In the absence of such specific user feedback details, the system can automatically generate multiple variations for the recipient to choose from.

[0051] In some implementations, the system provides a feedback loop by enhancing the thumb-up and thumb-down buttons for each transformed design content item in the user interface 302. If the recipient dislikes a transformed design content item, the system can ask why and use user feedback data to improve one or more generative models 126. A thumb-down click can also prompt the user to indicate whether the transformed design content item is too bright, too dark, too large, too small, etc.

[0052] LMM 126a can utilize GPT-4V, Imagen, Contrastive Language Image Pre-training (CLIP), Flago, Percepver, Multi-Task Unified Model (MUM), etc., based on considerations such as open source, photorealism, creative control, computational requirements, ease of use, and licensing. One or more generative models 126 can be included as part of the application service platform 110, or they can be external models invoked by the application service platform 110. In implementations utilizing models other than one or more generative models 126, these models can be included as part of the application service platform 110, or they can be external models invoked by the application service platform 110.

[0053] Request processing unit 122 also coordinates the communication and exchange of data between components of application service platform 110, as discussed in the following example. Request processing unit 122 receives user requests to generate personalized batch design content items in local application 114 or browser application 112.

[0054] The cue word construction unit 124 can reformat or otherwise normalize any information to be included in the cue words into a normalized format recognizable by one or more generative models 126. In some implementations, the generative models 126 are trained using training data in this normalized format, and the cue words provided to the generative models 126 using this format can improve the quality of the output provided by the generative models 126.

[0055] Some common formats recognized by LMM include JPEG (Joint Image Experts Group), PNG (Portable Web Graphics), TIFF (Tagged Image File Format), BMP (Bitmap Image File), GIF (Graphics Interchange Format), PSD (Photoshop Document), RAW, SVG (Scalable Vector Graphics), WEBP, OpenEXR, etc.

[0056] The system can instruct one or more generative models 126 to generate single cue words (i.e., a single example or instruction that guides the LMM's response) or multiple cue words (i.e., multiple examples or instructions that give the LMM more context and improve its understanding of the task) for generating personalized batch design content items.

[0057] In some implementations, when the user data from the user database 128 is already in a format that can be directly processed by one or more generative models 126, the prompt word construction unit 124 does not need to convert the user data. In other implementations, when the user data does not have a format that can be directly processed by one or more generative models 126, the prompt word construction unit 124 converts the user data into a format that can be directly processed by one or more generative models 126. Some common standardized formats recognized by the language model include plain text, HTML, JSON, XML, etc. In one embodiment, the system converts the user data into JSON, which is a lightweight and efficient data exchange format. For example, the prompt word construction unit 124 can convert the user data into a format that can be directly processed by LMM 126a, for example, to adjust personalized design content items rejected by the user. Thus, the user data can be considered as adjusting personalized design content items rejected by the user, such as larger / smaller object / background / object size, higher / lower resolution, brighter / darker colors, higher / lower object similarity, higher / lower style similarity, etc. Other implementations may include instructions other than or in place of one or more of these instructions.

[0058] In some implementations, the application service platform 110 includes a monitoring service that analyzes one or more user request / prompt words, design content items generated by one or more generative models 126, and / or user data obtained from the user database 128 to ensure that the application service platform 110 does not generate or utilize potentially harmful or intrusive content.

[0059] If potentially harmful or offensive content is detected in user data obtained from user database 128, the monitoring service provides client device 105 with a notification instructing (one or more) prompt words to block the user data forming a meta-prompt. In some implementations, request processing unit 122 discards any user data containing potentially harmful or offensive content and passes any remaining content that has not yet been discarded to request processing unit 122 as input to prompt word construction unit 124. In other implementations, prompt word construction unit 124 discards any content containing potentially harmful or offensive content and passes any remaining content that has not yet been discarded as input to (one or more) generative models 126.

[0060] In one embodiment, the prompt word construction unit 124 submits one or more prompt words and / or one or more meta prompt words to the monitoring service to ensure that the prompt words do not include any potentially harmful or offensive content. The prompt word construction unit 124 stops processing the one or more user prompt words and / or one or more meta prompt words in response to the monitoring service determining that one or more user prompt words and / or visual content data contain potentially harmful or offensive content.

[0061] The prompt word construction unit 124 can stop converting design content items in response to a monitoring service determining that the graphic design includes potentially harmful or offensive content. In response to determining that personalized batch design content items include potentially harmful or offensive content, the monitoring service generates a blocked content notification and provides the notification to the prompt word construction unit 124. The prompt word construction unit 124 may attempt to correct and resubmit the text prompt. If the monitoring service does not identify any problems with the converted design content items, the prompt word construction unit 124 provides the converted design content items to the request processing unit 122. Depending on which application is the source of the initial design content request, the request processing unit 122 provides the converted design content items to the local application 114 or the browser application 112. The technical benefit of this approach is that the monitoring service provides security measures for both user-created and model-created content to ensure that prohibited offensive or potentially offensive content is not presented to the user in the local application 114 or the browser application 112.

[0062] In some implementations, the user database 128 can be implemented on the application service platform 110. In other implementations, at least a portion of the user database 128 is implemented on an external server accessible by the prompt word construction unit 124.

[0063] In some implementations, application service platform 110 adheres to privacy guidelines and rules governing the use of user data included in user database 128 to ensure users have control over how application service platform 110 utilizes their data. Users are given the opportunity to opt-in to application service platform 110, allowing it to access user data and enabling generative models(s)126 to generate transformed design content items. In some implementations, when an application such as native application 114 or browser application 112 first presents the AI ​​assistant to the user, a message is presented to the user instructing them to opt-in to allow application service platform 110 access to user data included in user database 128 to support design content item transformation functionality. Users can opt-in to allow application service platform 110 to access all or a subset of the user data included in user database 128. Furthermore, users can modify their membership status at any time by accessing their user data and selectively opting out of allowing application service platform 110 to access and utilize user data from user database 128, either as a whole or individually.

[0064] In one embodiment, metadata can be generated for personalized design content items to facilitate later retrieval based on user queries. For example, the metadata could detail whether the transformed design content item relates to thank-you cards and thank-you songs. Therefore, the metadata can be used to match queries from the same user related to thank-you cards to personalized design content items.

[0065] The technical advantage of this approach is that it eliminates the need to retrieve recipient consumption settings again, and it avoids regenerating the converted design content items each time a user submits a prompt for batch conversion of the same design content items. The request processing unit 122 maintains user session information in the persistent storage of the application service platform 110 and retrieves style element data from the user session information in response to each subsequent prompt submitted by the user. The request processing unit 122 then provides the newly received user prompt(s) and recipient consumption settings(s) to the prompt(s) construction unit 124 or LMM 126a for batch conversion of design content items as discussed in the previous examples.

[0066] The visual content library 142 discussed above (storing, for example, initial / transformed design content items), requests, prompts, and responses 144, extracted / inferred user data 146 (e.g., user preferences), and other asset data 148 can be stored in the enterprise data storage device 140. The extracted / inferred user data 146 (e.g., user preferences) can be collected via the user feedback loop discussed above and temporarily linked to the user ID during a user session and stored in a cache. After the user session, the extracted / inferred user data 146 is delinked from the user ID to metadata of the transformed design content items and stored in the visual content library 142. Additionally, the extracted / inferred user data 146 linked to the user ID is saved back to the user database 128.

[0067] Depending on the entity's needs and IT infrastructure, the enterprise data storage device 140 can be physical and / or virtual. Examples of physical enterprise data storage systems include Network Attached Storage (NAS), Storage Area Network (SAN), Direct Attached Storage (DAS), tape libraries, hybrid storage arrays, object storage, etc. Examples of virtual enterprise data storage systems include Virtual SAN (vSAN), Software-Defined Storage (SDS), cloud storage, Hyper-Converged Infrastructure (HCI), network virtualization and Software-Defined Networking (SDN), container storage, etc.

[0068] Figure 4 This is a flowchart illustrating an example process for creating batch design content based on individual consumer needs / preferences using AI, according to the techniques disclosed herein. Process 400 can be implemented by the application service platform 110 or its components shown in the foregoing example. Process 400 can be implemented in, for example, an example machine including a processor and memory, such as in... Figure 6 As shown in the diagram. Thus, the application service platform 110 is able to provide units for implementing the various parts of process 400, as well as units for implementing embodiments of other processes described herein in conjunction with other components of the example computing environment 100. Although process 400 is illustrated and described as a series of steps, it is contemplated that various embodiments of process 400 can be performed in any order or combination, and it is not necessary to include all the steps illustrated.

[0069] In one embodiment, for example, in step 402, request processing unit 122 receives data from a first device (e.g., client device 105) associated with a sender (e.g., Rolly) based on multiple receivers (e.g., as in...). Figure 2 R1, R2, R3, R4 or in Figure 3BThe request to personalize and distribute designed content items (e.g., initial designed content item 201 or thank-you card 302b) is made by at least one of the content consumption needs or content consumption preferences of recipients Julie Susie, Alex, and Derak (203), and at least one of the content consumption needs or content consumption preferences includes a data type. For example, the data type includes text, images, audio, or video.

[0070] In other embodiments, at least one of the content consumption needs or the content consumption preferences further includes at least one of design attributes or communication channels. For example, the design attributes include font type, font size, color palette, texture, composition, layout, level of detail, level of abstraction, overall mood or atmosphere, and the communication channels include at least one of email, text message, voice message, video message, or file sharing.

[0071] In step 404, the prompt word construction unit (e.g., prompt word construction unit 124) constructs a first prompt word (e.g., a meta-prompt word in Table 1 or Table 2) by appending at least one of the designed content items and content consumption needs or content consumption preferences to a first instruction string. The first instruction string includes instructions for a generative model (e.g., LMM 126a) to convert the designed content items into personalized content items for each receiver based on at least one of the corresponding receiver's content consumption needs or content consumption preferences. For example, the generative model is a multimodal model, and the personalized content items (e.g., the three converted content items rendered in FIG3) are provided to the second device via a Uniform Resource Locator.

[0072] For example, content consumption demand may be caused by at least one of the following: physical disability (e.g., visual impairment, mobility limitation, hearing impairment, learning disability, etc.), language impairment, or cognitive impairment of the corresponding recipient. The content consumption preference may be caused by at least one of the following: physical preference, language preference, or cognitive preference of the corresponding recipient. In step 406, the prompt word construction unit provides a first prompt word as input to the generative model and receives personalized content items as output from the generative model.

[0073] In step 408, request processing unit 122 provides personalized content items to a second device associated with the corresponding recipient. In step 410, request processing unit 122 causes the user interface of the second device to display the personalized content items (e.g., the three transformed content items presented in FIG. 3). In some implementations, the instructions for the generative model include instructions to extract audio transcripts (e.g., a narration of “Thank you Julie., from Rolly” and / or a narration of thank you song 304b) from the designed content items, based on at least one of the corresponding recipient’s content consumption needs or content consumption preferences being audio, or instructions to generate audio descriptions (e.g., a thank-you card designed in a surreal holographic pastel style, etc.), and the user interface of the second device displays the audio transcripts or the audio descriptions.

[0074] In another embodiment, the request processing unit 122 receives at least one user feedback regarding a personalized content item via a user interface of the second device. For example, the user feedback is collected via a user selection of at least one of the following: a thumbs-up tab, a thumbs-down tab, a neutral tab, or a generate more images tab, text input, or a combination thereof. The prompt word construction unit constructs a second prompt word by appending the feedback and the personalized content item to a second instruction string, the second instruction string including instructions for the generative model to generate at least one other personalized content item based on the feedback and the personalized attribute item by adjusting one or more attributes of the personalized content item based on the feedback, while satisfying at least one of content consumption needs or content consumption preferences.

[0075] In some implementations, the processor is embedded in a first / sender device, a second / receiver device, or a server (e.g., an application service platform 110 or in the cloud), while the first device (e.g., Figure 3B ), second equipment (e.g., Figure 3C At least one of the following: (a) or (b) servers, determines at least one of the content consumption needs or content consumption preferences of the corresponding recipient.

[0076] In one embodiment, when a user signs up for a platform (e.g., a web-based collaboration platform, social media platform, gaming platform, etc.) or an application (e.g., Microsoft Copilot®, a team collaboration application, etc.) or when a user activates a platform or application, request processing unit 122 stores the user's content consumption needs and / or preferences from the user profile. Then, in addition to AI-based personalized batch content creation, request processing unit 122 applies the user profile to at least one function of the platform or application (e.g., email, chat and video conferencing, file sharing, etc.).

[0077] The system allows users to batch convert design content items for different recipients, thereby simplifying the user's creative process. This ease of use improves user productivity and utilization, and attracts more non-technical users. By automating the creation of personalized batch design content, the system eliminates reliance on user-generated personalized prompts for design content items. This solution makes the personalized batch design content creation process more efficient and open. The system can apply personalized batch design content creation to a range of data types, including images, images with text, videos, animations, etc., thereby enhancing the accessibility of the design content creation platform / application for users with different content consumption needs and / or preferences.

[0078] Enterprises employing open-source generative models for data have security and privacy considerations and strategies, such as data anonymization, data isolation, providing secure access, model protection, using a secure environment, encryption, regular auditing, compliance with laws and regulations, data retention policies, conducting privacy impact assessments, user education, performing regular updates, providing disaster recovery and backups, providing incident response plans, and third-party audits. By following these security and privacy best practices, Example Computing Environment 100 can minimize the risks associated with using open-source generative models while protecting enterprise data from unauthorized access or exposure.

[0079] In one embodiment, the application service platform 110 can store enterprise data separately from the generative model training data to reduce the risk of unintentional leakage of sensitive information during model generation. The application service platform 110 can restrict access to both the generative model and the enterprise data. The application service platform 110 can also implement appropriate access controls, strong authentication, and authorization mechanisms to ensure that only authorized personnel can interact with the selected model and enterprise data.

[0080] Application service platform 110 can also run one or more generative models 126 in a secure computing environment. Furthermore, application service platform 110 employs robust network security, firewalls, and intrusion detection systems to defend against external threats. Application service platform 110 can encrypt enterprise data and any data in transit. Application service platform 110 can also employ encryption standards used for data storage and transmission to prevent data corruption.

[0081] Furthermore, the application service platform 110 can implement robust security measures around the generative model(s) 126 themselves, such as routine security audits, code reviews, and ensuring that the models and security patches are up-to-date. The application service platform 110 can periodically audit the usage and access logs of the generative models to detect any unauthorized or anomalous activity. The application service platform 110 can also ensure that any use of the open-source generative models complies with relevant data protection regulations, such as GDPR, HIPAA, or other industry-specific compliance standards.

[0082] Application Service Platform 110 can establish data retention and deletion policies to ensure that generated data (especially user data) is no longer stored than necessary, minimizing the risk of data exposure. Application Service Platform 110 can perform Privacy Impact Assessments (PIAs) to identify and mitigate potential privacy risks associated with the use of generative models. Application Service Platform 110 also provides mechanisms for training and educating users on the proper handling of enterprise data and the responsible use of generative models. Furthermore, Application Service Platform 110 stays up-to-date with evolving security threats and best practices crucial for ongoing data protection.

[0083] This article presents a combination Figure 1-4 Detailed examples of the systems, devices, and techniques described are provided to illustrate the invention and its benefits. Such examples of use should not be construed as limiting the logical process embodiments of this disclosure, nor should variations from those user interface methods described herein be considered outside the scope of this disclosure. It should be understood that references to displaying or presenting items (such as, but not limited to, presenting images on a display device, presenting audio via one or more speakers, and / or vibrating the device) include issuing instructions, commands, and / or signals that cause or are reasonably expected to cause the device or system to display or present items. In some embodiments, in Figure 1-4 The various features described herein are implemented in the corresponding modules, which may also be referred to as and / or include logic, components, units, and / or mechanisms. Modules may constitute software modules (e.g., code embodied in a machine-readable medium) or hardware modules.

[0084] In some examples, hardware modules may be implemented mechanically, electronically, or using any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic configured to perform a specific operation. For instance, a hardware module may include a dedicated processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). A hardware module may also include programmable logic or circuitry temporarily configured by software to perform a specific operation, and may include a portion of machine-readable medium data and / or instructions for such a configuration. For example, a hardware module may include software contained within a programmable processor configured to execute a software instruction set. It should be understood that the decision to implement a hardware module mechanically in dedicated and permanently configured circuitry or in temporarily configured circuitry (e.g., software-configured) may be driven by cost, time, support, and engineering considerations.

[0085] Therefore, the phrase "hardware module" should be understood to encompass tangible entities capable of performing specific operations and which can be configured or arranged in some physical manner—that is, entities physically constructed, permanently configured (e.g., hardwired), and / or temporarily configured (e.g., programmed) to operate or perform the specific operations described herein. As used herein, "hardware-implemented module" refers to a hardware module. Consider examples where hardware modules are temporarily configured (e.g., programmed), and each of the hardware modules need not be configured or instantiated at any given time. For example, in cases where the hardware module includes a programmable processor configured by software as a dedicated processor, the programmable processor can be configured as different dedicated processors (e.g., including different hardware modules) at different times. The software can accordingly configure the processor or processor, for example, constituting a specific hardware module at one time and different hardware modules at different times. A hardware module implemented using one or more processors can be referred to as "processor-implemented" or "computer-implemented."

[0086] Hardware modules are capable of providing information to and receiving information from other hardware modules. Therefore, the described hardware modules can be considered communicatively coupled. In the presence of multiple hardware modules, communication can be achieved through signal transmission between or among the two or more hardware modules (e.g., via appropriate circuitry and buses). In embodiments where multiple hardware modules are configured or instantiated at different times, such communication between hardware modules can be achieved, for example, by storing and retrieving information in a memory device accessible to the multiple hardware modules. For instance, one hardware module may perform an operation and store the output in a memory device, and another hardware module may then access the memory device to retrieve and process the stored output.

[0087] In some examples, at least some of the operations of the method can be executed by one or more processors or modules implemented by processors. Furthermore, one or more processors can operate to support the execution of related operations in a “cloud computing” environment or “Software as a Service” (SaaS). For example, at least some of the operations can be executed by multiple computers (as an example of a machine including processors) and / or across multiple computers, and these operations can be accessed via a network (e.g., the Internet) and / or via one or more software interfaces (e.g., application programming interfaces (APIs)). The performance of a particular operation within the operations can be distributed among processors, residing not only within a single machine but also deployed across several machines. Processors or modules implemented by processors can be in a single geographic location (e.g., in a home or office environment or a server cluster) or can be distributed across multiple geographic locations.

[0088] Figure 5 This is a block diagram 500 illustrating an example software architecture 502, the various parts of which can be used in conjunction with various hardware architectures described herein, which can implement any of the features described above. Figure 5 This is a non-limiting example of a software architecture, and it should be understood that many other architectures can be implemented to facilitate the functionality described herein. Software architecture 502 can be implemented in, for example... Figure 6 The hardware of the machine 600 is executed, and the machine 600 specifically includes a processor 610, a memory 630, and an input / output (I / O) component 650. A representative hardware layer 504 is illustrated and can be represented, for example... Figure 6 The machine 600. A representative hardware layer 504 includes a processing unit 506 and associated executable instructions 508. The executable instructions 508 represent executable instructions of the software architecture 502, including implementations of methods, modules, etc., as described herein. Hardware layer 504 also includes a memory / storage device 510, which further includes the executable instructions 508 and accompanying data. Hardware layer 504 may also include other hardware modules 512. The instructions 508 stored by the processing unit 506 may be a portion of the instructions 508 held by the memory / storage device 510.

[0089] The example software architecture 502 can be conceptualized as layers, each providing various functionalities. For example, software architecture 502 may include layers and components such as an operating system (OS) 514, libraries 516, frameworks 518, applications 520, and a presentation layer 544. Operationally, applications 520 and / or other components within a layer can invoke API calls 524 to other layers and receive corresponding results 526. The illustrated layers are representative in nature, and other software architectures may contain additional or different layers. For example, some mobile or dedicated operating systems may not provide a framework / middleware 518.

[0090] OS 514 can manage hardware resources and provide public services. OS 514 may include, for example, a kernel 528, services 530, and drivers 532. Kernel 528 can act as an abstraction layer between hardware layer 504 and other software layers. For example, kernel 528 may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, etc. Services 530 can provide other public services to other software layers. Drivers 532 can be responsible for controlling the underlying hardware layer 504 or interfacing with the underlying hardware layer 504. For example, depending on the hardware and / or software configuration, drivers 532 may include display drivers, camera drivers, memory / storage drivers, peripheral device drivers (e.g., via Universal Serial Bus (USB)), network and / or wireless communication drivers, audio drivers, etc.

[0091] Library 516 provides common infrastructure that can be used by application 520 and / or other components and / or layers. Library 516 typically provides functionality used by other software modules to perform tasks, rather than interacting directly with OS 514. Library 516 may include system libraries 534 (e.g., the C standard library) that provide functions such as memory allocation, string manipulation, and file operations. Additionally, library 516 may include API libraries 536, such as media libraries (e.g., those supporting the rendering and manipulation of image, sound, and / or video data formats), graphics libraries (e.g., OpenGL libraries for rendering 2D and 3D graphics on a display), database libraries (e.g., SQLite or other relational database functionalities), and web libraries (e.g., WebKit that provides web browsing functionality). Library 516 may also include a wide variety of other libraries 538 to provide numerous functionalities for application 520 and other software modules.

[0092] Framework 518 (sometimes referred to as middleware) provides a more advanced public infrastructure that can be used by Application 520 and / or other software modules. For example, Framework 518 can provide various graphical user interface (GUI) functionalities, advanced resource management, or advanced location services. Framework 518 can provide a wide range of other APIs for Application 520 and / or other software modules.

[0093] Application 520 includes built-in application 540 and / or third-party application 542. Examples of built-in application 540 may include, but are not limited to: contact application, browser application, location application, media application, messaging application, and / or game application. Third-party application 542 may include any application developed by an entity other than a platform-specific vendor. Application 520 may use the functionality available via OS 514, library 516, framework 518, and presentation layer 544 to create a user interface for interaction with the user.

[0094] Some software architectures use virtual machines, such as the one illustrated by virtual machine 548. Virtual machine 548 provides an execution environment where applications / modules can run as if they were on a hardware machine (e.g., Figure 6 The virtual machine 548 can be run on the same machine 600. It can be hosted by a host OS (e.g., OS 514) or a hypervisor, and can have a virtual machine monitor 546 that manages the operation of the virtual machine 548 and interoperates with the host operating system. A software architecture different from the external software architecture 502 can run within the virtual machine 548, such as an OS 550, libraries 552, frameworks 554, applications 556, and / or a presentation layer 558.

[0095] Figure 6 This is a block diagram illustrating components of an example machine 600 configured to read instructions from a machine-readable medium (e.g., a machine-readable storage medium) and perform any of the features described herein. The example machine 600 is in the form of a computer system in which instructions 616 (e.g., in the form of software components) can be executed to cause the machine 600 to perform any of the features described herein. Thus, instructions 616 can be used to implement the modules or components described herein. Instructions 616 cause the unprogrammed and / or unconfigured machine 600 to perform specific machine operations configured to perform the described features. The machine 600 can be configured to operate as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, the machine 600 can operate as a server machine or client machine in a server-client network environment, or as a node in a peer-to-peer or distributed network environment. The machine 600 can be embodied as, for example, a server computer, client computer, personal computer (PC), tablet computer, laptop computer, netbook, set-top box (STB), gaming and / or entertainment system, smartphone, mobile device, wearable device (e.g., smartwatch), and Internet of Things (IoT) device. Furthermore, although only a single machine 600 is shown, the term "machine" includes a collection of machines that execute instructions 616 individually or in combination.

[0096] Machine 600 may include processor 610, memory 630, and I / O components 650, which may be communicatively coupled via, for example, bus 602. Bus 602 may include multiple buses coupling various components of machine 600 via various bus technologies and protocols. In examples, processor 610 (including, for example, a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), ASIC, or suitable combinations thereof) may include one or more processors 612a to 612n capable of executing instructions 616 and processing data. In some examples, one or more processors 610 may execute instructions provided or recognized by one or more other processors 610. The term "processor" includes multi-core processors, which include cores capable of executing instructions simultaneously. Although Figure 6 Multiple processors are shown, but machine 600 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors each with a single core, multiple processors each with multiple cores, or any combination thereof. In some examples, machine 600 may include multiple processors distributed among multiple machines.

[0097] Memory / storage device 630 may include main memory 632, static memory 634, or other memory and storage unit 636, both accessible by processor 610, such as via bus 602. Storage unit 636 and memory 632 store instructions 616 embodying any one or more of the functions described herein. Memory / storage device 630 may also store temporary, intermediate, and / or long-term data of processor 610. Instructions 616 may also reside wholly or partially within memory 632, 634, storage unit 636, at least one of processor 610 (e.g., within a command buffer or cache memory), at least one of I / O components 650, or any suitable combination thereof. Thus, memory 632, 634, storage unit 636, memory in processor 610, and memory in I / O component 650 are examples of machine-readable media.

[0098] As used herein, "machine-readable medium" means a device capable of temporarily or permanently storing instructions and data that cause machine 600 to operate in a particular manner, and may include, but is not limited to, random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical storage media, magnetic storage media and devices, cache memory, network-accessible or cloud storage, other types of storage and / or any suitable combination thereof. The term "machine-readable medium" applies to a single medium or combination of media used to store instructions (e.g., instruction 616) for execution by machine 600, such that when executed by one or more processors 610 of machine 600, the instructions cause machine 600 to perform one or more of the features described herein. Therefore, "machine-readable medium" can refer to a single storage device, as well as a "cloud-based" storage system or storage network comprising multiple storage devices or equipment. The term "machine-readable medium" excludes signals of its own purported nature.

[0099] I / O component 650 may include a wide variety of hardware components suitable for receiving input, providing output, generating output, sending information, exchanging information, capturing measurements, etc. The specific I / O component 650 included in a particular machine will depend on the type and / or function of the machine. For example, mobile devices such as mobile phones may include touch input devices, while headless servers or IoT devices may not include such touch input devices. Figure 6 The specific examples of I / O components shown are by no means limiting, and other types of components may be included in machine 600. The grouping of I / O components 650 is only for the purpose of simplifying the discussion and is by no means limiting. In various examples, I / O components 650 may include user output components 652 and user input components 654. User output components 652 may include, for example, display components (e.g., liquid crystal display (LCD) or projectors) for displaying information, acoustic components (e.g., speakers), haptic components (e.g., vibration motors or force feedback devices), and / or other signal generators. User input components 654 may include, for example, alphanumeric input components (e.g., keyboards or touchscreens), pointing components (e.g., mouse devices, touchpads, or other pointing instruments), and / or haptic input components (e.g., physical buttons or touchscreens that provide the position and / or force of touch or touch gestures), configured to receive various user inputs, such as user commands and / or selections.

[0100] In some examples, I / O component 650 may include biometric component 656, motion component 658, environmental component 660 and / or position component 662, as well as various other physical sensor components. Biometric component 656 may include components for detecting bodily expressions (e.g., facial expressions, vocal expressions, hand or body gestures, or eye tracking), measuring biosignals (e.g., heart rate or brain waves), and identifying people (e.g., via voice-based, retinal, fingerprint, and / or face-based recognition). Motion component 658 may include, for example, accelerometers (e.g., accelerometers) and rotation sensors (e.g., gyroscopes). Environmental component 660 may include, for example, lighting sensors, temperature sensors, humidity sensors, pressure sensors (e.g., barometers), acoustic sensors (e.g., microphones for detecting ambient noise), proximity sensors (e.g., infrared sensing of nearby objects), and / or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. The location component 662 may include, for example, a location sensor (e.g., a Global Positioning System (GPS) receiver), an altitude sensor (e.g., a barometric pressure sensor from which altitude can be derived), and / or an orientation sensor (e.g., a magnetometer).

[0101] I / O component 650 may include communication component 664, which implements various technologies operable to couple machine 600 to network 670 and / or device 680 via corresponding communication couplers 672 and 682. Communication component 664 may include one or more network interface components or other suitable devices to interface with network 670. Communication component 664 may include components, for example, adapted to provide wired communication, wireless communication, cellular communication, near field communication (NFC), Bluetooth communication, Wi-Fi and / or communication via other modes. Device 680 may include other machines or various peripheral devices (e.g., via USB coupling).

[0102] In some examples, communication component 664 may detect identifiers or include components suitable for detecting identifiers. For example, communication component 664 may include a radio frequency identification (RFID) tag reader, an NFC detector, an optical sensor (e.g., a one-dimensional or multi-dimensional barcode or other optical code), and / or an acoustic detector (e.g., a microphone for identifying audio signals of the tag). In some examples, location information may be determined based on information from communication component 664, such as, but not limited to, geographic location via Internet Protocol (IP) address, geographic location via Wi-Fi, cellular, NFC, Bluetooth, or other wireless station identification and / or signal triangulation.

[0103] In the foregoing detailed description, numerous specific details have been illustrated with examples to provide a thorough understanding of the relevant teachings. However, it is evident that these teachings can be practiced without such details. In other instances, well-known methods, processes, components, and / or circuits have been described in a relatively high-level but undetailed manner to avoid unnecessarily obscuring aspects of these teachings.

[0104] Although various embodiments have been described, the description is intended to be exemplary and not limiting, and it should be understood that many more embodiments and implementations are possible within the scope of the embodiments. While many possible combinations of features are shown in the drawings and discussed in this detailed description, many other combinations of the disclosed features are possible. Unless specifically limited, any feature of any embodiment may be used in combination with or substituted for any other feature or element in any other embodiment. Therefore, it should be understood that any feature shown and / or discussed in this disclosure can be implemented together in any suitable combination. Therefore, the embodiments are not limited except for the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0105] Typically, the functions described in this article (e.g., Figures 1 to 6 The features shown herein can be implemented using software, firmware, hardware (e.g., fixed logic, finite state machines, and / or other circuitry), or a combination of these implementations. In the case of a software implementation, the program code performs a specified task when executed on a processor (e.g., a CPU or CPU). The program code can be stored in one or more machine-readable storage devices. The techniques described herein are characterized by system independence, meaning that these techniques can be implemented on a variety of computing systems with various processors. For example, an implementation may include an entity (e.g., software) that causes hardware to perform operations (e.g., processor function blocks, etc.). For example, a hardware device may include a machine-readable medium that can be configured to maintain instructions that cause the hardware device (including an operating system running thereon and associated hardware) to perform operations. Thus, the instructions can be used to configure the operating system and associated hardware to perform operations, and thereby configure or otherwise adapt the hardware device to perform the aforementioned functions. The instructions can be provided by the machine-readable medium to the hardware elements executing the instructions in various different configurations.

[0106] In the following sections, further features, characteristics, and advantages of the invention will be described by way of items: Project 1: A data processing system, comprising: processor, and A machine-readable storage medium storing executable instructions that, when executed by the processor, cause the processor, alone or in combination with other processors, to perform the following operations: A request is received from a first device associated with a sender to personalize and distribute designed content items based on at least one of a plurality of recipients’ content consumption needs or content consumption preferences, wherein the at least one of the content consumption needs or content consumption preferences includes a data type; The first prompt word is constructed by the prompt word construction unit by appending at least one of the content consumption needs or the content consumption preferences and the designed content item to a first instruction string. The first instruction string includes instructions for the generative model to convert the designed content item into a personalized content item for each receiver based at least one of the corresponding receiver's content consumption needs or content consumption preferences. The first prompt word is provided as input to the generative model, and the personalized content item is received from the generative model as output. Provide the personalized content items to a second device associated with the corresponding recipient; and This causes the user interface of the second device to display the personalized content items. Project 2. The data processing system according to Project 1, wherein the machine-readable storage medium further includes instructions configured to cause the processor to perform the following operations, either alone or in combination with other processors: Determine at least one of the content consumption needs or content consumption preferences of the corresponding recipient at at least one of the first device, the second device, or the server. Project 3. A data processing system according to any one of Project 1 or Project 2, wherein the content consumption demand is caused by at least one of the physical disability, language impairment, or cognitive impairment of the corresponding recipient. Project 4. The data processing system according to any one of Projects 1-3, wherein the content consumption preference is caused by at least one of the corresponding receiver's physical preferences, language preferences, or cognitive preferences. Item 5. A data processing system according to any one of Items 1-4, wherein the processor is embedded in the first device, the second device, or the server. Project 6. A data processing system according to any one of Projects 1-5, wherein the machine-readable storage medium further includes instructions configured to cause the processor to perform the following operations, either alone or in combination with other processors: At least one user feedback item regarding the personalized content is received via the user interface of the second device. Item 7. A data processing system according to any one of items 1-6, wherein the machine-readable storage medium further includes instructions configured to cause the processor to perform the following operations, either alone or in combination with other processors: The second prompt word is constructed by the prompt word construction unit by appending the feedback and the personalized content item to a second instruction string, the second instruction string including instructions for the generative model to generate at least one other personalized content item based on the feedback and the personalized content item by adjusting one or more attributes of the personalized content item based on the feedback, while satisfying at least one of the content consumption needs or the content consumption preferences. Item 8. A data processing system according to any one of Items 1-7, wherein the user feedback is collected via user selection of at least one of the following: thumb up tab, thumb down tab, neutral tab, or generate more images tab, text input, or a combination thereof. Project 9. A data processing system according to any one of Projects 1-8, wherein the generative model is a multimodal model. Item 10. A data processing system according to any one of items 1-9, wherein the personalized content item is provided to the second device via a Uniform Resource Locator. Project 11. A data processing system according to any one of Projects 1-10, wherein the instructions for the generative model include instructions for: extracting audio transcripts or generating an audio description based on the designed content item, based on at least one of the content consumption needs or content consumption preferences of the corresponding receiver being audio, and The user interface of the second device presents the audio transcription or the audio description. Item 12. A data processing system according to any one of Items 1-11, wherein the data type includes text, image, audio or video. Item 13, a method comprising: A request is received from a first device associated with a sender to personalize and distribute designed content items based on at least one of a plurality of recipients’ content consumption needs or content consumption preferences, wherein the at least one of the content consumption needs or content consumption preferences includes a data type; The first prompt word is constructed by the prompt word construction unit by appending at least one of the content consumption needs or the content consumption preferences and the designed content item to a first instruction string. The first instruction string includes instructions for the generative model to convert the designed content item into a personalized content item for each receiver based at least one of the corresponding receiver's content consumption needs or content consumption preferences. The first prompt word is provided as input to the generative model, and the personalized content item is received from the generative model as output. Provide the personalized content items to a second device associated with the corresponding recipient; and This causes the user interface of the second device to display the personalized content items. Item 14. The method described in Item 13 further includes: Determine at least one of the content consumption needs or content consumption preferences of the corresponding recipient at at least one of the first device, the second device, or the server. Item 15. The method according to any one of Item 13 or Item 14, wherein the content consumption demand is caused by at least one of the physical disability, language impairment, or cognitive impairment of the corresponding recipient. Item 16. The method according to any one of items 13-15, wherein the personalized content item is provided to the second device via a Uniform Resource Locator. Item 17. A non-transitory computer-readable medium having instructions stored thereon, the instructions causing a programmable device to perform the following functions when executed: A request is received from a first device associated with a sender to personalize and distribute designed content items based on at least one of a plurality of recipients’ content consumption needs or content consumption preferences, wherein the at least one of the content consumption needs or content consumption preferences includes a data type; The first prompt word is constructed by the prompt word construction unit by appending at least one of the content consumption needs or the content consumption preferences and the designed content item to a first instruction string. The first instruction string includes instructions for the generative model to convert the designed content item into a personalized content item for each receiver based at least one of the corresponding receiver's content consumption needs or content consumption preferences. The first prompt word is provided as input to the generative model, and the personalized content item is received from the generative model as output. Provide the personalized content items to a second device associated with the corresponding recipient; and This causes the user interface of the second device to display the personalized content items. Item 18. The non-transitory computer-readable medium according to Item 17, wherein the instructions, when executed, also cause the programmable device to perform the following functions: Determine at least one of the content consumption needs or content consumption preferences of the corresponding recipient at at least one of the first device, the second device, or the server. Item 19. A non-transitory computer-readable medium according to any one of Item 17 or Item 18, wherein the content consumption demand is caused by at least one of a physical disability, language impairment, or cognitive impairment of the respective recipient. Item 20, a non-transitory computer-readable medium according to any one of items 16-19, wherein the personalized content item is provided to the second device via a Uniform Resource Locator.

[0107] In the foregoing detailed description, numerous specific details have been set forth in order to provide a thorough understanding of the relevant teachings. It will be apparent to those skilled in the art, upon reading this specification, that various aspects can be practiced without these details. In other instances, well-known methods, processes, components, and / or circuits have been described in relatively high-level but undetailed terms to avoid unnecessarily obscuring aspects of the teachings.

[0108] While what is considered the best model and / or other examples has been described above, it should be understood that various modifications may be made therein, and the subject matter disclosed herein can be implemented in various forms and examples, and the teachings can be applied to many applications, only some of which are described herein. The appended claims are intended to cover any and all applications, modifications, and variations that fall within the true scope of this teaching.

[0109] Unless otherwise stated, all measurements, values, ratings, locations, sizes, dimensions, and other specifications set forth in this specification (including those in the following claims) are approximate, not precise. They are intended to have a reasonable range consistent with the functions they address and the conventions of the field to which they belong.

[0110] The scope of protection is defined solely by the following claims. This scope is intended and should be interpreted in accordance with the ordinary meaning of the language used in the claims, when interpreted in accordance with this specification and the examination history followed, and covers all structural and functional equivalents. Nevertheless, none of the claims are intended to cover subject matter that fails to meet the requirements of sections 101, 102, or 103 of the Patent Law, nor should they be interpreted in this manner. Any unintended encirclement of such subject matter is hereby claimed.

[0111] Apart from what has just been stated above, nothing that has been stated or shown is intended or should be construed as giving rise to any component, step, feature, object, benefit, advantage, or equivalent to any other thing that is known to the public, whether or not it is stated in the claims.

[0112] It will be understood that the terms and expressions used herein have the general meaning consistent with those in the relevant query and field of study, unless otherwise specified herein. Relational terms such as "first" and "second" may be used only to distinguish one entity or action from another, without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but may include other elements not expressly listed or inherent to such a process, method, article, or apparatus. Without further constraints, an element modified by "a" or "an" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes that element. Furthermore, the subsequent limitation returning to "the element" or "the element performing certain functions" indicates that "the element" or "the element," alone or in combination with additional identical elements in the process, method, article, or apparatus, is capable of performing all said functions.

[0113] An abstract of this disclosure is provided to allow the reader to quickly determine the nature of the technical disclosure. It should be understood that it is not intended to interpret or limit the scope or meaning of the claims. Furthermore, as can be seen from the foregoing detailed description, various features have been grouped together in various examples for the purpose of simplifying this disclosure. This approach of the disclosure should not be construed as reflecting an intention to require more features than are expressly recited in each claim. Rather, as reflected in the following claims, the subject matter of the invention lies in all features less than those in a single disclosed example. Therefore, the following claims are incorporated into the detailed description, wherein each claim is, in itself, a separately claimed subject matter.

Claims

1. A data processing system (600), comprising: Processor (610); as well as A machine-readable storage medium (630) in communication with the processor, the memory including executable instructions (616) that, when executed by the processor, cause the data processing system to perform the following operations: A request is received from a first device (105) associated with the sender to personalize and distribute a design content item (201) based on at least one of the content consumption needs or content consumption preferences of a plurality of receivers (203), wherein the at least one of the content consumption needs or content consumption preferences includes a data type; The first prompt word is constructed by the prompt word construction unit (124) by attaching at least one of the content consumption needs or the content consumption preferences and the designed content item to a first instruction string, the first instruction string including instructions for the generative model (126a) to convert the designed content item into a personalized content item for each receiver based at least one of the corresponding receiver's content consumption needs or content consumption preferences. The first prompt word is provided as input to the generative model, and the personalized content item is received from the generative model as output. The personalized content item is provided to a second device (105) associated with the corresponding recipient; and This causes the user interface (318) of the second device to display the personalized content items.

2. The data processing system according to claim 1, wherein, The machine-readable storage medium also includes instructions configured to cause the processor, alone or in combination with other processors, to perform the following operations: Determine at least one of the content consumption needs or content consumption preferences of the corresponding recipient at at least one of the first device, the second device, or the server.

3. The data processing system according to any one of claim 1 or claim 2, wherein, The content consumption demand is caused by at least one of the physical disability, language impairment, or cognitive impairment of the corresponding recipient.

4. The data processing system according to any one of claims 1-3, wherein, The content consumption preference is caused by at least one of the corresponding recipient's physical preferences, language preferences, or cognitive preferences.

5. The data processing system according to any one of claims 1-4, wherein, The processor is embedded in the first device, the second device, or the server.

6. The data processing system according to any one of claims 1-5, wherein, The machine-readable storage medium also includes instructions configured to cause the processor, alone or in combination with other processors, to perform the following operations: At least one user feedback regarding the personalized content item is received via the user interface of the second device.

7. The data processing system according to any one of claims 1-6, wherein, The machine-readable storage medium also includes instructions configured to cause the processor, alone or in combination with other processors, to perform the following operations: The second prompt word is constructed by the prompt word construction unit by appending the feedback and the personalized content item to a second instruction string, the second instruction string including instructions for the generative model to generate at least one other personalized content item based on the feedback and the personalized content item by adjusting one or more attributes of the personalized content item based on the feedback, while satisfying at least one of the content consumption needs or the content consumption preferences.

8. The data processing system according to any one of claims 1-7, wherein, The user feedback is collected via user selection of at least one of the following: thumbs up tab, thumbs down tab, neutral tab, or generate more images tab, text input, or a combination thereof.

9. The data processing system according to any one of claims 1-8, wherein, The generative model is a multimodal model.

10. The data processing system according to any one of claims 1-9, wherein, The personalized content items are provided to the second device via a Uniform Resource Locator.

11. The data processing system according to any one of claims 1-10, wherein, The instructions for the generative model include instructions for extracting audio transcripts or instructions for generating audio descriptions based on the designed content items, provided that at least one of the content consumption needs or content consumption preferences of the corresponding receiver is audio. The user interface of the second device presents the audio transcription or the audio description.

12. The data processing system according to any one of claims 1-11, wherein, The data types include text, images, audio, or video.

13. A method comprising: A request is received from a first device (105) associated with the sender to personalize and distribute a design content item (201) based on at least one of the content consumption needs or content consumption preferences of a plurality of receivers (203), wherein the at least one of the content consumption needs or content consumption preferences includes a data type; The first prompt word is constructed by the prompt word construction unit (124) by attaching at least one of the content consumption needs or the content consumption preferences and the designed content item to a first instruction string, the first instruction string including instructions for the generative model (126a) to convert the designed content item into a personalized content item for each receiver based at least one of the corresponding receiver's content consumption needs or content consumption preferences. The first prompt word is provided as input to the generative model, and the personalized content item is received from the generative model as output. The personalized content item is provided to a second device (105) associated with the corresponding recipient; and This causes the user interface (318) of the second device to display the personalized content items.

14. The method of claim 13, further comprising: Determine at least one of the content consumption needs or content consumption preferences of the corresponding recipient at at least one of the first device, the second device, or the server.

15. The method according to any one of claim 13 or claim 14, wherein, The content consumption demand is caused by at least one of the physical disability, language impairment, or cognitive impairment of the corresponding recipient.

16. The method according to any one of claims 13-15, wherein, The personalized content items are provided to the second device via a Uniform Resource Locator.

17. A non-transitory computer-readable medium (630) having instructions (616) stored thereon, said instructions, when executed, causing a programmable device (600) to perform the following functions: A request is received from a first device (105) associated with the sender to personalize and distribute designed content items (201) based on at least one of the content consumption needs or preferences of a plurality of receivers (203), wherein, At least one of the content consumption needs or the content consumption preferences includes a data type; The first prompt word is constructed by the prompt word construction unit (124) by attaching at least one of the content consumption needs or the content consumption preferences and the designed content item to a first instruction string, the first instruction string including instructions for the generative model (126a) to convert the designed content item into a personalized content item for each receiver based at least one of the corresponding receiver's content consumption needs or content consumption preferences. The first prompt word is provided as input to the generative model, and the personalized content item is received from the generative model as output. The personalized content item is provided to the second device associated with the corresponding recipient (105). as well as This causes the user interface (318) of the second device to display the personalized content items.

18. The non-transitory computer-readable medium according to claim 17, wherein, When the instructions are executed, the programmable device also performs the following functions: Determine at least one of the content consumption needs or content consumption preferences of the corresponding recipient at at least one of the first device, the second device, or the server.

19. The non-transitory computer-readable medium according to any one of claim 17 or claim 18, wherein, The content consumption demand is caused by at least one of the physical disability, language impairment, or cognitive impairment of the corresponding recipient.

20. The non-transitory computer-readable medium according to any one of claims 16-19, wherein, The personalized content items are provided to the second device via a Uniform Resource Locator.

Images