Interaction method and device, medium, equipment and computer program product
By generating images related to text in electronic reading, the problem of user attention is solved, user participation and reading experience are improved, the matching between images and text is achieved, and reading interest is enhanced.
Patent Information
- Application Number
- CN202411111776.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-08
AI Technical Summary
During the electronic reading process, due to the long novel, the user's attention is reduced and the reading experience is poor, and the existing technology is difficult to effectively improve user interactivity and reading interest.
By receiving image generation instructions during text reading, generating and displaying target object feature information related to text, the image is automatically generated and user interactivity and reading experience are improved.
It improves user participation and reading interest, saves the amount of operation of users in obtaining generated images, ensures the matching between the images and text, and enhances the reading experience.
Smart Images

Figure CN118915943A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular, to an interaction method, apparatus, medium, device, and computer program product. Background Art
[0002] With the development of the times, more and more users read through electronic reading devices or reading applications. However, during the reading process, users may reduce their concentration due to problems such as the length of the novel, resulting in a poor reading experience. Summary of the invention
[0003] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.
[0004] In a first aspect, the present disclosure provides an interaction method, the method comprising: If an image generation instruction of the associated delivery content is received during the text display process, the target delivery content is displayed; In response to determining that the display of the target delivery content is completed, obtaining a target text corresponding to the image generation instruction from the text; Generating target feature information of a target object corresponding to the target text according to the target text; Based on the target feature information, a generated image corresponding to the target object is generated and displayed.
[0005] In a second aspect, the present disclosure provides an interactive device, the device comprising: A first display module is used to display the target delivery content if an image generation instruction related to the delivery content is received during the text display process; An acquisition module, configured to acquire a target text corresponding to the image generation instruction from the text in response to determining that the display of the target delivery content is completed; A generating module, used for generating target feature information of a target object corresponding to the target text according to the target text; A processing module is used to generate and display a generated image corresponding to the target object based on the target feature information.
[0006] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when the computer program is executed by a processing device.
[0007] In a fourth aspect, the present disclosure provides an electronic device, including: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect.
[0008] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, which implements the steps of the method described in the first aspect when executed by a processor.
[0009] In the above technical solution, interaction can be performed during the text reading process, which improves the diversity of user interaction during the text reading process, so that users can watch the delivered content and obtain the generated image during the text reading process, which is convenient for increasing the browsing volume of the delivered content and improving user participation. At the same time, during the image generation process, there is no need for users to understand and summarize the feature information of the object. The target feature information can be automatically generated based on the read text, and then the generated image can be obtained, saving the user the operation and workload of obtaining the generated image. At the same time, it can also ensure the matching degree between the generated image and the text to a certain extent, avoiding the problem that the user's description of the feature is biased and it is difficult to obtain the image required by the user, further improving the user interaction experience, and at the same time, combined with the generated image, it can increase the user's understanding of the text, which can effectively improve the user's reading interest and viewing interest in the target delivered content, and improve the diversity of interaction during the user's reading process.
[0010] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 It is a flowchart of an interaction method provided according to an embodiment of the present disclosure.
[0012] Figure 2 It is a schematic diagram of a text display interface provided according to an embodiment of the present disclosure.
[0013] Figure 3 It is a schematic diagram of an interactive interface provided according to an embodiment of the present disclosure.
[0014] Figure 4 It is a schematic diagram of an interactive interface provided according to another embodiment of the present disclosure.
[0015] Figure 5 It is a schematic diagram of an image display interface provided according to an embodiment of the present disclosure.
[0016] Figure 6 It is a block diagram of an interaction device provided according to an embodiment of the present disclosure.
[0017] Figure 7 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION
[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0019] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0020] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.
[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0024] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0025] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
[0026] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0027] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.
[0028] At the same time, it is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.
[0029] Figure 1 As shown, it is a flow chart of an interactive method provided according to an embodiment of the present disclosure. Figure 1 As shown, the method may include: In step 11, if an image generation instruction of the associated delivery content is received during the text display process, the target delivery content is displayed.
[0030] Among them, the text can be a text for users to read, such as a novel or online introduction material. Taking the text as a novel as an example, when the user reads the text for a long time, his attention may be reduced. In this embodiment, interaction can be provided to the user during the display of the text. For example, an interaction mark can be displayed in the text display interface, and the user can trigger the interaction operation by clicking on the interaction mark. As another example, a gesture operation that triggers the interaction can be pre-set, and when it is detected that the user inputs the gesture operation in the text display interface, the corresponding interaction can be performed. It can be set based on the actual application scenario, and the present disclosure is not limited to this.
[0031] As an example, an image generation mark may be displayed in the text display interface, and the user may click on the image generation mark to trigger image generation. Figure 2As shown, the image generation mark 21 can prompt the user to confirm the display of the delivery content. If the user clicks to confirm, it means that the user confirms that the delivery content can be displayed. In this scenario, the image generation instruction for generating the associated delivery content can be triggered, and then the delivery content can be jumped to display, so that the user can browse the delivery content during the text reading process, increasing the diversity of interaction. Among them, the delivery content can be video content, audio content, etc. Among them, the image generation mark can be displayed in the form of a floating mark in each text display interface, and can also be displayed in the text display interface corresponding to the last page of each chapter in the text, so as to improve the continuity of the user's text reading.
[0032] In step 12, in response to determining that the display of the target delivery content is completed, a target text corresponding to the image generation instruction is obtained from the text.
[0033] As an example, detection can be performed based on a commonly used method for detecting display of delivery content in the art, such as determining whether the display duration corresponding to the target delivery content meets the target duration. The target duration can be the total duration of the target delivery content, or it can be a duration set based on an actual application scenario, and there is no limitation on this. When it is determined that the display duration corresponding to the target delivery content meets the target duration, it is determined that the display of the target delivery content is completed, and then the target text corresponding to the image generation instruction is obtained to trigger the task of image generation. The target text can be all the text in the text, or it can be part of the text in the text.
[0034] In step 13, target feature information of a target object corresponding to the target text is generated according to the target text.
[0035] As an example, text analysis can be performed based on the target text to determine various objects appearing in the target text. For example, the type of the object can be set in advance, such as a human character, an animal character, an environment, etc. Then, objects of the above various types and their feature information can be determined from the target text.
[0036] For example, the target text can be processed by a natural language processing (NLP) algorithm to extract key information from the target text to generate feature information of the object, such as summary information, and obtain an image description for image generation. If the object can be a person, the target feature information can be information used to describe the character, such as age, personality, appearance, clothing, etc. For example, the algorithm can include but is not limited to: an information extraction algorithm to identify and extract key information such as names, events, and attributes from the unstructured text of the target text; a text summarization algorithm to generate a summary based on the target text to obtain a summary of the role description of the object; a sentiment analysis algorithm to determine the emotions and attitudes of the object by identifying and processing emotional words in the target text; an entity relationship extraction algorithm to identify and understand the relationship between different entities (such as people, places, and events) in the target text. The specific implementation of the above algorithm is a general algorithm in this field and will not be repeated here.
[0037] Among them, the object with the most corresponding text content in the target text can be used as the target object, or the key object corresponding to the text can be determined based on the attribute information of the text, such as the protagonist, and the characteristic information of the protagonist that can be determined can be used as the target characteristic information of the target object.
[0038] In step 14, based on the target feature information, a generated image corresponding to the target object is generated and displayed.
[0039] As an example, the target feature can be input into an image generation model to obtain a corresponding generated image. The image generation model can be pre-trained based on a diffusion model, a large language model, etc., and its training method can be implemented by a general training method, which is not limited here.
[0040] There may be one or more generated images. If there is one generated image, the generated image may be directly displayed in the interactive interface. If there are multiple generated images, each generated image may be displayed as an option in the interactive interface, so that the user can select a generated image that meets his needs from the multiple generated images to download or share, etc., which is convenient for the user's operation.
[0041] Therefore, in the above technical solution, interaction can be performed during the text reading process, which improves the diversity of user interaction during the text reading process, so that users can watch the delivered content and obtain the generated image during the text reading process, which is convenient for increasing the browsing volume of the delivered content and improving user participation. At the same time, during the image generation process, there is no need for users to understand and summarize the feature information of the object. The target feature information can be automatically generated based on the read text, and then the generated image can be obtained, saving the user the operation and workload of obtaining the generated image. At the same time, it can also ensure the matching degree between the generated image and the text to a certain extent, avoiding the problem that the user's description of the feature is biased and it is difficult to obtain the image required by the user, further improving the user interaction experience, and at the same time, combined with the generated image, it can increase the user's understanding of the text, which can effectively improve the user's reading interest and viewing interest in the target delivered content, and improve the diversity of interaction during the user's reading process.
[0042] In a possible embodiment, the text includes multiple chapters, and the multiple chapters in the text can be obtained based on the directory corresponding to the text. Accordingly, in response to determining that the display of the target delivery content is completed, obtaining the target text corresponding to the image generation instruction from the text may include: In response to determining that the presentation of the delivered content is complete, a current chapter presented in the text is determined.
[0043] The current chapter may be determined based on the page corresponding to the currently displayed text in the text. If each chapter is associated with a corresponding page range in the text, the chapter in the range to which the page corresponding to the currently displayed text belongs may be taken as the current chapter.
[0044] The text of the current chapter and the first N chapters corresponding to the current chapter are used as the target text, where N is an integer greater than or equal to zero, which can be set based on the actual application scenario and is not limited in the present disclosure.
[0045] As an example, the text of the current chapter can be used as the target text, so that the target feature information of the target object can be generated based on the text of the target object in the current chapter, thereby improving the matching degree between the generated image and the text of the current reading chapter. As another example, the text of the current chapter and its consecutive historical chapters can be used as the target text. If the current chapter is the Mth chapter, the text of the Mth chapter, the M-1th chapter, the M-2th chapter to the MN chapter can be used as the target text to improve the completeness of the description of the target object in the target text, to provide more comprehensive feature information for image generation, and at the same time to characterize the continuity between multiple generated images corresponding to the same object during the text reading process, thereby improving the accuracy and efficiency of image generation.
[0046] In a possible embodiment, generating target feature information of a target object corresponding to the target text according to the target text may include: Based on the target text and a pre-trained text analysis model, candidate objects corresponding to the target text and feature information of the candidate objects are obtained.
[0047] The text analysis model may be the model implemented based on the NLP algorithm described above to analyze the target text.
[0048] As an example, the candidate objects can be pre-set, such as for each text, the objects in the text that can generate images can be pre-set, such as the protagonists A1, A2, supporting roles B1-Bm, and environments C1-Cn, which can be pre-configured based on the introduction information of the text or the configuration information about the role. In this embodiment, the target text can be input into the text analysis model to generate feature information of each candidate object by the text analysis model. It should be noted that if there is no text description for a candidate object in the target text, the feature information of the candidate object is empty. For example, supporting role B1 appears in Chapter 10, then when generating the feature information of the candidate object based on the text of Chapter 5, the feature information of supporting role B1 is empty.
[0049] Afterwards, in response to receiving a user's selection operation on the candidate object, the selected candidate object is determined as the target object.
[0050] As an example, each candidate object and its feature information can be displayed in the interactive interface so that the user can select and determine the object for which the image is to be generated. The user can select a candidate object by clicking on it, and then the candidate object can be used as the target object. Figure 3 As shown, the interactive interface may include feature information of characters A1 and A2, and the user may select a1 to select character A1, and then character A1 may be used as a target object to generate a generated image corresponding to the character.
[0051] Afterwards, target feature information of the target object is generated according to the feature information of the target object.
[0052] As an example, the characteristic information of the target object can be directly used as the target characteristic information for image generation. As another example, the characteristic information generated above may not conform to the user's understanding and configuration of the role. In the embodiment of the present disclosure, the characteristic information of the target object can be displayed, and the user can edit the characteristic information to obtain the target characteristic information that conforms to his intention and understanding. Thus, through the above technical solution, the characteristic information of each candidate object in the target text can be automatically generated based on the target text, without the need for the user to manually summarize, which greatly reduces the workload of manual extraction and generation, saves user operations, and can also provide users with more display of characteristic information of candidate objects that can be used for image generation, provide users with more comprehensive recommendations, effectively reduce the user's understanding cost in the interactive process of image generation, and improve the efficiency of image generation.
[0053] In a possible embodiment, generating target feature information of the target object according to the feature information of the target object includes: Display the feature information of the target object. As an example, the feature information can be displayed in the same editing area. As another example, the feature information can include information under multiple feature dimensions, wherein the multiple feature dimensions can be pre-set, such as theme dimension, role dimension, style dimension, etc., so as to improve the comprehensiveness and structured representation of feature extraction, and provide more accurate and effective data support for subsequent image generation. In this example scenario, the information of the target object under multiple feature dimensions can be displayed separately. Figure 4 As shown, the user can understand the feature information of the target object more directly, and can quickly determine whether the displayed feature information meets the needs and intentions of the user's image generation.
[0054] Accordingly, generating target feature information of the target object according to the feature information of the target object may include: In response to receiving the user's editing operation on the feature information, the feature information obtained after editing is determined as the target feature information. As an example, if the user believes that the currently displayed feature information does not meet his requirements, he can edit it in the editing area. When the feature information is displayed separately under multiple feature dimensions, if the user believes that the information under a certain feature dimension displayed does not meet his needs, he can edit it, such as deleting or modifying part of the description, or extracting part of the text from the reading text and adding it to the information under the corresponding feature dimension, so as to obtain the edited information.
[0055] like Figure 4As shown, the user can edit the information of the character dimension, and can edit it in the editing box 41, such as adding the feature of "curved eyebrows". The user can submit after the editing is completed. At this time, the information obtained after editing can be determined as the target feature information, that is, the target feature information includes the feature of "curved eyebrows".
[0056] Therefore, users can make modifications and adjustments based on the automatically generated feature information, allowing users to add, delete or change feature information to personalize the feature information according to their own understanding, so as to obtain target feature information that meets the user's intentions, so that images that meet the user's intentions can be generated in the future. At the same time, the diversity of generated images can be improved, the user's satisfaction with the subsequently obtained generated images can be improved, and the user's reading experience can be improved.
[0057] As an example, the delivery content may be an incentive advertisement. When determining the target delivery content to be displayed, it may be randomly selected from the candidate delivery content, thereby improving the randomness of the target delivery content and avoiding users from repeatedly viewing the same delivery content within a period of time, so as to ensure the viewing rate of the target delivery content.
[0058] As another example, if an image generation instruction of the associated delivery content is received during the text display process, displaying the target delivery content may include: If an image generation instruction associated with the delivered content is received during the display of the text, text feature information of the text is obtained.
[0059] The text feature information of the text may be brief description information corresponding to the text. For example, when the text is a novel, the novel is usually associated with brief description information to briefly describe the content of the text. In this embodiment, the text feature information may be the brief description information, which may be obtained through the attributes of the text. For another example, the attribute information of the text may be obtained, and the text feature information may be obtained by analyzing the attribute information based on the large language model and the attribute information.
[0060] Afterwards, the target delivery content is obtained and displayed based on the text feature information.
[0061] As an example, if the target delivery content is obtained from a content server, the text feature information can be sent to the content server so that the content server can match the candidate delivery content based on the text feature information, and distribute the candidate delivery content with the highest match as the target delivery content to display the target delivery content. As another example, if multiple candidate delivery contents are stored locally, matching can be performed based on the local candidate delivery content and text feature information, and the candidate delivery content with the highest match can be displayed as the target delivery content. Therefore, through the above technical solution, the delivery content can be displayed to the user during the user's reading process, thereby improving the viewing rate of the delivery content and the user's sense of participation, and providing support for the effective delivery of the subsequent delivery content.
[0062] In a possible embodiment, the method may further include: In response to receiving an upload operation for the generated image, the generated image, the target feature information and the target text are stored in association.
[0063] As an example, after obtaining the generated image, the user can trigger the generated image to share and interact. For example, the generated images can be displayed in the interactive interface, and the user can select one or more of them to upload, so that they can be displayed in association with the text and can be browsed by other users, such as being liked or commented on by other users, so as to realize interaction between multiple users.
[0064] For example, if the user selects to upload the generated image P1 in the interactive interface, the target text W1, the generated image P1 and the target feature information T1 may be stored in association with each other, so that information can be quickly acquired based on the association.
[0065] In the image display interface corresponding to the target text, each generated image associated with the target text and the target feature information of the generated image are displayed.
[0066] As an example, the image display interface corresponding to the target text may be displayed after all texts corresponding to the target text are displayed, that is, the images uploaded by different users for the target text are displayed on the last page of the target text.
[0067] like Figure 5 As shown, for the target text W1, user U1 uploads image P1, then image P1 and its target feature information T1 are displayed in area Q1; user U2 uploads image P2, then image P2 and its target feature information T2 are displayed in area Q2.
[0068] Therefore, through the above technical solution, different generated images obtained for the same target text can be displayed in a unified manner, further improving the user's multi-dimensional understanding of the target object in the target text, and at the same time, it can also enhance the user's interest in image generation, providing support for increasing the views of the delivered content.
[0069] In a possible embodiment, generating target feature information of a target object corresponding to the target text according to the target text may include: Based on the target text and the pre-trained text analysis model, the candidate object corresponding to the target text and the feature information of the candidate object are obtained. The specific implementation method of this step has been described in detail above and will not be repeated here.
[0070] The candidate object and the feature information of the candidate object are displayed in an image display interface corresponding to the target text.
[0071] Among them, as shown above, images already generated by other users and target feature information corresponding to image generation can be displayed in the image display interface. As an example, the feature information of the candidate object can be displayed preferentially before images already generated by other users and target feature information corresponding to image generation.
[0072] In response to receiving a user's selection of feature information displayed in the image display interface, target feature information of the target object is determined according to the selected feature information.
[0073] In this embodiment, the user can select feature information that meets his or her intention from the multiple feature information displayed based on his or her own understanding and preference. For example, the user can select the feature information of the candidate object, or select the target feature information of the image uploaded by the user. After the user confirms, the feature information selected by the user can be used as the target feature information of the target object for image generation.
[0074] Therefore, through the above-mentioned technical solution, the currently generated candidate object and the feature information of the candidate object and the target feature information applied by other users when generating images can be displayed to the user at the same time in the image display interface, providing a more comprehensive display for the user to confirm the target feature information required for the current image generation, and facilitating sharing and interaction between different users.
[0075] In a possible embodiment, generating and displaying a generated image corresponding to the target object based on the target feature information may include: The text is preprocessed based on the target feature information to obtain a processed text.
[0076] As an example, the text preprocessing may include performing spelling check on the target feature information, processing synonyms and phrases, etc., to maintain semantic consistency of the text in the target feature information.
[0077] Afterwards, key information is extracted from the processed text to obtain key information corresponding to the processed text.
[0078] As an example, in this step, key information for processing text, such as gender, age, posture, expression, clothing and other relevant features, can be extracted based on NLP technology. The dimensions of the key information can be pre-set based on the actual application scenario, and the present disclosure does not limit this.
[0079] Afterwards, a target prompt text is constructed based on the key information, and based on the target prompt text and a pre-trained image generation model, the generated image is generated and displayed.
[0080] As an example, in this step, the key information can be constructed based on the prompt template to obtain the target prompt text. As another example, it can also be based on a semantic conversion algorithm, such as converting the key information into the target prompt text through semantic analysis and semantic understanding. By constructing the target prompt text, the key information can be converted into structured information, such as structuring the key information into a sentence or a series of sentences with clear semantic descriptions. For example, the target prompt text prompt can contain a sentence with the same semantics as the key information, and the format of the sentence matches the input of the image generation model. For example, the target feature information is represented as "a young lady with a lovely smile and shining eyes, wearing a blue dress", and the target prompt text obtained by the above steps is represented as "depicting a young lady with a lovely smile and shining eyes, she is wearing a blue dress".
[0081] After that, the target prompt text can be input into the image generation model, so that the output of the image model is used as the generated image and displayed. Among them, the image generation model can be a model based on a generative adversarial network, which can be trained by pre-building training data containing prompt text and its corresponding image. The specific training method is not described in detail.
[0082] Furthermore, new training data can be obtained from the images displayed in the image display interface and their target feature information. For example, a portion of the image can be selected from the images displayed in the image display interface, and the corresponding prompt text can be obtained by processing the image based on the target feature information. The image generation model can then be updated based on the selected image and its corresponding prompt text, thereby further improving the accuracy of the image generation model.
[0083] Therefore, through the technical solution, by processing the target feature information to obtain the target prompt text, it is possible to provide a structured description text for image generation, improve the accuracy of the image generation model's understanding of the target prompt text, and thus improve the accuracy of the generated image.
[0084] Based on the same inventive concept, the present disclosure also provides an interactive device, such as Figure 6 As shown, the device 10 comprises: The first display module 100 is used to display the target delivery content if an image generation instruction related to the delivery content is received during the text display process; An acquisition module 200 is used to acquire a target text corresponding to the image generation instruction from the text in response to determining that the display of the target delivery content is completed; A generating module 300 is used to generate target feature information of a target object corresponding to the target text according to the target text; The processing module 400 is used to generate and display a generated image corresponding to the target object based on the target feature information.
[0085] Optionally, the generating module includes: A first processing submodule, configured to obtain a candidate object corresponding to the target text and feature information of the candidate object based on the target text and a pre-trained text analysis model; A first determination submodule, configured to, in response to receiving a user's selection operation on the candidate object, determine the selected candidate object as the target object; The first generating submodule is used to generate target feature information of the target object according to the feature information of the target object.
[0086] Optionally, the first generating submodule includes: A first display submodule, used to display feature information of the target object; The second determining submodule is configured to, in response to receiving an editing operation of the user on the feature information, determine the feature information obtained after editing as the target feature information.
[0087] Optionally, the first display module includes: A first acquisition submodule is used to acquire text feature information of the text if an image generation instruction associated with the delivery content is received during the display process of the text; The second acquisition submodule is used to acquire and display the target delivery content based on the text feature information.
[0088] Optionally, the device further comprises: A storage module, configured to store the generated image, the target feature information and the target text in association with each other in response to receiving an upload operation on the generated image; The second display module is used to display each generated image associated with the target text and target feature information of the generated image in an image display interface corresponding to the target text.
[0089] Optionally, the generating module includes: A first processing submodule, configured to obtain a candidate object corresponding to the target text and feature information of the candidate object based on the target text and a pre-trained text analysis model; A second display submodule, used to display the candidate object and the feature information of the candidate object in the image display interface corresponding to the target text; The third determination submodule is configured to determine target feature information of the target object according to the selected feature information in response to receiving a user's selection of feature information displayed in the image display interface.
[0090] Optionally, the text comprises a plurality of chapters; The acquisition module comprises: A fourth determination submodule, configured to determine a current chapter displayed in the text in response to determining that the display of the delivered content is completed; The fifth determination submodule is used to use the text of the current chapter and the previous N chapters corresponding to the current chapter as the target text, wherein N is an integer greater than or equal to zero.
[0091] Optionally, the processing module includes: A second processing submodule is used to perform text preprocessing based on the target feature information to obtain a processed text; A third processing submodule is used to extract key information from the processed text to obtain key information corresponding to the processed text; The fourth processing submodule is used to construct a target prompt text based on the key information, and to generate and display the generated image based on the target prompt text and a pre-trained image generation model.
[0092] Reference below Figure 7 , which shows a schematic diagram of the structure of an electronic device (such as a terminal device or a server) 600 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0093] like Figure 7 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0094] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.
[0095] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0096] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0097] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0098] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0099] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: if an image generation instruction of associated delivery content is received during the display of text, the electronic device displays the target delivery content; in response to determining that the display of the target delivery content is completed, the target text corresponding to the image generation instruction is obtained from the text; according to the target text, target feature information of the target object corresponding to the target text is generated; based on the target feature information, a generated image corresponding to the target object is generated and displayed.
[0100] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0101] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0102] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of the module does not limit the module itself in some cases. For example, the first display module may also be described as "a module that displays the target delivery content if an image generation instruction of the associated delivery content is received during the text display process."
[0103] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0104] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0105] According to one or more embodiments of the present disclosure, Example 1 provides an interaction method, wherein the method includes: If an image generation instruction of the associated delivery content is received during the text display process, the target delivery content is displayed; In response to determining that the display of the target delivery content is completed, obtaining a target text corresponding to the image generation instruction from the text; Generating target feature information of a target object corresponding to the target text according to the target text; Based on the target feature information, a generated image corresponding to the target object is generated and displayed.
[0106] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein generating target feature information of a target object corresponding to the target text according to the target text includes: Based on the target text and a pre-trained text analysis model, obtaining candidate objects corresponding to the target text and feature information of the candidate objects; In response to receiving a user's selection operation on the candidate object, determining the selected candidate object as the target object; According to the feature information of the target object, target feature information of the target object is generated.
[0107] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2, wherein generating target feature information of the target object according to the feature information of the target object includes: Displaying characteristic information of the target object; In response to receiving the editing operation of the user on the feature information, the feature information obtained after editing is determined as the target feature information.
[0108] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 1, wherein, in the process of displaying the text, if an image generation instruction of the associated delivery content is received, displaying the target delivery content includes: If an image generation instruction associated with the delivered content is received during the display of the text, text feature information of the text is obtained; The target delivery content is obtained and displayed based on the text feature information.
[0109] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 1, wherein the method further includes: In response to receiving an upload operation for the generated image, the generated image, the target feature information and the target text are stored in association with each other; In the image display interface corresponding to the target text, each generated image associated with the target text and the target feature information of the generated image are displayed.
[0110] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 5, wherein generating target feature information of a target object corresponding to the target text according to the target text includes: Based on the target text and a pre-trained text analysis model, obtaining candidate objects corresponding to the target text and feature information of the candidate objects; Displaying the candidate object and feature information of the candidate object in an image display interface corresponding to the target text; In response to receiving a user's selection of feature information displayed in the image display interface, target feature information of the target object is determined according to the selected feature information.
[0111] According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 1, wherein the text includes a plurality of chapters; In response to determining that the display of the target delivery content is completed, obtaining the target text corresponding to the image generation instruction from the text includes: In response to determining that the delivery content display is complete, determining a current chapter displayed in the text; The text of the current chapter and the first N chapters corresponding to the current chapter are used as the target text, where N is an integer greater than or equal to zero.
[0112] According to one or more embodiments of the present disclosure, Example 8 provides the method of Example 1, wherein the generating and displaying a generated image corresponding to the target object based on the target feature information includes: Perform text preprocessing based on the target feature information to obtain processed text; Extracting key information from the processed text to obtain key information corresponding to the processed text; A target prompt text is constructed based on the key information, and the generated image is generated and displayed based on the target prompt text and a pre-trained image generation model.
[0113] According to one or more embodiments of the present disclosure, Example 9 provides an interactive device, the device comprising: A first display module is used to display the target delivery content if an image generation instruction related to the delivery content is received during the text display process; An acquisition module, configured to acquire a target text corresponding to the image generation instruction from the text in response to determining that the display of the target delivery content is completed; A generating module, used for generating target feature information of a target object corresponding to the target text according to the target text; A processing module is used to generate and display a generated image corresponding to the target object based on the target feature information.
[0114] According to one or more embodiments of the present disclosure, Example 10 provides a computer-readable medium having a computer program stored thereon, and when the computer program is executed by a processing device, the steps of the method described in any one of Examples 1-8 are implemented.
[0115] According to one or more embodiments of the present disclosure, Example 11 provides an electronic device, including: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of any one of the methods described in Examples 1-8.
[0116] According to one or more embodiments of the present disclosure, Example 12 provides a computer program product, including a computer program, which implements the steps of any one of the methods of Examples 1-8 when executed by a processor.
[0117] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.
[0118] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0119] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Claims
1. An interactive method, characterized in that: The method comprises: If an image generation instruction of the associated delivery content is received during the text display process, the target delivery content is displayed; In response to determining that the display of the target delivery content is completed, obtaining a target text corresponding to the image generation instruction from the text; Generating target feature information of a target object corresponding to the target text according to the target text; Based on the target feature information, a generated image corresponding to the target object is generated and displayed.
2. The method according to claim 1, characterized in that The step of generating target feature information of a target object corresponding to the target text according to the target text includes: Based on the target text and a pre-trained text analysis model, obtaining candidate objects corresponding to the target text and feature information of the candidate objects; In response to receiving a user's selection operation on the candidate object, determining the selected candidate object as the target object; According to the feature information of the target object, target feature information of the target object is generated.
3. The method according to claim 2, characterized in that The step of generating target feature information of the target object according to the feature information of the target object includes: Displaying characteristic information of the target object; In response to receiving the editing operation of the user on the feature information, the feature information obtained after editing is determined as the target feature information.
4. The method according to claim 1, characterized in that: If an image generation instruction of the associated delivery content is received during the text display process, the target delivery content is displayed, including: If an image generation instruction associated with the delivered content is received during the display of the text, text feature information of the text is obtained; The target delivery content is obtained and displayed based on the text feature information.
5. The method according to claim 1, characterized in that The method further comprises: In response to receiving an upload operation for the generated image, the generated image, the target feature information and the target text are stored in association with each other; In the image display interface corresponding to the target text, each generated image associated with the target text and the target feature information of the generated image are displayed.
6. The method according to claim 5, characterized in that The step of generating target feature information of a target object corresponding to the target text according to the target text includes: Based on the target text and a pre-trained text analysis model, obtaining candidate objects corresponding to the target text and feature information of the candidate objects; Displaying the candidate object and feature information of the candidate object in an image display interface corresponding to the target text; In response to receiving a user's selection of feature information displayed in the image display interface, target feature information of the target object is determined according to the selected feature information.
7. The method according to claim 1, characterized in that Said text consists of a number of chapters; In response to determining that the display of the target delivery content is completed, obtaining the target text corresponding to the image generation instruction from the text includes: In response to determining that the delivery content display is complete, determining a current chapter displayed in the text; The text of the current chapter and the first N chapters corresponding to the current chapter are used as the target text, where N is an integer greater than or equal to zero.
8. The method according to claim 1, characterized in that The step of generating and displaying a generated image corresponding to the target object based on the target feature information includes: Perform text preprocessing based on the target feature information to obtain processed text; Extracting key information from the processed text to obtain key information corresponding to the processed text; A target prompt text is constructed based on the key information, and the generated image is generated and displayed based on the target prompt text and a pre-trained image generation model.
9. An interactive device, characterized in that: The device comprises: A first display module is used to display the target delivery content if an image generation instruction related to the delivery content is received during the text display process; An acquisition module, configured to acquire a target text corresponding to the image generation instruction from the text in response to determining that the display of the target delivery content is completed; A generating module, used for generating target feature information of a target object corresponding to the target text according to the target text; A processing module is used to generate and display a generated image corresponding to the target object based on the target feature information.
10. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method according to any one of claims 1 to 8 are implemented.
11. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 8.
12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.
