Picture insertion method and device, electronic equipment and storage medium
By automatically obtaining and inserting pictures that match the chapters in the big model, the problems of low image insertion efficiency and high error rate in the creation of long-form large-scale texts are solved, and efficient and accurate picture insertion is achieved.
Patent Information
- Application Number
- CN202410939446.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-29
AI Technical Summary
During the creation of a long-form text of a large model, users need to manually upload and insert pictures, resulting in increased labor and time costs, reduced processing efficiency, and prone to image insertion errors.
By obtaining the target image and automatically determining its insertion position, using the big model to parse the image content and match the chapter text, automatically complete the picture insertion, avoiding manual operations by users, and ensuring the relevance of the picture and the chapter content.
Save labor and time costs, improve processing efficiency, reduce image insertion errors, and improve the accuracy of insertion results.
Smart Images

Figure CN119047438A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to image insertion methods, devices, electronic devices and storage media in the fields of deep learning, large models, natural language processing and computer vision. Background Art
[0002] With the development of technology, large models have been widely used in different scenarios. Large models refer to large language models (LLM), which are deep learning models trained with a large amount of text data. They can generate natural language text or understand the meaning of natural language text. For example, large models can be used to create long texts, that is, to generate long texts. Summary of the invention
[0003] The present disclosure provides a method, device, electronic device and storage medium for inserting a picture.
[0004] A method for inserting a picture, comprising:
[0005] In response to determining that the user needs to insert a picture into any chapter in the target text, obtaining a target picture to be inserted;
[0006] In response to determining that the target image is related to the chapter text content of the chapter, the paragraph content that best matches the target image is selected from the paragraph contents included in the chapter text content as the target content, and the insertion position of the target image is determined based on the target content and inserted.
[0007] A picture inserting device comprises: a picture acquiring module and a picture inserting module;
[0008] The picture acquisition module is used to acquire a target picture to be inserted in response to determining that the user needs to insert a picture into any chapter in the target text;
[0009] The image insertion module is used to respond to determining that the target image is related to the chapter text content of the chapter, select the paragraph content that best matches the target image from the paragraph contents included in the chapter text content as the target content, determine the insertion position of the target image according to the target content, and insert it.
[0010] An electronic device, comprising:
[0011] at least one processor; and
[0012] a memory communicatively connected to the at least one processor; wherein,
[0013] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.
[0014] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method as described above.
[0015] A computer program product comprises a computer program / instruction, wherein the computer program / instruction implements the above method when executed by a processor.
[0016] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0018] Figure 1 This is a flow chart of the first embodiment of the picture insertion method disclosed in the present invention;
[0019] Figure 2 A schematic diagram of the topic input by the user and the generated long article outline described in the present disclosure;
[0020] Figure 3 A schematic diagram of the insertion effect after the target image is inserted in the manner described in the present disclosure;
[0021] Figure 4 This is a flow chart of a second embodiment of the picture insertion method disclosed in the present invention;
[0022] Figure 5 It is a schematic diagram of the composition structure of the picture insertion device embodiment 500 of the present disclosure;
[0023] Figure 6 A schematic block diagram of an electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0024] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0025] In addition, it should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0026] Figure 1 FIG. 1 is a flow chart of the first embodiment of the picture insertion method disclosed in the present invention. Figure 1 As shown, the following specific implementation methods are included.
[0027] In step 101, in response to determining that the user needs to insert a picture into any chapter in the target text, a target picture to be inserted is obtained.
[0028] In step 102, in response to determining that the target image is related to the chapter text content of the chapter, the paragraph content that best matches the target image is selected from the paragraph contents included in the chapter text content as the target content, and the insertion position of the target image is determined based on the target content and inserted.
[0029] When using large models to create long articles, users often have the need to upload pictures, but currently pictures are mainly uploaded based on manual operations of users, which increases manpower and time costs and reduces processing efficiency.
[0030] By adopting the scheme described in the embodiment of the disclosed scheme, the insertion position of the target image to be inserted can be automatically determined and inserted through a series of processing without the need for manual operation by the user, thereby saving manpower and time costs and improving processing efficiency. Moreover, the insertion position can be determined and inserted only after ensuring that the target image is related to the chapter text content of the chapter to be inserted, thereby avoiding the problem of image insertion errors as much as possible and further improving the accuracy of the insertion results.
[0031] When users use the big model to create long articles, they can enter the corresponding topic on the interactive interface according to actual needs, such as "Help me write a long article, the topic is: artificial breeding technology of giant pandas", and accordingly, the long article outline generated by the big model can be displayed to the user, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the topic input by the user and the generated long article outline described in the present disclosure. If necessary, the user can also modify the displayed long article outline.
[0032] In addition, the user can issue a picture insertion instruction for any chapter in the long article outline. Accordingly, the large model can generate the chapter text content according to the outline of the chapter, and then perform the picture insertion processing in the manner described in the present disclosure. Alternatively, the user can also generate a specific long article (long text) according to the long article outline, and then issue a picture insertion instruction to any chapter in the long text, and then perform the picture insertion processing in the manner described in the present disclosure.
[0033] The chapters usually refer to the contents under the second or third level headings. Figure 2 For example, "Chapter 2 Biological Characteristics of Giant Pandas" is a second-level title, and "2.1 Ecological Habits and Distribution" is a third-level title. Chapter text content refers to the text content of the entire chapter, and includes paragraph break information, etc.
[0034] For the target image uploaded by the user, it can be determined whether it is related to the chapter text content of the chapter to be inserted. In some embodiments of the present disclosure, the image content analysis result of the target image can be obtained, and the target text content can be determined according to the chapter text content, and then it can be determined whether the target image is related to the chapter text content according to the image content analysis result and the target text content.
[0035] The result of image content analysis is a piece of text information, which may include descriptive information about the image content, for example, there are two giant pandas in the picture, and they are sitting on a bamboo chair...
[0036] In some embodiments of the present disclosure, a picture uniform resource locator (URL) corresponding to a target picture may be obtained, and a picture content parsing result of the target picture may be generated based on the picture URL using a parsing model.
[0037] There is no restriction on how to obtain the image URL corresponding to the target image. For example, an existing implementation method can be used. In addition, there is no restriction on the specific model of the parsing model. For example, it can be a pre-trained model of explaining pictures. The number of characters included in the picture content parsing result is usually less than 100, and the content strictly conforms to the facts of the picture.
[0038] Accordingly, with the help of the parsing model, the image content parsing results of the target image can be obtained efficiently and accurately, thus laying a good foundation for subsequent processing.
[0039] In some embodiments of the present disclosure, a method for determining target text content based on chapter text content may include: in response to determining that the number of characters included in the chapter text content is greater than a first threshold, summarizing the chapter text content, and using the summary result as the target text content, the number of characters included in the summary result is less than or equal to the first threshold, and in response to determining that the number of characters included in the chapter text content is less than or equal to the first threshold, using the chapter text content as the target text content.
[0040] The specific value of the first threshold can be determined according to actual needs, such as 3000.
[0041] That is to say, we can first determine whether the chapter text content is an ultra-long text, that is, whether the text length exceeds 3,000 characters. If so, we can summarize the chapter text content (that is, content extraction) to obtain a summary result with a text length not exceeding 3,000, and then use the summary result as the target text content. If not, we can directly use the chapter text content as the target text content.
[0042] Through the above method, it is possible to effectively limit the text length of the target text content, thereby facilitating subsequent processing and avoiding the problem of excessive text length, thereby improving subsequent processing efficiency, etc.
[0043] Furthermore, whether the target image is related to the chapter text content can be determined based on the obtained image content analysis results and the target text content. In some embodiments of the present disclosure, the image content analysis results and the target text content can be used as inputs of a discriminant model to obtain an output discriminant result, which includes: the target image is related to the chapter text content, or the target image is not related to the chapter text content.
[0044] There is no restriction on the specific model of the discriminant model. For example, it can be the ERNIE-3.5-8K model, where ERNIE stands for Enhanced Language Representation with InformativeEntities. The ERNIE-3.5-8K model is a large-scale language model that covers a large amount of Chinese and English corpora and has strong reasoning capabilities. It can meet the requirements of most scenarios such as dialogue question and answer, creation generation, and plug-in application.
[0045] In the scheme disclosed in the present invention, with the help of the powerful reasoning ability of the discriminant model, a discriminant result of whether the target image is related to the chapter text content can be generated based on the input image content analysis results and the target text content. If so, subsequent processing can be continued; if not, processing can be terminated, and the user can be prompted through text, etc.
[0046] In some embodiments of the present disclosure, in response to the target image being related to the chapter text content, the identification result may further include: the name and caption information of the target image generated according to the image content analysis result.
[0047] The picture title refers to the name of the picture, such as "The living habits of giant pandas", and the caption information refers to the content annotation / explanation information of the picture, which can be extracted from the picture content analysis results, such as "The picture shows the interaction of giant pandas on bamboo chairs, reflecting their living environment and habits."
[0048] In addition, the chapter text content may be segmented into paragraphs. For example, the chapter text content may be segmented into paragraphs according to line breaks, thereby obtaining the segmented contents of each paragraph.
[0049] Afterwards, the paragraph content that best matches the target image can be selected from the segmented paragraph contents as the target content.
[0050] In some embodiments of the present disclosure, a first text vector representation corresponding to a target image can be obtained, and a second text vector representation corresponding to each paragraph content can be obtained respectively, and then the similarity between the first text vector representation and each second text vector representation can be obtained respectively, and the paragraph content with the highest similarity can be used as the target content.
[0051] For example, assuming that there are 4 paragraph contents, namely paragraph content a, paragraph content b, paragraph content c and paragraph content d, and the corresponding second text vector representations are second text vector representation a, second text vector representation b, second text vector representation c and second text vector representation d, then the similarity between the second text vector representation a and the first text vector representation, the similarity between the second text vector representation b and the first text vector representation, the similarity between the second text vector representation c and the first text vector representation, and the similarity between the second text vector representation d and the first text vector representation can be obtained respectively, and then the similarity with the largest value can be selected from the four obtained similarities, and the paragraph content corresponding to the similarity with the largest value can be used as the target content. Among them, the similarity between two text vector representations can be cosine similarity.
[0052] In some embodiments of the present disclosure, the acquired target image title and caption information may be spliced to obtain a splicing result, and an embedding model may be used to generate a text vector representation corresponding to the splicing result as a first text vector representation. In addition, the embedding model may be used to generate a second text vector representation corresponding to each paragraph content.
[0053] There is no restriction on how to perform splicing. For example, line breaks can be used for splicing, and the embedding model can be used to generate a text vector representation corresponding to the splicing result as the first text vector representation. In addition, each paragraph content can be traversed, and the embedding model can be used to generate a second text vector representation corresponding to each paragraph content.
[0054] It can be seen that through the above processing, the first text vector representation simultaneously integrates the name and caption information of the target image, thereby enabling the first text vector representation to more accurately reflect the image content of the target image. Accordingly, the target content is determined based on the similarity between the first text vector representation and the second text vector representation corresponding to each paragraph content, which can improve the accuracy of the determination result.
[0055] The embedding model may be an Embedding-V1 model, etc. The Embedding-V1 model is a text representation model based on the Wenxinda model technology, which can convert text into a vector form represented by numerical values.
[0056] Furthermore, the insertion position of the target image can be determined according to the target content and then inserted.
[0057] In some embodiments of the present disclosure, the name of the target image, the caption information, and the corresponding image URL may be inserted into the insertion position.
[0058] In addition, in some embodiments of the present disclosure, the name, caption information and corresponding image URL of the target image may be edited into a lightweight markup language (markdown) format or a fifth edition hierarchical data format (H5, HierarchicalData Format version 5) and then inserted into the insertion position.
[0059] Through the above processing, the image content inserted into the target text can include various information such as image name, caption information and image URL, thereby improving the image presentation effect. Moreover, through format editing, the inserted target image can be better compatible with the original text content, thereby further improving the image presentation effect.
[0060] In some embodiments of the present disclosure, a front position or a rear position adjacent to the position where the target content is located may be used as the insertion position of the target image.
[0061] Figure 3 FIG. 1 is a schematic diagram of the insertion effect after inserting the target image in the manner described in the present disclosure. Figure 3 As shown, assuming that the following content is the target content, it can be seen that the target image is inserted in front of the target content.
[0062] Through the adjacent setting, users can more conveniently view the target image and its corresponding target content.
[0063] Combined with the above introduction, Figure 4 FIG. 1 is a flow chart of the second embodiment of the picture insertion method disclosed in the present invention. Figure 4 As shown, the following specific implementation methods are included.
[0064] In step 401, in response to determining that the user needs to insert a picture into any chapter in the target text, a target picture to be inserted is obtained.
[0065] In step 402, the picture content analysis result of the target picture is obtained.
[0066] For example, the image URL corresponding to the target image may be obtained, and based on the image URL, the required image content parsing result may be generated using the parsing model.
[0067] In step 403, for the chapter text content of the chapter to be inserted, determine whether it is an extra-long text. If so, execute step 404; otherwise, execute step 405.
[0068] For example, if it is determined that the number of characters included in the chapter text content is greater than a first threshold, then it can be determined that the chapter text content is an extra-long text; otherwise, it is not an extra-long text.
[0069] In step 404 , the chapter text content is summarized, and the summary result is used as the target text content, and then step 406 is executed.
[0070] The number of characters included in the summary result is less than or equal to a first threshold.
[0071] In step 405 , the chapter text content is used as the target text content, and then step 406 is executed.
[0072] In step 406, the image content analysis result and the target text content are used as inputs of the discrimination model to obtain an output discrimination result.
[0073] In step 407, it is determined whether the target image is related to the chapter text content according to the discrimination result. If so, step 408 is executed, otherwise, the process ends.
[0074] In step 408, the chapter text content is segmented into paragraphs to obtain the content of each paragraph, and the embedding model is used to generate a second text vector representation corresponding to each paragraph content.
[0075] In step 409, the title and caption information of the target image included in the discrimination result are spliced to obtain a splicing result, and a first text vector representation corresponding to the splicing result is generated using an embedding model.
[0076] In step 410, the similarities between the first text vector representation and each second text vector representation are respectively obtained, and the paragraph content with the highest similarity is used as the target content.
[0077] In step 411, the target image, caption information and image URL are edited into markdown format and inserted into the front position adjacent to the target content, and then the process ends.
[0078] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the described order of actions, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure. In addition, for parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0079] The above is an introduction to the method embodiment. The following is a further explanation of the scheme disclosed in the present invention through an apparatus embodiment.
[0080] Figure 5 FIG. 5 is a schematic diagram of the structure of the picture insertion device embodiment 500 of the present disclosure. Figure 5 As shown, it includes: a picture acquisition module 501 and a picture insertion module 502.
[0081] The picture acquisition module 501 is used to acquire a target picture to be inserted in response to determining that the user needs to insert a picture into any chapter in the target text.
[0082] The image insertion module 502 is used to respond to determining that the target image is related to the chapter text content, select the paragraph content that best matches the target image from the paragraph contents included in the chapter text content as the target content, determine the insertion position of the target image based on the target content, and insert it.
[0083] By adopting the scheme described in the above-mentioned device embodiment, the insertion position of the target image to be inserted can be automatically determined and inserted through a series of processing without the need for manual operation by the user, thereby saving manpower and time costs and improving processing efficiency. Moreover, the insertion position can be determined and inserted only after ensuring that the target image is related to the chapter text content of the chapter to be inserted, thereby avoiding the problem of image insertion errors as much as possible and improving the accuracy of the insertion results.
[0084] In some embodiments of the present disclosure, the image insertion module 502 can obtain the image content analysis results of the target image, and can determine the target text content based on the chapter text content, and then can determine whether the target image is related to the chapter text content based on the image content analysis results and the target text content.
[0085] The picture content parsing result is a piece of text information, which may include description information about the picture content. In some embodiments of the present disclosure, the picture insertion module 502 may obtain the picture URL corresponding to the target picture, and may generate the picture content parsing result of the target picture based on the picture URL using the parsing model.
[0086] In addition, in some embodiments of the present disclosure, the way in which the picture insertion module 502 determines the target text content based on the chapter text content may include: in response to determining that the number of characters included in the chapter text content is greater than a first threshold, summarizing the chapter text content and using the summary result as the target text content, the number of characters included in the summary result is less than or equal to the first threshold, and in response to determining that the number of characters included in the chapter text content is less than or equal to the first threshold, using the chapter text content as the target text content.
[0087] Further, whether the target image is related to the chapter text content can be determined based on the obtained image content analysis result and the target text content. In some embodiments of the present disclosure, the image insertion module 502 can use the image content analysis result and the target text content as inputs to the discrimination model to obtain an output discrimination result, which includes: the target image is related to the chapter text content, or the target image is not related to the chapter text content.
[0088] In some embodiments of the present disclosure, in response to determining that the target image is related to the chapter text content, the image insertion module 502 can perform paragraph segmentation processing on the chapter text content to obtain segmented paragraph contents, and then select the paragraph content that best matches the target image from the segmented paragraph contents as the target content.
[0089] In some embodiments of the present disclosure, the image insertion module 502 can obtain a first text vector representation corresponding to the target image, and can respectively obtain a second text vector representation corresponding to each paragraph content, and then can respectively obtain the similarity between the first text vector representation and each second text vector representation, and can use the paragraph content with the highest similarity as the target content.
[0090] In some embodiments of the present disclosure, in response to the target image being related to the chapter text content, the discrimination result may also include: the name and caption information of the target image generated according to the image content analysis result; accordingly, the image insertion module 502 may splice the acquired name and caption information of the target image to obtain a splicing result, and may use an embedding model to generate a text vector representation corresponding to the splicing result as a first text vector representation; in addition, the embedding model may be used to generate second text vector representations corresponding to the contents of each paragraph.
[0091] Furthermore, the picture inserting module 502 may also determine the insertion position of the target picture according to the target content and insert the target picture.
[0092] In some embodiments of the present disclosure, the picture inserting module 502 may insert the name of the target picture, the caption information, and the corresponding picture URL into the insertion position.
[0093] In addition, in some embodiments of the present disclosure, the image insertion module 502 may edit the name, caption information, and corresponding image URL of the target image into a markdown format or an H5 format, and then insert the result into the insertion position.
[0094] Furthermore, in some embodiments of the present disclosure, the picture inserting module 502 may use the front position or the back position adjacent to the position where the target content is located as the insertion position of the target picture.
[0095] Figure 5 The specific working process of the illustrated device embodiment can refer to the relevant description in the aforementioned method embodiment and will not be described in detail.
[0096] The scheme disclosed in the present invention can be applied to the field of artificial intelligence, especially to the fields of deep learning, large models, natural language processing and computer vision. Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It has both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.
[0097] In addition, the texts and pictures in the embodiments described in this disclosure are not targeted at a specific user and do not reflect the personal information of a specific user. In the technical solution of this disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good customs.
[0098] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0099] Figure 6 A schematic block diagram of an electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0100] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 to a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0101] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0102] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI, Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP, Digital Signal Processing), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the methods described in the present disclosure. For example, in some embodiments, the methods described in the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the methods described in the present disclosure may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the method described in the present disclosure in any other appropriate manner (for example, by means of firmware).
[0103] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0104] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0105] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM, Electronically Programmable Read-Only Memory), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM, Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0106] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0107] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communication (e.g., a communication network) in any form or medium. Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0108] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0109] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0110] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A method for inserting a picture, comprising: In response to determining that the user needs to insert a picture into any chapter in the target text, obtaining a target picture to be inserted; In response to determining that the target image is related to the chapter text content of the chapter, the paragraph content that best matches the target image is selected from the paragraph contents included in the chapter text content as the target content, and the insertion position of the target image is determined based on the target content and inserted.
2. The method according to claim 1, wherein: Determining that the target image is related to the chapter text content of the chapter includes: Obtaining a picture content analysis result of the target picture; Determine the target text content according to the chapter text content; Whether the target image is related to the chapter text content is determined based on the image content analysis result and the target text content.
3. The method according to claim 2, wherein: The obtaining of the picture content analysis result of the target picture includes: The picture uniform resource locator corresponding to the target picture is obtained, and based on the picture uniform resource locator, a parsing model is used to generate the picture content parsing result.
4. The method according to claim 2, wherein: Determining the target text content according to the chapter text content includes: In response to determining that the number of characters included in the chapter text content is greater than a first threshold, summarizing the chapter text content, using the summary result as the target text content, and the number of characters included in the summary result is less than or equal to the first threshold; In response to determining that the number of characters included in the chapter text content is less than or equal to the first threshold, the chapter text content is used as the target text content.
5. The method according to claim 2, wherein: The determining whether the target image is related to the chapter text content according to the image content analysis result and the target text content includes: The image content analysis result and the target text content are used as inputs of a discrimination model to obtain an output discrimination result, wherein the discrimination result includes: the target image is related to the chapter text content, or the target image is not related to the chapter text content.
6. The method according to claim 5, wherein: The step of selecting the paragraph content that best matches the target image from the paragraph contents included in the chapter text content as the target content includes: Performing paragraph segmentation processing on the chapter text content to obtain the content of each paragraph; Obtaining a first text vector representation corresponding to the target image; Obtain the second text vector representation corresponding to each paragraph content respectively; The similarities between the first text vector representation and each second text vector representation are respectively obtained, and the paragraph content with the highest similarity is used as the target content.
7. The method according to claim 6, wherein: In response to the target image being related to the chapter text content, the determination result further includes: a name and illustration information of the target image generated according to the image content analysis result; The obtaining of the first text vector representation corresponding to the target image comprises: splicing the image name and the caption information to obtain a splicing result, and using an embedding model to generate a text vector representation corresponding to the splicing result as the first text vector representation; The obtaining of the second text vector representation corresponding to each paragraph content includes: using the embedding model to generate the second text vector representation corresponding to each paragraph content.
8. The method according to claim 7, wherein: The inserting includes: inserting the picture name, the graphic information and the picture uniform resource locator corresponding to the target picture into the inserting position.
9. The method according to claim 8, wherein: The inserting of the picture name, the caption information and the picture uniform resource locator corresponding to the target picture into the insertion position includes: editing the picture name, the caption information and the picture uniform resource locator into lightweight markup language format or fifth edition hierarchical data format, and then inserting them into the insertion position.
10. The method according to any one of claims 1 to 9, wherein: Determining the insertion position of the target image according to the target content includes: using a front position or a rear position adjacent to the position where the target content is located as the insertion position of the target image.
11. A picture inserting device, comprising: Picture acquisition module and picture insertion module; The picture acquisition module is used to acquire a target picture to be inserted in response to determining that the user needs to insert a picture into any chapter in the target text; The image insertion module is used to respond to determining that the target image is related to the chapter text content of the chapter, select the paragraph content that best matches the target image from the paragraph contents included in the chapter text content as the target content, determine the insertion position of the target image according to the target content, and insert it.
12. The device according to claim 11, wherein The image insertion module obtains the image content analysis result of the target image, determines the target text content according to the chapter text content, and determines whether the target image is related to the chapter text content according to the image content analysis result and the target text content.
13. The device according to claim 12, wherein: The image insertion module obtains the image uniform resource locator corresponding to the target image, and generates the image content parsing result based on the image uniform resource locator using a parsing model.
14. The device according to claim 12, wherein: In response to determining that the number of characters included in the chapter text content is greater than a first threshold, the picture insertion module summarizes the chapter text content and uses the summary result as the target text content, the number of characters included in the summary result is less than or equal to the first threshold, and in response to determining that the number of characters included in the chapter text content is less than or equal to the first threshold, uses the chapter text content as the target text content.
15. The device according to claim 12, wherein: The image insertion module uses the image content analysis result and the target text content as inputs of a discrimination model to obtain an output discrimination result, wherein the discrimination result includes: the target image is related to the chapter text content, or the target image is not related to the chapter text content.
16. The device according to claim 15, wherein: The image insertion module performs paragraph segmentation processing on the chapter text content to obtain the content of each paragraph, obtains the first text vector representation corresponding to the target image, obtains the second text vector representation corresponding to each paragraph content, and obtains the similarity between the first text vector representation and each second text vector representation, and takes the paragraph content with the highest similarity as the target content.
17. The device according to claim 16, wherein: In response to the target image being related to the chapter text content, the determination result further includes: a name and illustration information of the target image generated according to the image content analysis result; The image insertion module splices the image name and the caption information to obtain a splicing result, uses an embedding model to generate a text vector representation corresponding to the splicing result as the first text vector representation, and uses the embedding model to generate the second text vector representation corresponding to each paragraph content.
18. The device according to claim 17, wherein: The picture inserting module inserts the picture name, the illustration information and the picture uniform resource locator corresponding to the target picture into the inserting position.
19. The device according to claim 18, wherein: The picture inserting module edits the picture name, the illustration information and the picture uniform resource locator into a lightweight markup language format or a fifth edition hierarchical data format, and then inserts the edited picture into the inserting position.
20. The device according to any one of claims 11 to 19, wherein: The picture inserting module uses the front position or the back position adjacent to the position where the target content is located as the insertion position of the target picture.
21. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.
22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1 to 10.
23. A computer program product, comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 10 is implemented.
