An electronic picture book generation method, device and electronic equipment

Abstract

The present disclosure provides an electronic picture book generation method, apparatus and electronic device, relating to the field of artificial intelligence technology, and particularly to the field of text processing. The specific implementation solution is as follows: obtaining page information of each page in a paper picture book; using a picture book production model to determine objects in the page information and generating an animation effect for the objects; obtaining text in the page information and generating voice information for the text; and producing an electronic picture book based on the animation effect and the voice information.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and particularly to the fields of image processing and speech technology. Specifically, it relates to a method, apparatus, and electronic device for generating electronic picture books. Background Art

[0002] Education plays a crucial role in the development of society, and more and more people use picture books for the enlightenment education of infants and children. However, reading paper picture books has many drawbacks, such as being boring and having requirements for the cultural and reading levels of readers. Therefore, converting paper picture books into electronic picture books with rich content and vivid playback has been the goal pursued in the field of picture book processing. Summary of the Invention

[0003] The present disclosure provides a method, apparatus, and electronic device for generating electronic picture books.

[0004] According to a first aspect of the present disclosure, there is provided a method for generating an electronic picture book, the method comprising:

[0005] Obtaining page information of each page in a paper picture book;

[0006] Using a picture book production model to determine objects in the page information and generating an animation effect for the objects;

[0007] Obtaining text in the page information and generating voice information for the text;

[0008] Producing an electronic picture book based on the animation effect and the voice information.

[0009] According to a second aspect of the present disclosure, there is provided an electronic picture book generation apparatus, the electronic picture book generation apparatus comprising:

[0010] A first acquisition module for obtaining page information of each page in a paper picture book;

[0011] A generation module for using a picture book production model to determine objects in the page information and generating an animation effect for the objects;

[0012] A second acquisition module for obtaining text in the page information and generating voice information for the text;

[0013] A production module for producing an electronic picture book based on the animation effect and the voice information.

[0014] According to a third aspect of the present disclosure, there is provided an electronic device, comprising:

[0015] At least one processor; and a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned electronic picture book generation method.

[0017] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the above-mentioned electronic picture book generation method.

[0018] According to a fifth aspect of the present disclosure, there is provided a computer program product including a computer program / instructions, and the computer program / instructions, when executed by a processor, implement the above-mentioned electronic picture book generation method.

[0019] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0021] Figure 1 is an optional processing flow diagram of the electronic picture book generation method provided by the embodiments of the present disclosure;

[0022] Figure 2 is an optional processing flow diagram of using a picture book production model to determine an object in page information provided by the embodiments of the present disclosure;

[0023] Figure 3 is a processing flow diagram of generating an animation effect for an object provided by the embodiments of the present disclosure;

[0024] Figure 4 is a schematic diagram of a presentation form of extended information provided by the embodiments of the present disclosure;

[0025] Figure 5 is an optional processing flow diagram of the picture book playing method provided by the embodiments of the present disclosure;

[0026] Figure 6 is a schematic diagram of scanning or collecting the cover information of the first paper picture book provided by the embodiments of the present disclosure;

[0027] Figure 7 is a schematic diagram of an interface for displaying a matching result and prompting to start picture book reading provided by the embodiments of the present disclosure;

[0028] Figure 8 is an optional display schematic diagram of a text segment provided by the embodiments of the present disclosure;

[0029] Figure 9 It is a schematic diagram of the composition structure of the electronic picture book generation device provided by an embodiment of the present disclosure;

[0030] Figure 10 It is a block diagram of an electronic device for implementing the electronic picture book generation method of an embodiment of the present disclosure. Specific embodiments

[0031] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted below.

[0032] In the following description, "some embodiments" are involved, which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and they can be combined with each other without conflict.

[0033] In the following description, the terms "first / second / third" involved are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present disclosure belongs. The terms used herein are only for the purpose of describing the embodiments of the present disclosure and are not intended to limit the present disclosure.

[0035] Before describing the embodiments of the present disclosure in detail, relevant terms related to the present disclosure are explained.

[0036] Picture book: It is a type of book mainly composed of paintings and accompanied by a small amount of text. Picture books not only tell stories and impart knowledge, but also can comprehensively help children construct a spiritual world and cultivate multiple intelligences.

[0037] Three-dimensional animation: It can be unrestricted by time, space, location, conditions, and objects, and uses various forms of expression to present complex and abstract content, scientific principles, abstract concepts, etc. in a concentrated, simplified, vivid, and vivid form.

[0038] Augmented Reality (AR): It is a technology that calculates the position and angle of the camera image in real time and adds corresponding images. It is a new technology that "seamlessly" integrates real-world information and virtual-world information. The goal of this technology is to overlay the virtual world on the real world on the screen and interact with it.

[0039] Virtual Reality (VR), also known as virtual environment technology, is a brand-new practical technology developed in the 20th century. Virtual reality technology encompasses computer, electronic information, and simulation technologies. Its basic implementation method is to simulate a virtual environment by a computer to give people a sense of immersion in the environment.

[0040] Artificial Intelligence (AI): It is the theory, method, technology, and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. Machine Learning (ML) is the core of artificial intelligence, which specifically studies how a computer simulates or realizes human learning behavior to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve its own performance.

[0041] In the prior art, when using picture books to conduct enlightenment education for infants and children, the operation of reading picture books is usually carried out by the elderly. Due to limitations in cultural and reading levels, there are problems such as being unable to read English picture books, or being unable to fully recognize the words in Chinese picture books, or being unable to communicate in Mandarin, or the reading of picture books is not vivid and fluent enough.

[0042] Regarding the possible problems in reading paper picture books, a picture book reading machine can be purchased to read picture books through it. However, the picture books that the picture book reading machine can cover are limited, and its expandability in use is poor, the economic cost is high, and the popularity rate is low.

[0043] Figure 1 It is an optional processing flow schematic diagram of the electronic picture book generation method provided by the present disclosure, which may at least include the following steps:

[0044] Step S101, obtaining the page information of each page in the paper picture book.

[0045] In some optional embodiments, the page images of each page in the paper picture book can be collected by a camera device on an electronic device, and the page information included in the page images is detected.

[0046] In some alternative embodiments, the page information included in the detection page may at least include objects and text in the detection page; among them, the objects in the page may include objects in the page, such as people, animals, plants, buildings, items, and landscapes, etc.

[0047] Step S102, use the picture book production model to determine the objects in the page information and generate an animation effect for the objects.

[0048] In some alternative embodiments, the process of determining the objects in the page information may be a process of separating or segmenting the objects in the page from the image corresponding to the page.

[0049] In some alternative embodiments, an alternative processing flow for using the picture book production model to determine the objects in the page information, such as Figure 2 shown, may at least include:

[0050] Step S102a, the picture book production model detects the objects included in the page information.

[0051] In some embodiments, the picture book production model detects the foreground and background on the image corresponding to the page; among them, the foreground corresponds to the objects included in the page information.

[0052] Step S102b, the picture book production model determines the page area corresponding to the object.

[0053] In some embodiments, the picture book production model determines the page area corresponding to the foreground detected in step S102a and determines the page area corresponding to the foreground as the page area corresponding to the object.

[0054] In specific implementation, the foreground in the page can be located to determine the position of the foreground in the page, and the area formed by the position of the foreground in the page is the page area corresponding to the object. Among them, the positioning information of the foreground can be represented by the center point coordinates and width and height of the two-dimensional bounding box corresponding to the object.

[0055] Step S102c, the picture book production model determines the image of the object in the page information based on the page area.

[0056] In some embodiments, the picture book production model segments the page area corresponding to the foreground from the image corresponding to the page to obtain the image of the object in the page information.

[0057] In some embodiments, the processing flow for generating an animation effect for the object, such as Figure 3 shown, may at least include:

[0058] Step S102d, the picture book production model determines the attributes of the object.

[0059] In some embodiments, the attributes of an object may refer to the inherent characteristics of the object. As an example, if the object is a cloud, the attribute of the object is floating. If the object is a bird, the attribute of the object is flying. If the object is an eye, the attribute of the object is blinking.

[0060] Step S102e, the picture book production model generates an animation effect of the object based on the attributes of the object and the image.

[0061] In some embodiments, the picture book production model generates an animation effect of the object according to the attributes of the object and uses the image of the object in the page information determined in step S102c.

[0062] Among them, the animation effect of the object can be a three-dimensional animation effect. Such as leaves falling, clouds floating, blinking, etc.

[0063] In some embodiments, when the electronic device plays the text corresponding to the page, the animation effect of the object can be presented on the playing interface, making the playing of the picture book more vivid and improving the attractiveness of the picture book.

[0064] In the embodiments of the present disclosure, by using the vivid picture book production model to determine the objects in the page and generate the animation effect for the objects, the animation and interaction effects can be realized with a relatively low cost and a relatively simple process.

[0065] Step S103, obtain the text in the page information and generate voice information for the text.

[0066] In some embodiments, the text on the page can be recognized by optical character recognition (OCR) technology, such as recognizing the name, author, publication time, and release time on the cover and bottom of the picture book, and recognizing the text on the pages other than the cover and bottom of the picture book.

[0067] In some embodiments, if the text on the page is a language other than Chinese, the text on the page can also be converted into Chinese text.

[0068] In some embodiments, generating voice information for the text can convert the text on the page into voice information through text-to-speech conversion technology. When the text on the page is a language other than Chinese, the text on the page and the corresponding Chinese text of the text on the page can also be converted into voice information.

[0069] In some embodiments, since the audio information is obtained by converting the text, the audio information has a corresponding relationship with the text, and the corresponding relationship between the audio segments in the audio information and the text segments in the text can be established.

[0070] In some embodiments, the voice information may be children's voice information.

[0071] Step S104, create an e-picture book based on the animation effect and the voice information.

[0072] In some embodiments, the process of creating an e-picture book based on the animation effect and the voice information may include: playing the voice information in the e-picture book in the form of voice playback, and presenting the objects in the e-picture book by means of the animation effect.

[0073] Through the e-picture book generation method provided by the present application, any electronic device with an image acquisition function can be used to convert a paper picture book into an e-picture book through AI technology. The user can play the e-picture book through the electronic device, and the content in the picture book can be vividly played. Converting a paper picture book into an e-picture book through AI technology can further reduce costs and streamline the processing process compared with converting a paper picture book into an e-picture book through a digital method.

[0074] In some alternative embodiments, for the objects determined by the picture book production model in step S102, the e-picture book generation method may further include:

[0075] Step S102-1, the picture book production model obtains the extended information of the object.

[0076] In some embodiments, the extended information of the object may refer to the information not presented in the page of the picture book.

[0077] In specific implementation, the extended information can be obtained by the picture book production model in the way of keyword search and index key points based on the object.

[0078] As an example, the extended information may refer to the popular science knowledge or common sense of life corresponding to the object. For example, if the object is a fruit, the extended information may at least include: the English name of the fruit, the picture of the fruit, the place of origin, the nutritional value, etc. If the object is an animal, the extended information may at least include: the English name of the animal, the picture of the animal, the habits of the animal, the habitat of the animal, etc.

[0079] Step S102-2, the picture book production model generates an augmented reality effect for at least one piece of the extended information, and / or a virtual reality effect for at least one piece of the extended information.

[0080] In some embodiments, one presentation form of the extended information, such as Figure 4 shown, can be presented on a sub-page independent of the current page; wherein, Figure 4 the "rooster" and "hen" in

[0081] In some embodiments, the picture book production model can generate an enhanced display effect and / or a virtual display effect corresponding to the extended information, and introduce and explain the entity in combination with the enhanced display effect and / or the virtual display effect corresponding to the extended information, increasing the fun of playing the picture book.

[0082] In some embodiments, the extended information of each entity in the page can be generated, or the extended information of some entities in the page can be selectively generated; when generating the extended information of each entity in the page, the extended information of some entities can be presented according to the user's selection, and the enhanced display effect and / or the virtual display effect corresponding to some entities can be generated according to the user's selection.

[0083] It should be noted that the picture book production model in the embodiments of the present disclosure is a prior machine learning model generated based on historical picture book data. Through the prior machine learning model, a paper picture book can be converted into an electronic picture book with animation effects, AR effects, and VR effects.

[0084] In some alternative embodiments, the above Figures 1 to 4 described method can be used to generate electronic picture books corresponding to multiple paper picture books, and then an electronic picture book library can be constructed.

[0085] Based on the constructed electronic picture book library, a schematic diagram of an alternative processing flow of the picture book playing method provided in the embodiments of the present disclosure is shown in Figure 5 and includes at least the following steps:

[0086] Step S201, obtain the cover information of the first paper picture book.

[0087] In some embodiments, the cover information of the first paper picture book can be scanned or collected through a camera device on the electronic device; alternatively, the cover information of the first paper picture book can be scanned or collected through an independent camera device, and the independent camera device sends the cover information of the first paper picture book to the electronic device.

[0088] In some embodiments, a schematic diagram of scanning or collecting the cover information of the first paper picture book can be as shown in Figure 6 and is shown as follows.

[0089] Step S202, match the first electronic picture book corresponding to the first paper picture book in the electronic picture book library based on the cover information.

[0090] In some embodiments, the electronic device can identify the text in the cover information of the first paper picture book through OCR technology, and obtain the image in the cover information of the first knowledge picture book through the picture book production model. Search for the first electronic picture book in the electronic picture book library that matches the text in the cover information and the image in the cover information.

[0091] In the process of finding the matching first electronic picture book, the first electronic picture book can be found by calculating the similarity between the image in the cover information and the images in the cover information of each electronic picture book in the electronic picture book library; the first electronic picture book can also be found by comparing the consistency between the text in the cover information and the texts in the cover information of each electronic picture book in the electronic picture book library; the first electronic picture book can also be found by combining the method of calculating the similarity between the image in the cover information and the images in the cover information of each electronic picture book in the electronic picture book library and the method of comparing the consistency between the text in the cover information and the texts in the cover information of each electronic picture book in the electronic picture book library.

[0092] In some embodiments, if the first electronic picture book corresponding to the first paper picture book cannot be matched in the electronic picture book library, the first paper picture book can be converted into the first electronic picture book by using the electronic picture book generation method as Figure 1 shown and the first electronic picture book can be stored in the electronic picture book library. In this way, if the user scans the first paper picture book again, the first electronic picture book stored in the electronic picture book library can be directly played. In this way, any paper picture book can be scanned to generate a corresponding electronic version, improving the coverage of the electronic picture book library.

[0093] In some embodiments, after the matching first electronic picture book is found, the matching result can be displayed in the interface as Figure 7 shown and a prompt to start reading the picture book can be given.

[0094] Step S203, play the first electronic picture book.

[0095] In some embodiments, the first electronic picture book can be played through the default audio, where the default audio can refer to playing the content in the first electronic picture book through a child's voice.

[0096] In some embodiments, in response to the user's selection of the playing audio, the first electronic picture book can also be played based on the recorded audio.

[0097] In specific implementation, if the user triggers the electronic device to play the first electronic picture book, the user can be prompted on the interface of the electronic device about the way to play the first electronic picture book, for the user to choose to play the first electronic picture book through the default audio or based on the recorded audio. Among them, the recorded audio can be the audio of reading the first electronic picture book recorded by the user according to the text information in the first paper picture book.

[0098] Playing the first electronic picture book through the default child's voice makes the picture book reading more interesting. Playing the first electronic picture book through the recorded audio makes the picture book reading more amiable, and infants or children can hear the familiar voices of their parents or relatives.

[0099] In some embodiments, if the text in the first paper picture book is in Chinese, the first e-picture book can be directly played in Chinese. If the text in the first paper picture book is in a language other than Chinese, the other language can be translated into Chinese; when playing the first e-picture book, the first e-picture book can be played in a language other than Chinese, or the first e-picture book can be played in the translated Chinese.

[0100] In some embodiments, when playing the first e-picture book, based on the correspondence between the audio of the first e-picture book and the text of the first e-picture book, the text corresponding to the currently played audio can be displayed in a first style.

[0101] In specific implementation, since there is a correspondence between the audio of the first e-picture book and the text of the first e-picture book, each audio segment in the audio of the first e-picture book corresponds to a text segment in the text of the first e-picture book. When playing an audio segment of the first e-picture book, the text segment corresponding to the currently played audio segment is displayed in a first style on the interface of the electronic device. Among them, displaying in the first style can be highlighting, or can be boldfacing, or can also be changing the font color, font size, etc. of the text segment. The text segment corresponding to the currently unplayed audio segment is displayed in a second style, where the second style can be the default display style of the text.

[0102] In the embodiments of the present disclosure, an optional display schematic diagram of a text segment is as Figure 8 shown. By displaying the text segment corresponding to the currently played audio segment in a first style, the recognition rate of the text can be improved, which is convenient for users to track the reading or playing progress of the first e-picture book.

[0103] In some embodiments, when playing the first e-picture book, the picture book playing method provided by the embodiments of the present disclosure may further include:

[0104] Step S103a: Display the animation effect of at least one object in the page corresponding to the currently played audio.

[0105] In some embodiments, the three-dimensional animation effect of some objects in the page corresponding to the currently played audio can be displayed, or the three-dimensional animation effect of some objects in the page corresponding to the currently played audio can be displayed. In this way, the interest of knowledge can be increased and the reading interest of infants or children can be improved.

[0106] Step S103b, display the extended information of the object on a sub-page independent of the page to which the object belongs. In some embodiments, the extended information of the object may be displayed in an AR or VR manner on a sub-page independent of the page to which the object belongs. In this way, knowledge other than the content presented in the picture book can be expanded, and the knowledge scope learned by infants or children is extended. Presenting the extended information in an AR or VR manner can increase the interest of knowledge, improve the reading experience, and enrich the interaction form.

[0107] Embodiments of the present disclosure also provide an electronic picture book generation device. The schematic composition structure diagram of the electronic picture book generation device is as Figure 9 shown, including:

[0108] A first acquisition module 301, configured to acquire the page information of each page in a paper picture book;

[0109] A generation module 302, configured to determine the objects in the page information by using a picture book production model, and generate an animation effect for the objects;

[0110] A second acquisition module 303, configured to acquire the text in the page information and generate voice information for the text;

[0111] A production module 304, configured to produce an electronic picture book based on the animation effect and the voice information.

[0112] In some optional embodiments, the generation module 302 is configured to detect the objects included in the page information by using the picture book production model; determine the page area corresponding to the objects; and determine the images of the objects in the page information based on the page area.

[0113] In some optional embodiments, the generation module 302 is configured to determine the attributes of the objects by using the picture book production model; and generate an animation effect for the objects based on the attributes of the objects and the images.

[0114] In some optional embodiments, the electronic picture book generation device further includes: a processing module 305, configured to acquire the extended information of the objects by using the picture book production model, where the extended information is not presented in the picture book.

[0115] In some optional embodiments, the processing module 305 is configured to generate an augmented reality effect for at least one piece of the extended information by using the picture book production model, and / or a virtual reality effect for the at least one piece of the extended information.

[0116] In some optional embodiments, the second acquisition module 303 is configured to detect the text included in the page information by using the picture book production model; and convert the text into the audio information.

[0117] In some alternative embodiments, the second acquisition module 303 is further configured to establish a correspondence between the audio segments in the audio information and the text segments in the text.

[0118] In some alternative embodiments, the electronic picture book generating device further includes: a construction module 306, configured to construct an electronic picture book library, where the picture book library includes at least one electronic picture book.

[0119] In some alternative embodiments, the electronic picture book generating device further includes: a playing module 307, configured to acquire the cover information of a first physical picture book;

[0120] Match a first electronic picture book corresponding to the first physical picture book in the electronic picture book library based on the cover information.

[0121] In some alternative embodiments, the playing module 307 is further configured to play the first electronic picture book based on a default audio; or, in response to a user's selection of playing audio, play the first electronic picture book based on a recorded audio.

[0122] In some alternative embodiments, the playing module 307 is further configured to display the text corresponding to the currently played audio in a first style based on the correspondence between the audio of the first electronic picture book and the text of the first electronic picture book;

[0123] wherein, the text other than the text corresponding to the currently played audio is displayed in a second style.

[0124] In some alternative embodiments, the playing module 307 is further configured to display an animation effect of at least one object in the page corresponding to the currently played audio; and / or, display the extended information of the object on a sub-page independent of the page to which the object belongs.

[0125] It should be understood that in various embodiments of the present application, the magnitudes of the sequence numbers of the various implementation processes do not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0126] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0127] Figure 10FIG. 0 is a schematic block diagram of an exemplary electronic device 800 that can be used to implement embodiments of the present disclosure. In some alternative embodiments, the electronic device 800 may be a terminal device or a server. In some alternative embodiments, the electronic device 800 may implement the electronic picture book generation method provided in the embodiments of the present application by running a computer program. For example, the computer program may be a native program or a software module in an operating system; it may be a local (Native) application (Application, APP), that is, a program that needs to be installed in the operating system to run; it may also be a small program, that is, a program that only needs to be downloaded to the browser environment to run; it may also be a small program that can be embedded into any APP. In short, the above computer program may be any form of application program, module or plug-in.

[0128] In practical applications, the server that interacts with the electronic device 800 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Among them, cloud technology (Cloud Technology) refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data computing, storage, processing, and sharing. The server may also be a blade server. The electronic device 800 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart TV, a smart watch, etc., but is not limited thereto.

[0129] The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, vehicle-mounted terminals, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0130] As Figure 10As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to computer programs stored in a read-only memory (ROM) 802 or computer programs loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0131] A plurality of components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disc, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0132] The computing unit 801 can be various general-purpose and / or dedicated processing components with processing and computing capabilities. Some examples of the computing unit 801 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 executes the various methods and processes described above, such as the electronic picture book generation method. For example, in some alternative embodiments, the electronic picture book generation method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 808. In some alternative embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the electronic picture book generation method described above can be executed. Alternatively, in other embodiments, the computing unit 801 can be configured as the electronic picture book generation method by any other appropriate means (e.g., by means of firmware).

[0133] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0134] The program code for implementing the electronic picture book generation method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0137] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0138] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, or a server of a distributed system, or a server incorporating a blockchain.

[0139] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0140] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. An electronic picture book generation method, comprising: Obtaining page information of each page in a paper picture book; Using a picture book production model to determine objects in the page information and generating an animation effect for the objects; Obtaining the text in the page information and generating voice information for the text; Producing an electronic picture book based on the animation effect and the voice information; The using a picture book production model to determine objects in the page information includes: the picture book production model detecting objects included in the page information; the picture book production model determining the page area corresponding to the objects; the picture book production model segmenting the page area corresponding to the foreground from the image corresponding to the page to determine the image of the objects in the page information; The generating an animation effect for the objects includes: the picture book production model determining the attributes of the objects; the picture book production model generating the animation effect of the objects based on the attributes of the objects and the image; wherein, the attributes of the objects are the inherent characteristics of the objects.

2. The method according to claim 1, wherein, the method further includes: The picture book production model obtaining extended information of the objects, and the extended information is not presented in the picture book.

3. The method according to claim 2, wherein, the method further includes: The picture book production model generating an augmented reality effect for at least one piece of the extended information, and / or a virtual reality effect for the at least one piece of the extended information.

4. The method according to claim 1, wherein, the obtaining the text in the page information and generating voice information for the text includes: The picture book production model detecting the text included in the page information; The picture book production model converting the text into audio information.

5. The method according to claim 4, wherein, the method further includes: Establishing a correspondence relationship between the audio segments in the audio information and the text segments in the text.

6. The method according to any one of claims 1 to 5, wherein, the method further includes: Constructing an electronic picture book library, and the picture book library includes at least one electronic picture book.

7. The method according to claim 6, wherein, the method includes: Obtaining the cover information of a first paper picture book; Matching a first electronic picture book corresponding to the first paper picture book in the electronic picture book library based on the cover information.

8. The method according to claim 7, wherein, the method further includes: Playing the first electronic picture book based on default audio; Or, in response to the user's selection of the played audio, playing the first electronic picture book based on the recorded audio.

9. The method according to claim 8, wherein, the method further includes: Displaying the text corresponding to the currently played audio in a first style based on the correspondence relationship between the audio of the first electronic picture book and the text of the first electronic picture book; wherein, the text other than the text corresponding to the currently played audio is displayed in a second style.

10. The method according to claim 8, wherein, the method further includes at least one of the following: Displaying the animation effect of at least one object in the page corresponding to the currently played audio; Display the extended information of the object on a sub - page independent of the page to which the object belongs.

11. An electronic picture book generating device, the electronic picture book generating device comprises: A first acquisition module, configured to acquire the page information of each page in a paper picture book; A generation module, configured to use a picture book production model to determine the objects in the page information and generate an animation effect for the objects; A second acquisition module, configured to acquire the text in the page information and generate voice information for the text; A production module, configured to produce an electronic picture book based on the animation effect and the voice information; Wherein, the generation module is specifically configured to use the picture book production model to detect the objects included in the page information; determine the page area corresponding to the objects; use the picture book production model to segment the page area corresponding to the foreground from the image corresponding to the page to determine the image of the objects in the page information; use the picture book production model to determine the attributes of the objects; generate the animation effect of the objects based on the attributes of the objects and the image; wherein, the attributes of the objects are the inherent characteristics of the objects.

12. An electronic device, comprises: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the electronic picture book generating method according to any one of claims 1 to 10.

13. A non - transitory computer - readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the electronic picture book generating method according to any one of claims 1 to 10.

14. A computer program product, comprising computer programs / instructions, and when the computer programs / instructions are executed by a processor, the electronic picture book generating method according to any one of claims 1 to 10 is implemented.

Images