system

Abstract

An object of the system according to the embodiment is to effectively share an impression or recommendation of reading and enrich a reading experience on the SNS.SOLUTION: A system according to an embodiment includes an image acquisition unit, an OCR analysis unit, a review input unit, a phrase generation unit, a music impartment unit, and a moving image generation unit. The image acquisition unit photographs a cover image of the book read by the user. The OCR analysis unit analyzes the cover image acquired by the image acquisition unit using an OCR technique. The impression input unit receives an impression of a book and a keyword input by a user. The wording generation unit generates wording and an image on the basis of the impression and the keyword received by the impression input unit. The music imparting part imparts music on the basis of the words generated by the word generation part and the image. The video generation unit generates a short video by combining the music assigned by the music assignment unit, the words, and the images.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technologies have limited means for effectively sharing reading impressions and recommendations, leaving room for improvement in enriching the reading experience on social media.

[0005] The system according to the embodiment aims to effectively share reading impressions and recommendations, and to enrich the reading experience on SNS. [Means for solving the problem]

[0006] The system according to the embodiment includes an image acquisition unit, an OCR analysis unit, a review input unit, a text generation unit, a music assignment unit, and a video generation unit. The image acquisition unit captures a cover image of a book that a user has finished reading. The OCR analysis unit uses OCR technology to analyze the cover image acquired by the image acquisition unit. The review input unit accepts reviews and keywords about the book entered by the user. The text generation unit generates text and images based on the reviews and keywords accepted by the review input unit. The music assignment unit assigns music based on the text and images generated by the text generation unit. The video generation unit generates a short video by combining the music assigned by the music assignment unit with the text and images. [Effects of the Invention]

[0007] The system according to the embodiment can effectively share reading impressions and recommendations, enriching the reading experience on SNS. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The book POP generation system according to an embodiment of the present invention is a system in which a user takes a photo of the cover image of a book that the user has finished reading, a generation AI analyzes the image using OCR technology, generates text and images based on impressions and keywords, and generates a short video with music. This allows the book POP generation system to easily create book POPs and distribute them on social media.

[0029] A book POP generation system according to an embodiment includes an image acquisition unit, an OCR analysis unit, a review input unit, a text generation unit, a music assignment unit, and a video generation unit. The image acquisition unit captures a cover image of a book a user has finished reading. For example, the cover image can be captured using a smartphone camera and uploaded to an app. The image acquisition unit can also capture a screenshot of an e-book cover image. The OCR analysis unit analyzes the cover image acquired by the image acquisition unit using OCR technology. For example, Tesseract can be used as OCR technology to extract the title, author name, and cover image from the image. Google Cloud Vision can also be used for more accurate analysis. The review input unit accepts book reviews and keywords entered by the user. For example, reviews and keywords can be entered using a text input form. Reviews can also be entered using voice input. The text generation unit generates text and images based on the reviews and keywords accepted by the review input unit. For example, a generation AI can use GPT-3 to summarize the reviews and generate appropriate text. The generation AI can also use BERT to generate related images based on keywords. The music assignment unit assigns music based on the text and images generated by the text generation unit. For example, the generation AI performs emotional analysis and selects an inspiring song for a moving story. The generation AI can also select an appropriate song from a music library. The video generation unit generates a short video by combining the music assigned by the music assignment unit with the text and images. For example, the generation AI uses video editing software to place the text and images on a timeline and add music to the background. The generation AI can also automatically add effects to generate a visually appealing video. This allows the book POP generation system according to the embodiment to easily create book POPs and distribute them on social media. For example, users can post the generated short videos on Instagram or Twitter and share them with other users. Bookstores and publishers can also use this system for effective promotion.

[0030] The OCR analysis unit can analyze the design elements (color, font, layout) of the cover and extract design features. For example, the OCR analysis unit analyzes the color pattern of the cover and extracts specific color combinations and color schemes. For example, covers with a high red and black color scheme are associated with a specific genre. The OCR analysis unit also analyzes the font on the cover and extracts specific font styles. For example, handwritten fonts are often used in children's books. The OCR analysis unit also analyzes the layout of the cover and extracts design features. For example, layouts with a large title in the center are common in business books. This allows the analysis of the design elements of the cover and extraction of design features, making it possible to provide more detailed information.

[0031] The OCR analysis unit can capture images from multiple angles when photographing a cover image and generate a 3D model to perform a more detailed analysis. For example, the OCR analysis unit can analyze the three-dimensional design elements of the cover by photographing the cover image from multiple angles and generating a 3D model. For example, covers with embossing or special textures can be analyzed in detail. The OCR analysis unit also uses photogrammetry technology to generate the 3D model. For example, multiple images can be combined to generate a 3D model. Furthermore, the OCR analysis unit can use 3D scanning technology to analyze the detailed design elements of the cover. For example, laser scanning can be used to analyze the fine irregularities on the cover. This allows for more detailed analysis by photographing images from multiple angles and generating a 3D model.

[0032] The OCR analysis unit can link the OCR analysis results with other databases and automatically obtain additional information. For example, the OCR analysis unit links with a book database based on the OCR analysis results and automatically obtains detailed information about the book. For example, it obtains the book's synopsis and publication year as additional information. The OCR analysis unit also links with review sites to obtain book ratings and reviews. For example, it obtains user ratings and comments as additional information. Furthermore, the OCR analysis unit links with online bookstores to obtain book inventory and price information. For example, it obtains whether the book is in stock and the price as additional information. In this way, by linking the OCR analysis results with other databases and automatically obtaining additional information, more detailed information can be provided.

[0033] The OCR analysis unit can improve the accuracy of OCR analysis by using voice input when capturing a cover image, allowing the user to verbally input the title and author name. For example, the OCR analysis unit can improve the accuracy of OCR analysis by having the user verbally input the title and author name when capturing a cover image. For example, the user can input "The title is XX, and the author is △△" through voice input. The OCR analysis unit also analyzes the voice input using voice recognition technology. For example, voice recognition software can automatically analyze the voice and save it as text. The OCR analysis unit can also improve the accuracy of voice input by adjusting the microphone type and settings. For example, a high-sensitivity microphone can be used and noise-canceling functionality can be enabled. This allows the accuracy of OCR analysis to be improved by using voice input in combination.

[0034] In the impression input unit, when a user inputs impressions or keywords, the generation AI analyzes the user's input in real time and makes appropriate completion suggestions. For example, when a user inputs impressions or keywords, the generation AI analyzes the user's input in real time and makes appropriate completion suggestions. For example, if a user inputs "an inspiring story," the generation AI will make completion suggestions such as "a book that touches the heart." The impression input unit also uses GPT-3 as the generation AI. For example, GPT-3 analyzes the user's input and generates appropriate completion suggestions. Furthermore, the impression input unit uses BERT to make related completion suggestions based on keywords. For example, BERT analyzes keywords and suggests related phrases. This allows the generation AI to support user input by making completion suggestions in real time when users input impressions or keywords.

[0035] The impression input unit can automatically recommend other related books and content based on the impressions and keywords entered. For example, the impression input unit uses a generation AI to automatically recommend other related books based on the impressions and keywords entered by the user. For example, if the user enters "moving story," other books in the same genre will be recommended. The impression input unit also uses a recommendation algorithm to recommend related content. For example, collaborative filtering is used to recommend books that other users like. Furthermore, the impression input unit uses content-based filtering to recommend similar books. For example, books by the same author or on the same theme are recommended. This allows the user's interests to be broadened by automatically recommending other related books and content based on the impressions and keywords entered.

[0036] The impression input unit can also accommodate voice input and handwritten input for the input of impressions and keywords, thereby improving user convenience. The impression input unit, for example, can accommodate voice input for the input of impressions and keywords, allowing the user to input them verbally. For example, the user verbally inputs "a moving story." The impression input unit can also accommodate handwritten input, allowing the user to input them by hand. For example, the user inputs impressions by handwriting using a tablet. Furthermore, the impression input unit analyzes the handwritten input using handwriting recognition technology. For example, handwritten characters are automatically converted into text. This allows the input of impressions and keywords to be accommodated by voice input and handwritten input, thereby improving user convenience.

[0037] The impression input unit can add a function of sharing the input impression and keywords with other users and receiving feedback within a community. The impression input unit adds a function of, for example, sharing the impression and keywords input by the user with other users and receiving feedback. For example, sharing impressions and receiving comments from other users. The impression input unit also shares impressions using an online forum or a social networking group. For example, posting impressions within a specific community and receiving feedback. Furthermore, the impression input unit collects feedback using a reaction button. For example, receiving reactions such as "like" or "sympathy" to the impression. This allows the user to share impressions and keywords with other users and receive feedback within the community, thereby generating better content.

[0038] The music assignment unit can refer to the user's past music preference data when selecting music and provide individually customized music. The music assignment unit, for example, analyzes the user's past music preference data and selects music based on that data. For example, it prioritizes the selection of music from genres and artists that the user likes. The music assignment unit also uses past playback history as music preference data. For example, it selects music based on data on music that the user has played in the past. Furthermore, the music assignment unit selects music using user rating data. For example, it prioritizes the selection of music that the user has given high ratings to. In this way, by referring to the user's past music preference data and providing individually customized music, it is possible to select music that is more suited to the user.

[0039] The music assignment unit can incorporate music from different genres and cultural spheres when selecting music, allowing for music selection from a global perspective. For example, the music assignment unit incorporates music from different genres when selecting music. For example, it selects music from genres such as classical, jazz, and pop. The music assignment unit also incorporates music from different cultural spheres. For example, it selects Asian music and European music. Furthermore, the music assignment unit uses an international music library to select music from a global perspective. For example, it references global playlists from Spotify or Apple Music. This allows for music selection from a global perspective by incorporating music from different genres and cultural spheres.

[0040] The music assignment unit can add a function to disclose the music selection process to the user and receive feedback when the user selects a song. The music assignment unit, for example, adds a function to disclose the music selection process to the user and receive feedback when the user selects a song. For example, it receives ratings and comments on the songs selected by the user. The music assignment unit also publishes an explanation of the algorithm to increase the transparency of the selection process. For example, it explains to the user how the songs were selected. Furthermore, the music assignment unit publishes the selection criteria to serve as a reference when the user selects a song. For example, it publishes the tempo, rhythm, and results of emotion analysis of the songs. In this way, by disclosing the music selection process to the user and receiving feedback, it is possible to select a song that is more suitable for the user.

[0041] When generating a short video, the video generation unit can refer to the user's past posting data and provide an individually customized video template. The video generation unit, for example, analyzes the user's past posting data and customizes the video template based on that data. For example, it provides a template that matches the user's preferred style or theme. The video generation unit also selects a template based on the content of past posts. For example, it refers to the style and theme of videos the user has posted in the past. Furthermore, the video generation unit customizes the template based on the posting frequency and number of views. For example, it provides a simple template for users who post frequently, and a high-quality template for users with a high number of views. In this way, by referring to the user's past posting data and providing an individually customized video template, it is possible to generate a short video that is more suited to the user.

[0042] The video generation unit can add interactive functions to the video generation process that allow users to edit and modify in real time. For example, the video generation unit adds interactive functions to the video generation process that allow users to edit and modify in real time. For example, text and images in a video can be changed in real time. The video generation unit also provides real-time editing tools. For example, a user can add or delete scenes from a video. Furthermore, the video generation unit provides a collaborative editing function that allows multiple users to edit simultaneously. For example, a video can be created collaboratively with friends or colleagues. In this way, by adding interactive functions to the video generation process that allow users to edit and modify in real time, a video that better matches the user's intentions can be generated.

[0043] The video generation unit can automatically generate formats optimized for different social media platforms when generating short videos. For example, the video generation unit builds a system that automatically generates formats optimized for different social media platforms when generating short videos. For example, it automatically generates formats for Instagram, Twitter, and TikTok. The video generation unit also adjusts the format based on the specifications of each social media platform. For example, it generates a square format for Instagram and a vertical format for TikTok. Furthermore, the video generation unit applies different effects and filters to each social media platform. For example, it applies a specific filter for Instagram and adds a specific effect for TikTok. This allows users to easily post videos by automatically generating formats optimized for different social media platforms.

[0044] The video generation unit adds a collaboration function with other users in the video generation process, allowing them to jointly create videos. The video generation unit, for example, builds a system that adds a collaboration function with other users in the video generation process and allows them to jointly create videos. For example, it provides a function that allows multiple users to edit simultaneously. The video generation unit also provides a real-time chat function, allowing users to edit videos while communicating with each other. For example, videos can be created while exchanging opinions through chat. Furthermore, the video generation unit provides a collaborative editing tool, allowing users to divide up roles and edit videos. For example, one person edits text and another edits images. In this way, by adding a collaboration function with other users in the video generation process, videos can be jointly created.

[0045] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0046] The book POP generation system can further include a recommendation unit that analyzes a user's reading history and recommends related books based on data on books read in the past. For example, it can analyze the genres and themes of books read in the past by the user and recommend books in the same genre or theme. The recommendation unit can also recommend unread books based on the user's reading history. For example, if a user is reading books in a particular series, it can recommend unread books in that series. Furthermore, the recommendation unit can compare the reading history with that of other users and recommend books read by users with similar reading habits. This allows users to enjoy discovering new books.

[0047] In addition to analyzing cover images, the OCR analysis unit can also analyze images of the book's spine and back cover to extract more detailed information. For example, it can analyze the title and author name on the spine and match them with the cover image. It can also analyze the synopsis and reviews on the back cover and provide them as additional information. Furthermore, the OCR analysis unit can analyze the design elements (color, font, layout) of the book's spine and integrate them with the cover image to extract design features. This allows it to analyze the design elements of the entire book and provide more detailed information.

[0048] When analyzing cover images, the OCR analysis unit can adjust its analysis algorithm based on the book's genre and theme. For example, mystery novel covers contain many specific design elements, so the analysis algorithm can be optimized based on those elements. The OCR analysis unit can also learn the different font and layout characteristics of each genre to improve analysis accuracy. Furthermore, the OCR analysis unit can analyze color patterns and design elements based on the book's theme to provide more detailed information. This allows for more accurate analysis by adjusting the analysis algorithm based on the book's genre and theme.

[0049] The OCR analysis unit can automatically retrieve related information about a book based on the OCR analysis results and provide it to the user. For example, based on the book title and author name, it can retrieve related interview articles and other works by the author. The OCR analysis unit can also retrieve information about documentaries and movies related to the book's theme and provide it to the user. Furthermore, the OCR analysis unit can retrieve academic papers and research materials related to the book's content and provide them as additional information. This allows the system to provide users with a wider range of information by automatically retrieving related information based on the OCR analysis results and providing it to them.

[0050] The impression input unit can analyze the user's input when entering impressions or keywords and automatically suggest related quotes and famous sayings. For example, if you enter "inspirational story," it will suggest related quotes and famous sayings. The impression input unit can also use GPT-3 as a generative AI to analyze the user's input and generate appropriate quotes. Furthermore, the impression input unit can also use BERT to suggest related quotes from keywords. This allows it to support user input by automatically suggesting related quotes and famous sayings when entering impressions or keywords.

[0051] The impression input unit can automatically recommend other related books and content based on the impressions and keywords entered. For example, the generation AI automatically recommends other related books based on the impressions and keywords entered by the user. For example, if the user enters "moving story," other books in the same genre will be recommended. The impression input unit also uses a recommendation algorithm to recommend related content. For example, it uses collaborative filtering to recommend books that other users like. Furthermore, the impression input unit uses content-based filtering to recommend similar books. For example, it recommends books by the same author or on the same theme. This allows the user's interests to be broadened by automatically recommending other related books and content based on the impressions and keywords entered.

[0052] The processing flow of the first embodiment will be briefly explained below.

[0053] Step 1: The image acquisition unit takes a picture of the cover of the book the user has finished reading. For example, the cover image can be taken using a smartphone camera and uploaded to the app. The cover image of the e-book can also be taken as a screenshot. Step 2: The OCR analysis unit uses OCR technology to analyze the cover image acquired by the image acquisition unit. For example, Tesseract is used as the OCR technology to extract the title, author name, and cover image from the image. Google Cloud Vision can also be used to perform more accurate analysis. Step 3: The review input unit accepts the book review and keywords entered by the user. For example, the review and keywords can be entered using a text input form. The review can also be entered using voice input. Step 4: The text generation unit generates text and images based on the impressions and keywords received by the impression input unit. For example, the generation AI can use GPT-3 to summarize the impressions and generate appropriate text. The generation AI can also use BERT to generate related images from keywords. Step 5: The music assignment unit assigns music based on the text and images generated by the text generation unit. For example, the generation AI performs emotional analysis and selects an emotional song for an emotional story. The generation AI can also select an appropriate song from a music library. Step 6: The video generation unit generates a short video by combining the music added by the music addition unit with text and images. For example, the generation AI uses video editing software to place the text and images on a timeline and add music to the background. The generation AI can also automatically add effects to create a visually appealing video.

[0054] (Example 2) The book POP generation system according to an embodiment of the present invention is a system in which a user takes a photo of the cover image of a book that the user has finished reading, a generation AI analyzes the image using OCR technology, generates text and images based on impressions and keywords, and generates a short video with music. This allows the book POP generation system to easily create book POPs and distribute them on social media.

[0055] A book POP generation system according to an embodiment includes an image acquisition unit, an OCR analysis unit, a review input unit, a text generation unit, a music assignment unit, and a video generation unit. The image acquisition unit captures a cover image of a book a user has finished reading. For example, the cover image can be captured using a smartphone camera and uploaded to an app. The image acquisition unit can also capture a screenshot of an e-book cover image. The OCR analysis unit analyzes the cover image acquired by the image acquisition unit using OCR technology. For example, Tesseract can be used as OCR technology to extract the title, author name, and cover image from the image. Google Cloud Vision can also be used for more accurate analysis. The review input unit accepts book reviews and keywords entered by the user. For example, reviews and keywords can be entered using a text input form. Reviews can also be entered using voice input. The text generation unit generates text and images based on the reviews and keywords accepted by the review input unit. For example, a generation AI can use GPT-3 to summarize the reviews and generate appropriate text. The generation AI can also use BERT to generate related images based on keywords. The music assignment unit assigns music based on the text and images generated by the text generation unit. For example, the generation AI performs emotional analysis and selects an inspiring song for a moving story. The generation AI can also select an appropriate song from a music library. The video generation unit generates a short video by combining the music assigned by the music assignment unit with the text and images. For example, the generation AI uses video editing software to place the text and images on a timeline and add music to the background. The generation AI can also automatically add effects to generate a visually appealing video. This allows the book POP generation system according to the embodiment to easily create book POPs and distribute them on social media. For example, users can post the generated short videos on Instagram or Twitter and share them with other users. Bookstores and publishers can also use this system for effective promotion.

[0056] The OCR analysis unit can analyze the design elements (color, font, layout) of the cover and extract design features. For example, the OCR analysis unit analyzes the color pattern of the cover and extracts specific color combinations and color schemes. For example, covers with a high red and black color scheme are associated with a specific genre. The OCR analysis unit also analyzes the font on the cover and extracts specific font styles. For example, handwritten fonts are often used in children's books. The OCR analysis unit also analyzes the layout of the cover and extracts design features. For example, layouts with a large title in the center are common in business books. This allows the analysis of the design elements of the cover and extraction of design features, making it possible to provide more detailed information.

[0057] The OCR analysis unit can capture images from multiple angles when photographing a cover image and generate a 3D model to perform a more detailed analysis. For example, the OCR analysis unit can analyze the three-dimensional design elements of the cover by photographing the cover image from multiple angles and generating a 3D model. For example, covers with embossing or special textures can be analyzed in detail. The OCR analysis unit also uses photogrammetry technology to generate the 3D model. For example, multiple images can be combined to generate a 3D model. Furthermore, the OCR analysis unit can use 3D scanning technology to analyze the detailed design elements of the cover. For example, laser scanning can be used to analyze the fine irregularities on the cover. This allows for more detailed analysis by photographing images from multiple angles and generating a 3D model.

[0058] The OCR analysis unit can use an emotion estimation function to estimate a user's emotion from a cover image and adjust the accuracy of the OCR analysis based on that emotion. The OCR analysis unit, for example, analyzes a cover image to estimate the emotion felt by the user. For example, it estimates an emotional emotion from the color and design of the cover and adjusts the accuracy of the OCR analysis based on that emotion. The OCR analysis unit also uses facial expression recognition technology as its emotion estimation function. For example, it analyzes a user's facial expression to estimate their emotion. Furthermore, the OCR analysis unit uses voice analysis technology to estimate their emotion. For example, it analyzes the tone and speed of the user's voice to estimate their emotion. This allows for more accurate analysis by using the emotion estimation function to adjust the accuracy of the OCR analysis based on the user's emotion.

[0059] The OCR analysis unit can link the OCR analysis results with other databases and automatically obtain additional information. For example, the OCR analysis unit links with a book database based on the OCR analysis results and automatically obtains detailed information about the book. For example, it obtains the book's synopsis and publication year as additional information. The OCR analysis unit also links with review sites to obtain book ratings and reviews. For example, it obtains user ratings and comments as additional information. Furthermore, the OCR analysis unit links with online bookstores to obtain book inventory and price information. For example, it obtains whether the book is in stock and the price as additional information. In this way, by linking the OCR analysis results with other databases and automatically obtaining additional information, more detailed information can be provided.

[0060] The OCR analysis unit can improve the accuracy of OCR analysis by using voice input when capturing a cover image, allowing the user to verbally input the title and author name. For example, the OCR analysis unit can improve the accuracy of OCR analysis by having the user verbally input the title and author name when capturing a cover image. For example, the user can input "The title is XX, and the author is △△" through voice input. The OCR analysis unit also analyzes the voice input using voice recognition technology. For example, voice recognition software can automatically analyze the voice and save it as text. The OCR analysis unit can also improve the accuracy of voice input by adjusting the microphone type and settings. For example, a high-sensitivity microphone can be used and noise-canceling functionality can be enabled. This allows the accuracy of OCR analysis to be improved by using voice input in combination.

[0061] The OCR analysis unit can use the emotion estimation function to analyze the user's emotions in real time when taking a cover image and suggest the optimal timing for taking the image. For example, the OCR analysis unit can analyze the user's emotions in real time when taking a cover image and suggest the optimal timing for taking the image. For example, the OCR analysis unit can encourage the user to take the image when they are relaxed. The OCR analysis unit also uses facial expression recognition technology as its emotion estimation function. For example, it can analyze the user's facial expressions and estimate their emotions. Furthermore, the OCR analysis unit can estimate emotions using voice analysis technology. For example, it can analyze the tone and speed of the user's voice and estimate their emotions. This allows the emotion estimation function to suggest the optimal timing for taking the image based on the user's emotions, thereby enabling a better image to be obtained.

[0062] In the impression input unit, when a user inputs impressions or keywords, the generation AI analyzes the user's input in real time and makes appropriate completion suggestions. For example, when a user inputs impressions or keywords, the generation AI analyzes the user's input in real time and makes appropriate completion suggestions. For example, if a user inputs "an inspiring story," the generation AI will make completion suggestions such as "a book that touches the heart." The impression input unit also uses GPT-3 as the generation AI. For example, GPT-3 analyzes the user's input and generates appropriate completion suggestions. Furthermore, the impression input unit uses BERT to make related completion suggestions based on keywords. For example, BERT analyzes keywords and suggests related phrases. This allows the generation AI to support user input by making completion suggestions in real time when users input impressions or keywords.

[0063] The impression input unit can automatically recommend other related books and content based on the impressions and keywords entered. For example, the impression input unit uses a generation AI to automatically recommend other related books based on the impressions and keywords entered by the user. For example, if the user enters "moving story," other books in the same genre will be recommended. The impression input unit also uses a recommendation algorithm to recommend related content. For example, collaborative filtering is used to recommend books that other users like. Furthermore, the impression input unit uses content-based filtering to recommend similar books. For example, books by the same author or on the same theme are recommended. This allows the user's interests to be broadened by automatically recommending other related books and content based on the impressions and keywords entered.

[0064] The impression input unit uses an emotion estimation function to generate words and images based on the user's emotions, making it possible to create a POP that resonates more emotionally. The impression input unit, for example, uses the emotion estimation function to generate words based on the user's emotions. For example, if the user is deeply moved, it generates a phrase such as "A moving book that touches the heart." The impression input unit also uses facial expression recognition technology as its emotion estimation function. For example, it analyzes the user's facial expressions to estimate emotions. Furthermore, the impression input unit estimates emotions using voice analysis technology. For example, it analyzes the tone and speed of the user's voice to estimate emotions. In this way, the emotion estimation function can be used to generate words and images based on the user's emotions, making it possible to create a POP that resonates more emotionally.

[0065] The impression input unit can also accommodate voice input and handwritten input for the input of impressions and keywords, thereby improving user convenience. The impression input unit, for example, can accommodate voice input for the input of impressions and keywords, allowing the user to input them verbally. For example, the user verbally inputs "a moving story." The impression input unit can also accommodate handwritten input, allowing the user to input them by hand. For example, the user inputs impressions by handwriting using a tablet. Furthermore, the impression input unit analyzes the handwritten input using handwriting recognition technology. For example, handwritten characters are automatically converted into text. This allows the input of impressions and keywords to be accommodated by voice input and handwritten input, thereby improving user convenience.

[0066] The impression input unit can add a function of sharing the input impression and keywords with other users and receiving feedback within a community. The impression input unit adds a function of, for example, sharing the impression and keywords input by the user with other users and receiving feedback. For example, sharing impressions and receiving comments from other users. The impression input unit also shares impressions using an online forum or a social networking group. For example, posting impressions within a specific community and receiving feedback. Furthermore, the impression input unit collects feedback using a reaction button. For example, receiving reactions such as "like" or "sympathy" to the impression. This allows the user to share impressions and keywords with other users and receive feedback within the community, thereby generating better content.

[0067] The impression input unit can use the emotion estimation function to collect other users' emotional reactions to the input impressions and keywords and improve the generated content based on the feedback. The impression input unit, for example, uses the emotion estimation function to collect other users' emotional reactions to the input impressions and keywords. For example, it collects positive reactions to the impressions. The impression input unit also uses facial expression recognition technology as the emotion estimation function. For example, it analyzes the facial expressions of other users to estimate their emotions. Furthermore, the impression input unit estimates emotions using voice analysis technology. For example, it analyzes the tone and speed of other users' voices to estimate their emotions. In this way, by using the emotion estimation function to collect other users' emotional reactions and improving the generated content based on feedback, it is possible to generate content that resonates more with users.

[0068] The music assignment unit adjusts the tempo and rhythm of the music based on the emotional analysis of the cover image and text, thereby generating a video with a greater sense of unity. The music assignment unit adjusts the tempo of the music based on, for example, the emotional analysis of the cover image and text. For example, a song with a slow tempo is selected for a moving story. The music assignment unit also adjusts the rhythm to enhance the sense of unity of the video. For example, the rhythm of the music is adjusted to match the emotion of the text. Furthermore, the music assignment unit optimizes the tempo of the music by adjusting the BPM (beats per minute). For example, a song with a low BPM is selected for a moving scene. In this way, by adjusting the tempo and rhythm of the music based on the emotional analysis of the cover image and text, a video with a greater sense of unity can be generated.

[0069] The music assignment unit can refer to the user's past music preference data when selecting music and provide individually customized music. The music assignment unit, for example, analyzes the user's past music preference data and selects music based on that data. For example, it prioritizes the selection of music from genres and artists that the user likes. The music assignment unit also uses past playback history as music preference data. For example, it selects music based on data on music that the user has played in the past. Furthermore, the music assignment unit selects music using user rating data. For example, it prioritizes the selection of music that the user has given high ratings to. In this way, by referring to the user's past music preference data and providing individually customized music, it is possible to select music that is more suited to the user.

[0070] The music assigning unit uses the emotion estimation function to analyze the emotional response of the user when listening to music in real time and select the most suitable music. The music assigning unit, for example, uses the emotion estimation function to analyze the emotional response of the user when listening to music in real time. For example, if the user is moved, the music assigning unit selects a music that matches that emotion. The music assigning unit also uses facial expression recognition technology as the emotion estimation function. For example, it analyzes the user's facial expression and estimates the emotion. Furthermore, the music assigning unit analyzes changes in heart rate to estimate the emotion. For example, if the heart rate is elevated, it determines that the user is excited. In this way, the emotion estimation function can be used to analyze the emotional response of the user when listening to music in real time and select the most suitable music, thereby providing music that is more suitable for the user.

[0071] The music assignment unit can incorporate music from different genres and cultural spheres when selecting music, allowing for music selection from a global perspective. For example, the music assignment unit incorporates music from different genres when selecting music. For example, it selects music from genres such as classical, jazz, and pop. The music assignment unit also incorporates music from different cultural spheres. For example, it selects Asian music and European music. Furthermore, the music assignment unit uses an international music library to select music from a global perspective. For example, it references global playlists from Spotify or Apple Music. This allows for music selection from a global perspective by incorporating music from different genres and cultural spheres.

[0072] The music assignment unit can add a function to disclose the music selection process to the user and receive feedback when the user selects a song. The music assignment unit, for example, adds a function to disclose the music selection process to the user and receive feedback when the user selects a song. For example, it receives ratings and comments on the songs selected by the user. The music assignment unit also publishes an explanation of the algorithm to increase the transparency of the selection process. For example, it explains to the user how the songs were selected. Furthermore, the music assignment unit publishes the selection criteria to serve as a reference when the user selects a song. For example, it publishes the tempo, rhythm, and results of emotion analysis of the songs. In this way, by disclosing the music selection process to the user and receiving feedback, it is possible to select a song that is more suitable for the user.

[0073] The music assignment unit can use the emotion estimation function to collect the user's emotional response to the song selection and continuously improve the selection algorithm. The music assignment unit, for example, uses the emotion estimation function to collect the user's emotional response to the song selection. For example, it collects emotion scores when the user listens to the song. The music assignment unit also uses facial expression recognition technology as the emotion estimation function. For example, it analyzes the user's facial expression to estimate the emotion. Furthermore, the music assignment unit analyzes changes in heart rate to estimate the emotion. For example, if the heart rate is elevated, it is determined that the user is excited. In this way, by using the emotion estimation function to collect the user's emotional response to the song selection and continuously improving the selection algorithm, it is possible to provide songs that are more suited to the user.

[0074] When generating a short video, the video generation unit can refer to the user's past posting data and provide an individually customized video template. The video generation unit, for example, analyzes the user's past posting data and customizes the video template based on that data. For example, it provides a template that matches the user's preferred style or theme. The video generation unit also selects a template based on the content of past posts. For example, it refers to the style and theme of videos the user has posted in the past. Furthermore, the video generation unit customizes the template based on the posting frequency and number of views. For example, it provides a simple template for users who post frequently, and a high-quality template for users with a high number of views. In this way, by referring to the user's past posting data and providing an individually customized video template, it is possible to generate a short video that is more suited to the user.

[0075] The video generation unit can add interactive functions to the video generation process that allow users to edit and modify in real time. For example, the video generation unit adds interactive functions to the video generation process that allow users to edit and modify in real time. For example, text and images in a video can be changed in real time. The video generation unit also provides real-time editing tools. For example, a user can add or delete scenes from a video. Furthermore, the video generation unit provides a collaborative editing function that allows multiple users to edit simultaneously. For example, a video can be created collaboratively with friends or colleagues. In this way, by adding interactive functions to the video generation process that allow users to edit and modify in real time, a video that better matches the user's intentions can be generated.

[0076] The video generation unit uses the emotion estimation function to analyze the user's emotional response to the generated video and propose an optimal video composition. The video generation unit, for example, uses the emotion estimation function to analyze the user's emotional response to the generated video. For example, it collects emotion scores when the user watches the video. The video generation unit also uses facial expression recognition technology as the emotion estimation function. For example, it analyzes the user's facial expression to estimate the emotion. Furthermore, the video generation unit analyzes changes in heart rate to estimate the emotion. For example, if the heart rate is elevated, it is determined that the user is excited. In this way, by using the emotion estimation function to analyze the user's emotional response to the generated video and propose an optimal video composition, it is possible to generate a video that is more relatable to the user.

[0077] The video generation unit can automatically generate formats optimized for different social media platforms when generating short videos. For example, the video generation unit builds a system that automatically generates formats optimized for different social media platforms when generating short videos. For example, it automatically generates formats for Instagram, Twitter, and TikTok. The video generation unit also adjusts the format based on the specifications of each social media platform. For example, it generates a square format for Instagram and a vertical format for TikTok. Furthermore, the video generation unit applies different effects and filters to each social media platform. For example, it applies a specific filter for Instagram and adds a specific effect for TikTok. This allows users to easily post videos by automatically generating formats optimized for different social media platforms.

[0078] The video generation unit adds a collaboration function with other users in the video generation process, allowing them to jointly create videos. The video generation unit, for example, builds a system that adds a collaboration function with other users in the video generation process and allows them to jointly create videos. For example, it provides a function that allows multiple users to edit simultaneously. The video generation unit also provides a real-time chat function, allowing users to edit videos while communicating with each other. For example, videos can be created while exchanging opinions through chat. Furthermore, the video generation unit provides a collaborative editing tool, allowing users to divide up roles and edit videos. For example, one person edits text and another edits images. In this way, by adding a collaboration function with other users in the video generation process, videos can be jointly created.

[0079] The video generation unit can use the emotion estimation function to collect viewers' emotional reactions to the generated video and reflect them in generating the next video. The video generation unit, for example, uses the emotion estimation function to collect viewers' emotional reactions to the generated video. For example, it collects emotion scores when viewers watch the video. The video generation unit also uses facial expression recognition technology as the emotion estimation function. For example, it analyzes viewers' facial expressions to estimate emotions. Furthermore, the video generation unit analyzes viewers' comments and reaction buttons to estimate emotions. For example, it estimates emotions based on viewers' comments and the number of "Like" buttons. In this way, by using the emotion estimation function to collect viewers' emotional reactions to the generated video and reflecting them in generating the next video, it is possible to generate videos that are more relatable to viewers.

[0080] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0081] The book POP generation system can further include a recommendation unit that analyzes a user's reading history and recommends related books based on data on books read in the past. For example, it can analyze the genres and themes of books read in the past by the user and recommend books in the same genre or theme. The recommendation unit can also recommend unread books based on the user's reading history. For example, if a user is reading books in a particular series, it can recommend unread books in that series. Furthermore, the recommendation unit can compare the reading history with that of other users and recommend books read by users with similar reading habits. This allows users to enjoy discovering new books.

[0082] In addition to analyzing cover images, the OCR analysis unit can also analyze images of the book's spine and back cover to extract more detailed information. For example, it can analyze the title and author name on the spine and match them with the cover image. It can also analyze the synopsis and reviews on the back cover and provide them as additional information. Furthermore, the OCR analysis unit can analyze the design elements (color, font, layout) of the book's spine and integrate them with the cover image to extract design features. This allows it to analyze the design elements of the entire book and provide more detailed information.

[0083] When analyzing cover images, the OCR analysis unit can adjust its analysis algorithm based on the book's genre and theme. For example, mystery novel covers contain many specific design elements, so the analysis algorithm can be optimized based on those elements. The OCR analysis unit can also learn the different font and layout characteristics of each genre to improve analysis accuracy. Furthermore, the OCR analysis unit can analyze color patterns and design elements based on the book's theme to provide more detailed information. This allows for more accurate analysis by adjusting the analysis algorithm based on the book's genre and theme.

[0084] The OCR analysis unit can use its emotion estimation function to estimate a user's emotion from a cover image and automatically classify the genre and theme of a book based on that emotion. For example, it can estimate an emotional emotion from the color and design of the cover and classify the book as an emotional story based on that emotion. The OCR analysis unit can also use facial expression recognition technology as its emotion estimation function to analyze a user's facial expressions and estimate their emotion. Furthermore, the OCR analysis unit can also use voice analysis technology to estimate emotions and classify books based on those emotions. This allows the emotion estimation function to automatically classify book genres and themes based on the user's emotion, thereby providing more appropriate information.

[0085] The OCR analysis unit can automatically retrieve related information about a book based on the OCR analysis results and provide it to the user. For example, based on the book title and author name, it can retrieve related interview articles and other works by the author. The OCR analysis unit can also retrieve information about documentaries and movies related to the book's theme and provide it to the user. Furthermore, the OCR analysis unit can retrieve academic papers and research materials related to the book's content and provide them as additional information. This allows the system to provide users with a wider range of information by automatically retrieving related information based on the OCR analysis results and providing it to them.

[0086] The impression input unit can analyze the user's input when entering impressions or keywords and automatically suggest related quotes and famous sayings. For example, if you enter "inspirational story," it will suggest related quotes and famous sayings. The impression input unit can also use GPT-3 as a generative AI to analyze the user's input and generate appropriate quotes. Furthermore, the impression input unit can also use BERT to suggest related quotes from keywords. This allows it to support user input by automatically suggesting related quotes and famous sayings when entering impressions or keywords.

[0087] The impression input unit uses an emotion estimation function to generate words and images based on the user's emotions, making it possible to create POPs that empathize with a greater emotional impact. For example, if the emotion is strong, it generates words such as "A moving book that will resonate with you." The impression input unit can also use facial expression recognition technology as its emotion estimation function to analyze the user's facial expressions and estimate their emotions. Furthermore, the impression input unit can also use voice analysis technology to estimate emotions and generate words and images based on those emotions. This makes it possible to create POPs that empathize with a greater emotional impact by using the emotion estimation function to generate words and images based on the user's emotions.

[0088] The impression input unit can use the emotion estimation function to collect other users' emotional reactions to the input impressions and keywords and improve the generated content based on the feedback. For example, the emotion estimation function is used to collect other users' emotional reactions to the input impressions and keywords. The impression input unit can also use facial expression recognition technology as the emotion estimation function to analyze other users' facial expressions and estimate their emotions. Furthermore, the impression input unit can also use voice analysis technology to estimate emotions and improve the generated content based on the emotions. In this way, by using the emotion estimation function to collect other users' emotional reactions and improving the generated content based on the feedback, it is possible to generate content that resonates with users more.

[0089] The impression input unit can automatically recommend other related books and content based on the impressions and keywords entered. For example, the generation AI automatically recommends other related books based on the impressions and keywords entered by the user. For example, if the user enters "moving story," other books in the same genre will be recommended. The impression input unit also uses a recommendation algorithm to recommend related content. For example, it uses collaborative filtering to recommend books that other users like. Furthermore, the impression input unit uses content-based filtering to recommend similar books. For example, it recommends books by the same author or on the same theme. This allows the user's interests to be broadened by automatically recommending other related books and content based on the impressions and keywords entered.

[0090] The impression input unit can use the emotion estimation function to collect other users' emotional reactions to the input impressions and keywords and improve the generated content based on the feedback. For example, the emotion estimation function is used to collect other users' emotional reactions to the input impressions and keywords. The impression input unit can also use facial expression recognition technology as the emotion estimation function to analyze other users' facial expressions and estimate their emotions. Furthermore, the impression input unit can also use voice analysis technology to estimate emotions and improve the generated content based on the emotions. In this way, by using the emotion estimation function to collect other users' emotional reactions and improving the generated content based on the feedback, it is possible to generate content that resonates with users more.

[0091] The processing flow of the second embodiment will be briefly explained below.

[0092] Step 1: The image acquisition unit takes a picture of the cover of the book the user has finished reading. For example, the cover image can be taken using a smartphone camera and uploaded to the app. The cover image of the e-book can also be taken as a screenshot. Step 2: The OCR analysis unit uses OCR technology to analyze the cover image acquired by the image acquisition unit. For example, Tesseract is used as the OCR technology to extract the title, author name, and cover image from the image. Google Cloud Vision can also be used to perform more accurate analysis. Step 3: The review input unit accepts the book review and keywords entered by the user. For example, the review and keywords can be entered using a text input form. The review can also be entered using voice input. Step 4: The text generation unit generates text and images based on the impressions and keywords received by the impression input unit. For example, the generation AI can use GPT-3 to summarize the impressions and generate appropriate text. The generation AI can also use BERT to generate related images from keywords. Step 5: The music assignment unit assigns music based on the text and images generated by the text generation unit. For example, the generation AI performs emotional analysis and selects an emotional song for an emotional story. The generation AI can also select an appropriate song from a music library. Step 6: The video generation unit generates a short video by combining the music added by the music addition unit with text and images. For example, the generation AI uses video editing software to place the text and images on a timeline and add music to the background. The generation AI can also automatically add effects to create a visually appealing video.

[0093] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0094] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0095] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0096] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0097] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0098] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0099] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0100] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0101] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0102] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0103] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0104] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0105] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0106] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0107] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0108] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0109] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0110] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0111] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0112] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0113] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0114] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0115] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0116] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0117] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0118] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0119] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0120] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0121] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 may also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0122] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0123] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0124] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0125] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0126] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0127] 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0128] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0129] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0130] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0131] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0132] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0133] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0134] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0135] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0136] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0137] In the robot 414, the processor 46 performs the identification process. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0138] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0139] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0140] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0141] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0142] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0143] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0144] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0145] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0146] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0147] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0148] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0149] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0150] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0151] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0152] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0153] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0154] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0155] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0156] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0157] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0158] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0159] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0160] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. an image acquisition unit that captures a cover image of a book that a user has finished reading; an OCR analysis unit that analyzes the cover image acquired by the image acquisition unit using OCR technology; an impression input unit that accepts impressions and keywords of a book input by a user; a word generation unit that generates words and images based on the impressions and keywords received by the impression input unit; a music assigning unit that assigns music based on the text and image generated by the text generating unit; a video generation unit that generates a short video by combining the music assigned by the music assignment unit with text and images. A system characterized by:

2. The OCR analysis unit When photographing the cover image, images are taken from multiple angles, and a 3D model is generated for more detailed analysis.

2. The system of claim 1.

3. The impression input unit When users input their thoughts or keywords, the AI ​​analyzes their input in real time and makes appropriate suggestions.

2. The system of claim 1.

4. The music providing unit Based on the emotional analysis of the cover image and text, the tempo and rhythm of the song are adjusted to create a more cohesive video.

2. The system of claim 1.

5. The video generation unit When generating the short video, the system refers to the user's past posting data and provides an individually customized video template.

2. The system of claim 1.

6. The OCR analysis unit The user's emotion is estimated from the cover image, and the accuracy of the OCR analysis is adjusted based on the emotion.

2. The system of claim 1.

7. The impression input unit Generate text and images based on the user's emotions to create a POP that resonates with them emotionally.

2. The system of claim 1.

8. The music providing unit The emotional response of the user when listening to the music is analyzed in real time, and the most suitable music is selected.

2. The system of claim 1.

Images