Real-time video creation system based on intelligent image-text fusion

Through a real-time video creation system based on intelligent graphic and text integration, the problems of low video creation efficiency and quality in the existing technology are solved, and high-quality videos are quickly generated, with natural and personalized fusion effects, reducing production costs.

CN119071594APending Publication Date: 2024-12-03BEIJING BITE YIPAI INFORMATION TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411229971.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing video creation technology has problems such as slow processing speed, insufficient fusion effect, and limited personalized customization in real-time video creation, and it usually requires professional team collaboration, which increases time and cost.

Method used

Provide a real-time video creation system based on intelligent fusion of graphics and text, including user-side unit, web-layer unit, gateway unit, service processing unit, data layer unit and operating environment unit. It quickly generates videos by uploading pictures and text, and supports asynchronous generation and pictures Clarity setting, using JavaCV and Tencent TTS to generate synchronous video and audio.

Benefits of technology

It improves the efficiency and quality of video creation, fast processing speed, natural fusion effect, large range of personalized customization, and no increase in production costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071594A_ABST
    Figure CN119071594A_ABST
Patent Text Reader

Abstract

The invention discloses a real-time video creation system based on intelligent image-text fusion, which is characterized in that a user side unit, a web layer unit, a gateway unit and a service processing unit are respectively connected with a data layer unit, the data layer unit is connected with a database unit, and the database unit is connected with an operation environment unit; the user side unit is connected with the web layer unit, the web layer unit is connected with the gateway unit, and the gateway unit is connected with the service processing unit. According to the method, the video can be quickly generated by uploading the picture and the character, if the picture content or the input character content is wrong, the picture content or the input character content is submitted after being modified again, asynchronous generation is realized, and the efficiency is improved; and if the PDF and the PPT are uploaded, when the PDF and the PPT are converted into the picture, the DPI attribute of the picture can be set, and the definition of the picture is improved, so that the definition of the video is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of video creation, and in particular to a real-time video creation system based on intelligent fusion of images and texts. Background Art

[0002] Video creation is one of the common activities in the fields of culture and entertainment. Especially in recent years, the popularization of digital video technology and shooting equipment has prompted video creation activities to gradually expand from professional fields to ordinary users.

[0003] With the rapid development of artificial intelligence technology, intelligent image and text fusion technology has been widely used in the field of video creation. Existing implementation solutions, such as Genmo and VEED.IO, can convert static images into dynamic videos through AI technology, and allow users to add text, audio and other elements to enhance the video expression. However, existing technologies still have limitations in real-time video creation, such as slow processing speed, unnatural fusion effect, and limited degree of personalized customization.

[0004] Existing video synthesis solutions, even for relatively simple videos, are completed by recording with a camera and then processing with a third-party video processing tool.

[0005] At the same time, video recording usually requires the collaboration of a team, including the joint efforts of professionals such as directors, photographers, and lighting technicians. After recording, the video often needs to be edited and post-processed through professional video editing software, such as removing unnecessary beginnings and endings, or performing color correction, adding special effects, etc. These tasks are usually performed by video editors with professional skills. If technical problems or poor performance are encountered during the recording process, reshooting may be required, which will not only consume more time and resources, but also increase production costs. Summary of the invention

[0006] The technical problem to be solved by the present invention is to overcome the defects of low creation efficiency and quality of videos in the prior art and to provide a real-time video creation system based on intelligent fusion of images and texts.

[0007] The present invention solves the above technical problems through the following technical solutions:

[0008] The present invention provides a real-time video creation system based on intelligent fusion of images and texts, the real-time video creation system comprises a user end unit, a web layer unit, a gateway unit, a business processing unit, a data layer unit, a database unit and an operating environment unit, the user end unit, the web layer unit, the gateway unit and the business processing unit are respectively connected to the data layer unit, the data layer unit is connected to the database unit, and the database unit is connected to the operating environment unit;

[0009] The user terminal unit is connected to the web layer unit, the web layer unit is connected to the gateway unit, and the gateway unit is connected to the service processing unit.

[0010] In this technical solution, you can quickly generate a video by uploading pictures and text. If the picture content or the input text content is incorrect, you only need to modify it and submit it again. Asynchronous generation improves efficiency.

[0011] If you upload PDF and PPT, you can set the DPI property of the image when converting it to an image to improve the clarity of the image, thereby improving the clarity of the video;

[0012] Set the image clarity and JavaCV parameters to modify the generated video clarity.

[0013] Preferably, the user end unit comprises a registration and login module, an information association module and an information management and storage module, and the registration and login module is connected via the information association module and the information management and storage module.

[0014] In this technical solution, a registration and login module, an information association module and an information management and storage module are utilized to enable users to use the real-time video creation system.

[0015] Preferably, the user end unit further comprises a verification encryption module, and the verification encryption module is connected to the registration login module and the information management storage module respectively.

[0016] In this technical solution, the user information can be encrypted using the verification encryption module.

[0017] Preferably, the registration and login module includes a registration module, a login module and an information collection module.

[0018] In this technical solution, a registration and login module is used to allow users to register and log in to the real-time video creation system.

[0019] Preferably, the information collected by the information collection module is one or more of identity card, mobile phone number, face, and fingerprint.

[0020] Preferably, the web layer unit is H5.

[0021] In this technical solution, H5 is one of the versions of HTML.

[0022] Preferably, the gateway unit is API-SIX.

[0023] In this technical solution, API-SIX is a type of API gateway.

[0024] Preferably, the business processing unit includes a PDF to image module, a text to MP module, a multiple MP synthesis into one MP module, a picture and MP synthesis MP module, a message notification module and a PPT to image module.

[0025] In this technical solution, a business processing unit is used to intelligently fuse images and text to form a video.

[0026] Preferably, the database unit is MySQL.

[0027] In this technical solution, MySQL is a sophisticated SQL database management system.

[0028] Preferably, the operating environment unit is a virtual machine and docker.

[0029] In this technical solution, a virtual machine refers to a special software in the architecture of computer science, which can create an environment between a computer platform and an end user.

[0030] On the basis of being in accordance with the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present invention.

[0031] The positive and progressive effects of the present invention are:

[0032] The present invention can quickly generate a video by uploading pictures and texts. If the picture content or the input text content is wrong, it only needs to be revised and submitted again. The asynchronous generation improves efficiency.

[0033] The audio file generated by Tencent's TTS returns a time that is not a whole second. When generating a video based on the audio, the time is rounded up to ensure that the generated video and audio are synchronized.

[0034] If you upload PDF and PPT, when converting them to images, you can set the DPI attribute of the image to improve the clarity of the image, thereby improving the clarity of the video.

[0035] Set the image definition and JavaCV parameters to modify the generated video definition;

[0036] The video of the present invention has high creation efficiency and quality, fast processing speed, natural fusion effect, large personalized customization range, and will not increase production costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the overall structure of a real-time video creation system based on intelligent fusion of images and texts according to an embodiment of the present invention.

[0038] Figure 2 for Figure 1The diagram shown is a schematic diagram of the user end unit structure of a real-time video creation system based on intelligent fusion of graphics and text.

[0039] Figure 3 for Figure 1 The diagram shown is a structural diagram of a business processing unit of a real-time video creation system based on intelligent fusion of graphics and text.

[0040] Description of Reference Numerals

[0041] 1. User end unit; 11. Registration and login module; 111. Registration module; 112. Login module; 113. Information collection module; 12. Information association module; 13. Information management storage module; 14. Verification and encryption module;

[0042] 2. Web layer unit;

[0043] 3. Gateway unit;

[0044] 4. Business processing unit; 41. PDF to image module; 42. Text to MP3 module; 43. Multiple MP4s combined into one MP4 module; 44. Image and MP3 combined into MP4 module; 45. Message notification module; 46. PPT to image module;

[0045] 5. Data layer unit;

[0046] 6. Database unit;

[0047] 7. Operating environment unit. DETAILED DESCRIPTION

[0048] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0049] Figures 1 to 3 The structure diagram of an embodiment of the real-time video creation system based on intelligent fusion of images and texts of the present invention is shown. The real-time video creation system based on intelligent fusion of images and texts comprises a user end unit 1, a web layer unit 2, a gateway unit 3, a service processing unit 4, a data layer unit 5, a database unit 6 and an operating environment unit 7. The user end unit 1, the web layer unit 2, the gateway unit 3 and the service processing unit 4 are respectively connected to the data layer unit 5, the data layer unit 5 is connected to the database unit 6, and the database unit 6 is connected to the operating environment unit 7;

[0050] The user terminal unit 1 is connected to the web layer unit 2 , the web layer unit 2 is connected to the gateway unit 3 , and the gateway unit 3 is connected to the service processing unit 4 .

[0051] When in use, the user uploads PDF or EXCEL through the PC site, the system automatically determines the file type, and automatically divides the file into multiple pictures according to the file type. In this process, the DPI attribute of the picture is set according to the video resolution selected by the user to generate videos of different resolutions; or the user directly uploads the picture, and then enters the text to be played for each picture and submits it to the background processing.

[0052] The backend receives the request from the frontend, and generates the corresponding text into MP3 according to the tone and speech speed selected by the user; then uses the open source project JavaCV to generate an MP4 video file from a single image; the duration of the generated MP4 file must be the same as the duration of the MP3 audio, so that when merging the MP3 audio and MP4 video, there will be no inconsistency between the picture and the sound. After all the images are generated into video files, all the video files are combined into one video file, the temporary files generated in this process are deleted, and the final video file is uploaded to the file storage server; a generated message is sent to the user. The entire video generation step is completed.

[0053] If you find problems with the video content, such as text errors or other problems, you can simply modify the text and other information and regenerate the video.

[0054] PDF is a file format developed by Adobe Systems for file exchange in a way that is independent of applications, operating systems, and hardware. PDF files are based on the PostScript language image model and can guarantee precise colors and accurate printing results on any printer, that is, PDF will faithfully reproduce every character, color, and image of the original. PDF is an electronic file format that has nothing to do with the operating system platform, that is, PDF files are universal regardless of Windows, Unix, or Apple's Mac OS operating system. This feature makes it an ideal document format for electronic document distribution and digital information dissemination on the Internet. More and more electronic books, product descriptions, company announcements, network materials, and e-mails are beginning to use PDF format files.

[0055] JavaCV is a wrapper library based on the JavaCPP calling method (a layer of JNI encapsulation), which consists of multiple open source computer vision libraries. It encapsulates common libraries and utility classes in the field of computer vision, including FFmpeg, OpenCV, tensorflow, caffe, tesseract, libdc1394, OpenKinect, videoInput, and ARToolKitPlus. JavaCV is based on the Apache License Version 2.0 agreement and the GPLv2 agreement. JavaCV supports calling these interfaces on Java platforms including Windows, Linux, MacOS, Android, and IOS.

[0056] That is, when using it, use the one-to-one ratio of pictures and text, use Tencent TTS interface to synthesize text into MP3 audio files, use the open source JavaCV toolkit to write pictures to MP4 video files at frame rate, and the duration of MP4 video is equal to the duration of the corresponding MP3 audio file. Since the duration of MP3 audio files is not an integer number of seconds, such as 5.3 seconds, and videos must be generated according to integer seconds, for example, MP4 videos can only generate 6-second files, here the duration of MP3 audio files is rounded up, that is, 6 seconds, and video generation is also generated according to 6 seconds, so that each small video pit has less than 1 second of audio waiting time. After each picture and the corresponding text generate an MP4 video file, each MP4 file is merged into a final MP4 file using JavaCV, and the final generated MP4 file is uploaded to Tencent Cloud to notify the user that the video file merger is complete.

[0057] In this technical solution, you can quickly generate a video by uploading pictures and text. If the picture content or the input text content is incorrect, you only need to modify it and submit it again. Asynchronous generation improves efficiency.

[0058] If you upload PDF and PPT, you can set the DPI property of the image when converting it to an image to improve the clarity of the image, thereby improving the clarity of the video;

[0059] Set the image clarity and JavaCV parameters to modify the generated video clarity.

[0060] In the present invention, when uploading pictures, the system also supports converting PDF and PPT files into multiple picture formats for further processing;

[0061] Add corresponding text descriptions to each picture and combine these texts to generate short video clips;

[0062] The speech speed can be adjusted. The speed adjustment range provided by the system is 0.6 to 2.5 times, of which 1 times is the standard speed.

[0063] Provides voice selection function. After careful selection and testing, there are 4 high-quality voices to choose from, including 2 male and 2 female voices;

[0064] You can select the video resolution and file size as needed to meet different playback and sharing requirements;

[0065] After submitting the selected options, the system will combine the audio converted from the picture and the corresponding text into a small video in the background, and then combine multiple small videos into a complete MP4 video file;

[0066] After the video is produced, the system will automatically send a notification message to the user;

[0067] If the generated video needs to be modified, just modify the image or question and regenerate it.

[0068] The user terminal unit 1 comprises a registration and login module 11 , an information association module 12 and an information management and storage module 13 , and the registration and login module 11 is connected with the information management and storage module 13 via the information association module 12 .

[0069] In this technical solution, the registration and login module 11, the information association module 12 and the information management and storage module 13 are utilized to enable users to use the real-time video creation system.

[0070] The user end unit 1 is a PC station.

[0071] The user end unit 1 further comprises a verification and encryption module 14, and the verification and encryption module 14 is connected to the registration and login module 11 and the information management and storage module 13 respectively.

[0072] In this technical solution, the verification and encryption module 14 can be used to encrypt user information.

[0073] The registration and login module 11 includes a registration module 111 , a login module 112 and an information collection module 113 .

[0074] In this technical solution, the registration and login module 11 is used to allow users to register and log in to the real-time video creation system.

[0075] The information collected by the information collection module 113 is one or more of identity card, mobile phone number, face, and fingerprint.

[0076] The web layer unit 2 is H5.

[0077] In this technical solution, H5 is one of the versions of HTML.

[0078] HTML is Hypertext Markup Language, an application under Standard Generalized Markup Language. "Hypertext" means that a web page can contain pictures, links, and even non-text elements such as music and programs. The structure of Hypertext Markup Language includes the "head" part in English: Head, and the "body" part in English: Body. The "head" part provides information about the web page, and the "body" part provides the specific content of the web page.

[0079] HTML is a specification, a standard, which uses tags to mark various parts of the web page to be displayed. The web page file itself is a text file. By adding tags to the text file, you can tell the browser how to display the content, such as how to process text, how to arrange pictures, how to display pictures, etc. The browser reads the web page file in order, and then interprets and displays the marked content according to the tags. The error will not be pointed out for the tags with writing errors, and the interpretation and execution process will not be stopped. The compiler can only analyze the cause and location of the error through the display effect. However, it should be noted that different browsers may have different interpretations of the same tag, and therefore may have different display effects.

[0080] The gateway unit 3 is API-SIX.

[0081] In this technical solution, API-SIX is a type of API gateway.

[0082] The API Gateway is a server that is the only entry point to the system. The API Gateway encapsulates the internal architecture of the system and provides a customized API for each client. It may also have other responsibilities, such as authentication, monitoring, load balancing, caching, protocol conversion, current limiting and circuit breaking, and static response processing.

[0083] The business processing unit 4 includes a PDF to image module 41 , a text to MP3 module 42 , a multiple MP4s to one MP4 module 43 , a picture and MP3 to MP4 module 44 , a message notification module 45 and a PPT to image module 46 .

[0084] In this technical solution, the business processing unit 4 is used to intelligently fuse pictures and texts to form a video.

[0085] The database unit 6 is MySQL.

[0086] In this technical solution, MySQL is a sophisticated SQL database management system.

[0087] MySQL is a true multi-user, multi-threaded SQL database server. SQL structured query language is the most popular and standardized database language in the world. MySQL is implemented in a client / server structure, which consists of a server daemon mysqld and many different client programs and libraries.

[0088] SQL is a standardized language that makes it easier to store, update, and access information. For example, you can use SQL to retrieve product information and store customer information for a website, while MySQL is also fast and flexible enough to allow you to store log files and images.

[0089] The operating environment unit 7 is a virtual machine and docker.

[0090] In this technical solution, a virtual machine is a special software in the architecture of computer science, which can create an environment between the computer platform and the end user, and the end user operates the software based on the environment created by this software. In computer science, a virtual machine refers to the software implementation of a computer that can run programs like a real machine;

[0091] Virtual machine technology is a type of virtualization technology. The so-called virtualization technology is to transform things from one form into another. The most commonly used virtualization technology is the virtualization of memory in the operating system. During actual operation, the memory space required by the user may be much larger than the memory size of the physical machine. Using memory virtualization technology, users can virtualize part of the hard disk into memory, which is transparent to the user.

[0092] Docker is an open source application container engine that allows developers to package their applications and dependent packages into a portable image and then publish it to any popular Linux or Windows operating system machine, and can also achieve virtualization;

[0093] Docker's architecture includes three basic concepts:

[0094] Image: Docker image is equivalent to a root file system. For example, the official image ubuntu:16.04 contains a complete set of root file systems for the Ubuntu 16.04 minimum system.

[0095] Container: The relationship between an image and a container is like that between a class and an instance in object-oriented programming. An image is a static definition, and a container is an entity when the image is running. A container can be created, started, stopped, deleted, or paused.

[0096] Repository: The repository can be seen as a code control center for storing images.

[0097] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A real-time video creation system based on intelligent fusion of graphics and text, characterized by: The real-time video creation system comprises a user end unit (1), a web layer unit (2), a gateway unit (3), a service processing unit (4), a data layer unit (5), a database unit (6) and an operating environment unit (7); the user end unit (1), the web layer unit (2), the gateway unit (3) and the service processing unit (4) are respectively connected to the data layer unit (5); the data layer unit (5) is connected to the database unit (6); and the database unit (6) is connected to the operating environment unit (7); The user terminal unit (1) is connected to the web layer unit (2), the web layer unit (2) is connected to the gateway unit (3), and the gateway unit (3) is connected to the service processing unit (4).

2. The real-time video creation system based on intelligent fusion of images and texts as claimed in claim 1, characterized in that: The user terminal unit (1) comprises a registration and login module (11), an information association module (12) and an information management storage module (13); the registration and login module (11) is connected via the information association module (12) and the information management storage module (13).

3. The real-time video creation system based on intelligent fusion of graphics and text as claimed in claim 2, characterized in that: The user terminal unit (1) also includes a verification encryption module (14), and the verification encryption module (14) is connected to the registration and login module (11) and the information management storage module (13) respectively.

4. The real-time video creation system based on intelligent fusion of graphics and text as claimed in claim 2, characterized in that: The registration and login module (11) comprises a registration module (111), a login module (112) and an information collection module (113).

5. The real-time video creation system based on intelligent fusion of graphics and text as claimed in claim 4, characterized in that: The information collected by the information collection module (113) is one or more of identity card, mobile phone number, face and fingerprint.

6. The real-time video creation system based on intelligent fusion of graphics and text according to claim 1, characterized in that: The web layer unit (2) is H5.

7. The real-time video creation system based on intelligent fusion of graphics and text according to claim 1, characterized in that: The gateway unit (3) is API-SIX.

8. The real-time video creation system based on intelligent fusion of graphics and text according to claim 1, characterized in that: The business processing unit (4) comprises a PDF to picture module (41), a text to MP3 module (42), a multiple MP4s to one MP4 module (43), a picture and MP3 to MP4 module (44), a message notification module (45) and a PPT to picture module (46).

9. The real-time video creation system based on intelligent fusion of graphics and text according to claim 1, characterized in that: The database unit (6) is MySQL.

10. The real-time video creation system based on intelligent fusion of images and texts according to claim 1, characterized in that: The operating environment unit (7) is a virtual machine and a docker.