WYSIWYG expression screen design method

By using cellular mesh canvas and FFmpeg Wasm analysis technology in the emoticon screen design system, the existing system's poor compatibility and motion consistency problems are solved, and multi-platform preview and opposite-sex screen support are realized, design efficiency and compatibility are improved. .

CN118887318APending Publication Date: 2024-11-01ZHEJIANG YOULU ROBOT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410904366.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

The existing emoji screen design system has problems such as poor compatibility, not supporting the opposite sex screen, unable to configure multi-color display effects, poor consistency of motion and not supporting animation conversion, resulting in inapplicability and inefficiency.

Method used

Rendering with colorless honeycomb mesh canvas, use FFmpeg Wasm to parse images and videos in multiple formats, stretch, adapt, vertex line and downsample, generate standardized intermediate data and convert them into Burned C language code, supports multi-platform preview and opposite-sex screen configuration.

Benefits of technology

It improves platform compatibility, user customization capabilities, motion consistency and design efficiency, supports multiple people to design and preview simultaneously, significantly reduces the number of pixels processing, and improves computing efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118887318A_ABST
    Figure CN118887318A_ABST
Patent Text Reader

Abstract

The invention discloses a WYSIWYG (what you see is what you get) expression screen design method, which comprises the following steps of: generating an achromatic cellular grid canvas, and rendering the grid canvas; inputting images and videos in various formats into the rendered grid canvas, analyzing the images and videos in various formats by using FFmpeg Wasm as a front end, and converting the contents of the images and videos in various formats into a series of independent frame images; performing stretching and adaptation, vertex connection and downsampling and color calculation on the series of independent frame images, storing the frame images as standardized intermediate data, generating a programmable C language code, and deploying the programmable C language code on a display device. The method has the advantages of being wide in application range, accurate in image color processing, convenient to operate and capable of improving working efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of software design, in particular to a WYSIWYG expression screen design method. Background Art

[0002] The expression screen design system of the existing technology usually only supports pixel dot matrix solutions with fixed aspect ratios, such as square or rectangular structures such as 10×10 and 12×10. Users need to manually fill in the colors one by one to complete the design of a display image. If continuous dynamic images need to be designed, they can only be modified and generated one by one, and then the display image is converted into a code that can be recognized by the controllable hardware device, and then the developer modifies the code and burns it to the device.

[0003] The existing technology generally has the following problems: poor applicability and low efficiency.

[0004] 1. The existing technology is generally installed locally. Since the differences in installation and display of different systems are not taken into account, the compatibility of the expression screen design system in the existing technology is poor;

[0005] 2. It does not support unconventional hardware screens. Existing solutions can only support hardware structures that conform to squares and rectangles, and cannot support heterogeneous screens. For example, such solutions cannot be applied when the number of rows and columns is inconsistent, which limits the user's design scope.

[0006] 3. It is impossible to set a preset hardware rendering. The existing technology is acceptable for monochrome display, but the multi-color display rendering is obviously very different from the actual display;

[0007] 4. The output product cannot be configured. The existing technology can only configure the output of the preset type, and cannot configure or support the user to configure the output result by themselves;

[0008] 5. Poor consistency of motion effects. Existing technologies basically do not support setting display duration, brightness and other parameters one by one for content with continuous motion effects.

[0009] 6. It does not support animated image conversion. The existing technologies basically support single image conversion and increase the color difference. It also does not support animated image conversion, which increases the difficulty for users. Summary of the invention

[0010] The purpose of the present invention is to provide a WYSIWYG expression screen design method to solve the problems raised in the above background technology.

[0011] To achieve the above object, the present invention provides the following technical solutions:

[0012] In a first aspect, the present invention provides a WYSIWYG expression screen design method, comprising:

[0013] Generate a colorless honeycomb grid canvas, and render the grid canvas;

[0014] Input images and videos in various formats into the rendered grid canvas, use FFmpeg Wasm as the front end to parse images and videos in various formats, and convert the image and video content in various formats into a series of independent frame images;

[0015] The series of independent frame images are stretched and adapted, vertex connected and downsampled, and color calculated, and then stored as standardized intermediate data, and burnable C language code is generated and deployed on the display device.

[0016] Preferably, rendering the grid canvas includes:

[0017] The grid canvas is configured into a plurality of horizontal and vertical honeycomb cells, each of which has a side length of 15 pixels;

[0018] The actual size of the grid canvas is determined by calculating the maximum number of horizontal and vertical honeycomb cells.

[0019] Preferably, the images in multiple formats include pictures and picture series; and the videos in multiple formats include animations.

[0020] Preferably, the use of FFmpeg Wasm as a front end to parse images and videos in various formats includes:

[0021] Generate a virtual instruction set based on the WebAssembly language;

[0022] During the parsing process, at least 30 frames of images are processed per second, and each frame of image is refined into a separate set of pixel points.

[0023] Preferably, the stretching and fitting process includes:

[0024] In processing each frame of the image, drag the image along the honeycomb canvas to make the frame image fit the height and width of the honeycomb canvas.

[0025] Preferably, the vertex connection and downsampling process includes:

[0026] Connecting the vertices in each hexagon by lines to form a plurality of internal line segments, wherein the internal line segments include a plurality of edges and a plurality of diagonals;

[0027] Pixels are sampled at fixed intervals, and multiple sampling points are taken on each internal line segment.

[0028] Preferably, the process of color calculation and data storage includes:

[0029] Calculate the RGB value of each sampling point and take the average color as the default color data of the honeycomb cell;

[0030] Store the calculated default color data in a standardized intermediate data format;

[0031] The intermediate data is formatted into C language code for the robot including default color data, and the default color data is compressed using length encoding.

[0032] In a second aspect, the present invention provides a WYSIWYG expression screen design device, the device comprising:

[0033] A rendering module, used for generating a colorless grid canvas, and configuring the grid canvas into a plurality of horizontal and vertical honeycomb cells;

[0034] A parsing module, wherein the parsing module has a built-in FFmpeg Wasm language and is used to parse input images and videos in various formats to obtain a series of independent frame images;

[0035] A deployment module is used to process the frame image and store it as standardized intermediate data, generate burnable C language code, and deploy it to a display device.

[0036] In a third aspect, the present invention provides a storage medium having computer program instructions stored thereon, which implement the method as described above when the computer program instructions are executed by a processor.

[0037] In a fourth aspect, the present invention provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, wherein the method described above is implemented when the computer program instructions are executed by the processor.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. The present invention uses FFmpeg Wasm as the front end to parse images and videos in various formats, making the platform of the present invention highly compatible. This solution adopts an online approach and supports direct preview of effects and design solutions on multiple platforms;

[0040] 2. The present invention is based on the network and multiple users can design and preview effect pictures and effect animations at the same time;

[0041] 3. The device of the present invention has strong hardware compatibility, and not only supports users to configure the parameters of heterogeneous screens by themselves, but also supports users to configure the output products, and has strong applicability to expand application scenarios;

[0042] 4. The dynamic effect consistency of the present invention is strong. After the parameters of the hardware display are set, the preview interface content and the conversion tool will generate colors close to the actual display according to the set parameters, thereby improving the design efficiency;

[0043] 5. The present invention has strong software and hardware synergy. By adding the firmware library supporting this solution to the user device, the design content can be synchronized to the display hardware in real time, realizing what you see is what you get and improving design efficiency.

[0044] 6. The present invention significantly reduces the number of pixels that need to be processed through vertex connection and fixed-interval sampling methods, while ensuring the accuracy and consistency of color calculations, thereby improving calculation efficiency and precision compared to existing technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0046] Figure 2 is a sub-flow chart of a method according to an embodiment of the present invention;

[0047] Figure 3 It is a schematic diagram of a WYSIWYG expression screen design mechanism module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] See also Figure 1 , Figure 2 The present invention provides a technical solution: a WYSIWYG expression screen design method, the method comprising the following steps:

[0050] S100: generating a colorless honeycomb grid canvas, and rendering the grid canvas;

[0051] S200: Input images and videos in various formats into the rendered grid canvas, use FFmpeg Wasm as the front end to parse images and videos in various formats, and convert the image and video content in various formats into a series of independent frame images;

[0052] S300: stretching and adapting, connecting vertices and downsampling, and calculating colors of the series of independent frame images, storing them as standardized intermediate data, generating burnable C language codes, and deploying them on a display device.

[0053] In one embodiment of the present invention, step S100 includes the following steps:

[0054] S110: configuring the grid canvas into a plurality of horizontal and vertical honeycomb cells, each of which has a side length of 15 pixels;

[0055] S120: Determine the actual size of the grid canvas by calculating the maximum number of horizontal and vertical honeycomb cells.

[0056] In this embodiment, for example, the grid canvas is configured as 41 columns and 13 rows of hexagonal honeycomb cells, and the actual width and height of the grid canvas are calculated as follows:

[0057] Horizontal width: The horizontal spacing of the hexagons is 1.5 times the side length, that is, 22.5 pixels. The spacing of odd columns is 2 times the side length, and the spacing of even columns is 1 time the side length. The total width is (212+20)×15=930 pixels.

[0058] Vertical height: The vertical spacing of the hexagon is times the side length, that is, 25.98 pixels. The total height is 13×25.98≈337.74 pixels.

[0059] That is, the actual size of the grid canvas is 930×337.5 pixels.

[0060] In one embodiment of the present invention, S200 includes the following steps:

[0061] S210: Generate a virtual instruction set based on WebAssembly.

[0062] S220: During the parsing process, at least 30 frames of images are processed per second, and each frame of image is refined into a separate set of pixel points to ensure the efficiency of subsequent processing.

[0063] In this embodiment, the images in multiple formats include pictures and picture sequences; the videos in multiple formats include animations.

[0064] In this embodiment, WebAssembly is based on the web-based architecture design, can run with high performance across platforms, supports different operating systems and devices (such as Windows, macOS, Linux, iOS, Android, etc.), and can design emoticons anytime and anywhere through the browser without installing additional software, ensuring stable operation in the browser environment.

[0065] In one embodiment of the present invention, S300 includes the steps of:

[0066] S310: Stretch and adapt the series of independent frame images.

[0067] In this embodiment, in processing each frame of image, the image is dragged according to the honeycomb canvas so that the frame image fits the honeycomb canvas with a width of 930 pixels and a height of 337.5 pixels.

[0068] S320: Connecting vertices and downsampling the series of independent frame images: connecting the vertices in each hexagon by connecting lines to form a plurality of internal line segments, wherein the internal line segments include a plurality of edges and a plurality of diagonals.

[0069] In this embodiment, 15 internal line segments are formed by connecting the vertices in each hexagon, including 6 sides and 9 diagonals, and the purpose of connecting the lines is to perform downsampling. During the downsampling process, sampling is performed at a fixed interval of 1 pixel, and the sampling interval of 1 pixel can ensure the accuracy of pixel restoration.

[0070] In this embodiment, 30 sampling points are generated on each internal line segment, for a total of 450 sampling points.

[0071] S330: Perform color calculation on the adopted pixels and take the average color as the default color data of the honeycomb cells.

[0072] In this embodiment, the RGB values ​​obtained by sampling are used to accurately calculate the color of each hexagonal cell, and an average color processing method is used to ensure color uniformity and accuracy.

[0073] S340: Storing the default color data in a labeled intermediate data format.

[0074] In this embodiment, the design data of the honeycomb canvas is stored in JSON format, including various elements of the expression and their attributes, supporting animation frame sequences and layer information, and ensuring the integrity and portability of the data.

[0075] In this embodiment, the intermediate data format is used to generate C language code for the robot containing default color data, which is convenient for direct deployment into the robot's control system. The character string is compressed using the Run-Length Encoding (RLE) compression technology to compress the default color data to reduce storage space and transmission time.

[0076] Please refer to Figure 3 The embodiment of the present invention further provides a WYSIWYG expression screen design device, the device comprising:

[0077] The rendering module 100 is used to generate a colorless grid canvas and configure the grid canvas into a plurality of horizontal and vertical honeycomb cells.

[0078] In this embodiment, the rendering module 100 supports real-time preview effects, supports multi-level and multi-frame animation editing, and supports special-shaped honeycomb screen display.

[0079] The parsing module 200 has a built-in FFmpeg Wasm language and is used to parse input images and videos in various formats to obtain a series of independent frame images.

[0080] In this embodiment, the framework of the parsing module 200 is a Web-based design that can run across platforms and support different operating systems and devices (such as Windows, macOS, Linux, iOS, Android, etc.). Designers can design emoticons anytime and anywhere through a browser without installing additional software.

[0081] The deployment module 300 is used to process the frame image and store it as standardized intermediate data, generate burnable C language code, and deploy it to the display device.

[0082] In this embodiment, the deployment module 300 deploys the designed expression package to the expression screen, supports multiple platforms and display devices, and provides a simple deployment process.

[0083] The deployment process includes:

[0084] 1. Pipeline containerized deployment: Pipeline containerization technology is used to ensure that design tools can be deployed and run quickly and consistently in different environments. Through containerization technology, efficient continuous integration and continuous deployment (CI / CD) can be achieved to improve development and operation efficiency.

[0085] 2. Screen burning deployment: Provide screen burning tools to burn expression packages to the robot's expression screen. The burning tool automatically adapts to screens with different resolutions and display characteristics, supports wired and wireless burning methods, and has built-in error detection and repair functions to ensure the smooth progress of the burning process.

[0086] The WYSIWYG expression screen design method of the present invention can be implemented by an electronic device.

[0087] The electronic device may include a processor and a memory storing computer program instructions.

[0088] Specifically, the processor may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present invention.

[0089] The memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside a data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0090] The processor implements any one of the automatic typesetting methods for cut piece images in the above embodiments by reading and executing computer program instructions stored in the memory.

[0091] In one example, the electronic device may further include a communication interface and a bus, wherein the processor, the memory, and the communication interface are connected via the bus and communicate with each other.

[0092] The communication interface is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiments of the present invention.

[0093] Bus includes hardware, software or both, and the parts of electronic equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In suitable cases, bus may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0094] In addition, in combination with a WYSIWYG expression screen design method in the above embodiment, an embodiment of the present invention can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any WYSIWYG expression screen design method in the above embodiment is implemented.

[0095] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0096] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0097] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0098] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A WYSIWYG expression screen design method, characterized in that: include: Generate a colorless honeycomb grid canvas, and render the grid canvas; Input images and videos in various formats into the rendered grid canvas, use FFmpeg Wasm as the front end to parse images and videos in various formats, and convert the image and video content in various formats into a series of independent frame images; The series of independent frame images are stretched and adapted, vertex connected and downsampled, and color calculated, and then stored as standardized intermediate data, and burnable C language code is generated and deployed on the display device.

2. The WYSIWYG expression screen design method according to claim 1, characterized in that: The rendering of the grid canvas includes: The grid canvas is configured into a plurality of horizontal and vertical honeycomb cells, each of which has a side length of 15 pixels; The actual size of the grid canvas is determined by calculating the maximum number of horizontal and vertical honeycomb cells.

3. The WYSIWYG expression screen design method according to claim 2, characterized in that: The images in the various formats include pictures and picture sequences; the videos in the various formats include animations.

4. The WYSIWYG expression screen design method according to claim 3, characterized in that: The use of FFmpeg Wasm as a front-end to parse images and videos in various formats includes: Generate a virtual instruction set based on the WebAssembly language; During the parsing process, at least 30 frames of images are processed per second, and each frame of image is refined into a separate set of pixel points.

5. The WYSIWYG expression screen design method according to claim 4, characterized in that: The stretching and fitting process includes: In processing each frame of the image, drag the image along the honeycomb canvas to make the frame image fit the height and width of the honeycomb canvas.

6. The WYSIWYG expression screen design method according to claim 5, characterized in that: The vertex connection and downsampling process includes: Connecting the vertices in each hexagon by lines to form a plurality of internal line segments, wherein the internal line segments include a plurality of edges and a plurality of diagonals; Pixels are sampled at fixed intervals, and multiple sampling points are taken on each internal line segment.

7. The WYSIWYG expression screen design method according to claim 6, characterized in that: The process of color calculation and data storage includes: Calculate the RGB value of each sampling point and take the average color as the default color data of the honeycomb cell; Store the calculated default color data in a standardized intermediate data format; The intermediate data is formatted into C language code for the robot including default color data, and the default color data is compressed using length encoding.

8. A WYSIWYG expression screen design device, characterized in that: The device includes: A rendering module, used for generating a colorless grid canvas, and configuring the grid canvas into a plurality of horizontal and vertical honeycomb cells; A parsing module, wherein the parsing module has a built-in FFmpeg Wasm language and is used to parse input images and videos in various formats to obtain a series of independent frame images; A deployment module is used to process the frame image and store it as standardized intermediate data, generate burnable C language code, and deploy it to a display device.

9. A storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: At least one processor, at least one memory and computer program instructions stored in the memory, which implement the method according to any one of claims 1 to 7 when the computer program instructions are executed by the processor.