Visualization methods and devices for icon-based narrative programs for VR or AR.

Abstract

This invention discloses a visualization method and apparatus for VR or AR-based narrative programs using icons. The method includes extracting images, emotions, and actions of narrative objects from narrative content data for semantic modeling, mapping the semantic modeling to preset icons and color labels; wherein the color labels represent emotions through color saturation; formally describing the icons according to the narrative content data, assigning attribute data to the icon data, and visually describing the temporal relationship of the narrative objects corresponding to the icons with assigned attribute data within the narrative content data; optimizing the placement conflicts of the icons on the vertical and horizontal axes of the narrative space during visualization, and allocating regions of the icons based on temporal sequence to complete the visualization description and display it accordingly. This invention can intuitively reflect the narrative content and rhythm, thereby improving the user experience.

Description

Technical Field

[0001] This invention relates to the field of digital media, and more particularly to a visualization method and apparatus for expressing narrative programs based on icons for VR or AR. Background Technology

[0002] With the increasing maturity of software and hardware related to the metaverse and the establishment of disciplines such as virtual reality, VR / AR narrative programs have gradually become a mature industrial output. Creators use 3D models, scenes, and virtual characters to create virtual events, which are then presented in an immersive way through various VR / AR glasses. In the field of cultural heritage, virtual exhibition halls and interactive exhibits provide a better understanding of cultural heritage and historical events; in the field of education, students learn various kinds of knowledge intuitively through virtual environments and objects, aiding comprehension and strengthening memory; in the entertainment industry, VR / AR games and performances bring audiences into a more interesting world in a more immersive way. VR / AR narrative programs are similar to traditional films in that they tell a story on a single theme, but they are concise, with simple relationships between objects, and generally complete the entire explanation within a single scene.

[0003] The earliest methods for narrative temporal visualization were used to understand complex events and their relationships with timelines, helping people comprehend the development of events. Subsequently, temporal visualization techniques were developed to represent complex event data such as basketball games, workflows, and medical records, enhancing user understanding. It's easy to see that these scenarios using temporal visualization generally have long timelines, with object or attribute relationships hidden between many points in time, requiring visualization to aid analysis and discovery. VR / AR narrative programs, however, have short timelines and do not require uncovering these potential relationships. However, this presents another challenge: a story needs to be clearly presented and captivating within a short timeframe; yet viewers need to wear glasses to watch the entire program to grasp its content and pacing, which is not user-friendly for creators. Therefore, it is necessary to explore intuitive event visualization methods to construct a visual representation of the content and pacing for VR / AR narrative programs. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a visualization method and apparatus for icon-based narrative programs for VR or AR applications.

[0005] A first aspect of the present invention provides a visualization method for icon-based narrative programs for VR or AR, comprising:

[0006] Based on the narrative content data, images, emotions, and actions of the narrative objects are extracted for semantic modeling, and the semantic modeling is mapped to preset icons and color labels; wherein the color labels represent emotions through color saturation.

[0007] The icons are formally described based on the narrative content data, so that the icon data is assigned attribute data, and the temporal relationship of the narrative objects corresponding to the icons with assigned attribute data in the narrative content data is visualized.

[0008] When creating a visual description, the conflict between the vertical and horizontal placement of the icons in the narrative space is optimized, and the icons are assigned regions based on time sequence to complete the visual description and display it according to the visual description.

[0009] In one optional implementation, the step of formally describing the icon based on narrative content data, such that the icon data is assigned attribute data, includes:

[0010] The system obtains the name, size, coordinates, and emotion of the narrative object, as well as the change data of the narrative object; wherein the change data includes the change time of the narrative object and the change sequence of the narrative object, and the narrative object includes virtual objects and virtual characters;

[0011] Assign the name, size, coordinates, mood, and change data to the icon.

[0012] In one optional implementation, the icon includes a virtual object icon, a virtual character icon, and a scene change icon; assigning the name, size, coordinates, emotion, and change data to the icon includes:

[0013] When assigning the change data to the scene change icon, the scene change icon is displayed using two entities: an activity time block and a timeline. The color saturation of the activity time block is adjusted according to the attractiveness value of the virtual character icon or the virtual object icon. The two-dimensional virtual object corresponds to the first color, the three-dimensional virtual object corresponds to the second color, and the virtual character corresponds to both the first color and the second color.

[0014] In one optional implementation, the step of visually describing the temporal relationship of the narrative objects corresponding to the icons assigned attribute data in the narrative content data includes:

[0015] The icon is set according to the narrative object and placed in the scene, and a visual description is performed based on the attribute data;

[0016] Identify vertical and horizontal axis conflicts of the icons in the narrative space, optimize the vertical and horizontal axis conflicts, and group the icons.

[0017] The icons after grouping are assigned regions such that the total height of the displayed icons in each group does not exceed the height of the narrative space.

[0018] In one optional implementation, the step of identifying vertical and horizontal axis conflicts of the icons in the narrative space, optimizing the vertical and horizontal axis conflicts, and grouping the icons includes:

[0019] In the vertical axis direction, when different icons at the same distance from the camera overlap, the different icons are arranged close together and icons representing the same narrative object are grouped together;

[0020] When the lines connecting different icons intersect along the horizontal axis, the conflict between adjacent groups of icons placed on the same horizontal axis is calculated, and the non-conflicting icons are grouped together.

[0021] In one optional implementation, the step of allocating regions to the grouped icons such that the total height of the displayed groups of icons does not exceed the height of the narrative space includes:

[0022] Obtain the height of the largest icon in each group, calculate the base height of each group of icons in the narrative space, and place the icons of each group according to the base height.

[0023] In one optional implementation, the step of allocating regions to the grouped icons, such that the total height of the displayed groups of icons is not greater than the height of the narrative space, further includes:

[0024] If the position of any of the icons changes, the icons in the target group whose position has changed are compressed so that the height of the icons in all groups meets the height of the narrative space; the height of the icons whose position has not changed is set as the base height.

[0025] A second aspect of the present invention provides a visualization device for icon-based narrative programs for VR or AR, comprising:

[0026] The data acquisition module is used to extract images, emotions, and actions of narrative objects from narrative content data for semantic modeling, and map the semantic modeling to preset icons and color labels; wherein the color labels represent emotions through color saturation.

[0027] The visualization description module is used to formally describe the icons according to the narrative content data, so that the icon data is assigned attribute data, and to visualize the temporal relationship of the narrative objects corresponding to the icons with assigned attribute data in the narrative content data.

[0028] The display module is used to optimize the conflict between the vertical and horizontal axes of the icons in the narrative space when making a visual description, and to allocate the icons to regions based on time sequence, so as to complete the visual description and display it according to the visual description.

[0029] A third aspect of the present invention provides an electronic device comprising:

[0030] At least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor invokes the program instructions to execute the visualization method for icon-based narrative programs for VR or AR as described in the first aspect of the present invention.

[0031] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a computer, performs the visualization method for icon-based narrative programs for VR or AR as described in the first aspect of the present invention.

[0032] This invention formalizes the description of the icons based on narrative content data, assigning attribute data to the icon data. Then, it visualizes the temporal relationship between the narrative objects corresponding to the icons with assigned attribute data in the narrative content data, which can intuitively reflect the narrative content and rhythm, thereby improving the user experience. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart illustrating a visualization method for expressing narrative programs based on icons, oriented towards VR or AR, according to an embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram of a virtual object icon and a virtual character icon in an embodiment of the present invention.

[0036] Figure 3 This is a flowchart illustrating another visualization method for expressing narrative programs based on icons, oriented towards VR or AR, in an embodiment of the present invention.

[0037] Figure 4 This is a schematic diagram of an icon-based visual layout in an embodiment of the present invention.

[0038] Figure 5 This is a schematic diagram of a visualization device for VR or AR-based narrative programs, as described in an embodiment of the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] It should be understood that the terms "first," "second," and "third," etc., in the claims, specification, and drawings of this invention are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" used in the specification and claims of this invention indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0041] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0042] Please see Figure 1 The present invention provides a visualization method for expressing narrative programs based on icons for VR or AR, comprising the following steps.

[0043] Step 100: Extract images, emotions, and actions of the narrative objects from the narrative content data to perform semantic modeling, and map the semantic modeling to preset icons and color labels; wherein the color labels represent emotions through color saturation.

[0044] Step 200: The icons are formally described according to the narrative content data, so that the icon data is assigned attribute data, and the temporal relationship of the narrative objects corresponding to the icons with assigned attribute data in the narrative content data is visualized.

[0045] Step 300: Optimize the conflict between the vertical and horizontal placement of the icons in the narrative space when performing the visualization description, and allocate the icons to regions based on the time sequence to complete the visualization description and display it according to the visualization description.

[0046] Specifically, semantic modeling involves extracting information from the objects representing narrative content data, typically expressed as objects, characters, scenes, as well as characters' emotions, actions, and the state of objects.

[0047] The formal description of the narrative content data is divided into two types: parameter description and visualization description. The parameter description is further divided into four types: virtual object O, virtual character C, object transformation E, and scene S.

[0048] The name, size, coordinates, and emotion of the narrative object are obtained using a JSON script and the Unity engine, along with the change data of the narrative object. This change data includes object changes (E) and scene changes (S).

[0049] In one embodiment, the virtual object consists of an object name, coordinates, and size, wherein the name is obtained from a JSON script; the coordinates are the object's position in the scene; and the size is the bounding box size of the virtual object, obtained from the program via the Unity engine.

[0050] The virtual character is a special virtual object consisting of a character name, coordinates, size, and emotion. The name and emotion are obtained from a JSON script; the coordinates are the character's position in the scene; and the size is the character's bounding box size, obtained from the program through the Unity engine.

[0051] The object change consists of a start time, an end time, and an animation format. The start and end times are obtained from the program through the Unity engine, and the animation format is obtained from the JSON script.

[0052] The background of the scene is the location where the entire program takes place, which is represented by a panoramic image in Unity. The scene S is composed of a sequence of changes in different objects. After determining the visual representation of the objects and their changes in the scene, the scene organizes the narrative objects in a timeline and presents them in sequence, i.e., it is visualized.

[0053] Furthermore, the two-dimensional virtual object corresponds to the first color, the three-dimensional virtual object corresponds to the second color, and the virtual character corresponds to both the first and second colors. For example, the two-dimensional object uses a receding color; the three-dimensional object uses an aggressive color; and the narrative object uses a combination of both colors. The receding color is relative to the aggressive color. Aggressive colors are usually expressed by their saturation, such as red, while receding colors have low saturation, such as blue or yellow.

[0054] When mapping the semantic modeling to a preset icon, the name, size, coordinates, emotion, and change data are assigned to the icon. For example, based on the semantic information of the semantic modeling, icons are categorized into virtual object icons, virtual character icons, and scene change icons; the narrative objects include virtual objects and virtual characters, and the change data includes the change time of the narrative object and the change sequence of the narrative object.

[0055] The virtual object icons come in two forms: one is a standard icon style given for images, videos, prompts, etc.; the other is a 3D object, retrieved by searching the icon library using its name. The size of the virtual object icon is calculated using the formula Iconwidth = (w*h / W*H)*IconMax, where w and h are the length and width of the icon projected onto the screen, respectively; W and H are the length and width of the display, respectively. The maximum icon size is the optimal value to ensure that the icon wiring does not overlap during visualization generation, thus achieving the effect of near objects appearing larger and farther away appearing smaller. The color of the virtual object icon is typically black.

[0056] The proportions of the virtual character's icon are also calculated based on the method for calculating the size of virtual object icons. In addition, to reflect the character's special emotional changes, colored lines are used to express the virtual character's emotions, distinguishing them from the black lines of the object icons.

[0057] When the change data is assigned to the scene change icon, the scene change icon is displayed using two entities: an activity time block and a timeline; that is, the scene change icon is implemented through activity time blocks and a timeline. The shape of the activity time block reflects both the duration of the change and the attraction of the changes to the object represented by the icon over a period of time to the audience; the timeline represents the passage of time, reflecting the development of the narrative program as time flows by.

[0058] Then adjust the color saturation of the activity time block based on the attractiveness value of the virtual character icon or the virtual object icon.

[0059] For example, the color of the activity time block differs across three different objects (two-dimensional objects, three-dimensional objects, and virtual characters). Generally, two-dimensional objects use a receding color; three-dimensional objects use an aggressive color; and narrative objects use a combination of both. The activity blocks of different objects are filled with different saturations of this color based on the attractiveness values ​​of various animation effects.

[0060] For example, in the program introducing the Old Summer Palace (Yuanmingyuan), the scene at Haiyantang is represented by a two-dimensional icon, and throughout the program, it is represented by receding colors, such as blue or yellow. During the program, flames (virtual objects) appear in three dimensions, represented by aggressive colors, such as red. The narrator is a child named "Xiao Hai" (a virtual character). At the beginning of the program, "Xiao Hai" provides a narrative explanation, using receding colors. As the flames burn, "Xiao Hai's" emotions shift towards anger; at this point, the aggressive color is superimposed on the original receding color, and the saturation of the superimposed color is adjusted according to the magnitude of the attraction value.

[0061] In this invention, such as Figure 2 As shown, for the icons that have a time block attached, the time block icon is as follows: Figure 2 The icons represent rising, rotating, running, and waving colors. Therefore, when visually arranging these icons, it's necessary to combine the icons themselves with the timeline icons, the horizontal timeline, and the vertical axis.

[0062] The icon with an attached time block is represented by I. i =<X0,Y0,Max(Iconwidth,TimeX)Iconwidth+TimeY> Where X0 and Y0 are the coordinates of the bottom left corner of the icon, its width is determined by the width of the activity time block and the icon, and its height is the sum of the heights of the icon and the activity time block. X0 and TimeX are obtained from the time nodes in the program through the Unity engine, TimeY is set to a fixed value, and Iconwidth (i.e., attribute value) is represented by the maximum icon size IconMax. Therefore, the core of the visual layout lies in optimizing the layout on the vertical axis (i.e., height arrangement) to obtain the optimal IconMax and Y0 of each icon (i.e., horizontal arrangement).

[0063] For example, in the program introducing the Old Summer Palace, after the flames appear, there will be a burning animation effect. The burning animation effect is converted into the length of the activity time block according to the duration and attached to the bottom of the flame icon. At this time, the flame icon and the activity time block are integrated (as the icon described in this invention). The maximum length of the icon and the activity time block is calculated and the maximum length of the overall icon is set. The height of the flame icon and the activity time block are added together to get the height of the new icon.

[0064] like Figure 3 As shown, in step 300, the step of visually describing the temporal relationship of the narrative object corresponding to the icon with assigned attribute data in the narrative content data includes the following steps.

[0065] Step 310: Set the icon according to the narrative object and place the icon in the scene, and perform a visual description based on the attribute data.

[0066] Specifically, such as Figure 4 As shown in Figure 1, the initialized icon, which is then assigned attribute data, is placed in the scene and displayed in the narrative space. The narrative space is the display area of ​​the screen.

[0067] In the scene, all icons Ii from the initial set of N icons are sorted according to their distance from the camera. The icons are connected by lines, which can be either dashed or solid. Dashed lines represent the temporal relationship of the same object, connecting the same group of objects Om; dashed lines are generated by connecting two or more identical objects in the Unity interface. Solid lines represent the interaction process of the VR / AR program. After connecting an object change event with a solid line, if the interaction triggers another object change event, the interaction method is also mapped to an icon marked on the solid line; solid lines are generated by connecting two triggering objects in the Unity interface, and the interaction method is filled in.

[0068] Step 320: Identify the vertical and horizontal axis conflicts of the icons in the narrative space, optimize the vertical and horizontal axis conflicts, and group the icons.

[0069] The Unity engine can identify whether the icons overlap in the narrative space. For example, two icons overlap in the middle left position, and three icons overlap in the lower right position.

[0070] When optimizing for conflicts, if different icons at the same distance from the camera overlap along the vertical axis, the different icons are arranged close together, and icons representing the same narrative object are grouped together. In this invention, the vertical axis conflict refers to the overlap of different icons at the same distance from the camera; therefore, two objects can be arranged close together, and icons representing the same narrative object are grouped together. Figure 4 In the diagram, the initial icon 1 is changed to 2 to illustrate the vertical axis conflict handling. Multiple icons are superimposed vertically, and the multiple icons do not belong to the same narrative object (different narrative objects correspond to different icons). Then, the overlap is processed to be arranged vertically and grouped together.

[0071] When lines connecting different icons intersect along the horizontal axis, the conflict between adjacent groups of icons placed on the same horizontal axis is calculated, and icons without conflict are grouped together. For example... Figure 4 The diagram illustrates the handling of vertical axis conflict in section 2. Character 1 on the left and Character 1 on the right are connected by a dotted line, but their positions are staggered. Since they belong to the same narrative subject, they can be grouped together, ultimately optimized as follows: Figure 4 The diagram illustrates the handling of conflicts along the three horizontal axes, with the two figures on either side grouped together horizontally.

[0072] In this invention, the horizontal axis conflict refers to the conflict caused by the intersection of lines connecting different objects. Therefore, the conflict between two adjacent groups placed on a horizontal axis can be calculated, and objects without conflict can be grouped together. This process is repeated until they can no longer be merged. At this point, the grouping is {group1, group2, ..., group...} k There are k groups in total, and each group has numgroups. n Each element.

[0073] Step 330: Allocate regions to the grouped icons so that the total height of the displayed icons in each group is not greater than the height of the narrative space.

[0074] Obtain the height of the largest icon in each group, calculate the base height of each group of icons in the narrative space, and place the icons of each group according to the base height.

[0075] For example, search each group j The tallest icon I j-max Calculate each group proportionally j The Y-axis region in the allocated visualization view is used to obtain the basic Y0 of the placement area for each group of icons, with the optimal IconMax being the maximum area height. For example... Figure 4 In the narrative space, four groups of icons are placed along the height direction. By calculating the screen display size, the maximum height of each group of icons can be obtained, and thus the Y0 of each group of icons can be determined.

[0076] The layout fails when the height of the icon placement area exceeds the threshold IconMax. Since Y0 is fixed, icons I are placed in each group of areas according to X0. i Then, for the same object, adjust its height within the area based on its distance. The icon's height is determined by IconMax = (numgroup) j Calculate / ∑numgroup)*H0.

[0077] If any of the icons changes position, the icons in the affected group are compressed to ensure that the height of all icons in all groups meets the height of the narrative space; the height of the icons that do not change position is set to the base height. The final visual layout consists of four groups of icons, see... Figure 4 A schematic diagram of the allocation of the four regions.

[0078] For example, in the Yuanmingyuan program, three objects appear simultaneously: Haiyantang, flames, and a child named "Xiao Hai." These three objects are not the same, and their positions and sizes change over time. Therefore, each object in the entire program is divided into three groups. The group whose position does not change is assigned the largest icon, which is then assigned the group height IconMax. Xiao Hai's distance from the audience will decrease, changing his position on the vertical axis. The group height is calculated based on the overlap height of all icons in the group. When the combined height of the three groups exceeds the interface resolution, the group whose position changes is compressed.

[0079] This invention presents a visualization method for VR or AR narrative programs based on icon representation. By formally describing narrative content data and then visualizing it using an algorithm, the icon layout intuitively reflects the narrative content and rhythm, thereby improving the user experience. The visualization algorithm requires a temporal visualization layout of icon objects and activity time blocks, resolving conflicts on the horizontal and vertical axes. This constructs a visual explanation of the content and rhythm of the VR or AR narrative program, enhancing the user experience.

[0080] Please see Figure 5 The present invention also provides a visualization device for VR or AR-based icon-based narrative programs, comprising: a data acquisition module 51, a visualization description module 52, and a display module 53.

[0081] The data acquisition module 51 is used to extract images, emotions, and actions of narrative objects from narrative content data for semantic modeling, and map the semantic modeling to preset icons and color labels; wherein the color labels represent emotions through color saturation.

[0082] The visualization description module 52 is used to formally describe the icon according to the narrative content data, so that the icon data is assigned attribute data, and to visualize the temporal relationship of the narrative object corresponding to the icon with assigned attribute data in the narrative content data.

[0083] The display module 53 is used to optimize the conflict between the vertical and horizontal axes of the icons in the narrative space when making a visual description, and to allocate the icons to regions based on the time sequence, so as to complete the visual description and display it according to the visual description.

[0084] For a description of visualization devices for icon-based narrative programs for VR or AR, please refer to the visualization methods for icon-based narrative programs for VR or AR described above.

[0085] The present invention also provides an electronic device, comprising:

[0086] At least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor invokes the program instructions to execute the above-described visualization method for icon-based narrative programs for VR or AR.

[0087] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described visualization method for icon-based narrative programs for VR or AR.

[0088] It is understood that computer-readable storage media can include: any entity or device capable of carrying computer programs, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc. Computer programs include computer program code. Computer program code can be in the form of source code, object code, executable files, or certain intermediate forms, etc. Computer-readable storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.

[0089] In some embodiments of the present invention, the electronic device may include a controller or a processor. The controller is a microcontroller chip that integrates a processor, memory, communication module, etc. The processor may refer to the processor included in the controller. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0090] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0091] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0092] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A visualization method for icon-based narrative programs for VR or AR, characterized in that, include: Based on the narrative content data, images, emotions, and actions of the narrative objects are extracted for semantic modeling, and the semantic modeling is mapped to preset icons and color labels; The color tags described therein represent emotions through the saturation of the colors; The icons are formally described based on the narrative content data, so that the icon data is assigned attribute data, and the temporal relationship of the narrative objects corresponding to the icons with assigned attribute data in the narrative content data is visualized. When performing a visual description, the conflict between the vertical and horizontal axes of the icons in the narrative space is optimized, and the icons are allocated to different regions based on time sequence to complete the visual description and display it according to the visual description. The step of visually describing the temporal relationship of the narrative objects corresponding to the icons with assigned attribute data in the narrative content data includes: setting the icons according to the narrative objects and placing the icons in the scene, and visually describing them based on the attribute data; identifying vertical and horizontal axis conflicts of the icons in the narrative space, optimizing the vertical and horizontal axis conflicts, and grouping the icons; and allocating regions to the grouped icons so that the total height of the displayed groups of icons is not greater than the height of the narrative space. The process of identifying vertical and horizontal axis conflicts of the icons in the narrative space, optimizing the vertical and horizontal axis conflicts, and grouping the icons includes: in the vertical axis direction, when different icons at the same distance from the camera overlap, arranging the different icons close together and grouping icons representing the same narrative object into one group; in the horizontal axis direction, when the lines connecting different icons intersect, calculating the conflict between adjacent groups of icons placed on a horizontal axis, and grouping non-conflicting icons into one group. The step of allocating regions to the grouped icons so that the total height of the displayed icons in each group does not exceed the height of the narrative space includes: obtaining the height of the largest icon in each group, calculating the base height of each group of icons in the narrative space, and placing the icons in each group according to the base height; if an icon changes position, the icons in the target group whose position has changed are compressed so that the height of the icons in all groups meets the height of the narrative space; wherein the height of the icons whose position has not changed is set as the base height.

2. The visualization method for icon-based narrative programs for VR or AR according to claim 1, characterized in that, The step of formally describing the icon based on narrative content data, thereby assigning attribute data to the icon data, includes: The system obtains the name, size, coordinates, and emotion of the narrative object, as well as the change data of the narrative object; wherein the change data includes the change time of the narrative object and the change sequence of the narrative object, and the narrative object includes virtual objects and virtual characters; Assign the name, size, coordinates, mood, and change data to the icon.

3. The visualization method for icon-based narrative programs for VR or AR according to claim 2, characterized in that, The icons include virtual object icons, virtual character icons, and scene change icons; Assigning the name, size, coordinates, mood, and change data to the icon includes: When assigning the change data to the scene change icon, the scene change icon is displayed using two entities: an activity time block and a timeline. The color saturation of the activity time block is adjusted according to the attractiveness value of the virtual character icon or the virtual object icon. The two-dimensional virtual object corresponds to the first color, the three-dimensional virtual object corresponds to the second color, and the virtual character corresponds to both the first color and the second color.

4. A visualization device for VR or AR-based icon-based narrative programs, characterized in that, include: The data acquisition module is used to extract images, emotions, and actions of narrative objects from narrative content data for semantic modeling, and map the semantic modeling to preset icons and color labels; wherein the color labels represent emotions through color saturation. A visualization description module is used to formally describe the icons based on narrative content data, assigning attribute data to the icon data, and visually describing the temporal relationship of the narrative objects corresponding to the icons with assigned attribute data within the narrative content data. Specifically, visually describing the temporal relationship of the narrative objects corresponding to the icons with assigned attribute data within the narrative content data includes: setting the icon based on the narrative object and placing the icon in the scene; performing visualization description based on the attribute data; identifying vertical and horizontal axis conflicts of the icon in the narrative space; optimizing the vertical and horizontal axis conflicts; and... The icons are grouped; the grouped icons are assigned regions such that the total height of the displayed groups of icons is not greater than the height of the narrative space; the process of identifying vertical and horizontal axis conflicts of the icons in the narrative space, optimizing the vertical and horizontal axis conflicts, and grouping the icons includes: in the vertical axis direction, when different icons at the same distance from the camera overlap, arranging the different icons close together and grouping icons representing the same narrative object into one group; in the horizontal axis direction, when the lines connecting different icons intersect, calculating the conflict between adjacent groups of icons placed on a horizontal axis, and grouping non-conflicting icons into one group; The step of allocating regions to the grouped icons so that the total height of the displayed icons in each group does not exceed the height of the narrative space includes: obtaining the height of the largest icon in each group, calculating the base height of each group of icons in the narrative space, and placing the icons in each group according to the base height; if an icon changes position, compressing the icons in the target group whose position has changed so that the height of the icons in all groups meets the height of the narrative space; wherein the height of the icons whose position has not changed is set as the base height. The display module is used to optimize the conflict between the vertical and horizontal axes of the icons in the narrative space when making a visual description, and to allocate the icons to regions based on time sequence, so as to complete the visual description and display it according to the visual description.

5. An electronic device, characterized in that, include: At least one processor; And at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor invokes the program instructions to execute a visualization method for icon-based narrative programs for VR or AR as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a computer, performs a visualization method for VR or AR-oriented icon-based narrative programs as described in any one of claims 1 to 3.

Images