VR (Virtual Reality) or AR (Augmented Reality)-oriented visualization method and device for expressing narrative programs based on icons

Abstract

The invention discloses a VR or AR-oriented visualization method and device for expressing a narrative program based on an icon, and the method comprises the steps: extracting an image, emotion and motion of a narrative object according to narrative content data, carrying out the semantic modeling, and mapping the semantic modeling into a preset icon and a color label; wherein the color label represents the emotion through the saturation of the color; the icons are subjected to formalized description according to the narrative content data, so that the icon data are endowed with attribute data, and the sequential relation of the narrative objects corresponding to the icons endowed with the attribute data in the narrative content data is subjected to visual description; and optimizing the placement conflict of the icons in the longitudinal axis and the transverse axis of the narrative space during visual description, carrying out region distribution on the icons based on a time sequence, completing visual description, and carrying out display according to the visual description. According to the invention, the narrative content and rhythm can be intuitively reflected, so that the user impression is improved.

Description

Technical Field

[0001] The present invention relates to the field of digital media, and in particular to a visualization method and device for expressing narrative programs based on icons for VR or AR. Background Art

[0002] With the increasing maturity of metaverse-related software and hardware, and the establishment of majors such as virtual reality, VR / AR narrative programs have gradually become a mature industrial output. Creators use three-dimensional models, scenes, virtual characters, etc. to create virtual events, which are displayed in an immersive way in various VR / AR glasses. In the field of cultural relics and museums, virtual exhibition halls and interactive exhibits are used to better understand cultural heritage and historical events; in the field of education, students learn all kinds of knowledge intuitively through virtual environments and objects, which assists understanding and strengthens memory; in the entertainment industry, VR / AR games and performances bring the audience into a more interesting world in a more immersive way. VR / AR narrative programs are similar to traditional films that tell a theme, but they are short and concise, with simple object relationships, and generally all explanations are completed in one scene.

[0003] The method of narrative temporal visualization was first used for complex events and their relationship with the timeline to help people understand the development process of things. Later, temporal visualization technology was developed to express complex event data such as basketball games, workflows, and medical records to enhance user understanding. It is not difficult to see that these scenes using temporal visualization generally have a long timeline, and object relationships or attribute relationships are hidden between many time points. Visualization is needed to assist analysis and discovery, while VR / AR narrative programs have a short timeline and do not need to explore these potential relationships. However, it faces another problem: it needs to clearly interpret a story in a short time and attract people's attention; and the audience needs to wear glasses to watch the entire program before they can feel the content and rhythm of the program, which is not friendly to the creator's collection experience. Therefore, it is necessary to explore intuitive event visualization methods to construct a visual interpretation of content and rhythm for VR / AR narrative programs. Summary of the invention

[0004] In response to the above technical problems, the present invention provides a visualization method and device for expressing narrative programs based on icons for VR or AR.

[0005] A first aspect of the present invention provides a visualization method for VR or AR-oriented narrative programs based on icon expression, comprising:

[0006] Extracting images, emotions, and actions of narrative objects according to narrative content data for semantic modeling, and mapping the semantic modeling into preset icons and color labels; wherein the color labels represent emotions through color saturation;

[0007] Formalizing the icon according to the narrative content data so that the icon data is assigned attribute data, and visually describing the temporal relationship of the narrative objects corresponding to the icon assigned the attribute data in the narrative content data;

[0008] When performing visual description, the conflict between the vertical and horizontal axes of the icons in the narrative space is optimized, and the icons are allocated to areas based on the time sequence to complete the visual description and display it according to the visual description.

[0009] In an optional implementation manner, the step of formally describing the icon according to the narrative content data so that the icon data is assigned attribute data includes:

[0010] Acquire the name, size, coordinates, emotion of the narrative object, and acquire change data of the narrative object; wherein the change data includes the change time of the narrative object and the change sequence of the narrative object, and the narrative object includes a virtual object and a virtual character;

[0011] The name, size, coordinates, emotion and the change data are assigned to the icon.

[0012] In an optional implementation manner, the icon includes a virtual object icon, a virtual character icon, and a scene change icon; and assigning the name, size, coordinates, emotion, and the change data to the icon includes:

[0013] When the change data is assigned to the scene change icon, the scene change icon is displayed using two entities, an active time block and a timeline; and the color saturation of the active time block is adjusted according to the attractiveness value of the virtual character icon or the virtual object icon; wherein the two-dimensional virtual object corresponds to the first color, the three-dimensional virtual object corresponds to the second color, and the virtual character corresponds to the first color and the second color.

[0014] In an optional implementation manner, the visual description of the temporal relationship of the narrative objects corresponding to the icons assigned with attribute data in the narrative content data includes:

[0015] Setting the icon according to the narrative object and placing the icon in the scene, and performing a visual description based on the attribute data;

[0016] Identifying the longitudinal conflict and the transverse conflict of the icons in the narrative space, optimizing the longitudinal conflict and the transverse conflict, and grouping the icons;

[0017] Areas are allocated for the grouped icons so that the total height of the icons in each displayed group is no greater than the height of the narrative space.

[0018] In an optional implementation manner, the identifying the longitudinal axis conflict and the transverse axis conflict of the icons in the narrative space, optimizing the longitudinal axis conflict and the transverse axis conflict, and grouping the icons includes:

[0019] In the vertical axis direction, when different icons at the same distance from the camera overlap, the different icons are arranged closely together and the icons representing the same narrative object are grouped together;

[0020] In the horizontal direction, when the lines connecting different icons intersect, the conflict between two adjacent groups of icons placed on a horizontal axis is calculated, and icons without conflict are grouped together.

[0021] In an optional implementation manner, the performing of area allocation on the grouped icons so that the total height of the icons in each displayed group is not greater than the height of the narrative space includes:

[0022] The height of the largest icon in each group of icons is obtained, the basic height of each group of icons in the narrative space is calculated, and the icons of each group are placed according to the basic height.

[0023] In an optional implementation, the performing of area allocation on the grouped icons so that the total height of the icons in each displayed group is not greater than the height of the narrative space further includes:

[0024] If any of the icons changes position, the icons of the target group whose position has changed are compressed so that the heights of the icons of all groups meet the height of the narrative space; wherein the height of the icons that have not changed position is set to the basic height.

[0025] A second aspect of the present invention provides a visualization device for expressing narrative programs based on icons for VR or AR, comprising:

[0026] A data acquisition module is used to extract images, emotions, and actions of narrative objects according to narrative content data for semantic modeling, and map the semantic modeling into preset icons and color labels; wherein the color labels represent emotions through color saturation;

[0027] A visual description module, used to formally describe the icon according to the narrative content data, so that the icon data is assigned attribute data, and to visually describe the temporal relationship of the narrative object corresponding to the icon assigned with the attribute data in the narrative content data;

[0028] The display module is used to optimize the conflict between the vertical and horizontal axes of the icons in the narrative space when performing visual description, and to allocate areas to the icons based on time sequence, complete the visual description and display according to the visual description.

[0029] According to a third aspect of the present invention, there is provided an electronic device, comprising:

[0030] At least one processor; and at least one memory in communication with the processor, wherein: the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the visualization method for VR or AR-oriented narrative programs based on icon expression as described in the first aspect of the embodiment of the present invention.

[0031] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer, the visualization method for VR or AR-oriented narrative programs based on icon expression is performed as described in the first aspect of the embodiment of the present invention.

[0032] The present invention formally describes the icon according to the narrative content data so that the icon data is assigned attribute data, and then visualizes the temporal relationship of the narrative objects corresponding to the icons assigned with the attribute data in the narrative content data, which can intuitively reflect the narrative content and rhythm, thereby improving the user's perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 The present invention is a flowchart of a visualization method for expressing narrative programs based on icons for VR or AR in an embodiment of the present invention.

[0035] Figure 2 The figure is a schematic diagram of a virtual object icon and a virtual character icon according to an embodiment of the present invention.

[0036] Figure 3 The present invention is a flowchart of another visualization method for expressing narrative programs based on icons for VR or AR in an embodiment of the present invention.

[0037] Figure 4 It is a schematic diagram of an icon-based visualization layout in an embodiment of the present invention.

[0038] Figure 5 Schematic diagram of a visualization device for expressing narrative programs based on icons for VR or AR in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0040] It should be understood that the terms "first", "second", "third", etc. in the claims, specifications and drawings of the present invention are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0041] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the claims, the singular forms of "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in the specification of the present invention and the claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0042] See also Figure 1 The present invention provides a visualization method for expressing narrative programs based on icons for VR or AR, comprising the following steps.

[0043] Step 100: extracting images, emotions, and actions of narrative objects according to narrative content data to perform semantic modeling, and mapping the semantic modeling into preset icons and color labels; wherein the color labels represent emotions through color saturation.

[0044] Step 200: Formalizing the icon according to the narrative content data so that the icon data is assigned attribute data, and visually describing the temporal relationship of the narrative object corresponding to the icon assigned with the attribute data in the narrative content data.

[0045] Step 300: When performing visual description, the conflicts between the vertical and horizontal placements of the icons in the narrative space are optimized, and the icons are assigned regions based on the time sequence, the visual description is completed, and the display is performed according to the visual description.

[0046] Specifically, the semantic modeling is to extract information from the expression object of the narrative content data, which is usually expressed as objects, characters, scenes, as well as the emotions, actions of characters, states of objects, etc.

[0047] The formal description of the narrative content data is divided into two types: one is parameter description and the other is visual description. The parameter description is divided into four types, namely virtual object O, virtual character C, object change E and scene S.

[0048] The name, size, coordinates, emotion, and change data of the narrative object are obtained by using the Json script and the Unity engine. The change data includes object changes E and scenes S.

[0049] In one embodiment, the virtual object is composed of an object name, coordinates and size, wherein the name is obtained from a Json script; the coordinates are the position of the object in the scene, and the size is the size of the virtual object's bounding box, which is obtained from the program through the Unity engine.

[0050] The virtual character is a special virtual object, consisting of a character name, coordinates, size and emotion, wherein the name and emotion are obtained from a Json script; the coordinates are the position of the character in the scene, and the size is the size of the character's bounding box, which is obtained from the program through the Unity engine.

[0051] The object change consists of a start time, an end time, and an animation form. The start and end times are obtained from the program through the Unity engine, and the animation form is obtained from the Json script.

[0052] The background of the scene is the location where the entire program takes place, which is represented by a panoramic picture in Unity. The scene S is composed of a sequence of changes of different objects. After determining the visual expression of the objects and object changes in the scene, the scene organizes the narrative objects in a timeline and presents them in sequence, that is, visualization is performed.

[0053] Furthermore, the two-dimensional virtual object corresponds to a first color, the three-dimensional virtual object corresponds to a second color, and the virtual character corresponds to the first color and the second color. For example, the two-dimensional object uses a receding color; the three-dimensional object uses an offensive color; and the narrative object uses a superimposed color of the two. The receding color is relative to the offensive color. The offensive color is usually expressed by the size of the saturation, such as red, etc. The receding color has a low saturation, such as blue, yellow, etc.

[0054] When the semantic modeling is mapped to a preset icon, the name, size, coordinates, emotion and the change data are assigned to the icon. For example, the semantic information of the semantic modeling is divided into virtual object icons, virtual character icons and scene change icons; the narrative object includes virtual objects and virtual characters, and the change data includes the change time of the narrative object and the change sequence of the narrative object.

[0055] The icons of the virtual objects are divided into two forms, one is a given standard icon style of pictures, videos, prompts, etc.; the other is a three-dimensional object, which is searched and obtained in the icon library using its name. The size of the virtual object icon is calculated by the formula Iconwidth = (w*h / W*H) * IconMax, where w and h are the length and width of a certain icon projected onto the screen; W and H are the length and width of the display, respectively. The maximum value of the icon is the optimal value to ensure that the icon wiring does not overlap when the visualization is generated, thereby achieving the visualization effect of the icon being larger near and smaller far away; the color of the virtual object icon is usually black.

[0056] The icon ratio of the virtual character is also calculated according to the icon size calculation method of the virtual object. At the same time, in order to reflect the special emotional changes of the character, colored lines are used to express the emotions of the virtual character, which are different from the black lines of the object icon.

[0057] When the change data is assigned to the scene change icon, the scene change icon is displayed using two entities, an active time block and a timeline; that is, the scene change icon is realized through an active time block and a timeline. The form of the active time block reflects both the duration of the change and the attractiveness of the change of the object represented by the icon within a period of time to the audience; the timeline is the passage of time, and reflects the development process of the narrative program while the time passes.

[0058] Then, the color saturation of the active time block is adjusted according to the attractiveness value of the virtual character icon or the virtual object icon.

[0059] For example, the colors of the activity time blocks are different for three different objects (two-dimensional objects, three-dimensional objects, and virtual characters). Generally speaking, two-dimensional objects use retreating colors; three-dimensional objects use offensive colors; narrative objects use the superposition of the two colors; according to the attractiveness value of various animation effects of the objects, the activity blocks of different objects are filled with different saturations of the color.

[0060] For example: In the program introducing the Old Summer Palace, the scene takes place in Haiyan Hall, which is represented by a two-dimensional icon and is represented by a receding color throughout the program, such as blue or yellow. During the program, the flame (i.e., the virtual object) appears in a three-dimensional form and is represented by an offensive color, such as red, throughout the program. The program's narrator is a child "Xiaohai" (i.e., a virtual character). At the beginning of the program, "Xiaohai" gives a narrative explanation in a receding color. During the explanation, as the flame burns, "Xiaohai" becomes angry. At this time, the offensive color is superimposed on the original receding color, and the saturation of the superimposed color is adjusted according to the attraction value.

[0061] In the present invention, Figure 2 As shown, the icon of the time block is added to the icon, and the icon of the time block is as shown in Figure 2 Therefore, when visually laying out the icons, it is necessary to combine the icons, the icons of the time blocks, the horizontal axis timeline and the vertical axis for layout.

[0062] Icons with time blocks attached are indicated by I i = <X 0 , Y 0 , Max(Iconwidth, TimeX)Iconwidth+TimeY>, where X 0 and Y 0 is the coordinate of the lower left corner of the icon. Its width is determined by the active time block and the icon width, and its height is the sum of the icon and active time block heights. 0 TimeX and TimeY are obtained from the time node in the program through the Unity engine, and Timewidth is set to a fixed value. Iconwidth (i.e., attribute value) is expressed by the maximum icon size IconMax. Therefore, the core of the visual layout is to optimize the layout on the vertical axis (i.e., height arrangement) to obtain the best IconMax and Y value of each icon. 0 (ie arranged horizontally).

[0063] For example: in the program introducing the Old Summer Palace, there will be a burning animation effect after the flame appears. The burning animation effect is converted into the length of the active time block according to the duration and attached to the bottom of the flame icon. At this time, the flame icon and the active time block are integrated (as the icon described in the present invention). The maximum length of the icon and the active time block is calculated and the maximum length of the overall icon is set, and the height of the flame icon and the active time block is added together to obtain the height of the new icon.

[0064] like Figure 3 As shown, in step 300, the temporal relationship of the narrative objects corresponding to the icons assigned with attribute data in the narrative content data is visually described, including the following steps.

[0065] Step 310: setting the icon according to the narrative object and placing the icon in the scene, and performing a visual description based on the attribute data.

[0066] Specific as Figure 4 As shown in 1, the icons are initialized, and the icons with attribute data are placed in the scene and displayed in the narrative space, which is the display area of ​​the screen.

[0067] Placing in the scene means sorting all icons Ii in the N initial icon sets according to their distance from the camera; the icons are connected by lines, which are divided into two types: dotted lines and solid lines. The dotted line represents the temporal relationship of the same object, and the same group of the same object Om is connected by the dotted line; the dotted line is generated by connecting two or more of the same objects in the Unity interface. The solid line reflects the interactive process of the VR / AR program. After connecting an object change event with a solid line, the interaction triggers another object change event, and the interaction method is also mapped as an icon marked on the solid line; the solid line is generated by connecting two triggering objects in the Unity interface, and the interaction method is filled in at the same time.

[0068] Step 320: Identify the vertical axis conflict and the horizontal axis conflict of the icon in the narrative space, optimize the vertical axis conflict and the horizontal axis conflict, and group the icons.

[0069] The Unity engine can be used to identify whether the icons are superimposed in the narrative space. For example, two icons are superimposed in the middle of the left side, and three icons are superimposed in the lower right side.

[0070] When optimizing conflicts, in the longitudinal direction, when different icons at the same distance from the camera overlap, the different icons are arranged closely together and the icons representing the same narrative object are grouped together. In the present invention, the longitudinal conflict is a conflict in which different icons at the same distance from the camera overlap, so the two objects can be arranged closely together, and the icons representing the same narrative object can be grouped together. Figure 4 In the figure, the initialization icon 1 is changed to 2, which is an illustration of the conflict handling on the vertical axis. Multiple icons are superimposed vertically, and the multiple icons do not belong to the same narrative object (different narrative objects correspond to different icons). Then the overlap is processed to be arranged up and down in sequence and grouped together.

[0071] In the horizontal direction, when the lines connecting different icons intersect, the conflict between two adjacent groups of icons placed on a horizontal axis is calculated, and icons without conflict are grouped together. Figure 4 The schematic diagram of the conflict handling of the vertical axis 2 is shown in Figure 2. The left character 1 and the right character 1 are connected by a dotted line, but their positions are staggered. They are characters belonging to the same narrative object, so they can be grouped together and finally optimized as Figure 4 Schematic diagram of the horizontal axis conflict handling in the middle 3, where the characters 1 on both sides are divided into a group horizontally.

[0072] In the present invention, the horizontal axis conflict is the conflict caused by the intersection of the lines of different objects. Therefore, the conflict between two adjacent groups placed on a horizontal axis can be calculated, and the non-conflicting objects are grouped together, and the process is repeated until they cannot be merged. At this time, the grouping situation is {group 1 , group 2 …,group k}There are k groups in total, each group has numgroup n elements.

[0073] Step 330: Allocate areas for the grouped icons so that the total height of the icons in each displayed group is no greater than the height of the narrative space.

[0074] The height of the largest icon in each group of icons is obtained, the basic height of each group of icons in the narrative space is calculated, and the icons of each group are placed according to the basic height.

[0075] For example, search for each group j Icon I with the largest medium height j-max , calculate each group proportionally j The Y-axis area in the assigned visualization view is used to obtain the base Y area for each group of icon placements. 0 , the optimal IconMax is the maximum area height. Figure 4 In the narrative space, four groups of icons are placed in the height direction. By calculating the screen display size and placing four groups of icons, the maximum height of each group of icons can be obtained. Therefore, the Y value of each group of icons can be determined. 0 .

[0076] When the height of the icon placement area is greater than the threshold IconMax, the layout fails. 0 To confirm, press X in each group of areas. 0 Place Icon I i , and then adjust the area up and down for the same object according to the distance. The height of the icon is determined by IconMax = (numgroup j / ∑numgroup)*H 0 calculate.

[0077] If there is an icon that has changed position, the icons of the target group that has changed position will be compressed so that the height of the icons of all groups meets the height of the narrative space; the height of the icons that have not changed position is set to the basic height. The final visualization layout has four groups of icons, see Figure 4 Schematic diagram of the allocation of the four areas.

[0078] For example, in the Yuanmingyuan program, three objects, Haiyan Hall, flames, and the child "Xiaohai", will appear at the same time. The three objects are not the same object, and the positions and sizes of the three objects will change over time. Therefore, each object in the entire program process is divided into a group, a total of three groups; the group that has not changed position obtains the largest icon in the group, and the largest icon is assigned as the group height IconMax; Xiaohai will be shorter from the audience, and the position on the vertical axis will change. The height of the group is calculated based on the overlapping height of all icons in the entire group. When the sum of the heights of the three groups exceeds the interface resolution, the group with position changes in the three groups is compressed.

[0079] The VR or AR narrative program visualization method based on icon expression of the present invention can intuitively reflect the narrative content and rhythm by formally describing the narrative content data and then implementing the icon layout through the algorithm of visual display, thereby improving the user's perception. The algorithm of visual display needs to perform time-series visual layout of icon objects and activity time blocks, resolve conflicts on the horizontal and vertical axes, and construct a visual diagram of the content and rhythm for VR or AR narrative programs, thereby improving the user experience.

[0080] See also Figure 5 The present invention also provides a visualization device for expressing narrative programs based on icons for VR or AR, including: a data acquisition module 51, a visualization description module 52, and a display module 53.

[0081] The data acquisition module 51 is used to extract the image, emotion, and action of the narrative object according to the narrative content data to perform semantic modeling, and map the semantic modeling into a preset icon and color label; wherein the color label represents the emotion by the saturation of the color;

[0082] A visual description module 52, used to formally describe the icon according to the narrative content data, so that the icon data is assigned attribute data, and to visually describe the temporal relationship of the narrative object corresponding to the icon assigned with the attribute data in the narrative content data;

[0083] The display module 53 is used to optimize the conflict between the vertical and horizontal axes of the icons in the narrative space when performing visual description, and to allocate areas to the icons based on time sequence, complete the visual description and perform display according to the visual description.

[0084] For the description of the visualization device for expressing narrative programs based on icons for VR or AR, reference may be made to the visualization method for expressing narrative programs based on icons for VR or AR described above.

[0085] The present invention also provides an electronic device, comprising:

[0086] At least one processor; and at least one memory in communication with the processor, wherein: the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the above-mentioned visualization method for VR or AR-oriented narrative programs based on icon expression.

[0087] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the visualization method of the narrative program based on icon expression for VR or AR is implemented.

[0088] It can be understood that computer-readable storage media may include: any entity or device capable of carrying a computer program, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc. A computer program includes computer program code. The computer program code may be in source code form, object code form, executable file, or some intermediate form, etc. A computer-readable storage medium may include: any entity or device capable of carrying a computer program code, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.

[0089] In certain embodiments of the present invention, the electronic device may include a controller or a processor, and the controller is a single-chip microcomputer chip that integrates a processor, a memory, a communication module, etc. The processor may refer to the processor contained in the controller. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.

[0090] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0091] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0092] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A visualization method for expressing narrative programs based on icons for VR or AR, characterized in that: include: Extracting images, emotions, and actions of narrative objects according to narrative content data for semantic modeling, and mapping the semantic modeling into preset icons and color labels; The color label represents the emotion through the saturation of the color; Formalizing the icon according to the narrative content data so that the icon data is assigned attribute data, and visually describing the temporal relationship of the narrative objects corresponding to the icon assigned the attribute data in the narrative content data; When performing visual description, the conflict between the vertical and horizontal axes of the icons in the narrative space is optimized, and the icons are allocated to areas based on the time sequence to complete the visual description and display it according to the visual description.

2. The visualization method for VR or AR-oriented narrative programs based on icon expression according to claim 1, characterized in that: The step of formally describing the icon according to the narrative content data so that the icon data is assigned attribute data includes: Acquire the name, size, coordinates, emotion of the narrative object, and acquire change data of the narrative object; wherein the change data includes the change time of the narrative object and the change sequence of the narrative object, and the narrative object includes a virtual object and a virtual character; The name, size, coordinates, emotion and the change data are assigned to the icon.

3. The visualization method for VR or AR-oriented narrative programs based on icon expression according to claim 2, characterized in that: The icons include virtual object icons, virtual character icons, and scene change icons; The assigning the name, size, coordinates, emotion and the change data to the icon includes: When the change data is assigned to the scene change icon, the scene change icon is displayed using two entities, an active time block and a timeline; and the color saturation of the active time block is adjusted according to the attractiveness value of the virtual character icon or the virtual object icon; wherein the two-dimensional virtual object corresponds to the first color, the three-dimensional virtual object corresponds to the second color, and the virtual character corresponds to the first color and the second color.

4. The visualization method for VR or AR-oriented narrative programs based on icon expression according to claim 1, characterized in that: The step of visually describing the temporal relationship of the narrative objects corresponding to the icons assigned with attribute data in the narrative content data includes: Setting the icon according to the narrative object and placing the icon in the scene, and performing a visual description based on the attribute data; Identifying the longitudinal conflict and the transverse conflict of the icons in the narrative space, optimizing the longitudinal conflict and the transverse conflict, and grouping the icons; Areas are allocated for the grouped icons so that the total height of the icons in each displayed group is no greater than the height of the narrative space.

5. The visualization method for VR or AR-oriented narrative programs based on icon expression according to claim 4, characterized in that: The identifying the longitudinal axis conflict and the transverse axis conflict of the icons in the narrative space, optimizing the longitudinal axis conflict and the transverse axis conflict, and grouping the icons includes: In the vertical axis direction, when different icons at the same distance from the camera overlap, the different icons are arranged closely together and the icons representing the same narrative object are grouped together; In the horizontal direction, when the lines connecting different icons intersect, the conflict between two adjacent groups of icons placed on a horizontal axis is calculated, and icons without conflict are grouped together.

6. The visualization method for VR or AR-oriented narrative programs based on icon expression according to claim 4 or 5, characterized in that: The step of allocating regions for the grouped icons so that the total height of the icons in each displayed group is not greater than the height of the narrative space includes: The height of the largest icon in each group of icons is obtained, the basic height of each group of icons in the narrative space is calculated, and the icons of each group are placed according to the basic height.

7. The visualization method for VR or AR-oriented narrative programs based on icon expression according to claim 6, characterized in that: The step of allocating regions for the grouped icons so that the total height of the icons in each displayed group is no greater than the height of the narrative space further includes: If any of the icons changes position, the icons of the target group whose position has changed are compressed so that the heights of the icons of all groups meet the height of the narrative space; wherein the height of the icons that have not changed position is set to the basic height.

8. A visualization device for expressing narrative programs based on icons for VR or AR, characterized in that: include: A data acquisition module is used to extract images, emotions, and actions of narrative objects according to narrative content data for semantic modeling, and map the semantic modeling into preset icons and color labels; wherein the color labels represent emotions through color saturation; A visual description module, used to formally describe the icon according to the narrative content data, so that the icon data is assigned attribute data, and to visually describe the temporal relationship of the narrative object corresponding to the icon assigned with the attribute data in the narrative content data; The display module is used to optimize the conflict between the vertical and horizontal axes of the icons in the narrative space when performing visual description, and to allocate areas to the icons based on time sequence, complete the visual description and display according to the visual description.

9. An electronic device, characterized in that: include: at least one processor; And at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the visualization method for VR or AR-oriented narrative programs based on icons as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the visualization method for VR or AR-oriented narrative programs based on icon expression is executed as described in any one of claims 1 to 7.

Images