Virtual reality interactive methods, devices and storage media for historical and cultural heritage

Abstract

This disclosure relates to the field of computer technology, including virtual reality interaction methods, devices, and storage media for historical and cultural heritage. It involves determining associated 3D scene data based on the user's perception state, the first perceptual benefit and first resource cost of each candidate rendering state among multiple candidate rendering states for each rendering unit, and determining a target rendering strategy from at least two rendering strategies based on the first perceptual benefit and first resource cost. The 3D scene data is then rendered in 3D based on the target rendering strategy. This allows the user's perception state to serve as a unified control variable throughout the entire process, enabling coordinated regulation of the rendering accuracy, data transmission and scheduling strategies, and semantic generation and presentation of cultural content in the virtual reality scene of historical and cultural heritage. This achieves dynamic regulation of the virtual reality scene based on the user's perception state, avoiding information redundancy or insufficient presentation, thus saving equipment resources while meeting the user's interactive needs.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a virtual reality interaction method, device, and storage medium for historical and cultural heritage. Background Technology

[0002] With the increasing demand for cultural heritage protection and innovative experiences worldwide, virtual reality (VR) interactive experiences of historical and cultural heritage can break through the limitations of physical time and space, providing users with an immersive environment for cultural cognition and exploration. This effectively promotes the digital preservation, revitalization, and public education of cultural heritage, and has become a major means to improve the effectiveness of cultural dissemination.

[0003] Currently, traditional virtual reality interaction methods for historical and cultural heritage mostly adopt a static content library and rule-driven approach. That is, they rely on pre-built 3D model databases and preset interaction logic, processing the entire virtual scene through a unified rendering pipeline and interaction protocols. Specifically, during rendering, traditional Level of Detail (LOD) strategies for games or general virtual scenes are used, managing the geometric complexity and texture details of scene objects in a fixed or dynamic but homogeneous manner to ensure rendering efficiency. During interaction, the system often responds to user input through scripted rules, such as clicks, gazes, or controller actions, triggering predefined feedback content, such as text descriptions, audio explanations, or animated demonstrations.

[0004] However, the existing virtual reality interaction methods mentioned above have several shortcomings in virtual reality interaction scenarios for historical and cultural heritage: First, static content and rules are universal, making it difficult to capture and adapt to the multidimensional characteristics of individual users in real time, such as knowledge background, emotional tendencies, and interaction intentions. This results in mechanical repetition of experience content and a single feedback path, failing to provide personalized and differentiated cultural experiences. Second, in terms of rendering processing, historical and cultural heritage models often exhibit significant non-uniformity in spatial structure, model accuracy, and related information content. For example, the artifacts themselves, key components, and background environment differ greatly in geometric complexity and cultural importance. Existing uniform or static detail-level strategies cannot effectively distinguish the characteristics of "high-value key cultural information and low-value non-key areas." This leads to a large consumption of system equipment resources in areas that contribute little to cultural understanding, while the detailed artifacts and in-depth cultural information that users focus on fail to receive rendering quality and presentation effects commensurate with their importance. Furthermore, this uneven resource allocation and insufficient adaptation can further lead to slow VR application response and interaction delays, thereby reducing the smoothness and realism of the user experience. Summary of the Invention

[0005] In view of this, this disclosure proposes a virtual reality interaction method, device and storage medium for historical and cultural heritage. It can take the user's perception state as the core basis for virtual reality interaction and coordinate the control of rendering resources, transmission resources and cultural information presentation. It can solve the problems that traditional interaction methods cannot adapt to the user's personalized needs, the large difference in the accuracy of historical and cultural heritage models and the uneven depth of information content. It can effectively improve the cultural cognition experience under the condition of limited system resources.

[0006] According to one aspect of this disclosure, a virtual reality interactive method for historical and cultural heritage is provided, the method comprising:

[0007] Based on the posture data of the target user collected by the virtual reality device, the user perception state of the target user in the current three-dimensional scene is determined; wherein, the user perception state is used to indicate the object of the target user's attention in the current three-dimensional scene;

[0008] Obtain the three-dimensional scene data associated with the user's perceived state, wherein the three-dimensional scene data includes the historical and cultural heritage model indicated by the user's perceived state;

[0009] Obtain multiple candidate rendering states for each rendering unit in the historical and cultural heritage model, which includes multiple rendering units;

[0010] Determine the first perceptual benefit and the first resource cost corresponding to each candidate rendering state for each rendering unit; wherein, the first perceptual benefit corresponding to each rendering unit is positively correlated with the degree of correlation between the rendering unit and the user's perceptual state; and the first resource cost corresponding to each candidate rendering state is positively correlated with the device resources required by the candidate rendering state.

[0011] Based on the first perceived benefit and the first resource cost, the sum of the first perceived benefit and the sum of the first resource cost corresponding to the rendering strategy consisting of a candidate rendering state corresponding to each rendering unit are determined; wherein, the rendering strategy includes at least two types, and at least one rendering unit has a different candidate rendering state among different rendering strategies.

[0012] Based on the sum of the first resource costs and the sum of the first perceived benefits, a target rendering strategy is determined from at least two rendering strategies;

[0013] The three-dimensional scene data is rendered in three dimensions based on the target rendering strategy, so as to update the current three-dimensional scene in the three-dimensional scene through a virtual reality device.

[0014] In one possible implementation, the user perception state includes: the target user's gaze direction, the target user's spatial location, and the target user's perceptual behavior; correspondingly,

[0015] Determine the sum of the first-perceived benefits corresponding to the rendering strategy consisting of a candidate rendering state corresponding to each rendering unit, including:

[0016] When the perceived behavior is the first behavior, the weight of the rendering unit indicating the gaze direction and the spatial position is determined as the first weight, and the weight of the rendering unit not indicating the gaze direction and the spatial position is determined as the second weight; based on the first weight and the second weight, a weighted sum of the first perceived benefits corresponding to each rendering unit is determined, or, based on the first weight and the second weight, a weighted sum of the ratios of the first perceived benefits and the first resource costs corresponding to each rendering unit is determined to obtain the total first perceived benefits; wherein, the first behavior indicates that the object being followed by the target user remains unchanged within a preset time period, and the first weight is greater than the second weight;

[0017] When the perceived behavior is the second behavior, the weight of each rendering unit is determined as the third weight; based on the third weight, the weighted sum of the first perceived benefits corresponding to each rendering unit is determined, or, based on the third weight, the weighted sum of the ratio of the first perceived benefits to the first resource cost corresponding to each rendering unit is determined, to obtain the total first perceived benefits; wherein, the second behavior indicates that the object of the target user's attention is constantly changing, and the third weight is less than the first weight.

[0018] In one possible implementation, the 3D scene data further includes at least one textual information related to the historical and cultural heritage model, each textual information including multiple display levels; the method further includes:

[0019] When determining the display of each type of text information at different display levels, the second perceived benefit and the second resource cost corresponding to the hierarchical display strategy consisting of a display level corresponding to each type of text information, and the different hierarchical display strategies include at least one type of text information having a different display level;

[0020] Accordingly, determining the target rendering strategy from at least two rendering strategies based on the sum of the first resource costs and the sum of the first perceived benefits includes:

[0021] Based on the sum of the first perceived benefits and the second perceived benefits, the total perceived benefits are determined;

[0022] Based on the first total resource cost and the second total resource cost, determine the total resource cost;

[0023] Determine the target rendering strategy and target hierarchical display strategy that satisfy the preset resource constraints and maximize the perceived benefits.

[0024] In one possible implementation, the user-perceived state includes: the perceived behavior of the target user; correspondingly,

[0025] The step of determining the total perceived benefit based on the first total perceived benefit and the second total perceived benefit includes:

[0026] When the perceived behavior is the third behavior, the weight of the first perceived benefit sum is determined as the fourth weight and the weight of the second perceived benefit is determined as the fifth weight; based on the fourth weight and the fifth weight, the weighted sum of the first perceived benefit sum and the second perceived benefit sum is determined to obtain the perceived benefit sum.

[0027] The third line indicates that the target user is paying attention to the text information, and the fifth weight is greater than the fourth weight.

[0028] In one possible implementation, the user perception state includes: the target user's gaze direction and the target user's spatial location; correspondingly,

[0029] The second perceptual benefit and second resource cost corresponding to the hierarchical display strategy consisting of a display level for each type of text information when determining to display each type of text information at different display levels include:

[0030] Determine the semantic correlation between the rendering unit indicating the viewing direction and the spatial position and the text information corresponding to the hierarchical display strategy;

[0031] The semantic relevance is input into a pre-built text perception benefit model to obtain the second perception benefit;

[0032] The text information corresponding to the hierarchical display strategy is input into a pre-constructed text resource cost model to obtain the second resource cost.

[0033] In one possible implementation, the information input to the text-aware benefit model also includes at least one of the following:

[0034] The intensity of attention in the user's perceived state includes the intensity of the target user's attention to the historical and cultural heritage model;

[0035] The perceived behavior of the target user;

[0036] The importance of each type of textual information.

[0037] In one possible implementation, the user perception state includes: the target user's gaze direction and the target user's spatial location; correspondingly,

[0038] The determination of the first perceived benefit and first resource cost corresponding to each candidate rendering state of each rendering unit includes:

[0039] Determine the spatial relationship between the rendering unit, the viewing direction, and the spatial position;

[0040] The spatial relationship is input into a pre-built unit perception benefit model to obtain the first perception benefit;

[0041] The rendering parameters corresponding to the candidate rendering state are input into a pre-built unit resource cost model to obtain the first resource cost.

[0042] In one possible implementation, the information input to the unit-aware benefit model further includes at least one of the following:

[0043] The intensity of attention in the user's perceived state includes the intensity of the target user's attention to the historical and cultural heritage model;

[0044] The perceived behavior of the target user;

[0045] The importance of the historical and cultural heritage model.

[0046] According to another aspect of this disclosure, a virtual reality interactive device for historical and cultural heritage is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described method.

[0047] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method.

[0048] According to another aspect of this disclosure, a computer program product is provided, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above-described method.

[0049] This method determines the associated 3D scene data based on the user's perception state, the first perceptual benefit and first resource cost of each candidate rendering state among multiple candidate rendering states of each rendering unit, and determines the target rendering strategy from at least two rendering strategies based on the first perceptual benefit and first resource cost. The 3D scene data is then rendered in 3D based on the target rendering strategy. This makes the user's perception state a unified control variable throughout the entire process, collaboratively regulating the rendering accuracy, data transmission and scheduling strategies, and semantic generation and presentation methods of the virtual reality scene of historical and cultural heritage. This constructs a real-time closed-loop interactive mechanism across perception, rendering, transmission, and cultural information services. It enables dynamic adjustment of the virtual reality scene based on the user's perception state, avoiding information redundancy or insufficient presentation. This saves equipment resources and meets the user's interactive needs, significantly improving the cultural dissemination effect and user experience quality of immersive displays of historical and cultural heritage.

[0050] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0051] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0052] Figure 1 A flowchart illustrating a virtual reality interaction method for historical and cultural heritage according to an embodiment of the present disclosure;

[0053] Figure 2 A block diagram of a virtual reality interactive device for historical and cultural heritage according to an embodiment of the present disclosure is shown.

[0054] Figure 3 A block diagram of a virtual reality interactive device for historical and cultural heritage according to another embodiment of the present disclosure is shown. Detailed Implementation

[0055] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0056] As used herein, the terms “comprising,” “including,” “having,” or variations thereof are open-ended and include one or more of the stated features, integrals, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integrals, elements, steps, components, functions, or groups thereof.

[0057] When an element is referred to as “connected,” “coupled,” “responding,” or a variation thereof relative to another element, it may be directly connected, coupled, or responding to another element, or there may be an intermediate element present.

[0058] Although the terms first, second, third, etc., may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another. Therefore, without departing from the teachings of the inventive concept, a first element / operation in some embodiments may be referred to as a second element / operation in other embodiments.

[0059] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0060] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0061] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions.

[0062] Figure 1 A flowchart illustrating a virtual reality interaction method for historical and cultural heritage according to an embodiment of this disclosure is provided. This embodiment describes the method using an electronic device as an example. The electronic device can be a virtual reality device for use by a target user, or a user terminal or server communicatively connected to the virtual reality device. This embodiment does not limit the implementation of the electronic device. The target user refers to an object that needs to interact with the virtual display scene. For example... Figure 1 As shown, the method includes:

[0063] Step 101: Based on the posture data of the target user collected by the virtual reality device, determine the user perception state of the target user in the current three-dimensional scene.

[0064] The user perception state is used to indicate the object that the target user is focusing on in the current 3D scene. Indicatively, the user perception state includes at least: the target user's gaze direction and the target user's spatial location.

[0065] The target user's gaze direction can be determined based on the target user's head posture data, eye movement data, and user gaze point detection technology. Accordingly, posture data includes head posture data and eye movement data. For example, an eye-tracking module in a virtual reality device (such as a near-infrared camera) captures real-time images of the user's eyes, extracts the eye movement features from these images, and obtains eye movement data. This eye movement data includes, but is not limited to, the pupil center position, corneal reflection point, pupil size, and shape. Head posture data is collected based on the inertial measurement unit (IMU) in the virtual reality device. Based on the pupil-corneal reflection vector difference method, the two-dimensional gaze direction of the eyeball relative to the acquisition plane of the eye-tracking module is determined based on the eye movement data. The two-dimensional gaze direction and head posture data are then converted to the virtual world coordinate system of the virtual reality scene to obtain the target user's gaze direction.

[0066] The spatial location of the target user is determined based on the target user's positioning data. Accordingly, the attitude data includes the target user's positioning data, which can be collected by the positioning system in the virtual reality device, or it can be calculated based on inertial acceleration data collected by the IMU. For example, the target user's spatial location is the position coordinates obtained after converting the positioning data to the virtual world coordinate system of the virtual reality scene.

[0067] In other implementations, the user perception state may also include other elements, such as attention intensity and perception behavior.

[0068] Here, attention intensity refers to the degree to which a target user focuses on various components (including historical and cultural heritage models) within the current 3D scene. For example, attention intensity can be represented as the accumulated perception of each component by the target user over a preset time period. For instance, attention intensity can be expressed by the following formula:

[0069] ;

[0070] Among them, h t Indicates the intensity of attention; t represents the preset time length; w(τ) represents the time weighting function, used to indicate the attention contribution at each time point τ within the preset time length; U represents the component; r τ Indicates the direction of the target user's gaze; This indicates the direction r of the target user's gaze. τ Does it point to component U?

[0071] Perceptual behaviors are used to indicate changes in the direction of gaze. For example, perceptual behaviors include, but are not limited to, first behaviors, second behaviors, and third behaviors.

[0072] The first behavior indicates that the object of the target user's attention remains unchanged within a preset time period; in other words, the first behavior is a stable observation behavior. At this time, the target user's eye movement data is relatively stable within the preset time period, the change in head posture data is within a preset range, and the target user's spatial position remains essentially unchanged. Based on this, the electronic device can determine whether the perceived behavior is the first behavior based on the changes in the target user's gaze direction and spatial position. For example, if the change in gaze direction is within a first preset range and the change in spatial position is within a second preset range, then the perceived behavior is determined to be the first behavior; if the change in gaze direction exceeds the first preset range or the change in spatial position exceeds the second preset range, then the perceived behavior is determined not to be the first behavior.

[0073] The second behavior indicates that the object of the target user's attention is constantly changing, or in other words, the second behavior is a saccade. In this case, the target user's gaze direction changes rapidly, and the attention dwell time on each component is short. Based on this, the electronic device can determine whether the perceived behavior is the second behavior based on the speed of the target user's gaze direction change and / or the attention intensity of each component. For example: if the speed of the target user's gaze direction change is greater than a speed threshold, and the attention intensity of each component is less than an intensity threshold, then the perceived behavior is determined to be the second behavior; if the speed of the target user's gaze direction change is less than or equal to the speed threshold, or the attention intensity of each component is greater than or equal to the intensity threshold, then the perceived behavior is determined not to be the second behavior.

[0074] The third behavior indicates that the target user is focusing on text information, or in other words, the third behavior is information reading behavior. In this case, the target user's gaze direction points to the text information in the current 3D scene, and moves periodically within the text information's display area. Based on this, the electronic device can determine whether the perceived behavior is the third behavior based on whether the target user's gaze direction points to text information and whether the change in that gaze direction exceeds a first preset range but falls within a third preset range. For example: if the target user's gaze direction points to text information, and the change in that gaze direction exceeds the first preset range but falls within the third preset range, then it is determined whether the perceived behavior is the third behavior; if the target user's gaze direction does not point to text information, or the change in that gaze direction does not exceed the first preset range, or the change in that gaze direction exceeds the third preset range, then it is determined that the perceived behavior is not the third behavior.

[0075] The above classification of perceptual behaviors is only illustrative. In actual implementation, perceptual behaviors may include more or fewer behavior categories. The judgment method for each perceptual behavior can be determined based on the behavioral characteristics of that perceptual behavior. This embodiment does not limit the classification and judgment methods of perceptual behaviors.

[0076] Taking the user's perceived state, which includes the target user's gaze direction, spatial location, attention intensity, and perceptual behavior, as an example, the user's perceived state P at any given time... t It can be represented as:

[0077] P t =(x t ,r t , h t ,s t );

[0078] Where, x t r represents the spatial location of the target user at time t. t h represents the direction of the target user's gaze at time t. t s represents the intensity of attention of the target user at time t. t This represents the perceived behavior of the target user at time t.

[0079] Step 102: Obtain the three-dimensional scene data associated with the user's perceived state. The three-dimensional scene data includes the historical and cultural heritage model indicating the user's perceived state.

[0080] Historical and cultural heritage models refer to high-precision three-dimensional scanning data of historical and cultural heritage.

[0081] In one example, acquiring 3D scene data associated with the user's perceived state includes: determining the historical and cultural heritage model corresponding to the user's perceived spatial location based on the correspondence between spatial location and historical and cultural heritage models. Alternatively, acquiring the historical and cultural heritage model indicating the user's gaze direction in the current 3D scene.

[0082] Optionally, the historical and cultural heritage model is also associated with auxiliary data, which is used to enhance the virtual reality display effect and cultural dissemination effect of the historical and cultural heritage model. Accordingly, the 3D scene data may also include this auxiliary data. Auxiliary data includes, but is not limited to: spatial data of the space where the historical and cultural heritage corresponding to the historical and cultural heritage model is located, and / or textual information used to introduce the historical and cultural heritage, etc. This embodiment does not limit the type of auxiliary data.

[0083] Accordingly, obtaining the 3D scene data associated with the user's perceived state also includes: obtaining the auxiliary data associated with the historical and cultural heritage model corresponding to the user's perceived state based on the relationship between the historical and cultural heritage model and the auxiliary data.

[0084] Step 103: Obtain multiple candidate rendering states for each rendering unit in the historical and cultural heritage model. The historical and cultural heritage model includes multiple rendering units.

[0085] A rendering unit is the smallest unit used for rendering control, transmission scheduling, and resource allocation in a virtual reality display device for historical and cultural heritage. In this embodiment, the rendering unit is also the basic object for electronic devices to perform level-of-detail control, perception benefit assessment, and resource cost calculation.

[0086] In this embodiment, the data in the 3D scene data, excluding text information, is divided into multiple rendering units. Each rendering unit corresponds to a portion of the historical and cultural heritage model or the spatial data associated with it, and each rendering unit can be presented in multiple candidate rendering states. Therefore, for any given rendering unit U... i Its candidate rendering state can be represented as:

[0087] U i (l), l∈{l0,l1,…,l K};

[0088] Among them, l0~ l K The classification of candidate rendering states is represented by K, where K is a natural number; for example, l0 represents geometric precision, l1 represents texture resolution, and l2 represents lighting complexity. Each candidate rendering state classification can have multiple rendering parameters, l0~l K These collectively constitute l (i.e., candidate rendering states). Different candidate rendering states include those with different rendering parameters for at least one category, or those with different classifications. For example, a candidate rendering state l may include categories l0 to l1. K Another candidate rendering state l' also includes categories l0~ l K However, for these two candidate rendering states, there is at least one classification. m The rendering parameters are different. A candidate rendering state "l" can include different categories l2~l than candidate rendering state l. K .

[0089] Step 104: Determine the first perceived benefit and the first resource cost corresponding to each candidate rendering state for each rendering unit.

[0090] Among them, the first perceptual benefit corresponding to each rendering unit is positively correlated with the degree of correlation between the rendering unit and the user's perceptual state; the first resource cost corresponding to each candidate rendering state is positively correlated with the device resources required for the candidate rendering state.

[0091] In the virtual interactive presentation of historical and cultural heritage, the current 3D scene can display at least two models of historical and cultural heritage. These different models exhibit significant differences in cultural value, spatial salience, and user cognitive contribution. Furthermore, due to the diverse sources and varying levels of precision of these models, coupled with the limited resources (such as computing, storage, and network bandwidth) of virtual reality devices during rendering and data transmission, it is difficult to simultaneously present all elements of the current 3D scene at the highest level of detail. Therefore, this embodiment departs from the traditional approach of using resolution, frame rate, or geometric complexity as a single optimization objective. Instead, it introduces perceptual benefit as an evaluation metric for the display of historical and cultural heritage models. This quantifies the contribution of different candidate rendering states to the user's cultural cognitive experience. Simultaneously, resource constraints are considered to ensure that the selected rendering state does not exceed device resource limitations. This guarantees that the selected rendering state prioritizes rendering units with higher perceptual benefits, thereby improving the display effect of the virtual reality scene.

[0092] In one example, given that the user's perceived state includes the target user's gaze direction and spatial location, the first perceptual benefit and first resource cost corresponding to each candidate rendering state for each rendering unit are determined, including:

[0093] Determine the spatial relationship between the rendering unit and the viewing direction and spatial position; input the spatial relationship into a pre-built unit perception benefit model to obtain the first perception benefit; input the rendering parameters corresponding to the candidate rendering state into a pre-built unit resource cost model to obtain the first resource cost.

[0094] The first perceived benefit is used to indicate the contribution of the rendering unit's detail enhancements to the user's cultural cognitive experience.

[0095] Optionally, the spatial relationship includes: the position coordinates, viewing direction, and spatial position of the rendering unit in the virtual world coordinate system; or, whether the position coordinates of the rendering unit in the virtual world coordinate system intersect with the virtual area formed by the viewing direction and spatial position in the current 3D scene; or, the relative positional relationship between the position coordinates of the rendering unit in the virtual world coordinate system and the viewing direction, and the relative positional relationship between the position coordinates of the rendering unit in the virtual world coordinate system and the spatial position. This embodiment does not limit the implementation method of the spatial relationship.

[0096] Specifically, the closer the rendering unit is to the virtual area formed by the viewing direction and spatial position, the greater the first perceptual benefit of that rendering unit.

[0097] Optionally, the information of the input unit-perceived benefit model also includes at least one of the following:

[0098] The intensity of attention in the user's perception state; at this time, if the spatial relationship indicates that the rendering unit is closer to the virtual area formed by the viewing direction and spatial position, and the intensity of attention is greater, then the first perception benefit corresponding to the rendering unit is greater.

[0099] The target user's perceptual behavior; at this time, if the spatial relationship indicates that the rendering unit is closer to the virtual area formed by the viewing direction and spatial position, and the perceptual behavior is the first behavior, then the first perceptual benefit corresponding to the rendering unit is greater; while for the same distance, the first perceptual benefit corresponding to the first behavior is greater than the first perceptual benefit corresponding to the second and third behaviors.

[0100] The importance of the historical and cultural heritage model; whereby the importance of the historical and cultural heritage model is used to indicate its significance within the entire 3D scene data, and this importance can be quantified using a preset numerical value. In this case, if the spatial relationship indicator rendering unit is closer to the virtual area formed by the viewing direction and spatial location, and the greater its importance, the greater the first perceptual benefit corresponding to that rendering unit.

[0101] Optionally, the unit-aware revenue model can be constructed based on a rule-based model, a statistical model, or a machine learning model. This embodiment does not limit the way the unit-aware revenue model is built. For example, the unit-aware revenue model can be represented as:

[0102] G i (l∣P t );

[0103] Among them, G i (·) represents the i-th rendering unit U i The corresponding first perceived benefit, l represents the i-th rendering unit U i A candidate rendering state, P t This indicates the user's perceived state.

[0104] Optionally, the unit resource cost model can be constructed based on a rule model, a statistical model, or a machine learning model, and takes the rendering parameters corresponding to the candidate rendering state as input. The first resource cost output is used to indicate the device resources required to render in the candidate rendering state. These device resources include, but are not limited to, computing resources, storage resources, and / or network bandwidth resources.

[0105] Alternatively, the unit resource cost model can also be established based on other parameters related to rendering, such as the number of model parameters in the rendering model.

[0106] For example, the unit resource cost model can be expressed by the following formula:

[0107] Ci (l);

[0108] Among them, C i (·) represents the i-th rendering unit U i The corresponding first resource cost, l represents the i-th rendering unit U i A candidate rendering state.

[0109] Step 105: Based on the first perceived benefit and the first resource cost, determine the sum of the first perceived benefit and the sum of the first resource cost corresponding to the rendering strategy consisting of a candidate rendering state corresponding to each rendering unit; wherein, the rendering strategy includes at least two types, and there are at least one different candidate rendering state of the rendering unit among the different rendering strategies.

[0110] Each rendering strategy can be represented as {U i (l)},l∈{l0,l1,…,l K}, where i is a positive integer starting from 1, and its maximum value is the number of rendering units. Within the same rendering strategy, different rendering units may correspond to the same or different candidate rendering states; that is, U1(l), U2(l), ... within the same rendering strategy can be the same or different. Different rendering strategies have at least one U i (l) Different.

[0111] The electronic device sums the first resource costs corresponding to each candidate rendering state of each rendering unit to obtain the total first resource cost.

[0112] The methods for determining the total benefit of the first perception include, but are not limited to, one of the following:

[0113] The first method is to sum the first perception gains corresponding to each candidate rendering state of each rendering unit to obtain the total first perception gains.

[0114] Accordingly, the sum of the benefits of the first perception can be expressed as:

[0115] ;

[0116] Among them, G i (·) represents the i-th rendering unit U i The corresponding first perceived benefit is i, which is a positive integer starting from 1, and its maximum value is the number of rendering units.

[0117] The second method is to sum the ratios of the first perception benefit to the first resource cost corresponding to each candidate rendering state of each rendering unit to obtain the total first perception benefit.

[0118] Accordingly, the sum of the benefits of the first perception can be expressed as:

[0119] ;

[0120] in, This represents the first perceived benefit of the i-th rendering unit, where i is a positive integer starting from 1, and its maximum value is the number of rendering units; C i (l) represents the first resource cost of the i-th rendering unit.

[0121] The third approach: If the user's perceived state includes the target user's gaze direction, the target user's spatial location, and the target user's perceived behavior, then when the perceived behavior is the first behavior, the weight of the rendering unit indicating the gaze direction and spatial location is determined as the first weight, and the weight of the rendering unit without gaze direction and spatial location indication is determined as the second weight. Based on the first and second weights, a weighted sum of the first perceived benefits corresponding to each rendering unit is determined, or, based on the first and second weights, a weighted sum of the ratio of the first perceived benefits to the first resource costs corresponding to each rendering unit is determined, to obtain the total first perceived benefits; wherein, the first weight is greater than the second weight. When the perceived behavior is the second behavior, the weight of each rendering unit is determined as the third weight; based on the third weight, a weighted sum of the first perceived benefits corresponding to each rendering unit is determined, or, based on the third weight, a weighted sum of the ratio of the first perceived benefits to the first resource costs corresponding to each rendering unit is determined, to obtain the total first perceived benefits; wherein, the third weight is less than the first weight.

[0122] Optionally, the rendering unit indicated by the viewing direction and spatial position refers to the rendering unit that intersects with the virtual area formed by the viewing direction and spatial position. The first weights of different rendering units indicated by the viewing direction and spatial position may be the same or different. If the first weights of different rendering units indicated by the viewing direction and spatial position are different, the larger the area where the virtual area formed by the viewing direction and spatial position intersects with the rendering unit, the greater the first weight. The second weights of different rendering units not indicated by the viewing direction and spatial position may be the same or different. If the second weights of different rendering units not indicated by the viewing direction and spatial position are different, the closer the distance between the virtual area formed by the viewing direction and spatial position and the rendering unit, the greater the second weight. This embodiment does not limit the setting method of the first weight and the second weight.

[0123] Accordingly, when the perceptual behavior is the first behavior, the sum of the benefits of the first perceptual behavior can be expressed as:

[0124] + ;

[0125] Or:

[0126] + ;

[0127] in, The first perceptual gain of the m-th rendering unit represents the direction of gaze and spatial location indication. This represents the first resource cost of the m-th rendering unit, which indicates the view direction and spatial location; M is the number of rendering units that indicate the view direction and spatial location. This represents the first perceptual gain of the nth rendering unit, where the view direction and spatial location are not indicated. This represents the first resource cost of the nth rendering unit where the view direction and spatial location are not indicated; N is the number of rendering units where the view direction and spatial location are not indicated. Indicates the first weight. This indicates the second weight.

[0128] In this way, when the target user is perceiving 3D scene data in the first line, the electronic device can increase the weight of the target user's first perception benefit of focusing on the historical and cultural heritage model and its local details, and prioritize the enhancement of its geometric and textural detail levels to support the user's in-depth understanding of the detailed structure and artistic features of the cultural heritage.

[0129] Optionally, the weights of each rendering unit can be the same or different from the third weight. For example, the third weight of the rendering unit corresponding to the edge of the historical and cultural heritage model is greater than the third weight of the rendering unit corresponding to the non-edge of the historical and cultural heritage model. In this way, when the target user perceives 3D scene data in the second row, the electronic device can reduce the weight of the first perception benefit of the historical and cultural heritage model and its local details, and instead emphasize the continuous presentation of the overall spatial structure and key contour features, so as to support the user's rapid understanding of the spatial pattern of the cultural heritage.

[0130] When the perceptual behavior is the second behavior, the total benefit of the second perceptual behavior can be expressed as:

[0131] ;

[0132] Or:

[0133] ;

[0134] in, This represents the first perceived benefit of the i-th rendering unit, where i is a positive integer starting from 1, and its maximum value is the number of rendering units. Indicates the third weight; C i (l) represents the first resource cost of the i-th rendering unit.

[0135] Step 106: Determine the target rendering strategy from at least two rendering strategies based on the sum of the first resource costs and the sum of the first perceived benefits.

[0136] In one example, determining a target rendering strategy from at least two rendering strategies based on the first total resource cost and the first total perceived benefit includes: determining a target rendering strategy from at least two rendering strategies that ensures the first total resource cost does not exceed resource constraints and maximizes the first total perceived benefit.

[0137] Accordingly, the process of selecting a target rendering strategy can be represented as:

[0138] max G'; and maxC'≤S;

[0139] Where max represents finding the maximum value, G' represents the sum of the first perception benefits, C' represents the sum of the first resource costs, and S represents the preset resource constraints.

[0140] In another example, the 3D scene data also includes various textual information related to the historical and cultural heritage model; in this case, different textual information displayed at different display levels will have different second perceptual benefits and second resource costs. Accordingly, when selecting a target rendering strategy, the second perceptual benefits and second resource costs can be combined.

[0141] At this point, the method further includes determining the second perceived benefit and the second resource cost corresponding to the hierarchical display strategy consisting of a display level corresponding to each type of text information when displaying each type of text information at different display levels.

[0142] Among them, different display strategies include at least one type of text information being displayed at a different level.

[0143] For example, text information includes historical background information, academic research results information, and artistic interpretation information. Each type of text information includes multiple display levels. Historical background information includes display levels 1, 2, and 3. The higher the display level, the more detailed the corresponding text information.

[0144] The various textual information of the historical and cultural heritage model can be represented as: C i,j , where i and j are both positive integers, representing the j-th type of cultural information in the i-th historical and cultural heritage model.

[0145] Accordingly, the hierarchical display strategy can be represented as { C i,j Different display strategies include at least one C i,j The display levels differ. Within the same display strategy, different types of cultural information may correspond to the same or different display levels.

[0146] For example, the user's perceived state includes: the target user's gaze direction and the target user's spatial location; accordingly,

[0147] When displaying each type of text information at different display levels, the second perceptual benefit and the second resource cost corresponding to the hierarchical display strategy consisting of a display level for each type of text information include:

[0148] The semantic correlation between the rendering unit indicating the viewing direction and spatial position and the text information corresponding to the hierarchical display strategy is determined; the semantic correlation is input into a pre-built text perception benefit model to obtain the second perception benefit; the text information corresponding to the hierarchical display strategy is input into a pre-built text resource cost model to obtain the second resource cost.

[0149] Optionally, a semantic space model is pre-created in the electronic device. The text information of each display level and the unit information of the rendering unit indicating the viewing direction and spatial position are input into the semantic space model to obtain semantic relevance. Alternatively, semantic relevance is determined based on the similarity between the rendering unit and its corresponding template text information and the text information of each display level. The greater the similarity, the stronger the semantic relevance.

[0150] The unit information of the rendering unit can be a unit identifier or the location of the rendering unit, etc. This embodiment does not limit the implementation method of the unit information.

[0151] The semantic space model can be a knowledge graph between pre-built rendering units and text information, or it can be a mathematical model built based on a machine learning model. This embodiment does not limit the implementation method of the semantic space model.

[0152] Optionally, the text-aware benefit model and the unit-aware benefit model may be of the same or different model types, and can be built based on rule-based models, statistical models, or machine learning models. This embodiment does not limit the implementation method of the text-aware benefit model. The stronger the semantic relevance, the greater the second perceptual benefit output by the text-aware benefit model. For example, the text-aware benefit model can be represented as:

[0153] G i,j info (d∣P t );

[0154] Among them, G i,j info (·) represents the display strategy of the d-th level corresponding to the cultural information of the j-th display level of the i-th historical and cultural heritage model, P t This indicates the user's perceived state.

[0155] Optionally, the information input to the text-aware benefit model may also include at least one of the following:

[0156] The intensity of attention in the user's perceived state; at this time, the greater the intensity of attention of the target user to the historical and cultural heritage model, the greater the second perception benefit;

[0157] The target user's perceived behavior; at this point, with other information in the input model being the same, the second perceived benefit corresponding to the first behavior is greater than the second perceived benefit corresponding to the second behavior.

[0158] The importance of each piece of textual information is used to indicate its significance within the overall narrative structure of the historical and cultural heritage model. This importance can be quantified using a preset numerical value. The greater the importance of the textual information, the greater the second perceptual benefit.

[0159] During the presentation of cultural information, virtual reality devices also consume device resources. Therefore, a second resource cost needs to be determined based on a text resource cost model. Optionally, the model type of the text resource cost model may be the same as or different from the model type of the unit resource cost model. The text resource cost model can be constructed based on a rule-based model, a statistical model, or a machine learning model. This embodiment does not limit the method of establishing the text resource cost model.

[0160] For example, the text resource cost model can be represented as:

[0161] C i，j info (d);

[0162] Among them, C i，j info (·) represents the second resource cost of the display strategy of the d-th level corresponding to the cultural information of the j-th display level of the i-th historical and cultural heritage model.

[0163] The second resource cost is used to indicate the display of text information content C using the d-th level display strategy. i，j The required device resources and their impact on user perception. Specifically, the greater the amount of information in the text corresponding to the hierarchical display strategy, the greater the resource cost.

[0164] Accordingly, based on the sum of the first resource costs and the sum of the first perceived benefits, a target rendering strategy is determined from at least two rendering strategies, including:

[0165] Based on the sum of the first and second perceived benefits, determine the total perceived benefits; based on the sum of the first and second resource costs, determine the total resource costs; determine the target rendering strategy and target level display strategy that satisfy the preset resource constraints and maximize the total perceived benefits.

[0166] The electronic device sums the first resource cost and the second resource cost to obtain the total resource cost.

[0167] The methods for determining the total perceived benefits include, but are not limited to, the following:

[0168] The first method: Determine the sum of the first perceptual benefits and the sum of the second perceptual benefits to obtain the total perceptual benefits.

[0169] Accordingly, the total perceived benefit can be expressed by the following formula:

[0170] G'+ ∑G i,j info (d∣P t );

[0171] Among them, G i,j info (·) represents the display strategy of the d-th level corresponding to the cultural information of the j-th display level of the i-th historical and cultural heritage model, P t Let i and j represent the user's perception state, where i and j are both positive integers; G' represents the sum of the first perception benefits of the i-th historical and cultural heritage model.

[0172] The second method involves determining the sum of the ratios between the total first perceived benefit, the second perceived benefit, and the second resource cost to obtain the total perceived benefit.

[0173] Accordingly, the total perceived benefit can be expressed by the following formula:

[0174] G'+ ∑(G i,j info (d∣P t ) / C i，j info (d)).

[0175] Among them, C i，j info (d) represents the second resource cost of the display strategy of the d-th level corresponding to the cultural information of the j-th display level of the i-th historical and cultural heritage model.

[0176] The third approach: If the user's perceived state includes the target user's perceived behavior, then when the perceived behavior is the third behavior, the weight of the first perceived benefit sum is determined as the fourth weight, and the weight of the second perceived benefit is determined as the fifth weight; based on the fourth and fifth weights, the weighted sum of the first perceived benefit sum and the second perceived benefit sum is determined to obtain the perceived benefit sum; where the fifth weight is greater than the fourth weight.

[0177] Accordingly, when the perceived action is the first action, the total perceived benefit can be expressed as:

[0178] ηG'+ λ∑G i,j info (d∣P t );

[0179] Or:

[0180] ηG'+ λ∑(G i,j info (d∣P t ) / C i，j info (d));

[0181] Where η represents the fourth weight, λ represents the fifth weight, and λ > η. G i,j info (·) represents the display strategy of the d-th level corresponding to the cultural information of the j-th display level of the i-th historical and cultural heritage model, P t Represents the user's perception state, where i and j are both positive integers; G' represents the sum of the first perception benefits of the i-th historical and cultural heritage model; C i，j info (d) represents the second resource cost of the display strategy of the d-th level corresponding to the cultural information of the j-th display level of the i-th historical and cultural heritage model.

[0182] In this way, when the target user is perceiving 3D scene data in the third row, the electronic device can increase the weight of the text information that the target user is currently focusing on, thereby improving the target user's systematic understanding of the text information.

[0183] Accordingly, the process of selecting the target rendering strategy and the target hierarchy display strategy can be represented as follows:

[0184] max G' '; and max C' '≤ S;

[0185] Where max represents finding the maximum value, G'' represents the total perceived benefit, C'' represents the total resource cost, and S represents the preset resource constraints.

[0186] Step 107: Perform 3D rendering on the 3D scene data based on the target rendering strategy, so as to update the current 3D scene in the 3D scene through the virtual reality device.

[0187] The electronic device calls the rendering engine of the virtual reality scene and performs 3D rendering on the rendering units in the 3D scene data, excluding text information, according to each rendering parameter in the target rendering strategy, to obtain the updated current 3D scene.

[0188] When the 3D scene data also includes text information, the rendering engine is invoked to perform 3D rendering on the text information in the 3D scene data according to the display level in the target level display strategy, so as to obtain the updated current 3D scene.

[0189] In summary, the virtual reality interaction method for historical and cultural heritage provided in this embodiment determines the associated 3D scene data based on the user's perception state, the first perception benefit and the first resource cost of each candidate rendering state among multiple candidate rendering states of each rendering unit, and determines the target rendering strategy from at least two rendering strategies based on the first perception benefit and the first resource cost; and performs 3D rendering of the 3D scene data based on the target rendering strategy. This makes the user's perception state a unified control variable throughout the entire process of the method, and coordinates the rendering accuracy, data transmission and scheduling strategy, and semantic generation and presentation of cultural content in the virtual reality scene of historical and cultural heritage. This constructs a real-time closed-loop interaction mechanism across perception, rendering, transmission, and cultural information services. It enables dynamic adjustment of the virtual reality scene according to the user's perception state, avoiding information redundancy or insufficient presentation. This saves equipment resources and meets the user's interaction needs, thereby significantly improving the cultural dissemination effect and user experience quality of immersive displays of historical and cultural heritage.

[0190] Figure 2 A block diagram of a virtual reality interactive device for historical and cultural heritage according to an embodiment of the present disclosure is shown. The device includes: a perception state determination module 210, a scene data acquisition module 220, a rendering state acquisition module 230, a perception benefit determination module 240, a benefit sum determination module 250, a rendering strategy determination module 260, and a three-dimensional scene rendering module 270.

[0191] The perception state determination module 210 is used to determine the user perception state of the target user in the current three-dimensional scene based on the posture data of the target user collected by the virtual reality device; wherein, the user perception state is used to indicate the object of the target user's attention in the current three-dimensional scene;

[0192] Scene data acquisition module 220 is used to acquire three-dimensional scene data associated with the user's perception state, the three-dimensional scene data including the historical and cultural heritage model indicated by the user's perception state;

[0193] The rendering state acquisition module 230 is used to acquire multiple candidate rendering states for each rendering unit in the historical and cultural heritage model, which includes multiple rendering units.

[0194] The perceived benefit determination module 240 is used to determine the first perceived benefit and the first resource cost corresponding to each candidate rendering state of each rendering unit; wherein, the first perceived benefit corresponding to each rendering unit is positively correlated with the degree of correlation between the rendering unit and the user's perceived state; and the first resource cost corresponding to each candidate rendering state is positively correlated with the device resources required by the candidate rendering state.

[0195] The total benefit determination module 250 is used to determine the total first perceived benefit and the total first resource cost corresponding to a rendering strategy consisting of a candidate rendering state corresponding to each rendering unit, based on the first perceived benefit and the first resource cost; wherein, the rendering strategy includes at least two types, and at least one rendering unit has a different candidate rendering state among the different rendering strategies.

[0196] The rendering strategy determination module 260 is used to determine a target rendering strategy from at least two rendering strategies based on the sum of the first resource costs and the sum of the first perceived benefits.

[0197] The 3D scene rendering module 270 is used to perform 3D rendering on the 3D scene data based on the target rendering strategy, so as to update the current 3D scene in the 3D scene through a virtual reality device.

[0198] Optionally, the user perception state includes: the target user's gaze direction, the target user's spatial location, and the target user's perception behavior; correspondingly, the total revenue determination module 250 is used for:

[0199] When the perceived behavior is the first behavior, the weight of the rendering unit indicating the gaze direction and the spatial position is determined as the first weight, and the weight of the rendering unit not indicating the gaze direction and the spatial position is determined as the second weight; based on the first weight and the second weight, a weighted sum of the first perceived benefits corresponding to each rendering unit is determined, or, based on the first weight and the second weight, a weighted sum of the ratios of the first perceived benefits and the first resource costs corresponding to each rendering unit is determined to obtain the total first perceived benefits; wherein, the first behavior indicates that the object being followed by the target user remains unchanged within a preset time period, and the first weight is greater than the second weight;

[0200] When the perceived behavior is the second behavior, the weight of each rendering unit is determined as the third weight; based on the third weight, the weighted sum of the first perceived benefits corresponding to each rendering unit is determined, or, based on the third weight, the weighted sum of the ratio of the first perceived benefits to the first resource cost corresponding to each rendering unit is determined, to obtain the total first perceived benefits; wherein, the second behavior indicates that the object of the target user's attention is constantly changing, and the third weight is less than the first weight.

[0201] Optionally, the three-dimensional scene data also includes at least one textual information related to the historical and cultural heritage model, each textual information including multiple display levels;

[0202] The perceived benefit determination module 240 is also used to determine the second perceived benefit and the second resource cost corresponding to the hierarchical display strategy consisting of a display level corresponding to each type of text information when displaying each type of text information at different display levels. The different hierarchical display strategies include at least one type of text information with a different display level.

[0203] Accordingly, the rendering strategy determination module 260 is further configured to:

[0204] Based on the sum of the first perceived benefits and the second perceived benefits, the total perceived benefits are determined;

[0205] Based on the first total resource cost and the second total resource cost, determine the total resource cost;

[0206] Determine the target rendering strategy and target hierarchical display strategy that satisfy the preset resource constraints and maximize the perceived benefits.

[0207] Optionally, the user perception state includes: the target user's perception behavior; correspondingly,

[0208] The total revenue determination module 250 is also used for:

[0209] When the perceived behavior is the third behavior, the weight of the first perceived benefit sum is determined as the fourth weight and the weight of the second perceived benefit is determined as the fifth weight; based on the fourth weight and the fifth weight, the weighted sum of the first perceived benefit sum and the second perceived benefit sum is determined to obtain the perceived benefit sum.

[0210] The third line indicates that the target user is paying attention to the text information, and the fifth weight is greater than the fourth weight.

[0211] Optionally, the user's perceived state includes: the target user's gaze direction and the target user's spatial location; correspondingly,

[0212] The perceived benefit determination module 240 is further configured to:

[0213] Determine the semantic correlation between the rendering unit indicating the viewing direction and the spatial position and the text information corresponding to the hierarchical display strategy;

[0214] The semantic relevance is input into a pre-built text perception benefit model to obtain the second perception benefit;

[0215] The text information corresponding to the hierarchical display strategy is input into a pre-constructed text resource cost model to obtain the second resource cost.

[0216] Optionally, the information input to the text-aware benefit model may also include at least one of the following:

[0217] The intensity of attention in the user's perceived state includes the intensity of the target user's attention to the historical and cultural heritage model;

[0218] The perceived behavior of the target user;

[0219] The importance of each type of textual information.

[0220] Optionally, the user's perceived state includes: the target user's gaze direction and the target user's spatial location; correspondingly,

[0221] The perceived benefit determination module 240 is further configured to:

[0222] Determine the spatial relationship between the rendering unit, the viewing direction, and the spatial position;

[0223] The spatial relationship is input into a pre-built unit perception benefit model to obtain the first perception benefit;

[0224] The rendering parameters corresponding to the candidate rendering state are input into a pre-built unit resource cost model to obtain the first resource cost.

[0225] Optionally, the information input into the unit-aware benefit model may also include at least one of the following:

[0226] The intensity of attention in the user's perceived state includes the intensity of the target user's attention to the historical and cultural heritage model;

[0227] The perceived behavior of the target user;

[0228] The importance of the historical and cultural heritage model.

[0229] For details, please refer to the above method implementation examples.

[0230] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0231] This disclosure also provides a virtual reality interactive device for historical and cultural heritage, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0232] This disclosure also provides a non-volatile computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0233] This disclosure also provides a computer program product, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above method.

[0234] Figure 3 This is a block diagram illustrating a virtual reality interactive device 1900 for historical and cultural heritage according to an exemplary embodiment. For example, device 1900 may be provided as a server or terminal device. (Refer to...) Figure 3 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0235] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output interface 1958 (I / O interface). Device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM macOS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0236] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of the device 1900 to perform the above-described method.

[0237] Computer-readable storage media can be tangible devices capable of holding and storing programs / instructions used by instruction execution devices. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0238] The computer program (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage medium in the respective computing / processing device.

[0239] The computer program (or computer program instructions) used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions to implement various aspects of this disclosure.

[0240] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0241] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0242] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0243] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0244] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A virtual reality interactive method for historical and cultural heritage, characterized in that, The method includes: Based on the posture data of the target user collected by the virtual reality device, the user perception state of the target user in the current three-dimensional scene is determined; wherein, the user perception state is used to indicate the object of the target user's attention in the current three-dimensional scene; Obtain the three-dimensional scene data associated with the user's perceived state, wherein the three-dimensional scene data includes the historical and cultural heritage model indicated by the user's perceived state; Obtain multiple candidate rendering states for each rendering unit in the historical and cultural heritage model, which includes multiple rendering units; Determine the first perceptual benefit and the first resource cost corresponding to each candidate rendering state for each rendering unit; wherein, the first perceptual benefit corresponding to each rendering unit is positively correlated with the degree of correlation between the rendering unit and the user's perceptual state; and the first resource cost corresponding to each candidate rendering state is positively correlated with the device resources required by the candidate rendering state. Based on the first perceived benefit and the first resource cost, the sum of the first perceived benefit and the sum of the first resource cost corresponding to the rendering strategy consisting of a candidate rendering state corresponding to each rendering unit are determined; wherein, the rendering strategy includes at least two types, and at least one rendering unit has a different candidate rendering state among different rendering strategies. Based on the sum of the first resource costs and the sum of the first perceived benefits, a target rendering strategy is determined from at least two rendering strategies; The three-dimensional scene data is rendered in three dimensions based on the target rendering strategy, so as to update the current three-dimensional scene in the three-dimensional scene through a virtual reality device.

2. The method according to claim 1, characterized in that, The user's perceived state includes: the target user's gaze direction, the target user's spatial location, and the target user's perceptual behavior; correspondingly, Determine the sum of the first-perceived benefits corresponding to the rendering strategy consisting of a candidate rendering state corresponding to each rendering unit, including: When the perceived behavior is the first behavior, the weight of the rendering unit indicating the gaze direction and the spatial position is determined as the first weight, and the weight of the rendering unit not indicating the gaze direction and the spatial position is determined as the second weight; based on the first weight and the second weight, a weighted sum of the first perceived benefits corresponding to each rendering unit is determined, or, based on the first weight and the second weight, a weighted sum of the ratios of the first perceived benefits and the first resource costs corresponding to each rendering unit is determined to obtain the total first perceived benefits; wherein, the first behavior indicates that the object being followed by the target user remains unchanged within a preset time period, and the first weight is greater than the second weight; When the perceived behavior is the second behavior, the weight of each rendering unit is determined as the third weight; based on the third weight, the weighted sum of the first perceived benefits corresponding to each rendering unit is determined, or, based on the third weight, the weighted sum of the ratio of the first perceived benefits to the first resource cost corresponding to each rendering unit is determined, to obtain the total first perceived benefits; wherein, the second behavior indicates that the object of the target user's attention is constantly changing, and the third weight is less than the first weight.

3. The method according to claim 1, characterized in that, The 3D scene data also includes at least one type of text information related to the historical and cultural heritage model, each type of text information including multiple display levels; the method further includes: When determining the display of each type of text information at different display levels, the second perceived benefit and the second resource cost corresponding to the hierarchical display strategy consisting of a display level corresponding to each type of text information, and the different hierarchical display strategies include at least one type of text information having a different display level; Accordingly, determining the target rendering strategy from at least two rendering strategies based on the sum of the first resource costs and the sum of the first perceived benefits includes: Based on the sum of the first perceived benefits and the second perceived benefits, the total perceived benefits are determined; Based on the first total resource cost and the second total resource cost, determine the total resource cost; Determine the target rendering strategy and target hierarchical display strategy that satisfy the preset resource constraints and maximize the perceived benefits.

4. The method according to claim 3, characterized in that, The user's perceived state includes: the target user's perceived behavior; correspondingly. The step of determining the total perceived benefit based on the first total perceived benefit and the second total perceived benefit includes: When the perceived behavior is the third behavior, the weight of the first perceived benefit sum is determined as the fourth weight and the weight of the second perceived benefit is determined as the fifth weight; based on the fourth weight and the fifth weight, the weighted sum of the first perceived benefit sum and the second perceived benefit sum is determined to obtain the perceived benefit sum. The third line indicates that the target user is paying attention to the text information, and the fifth weight is greater than the fourth weight.

5. The method according to claim 3, characterized in that, The user's perceived state includes: the target user's gaze direction and the target user's spatial location; correspondingly... The second perceptual benefit and second resource cost corresponding to the hierarchical display strategy consisting of a display level for each type of text information when determining to display each type of text information at different display levels include: Determine the semantic correlation between the rendering unit indicating the viewing direction and the spatial position and the text information corresponding to the hierarchical display strategy; The semantic relevance is input into a pre-built text perception benefit model to obtain the second perception benefit; The text information corresponding to the hierarchical display strategy is input into a pre-constructed text resource cost model to obtain the second resource cost.

6. The method according to claim 5, characterized in that, The information input into the text-aware benefit model also includes at least one of the following: The intensity of attention in the user's perceived state includes the intensity of the target user's attention to the historical and cultural heritage model; The perceived behavior of the target user; The importance of each type of textual information.

7. The method according to any one of claims 1 to 6, characterized in that, The user's perceived state includes: the target user's gaze direction and the target user's spatial location; correspondingly... The determination of the first perceived benefit and first resource cost corresponding to each candidate rendering state of each rendering unit includes: Determine the spatial relationship between the rendering unit, the viewing direction, and the spatial position; The spatial relationship is input into a pre-built unit perception benefit model to obtain the first perception benefit; The rendering parameters corresponding to the candidate rendering state are input into a pre-built unit resource cost model to obtain the first resource cost.

8. The method according to claim 7, characterized in that, The information input into the unit-aware benefit model also includes at least one of the following: The intensity of attention in the user's perceived state includes the intensity of the target user's attention to the historical and cultural heritage model; The perceived behavior of the target user; The importance of the historical and cultural heritage model.

9. A virtual reality interactive device for historical and cultural heritage, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.

10. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images