Scene adjustment method and apparatus, device, and storage medium

Abstract

The present invention provides a scene adjustment method and apparatus, a device, and a storage medium. The method comprises: acquiring status information of a target object in a virtual scene; on the basis of the status information of the target object, adjusting environment information in the virtual scene.

Description

Scene adjustment method, device, equipment and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on and claims priority to the Chinese patent application with application number 202310870291.9 and application date July 14, 2023. The disclosed content of the Chinese patent application is hereby introduced as a whole into this application. Technical Field

[0003] The embodiments of the present application relate to the field of computer technology, and in particular to a scene adjustment method, apparatus, device, and storage medium. Background Art

[0004] With the development of Extended Reality (XR) technology, users can watch various virtual live broadcasts through electronic devices such as XR devices, Virtual Reality (VR) devices or Augmented Reality (AR), thereby obtaining an immersive live broadcast experience that cannot be experienced on traditional electronic devices such as mobile phones or tablets.

[0005] Summary of the Invention

[0006] The present application provides a scene adjustment method, apparatus, device and storage medium.

[0007] In a first aspect, an embodiment of the present application provides a scene adjustment method, comprising:

[0008] Obtaining status information of target objects in the virtual scene;

[0009] Adjusting environmental information within the virtual scene according to the state information of the target object.

[0010] In a second aspect, an embodiment of the present application provides a scene adjustment device, comprising:

[0011] An information acquisition module is used to obtain status information of target objects in a virtual scene;

[0012] The scene adjustment module is used to adjust the environmental information in the virtual scene according to the state information of the target object.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0014] A processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to execute the scene adjustment method described in the embodiment of the first aspect or its various implementations.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute the scene adjustment method as described in the embodiment of the first aspect or its various implementations.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product comprising program instructions, which, when executed on an electronic device, enables the electronic device to execute the scene adjustment method as described in the embodiment of the first aspect or its various implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] FIG1 is a schematic diagram of a flow chart of a scene adjustment method provided in an embodiment of the present application;

[0019] FIG2 is a schematic diagram of setting up a performance stage and various decorative objects in a virtual scene according to an embodiment of the present application;

[0020] FIG3 is a flow chart of another scene adjustment method provided in an embodiment of the present application;

[0021] FIG4 is a flow chart of another scene adjustment method provided in an embodiment of the present application;

[0022] FIG5 is a flow chart of another scene adjustment method provided in an embodiment of the present application;

[0023] FIG6 is a schematic diagram of adjusting any environmental parameter on a control panel according to an embodiment of the present application;

[0024] FIG7 is a schematic block diagram of a scene adjustment device provided in an embodiment of the present application;

[0025] FIG8 is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0028] Considering that users currently watch various virtual live broadcasts based on electronic devices such as XR devices, virtual reality (VR) devices or augmented reality (AR), such as live music performances or DJ live broadcasts, the environmental information in the performance space where the performers are located is usually expressed in a relatively simple form, which cannot meet the diverse viewing needs of users, resulting in limited visual experience for users. To this end, the present application provides a scene adjustment solution to enrich the expression of environmental information in the performance space, thereby providing users with more colorful visual effects and artistic appeal to meet the diverse viewing needs of users and enhance their visual experience. At the same time, it can also enhance the attractiveness and viewing value of music performances.

[0029] To facilitate understanding of the embodiments of the present application, before describing each embodiment of the present application, some concepts involved in all embodiments of the present application are first appropriately explained as follows:

[0030] 1) Virtual Reality (VR), a technology for creating and experiencing a virtual world, computationally generates a virtual environment, which is a multi-source information (the virtual reality mentioned in this article includes at least visual perception, and can also include auditory perception, tactile perception, motion perception, and even taste perception, olfactory perception, etc.), realizing the fusion of virtual environments, interactive three-dimensional dynamic vision and simulation of entity behavior, allowing users to immerse themselves in a simulated virtual reality environment, and realizing applications in various virtual environments such as maps, games, videos, education, medical care, simulation, collaborative training, sales, assisted manufacturing, maintenance and repair.

[0031] 2) Virtual reality devices (VR devices), terminals that achieve virtual reality effects, can usually be provided in the form of glasses, head-mounted displays (HMDs), or contact lenses to achieve visual perception and other forms of perception. Of course, the form of virtual reality devices is not limited to this and can be further miniaturized or enlarged according to actual needs.

[0032] Optionally, the virtual reality devices described in the embodiments of this application may include but are not limited to the following types:

[0033] 2.1) PC-based virtual reality (PCVR) devices use the PC to perform calculations and output data related to virtual reality functions. External PC-based virtual reality devices use the data output by the PC to achieve virtual reality effects.

[0034] 2.2) Mobile VR devices support the configuration of mobile terminals (such as smartphones) in various ways (such as head-mounted displays with dedicated card slots). Through wired or wireless connection with the mobile terminal, the mobile terminal performs relevant calculations for VR functions and outputs data to the mobile VR device, such as viewing VR videos through the mobile terminal's app.

[0035] 2.3) All-in-one virtual reality devices have a processor for performing relevant calculations for virtual functions, and thus have independent virtual reality input and output functions. They do not need to be connected to a PC or mobile terminal and have a high degree of freedom of use.

[0036] 3) Augmented Reality (AR): A technology that uses real-time calculations of the camera's pose parameters in the real world (also known as the 3D world or the real world) as the camera captures images. Based on these pose parameters, virtual elements are added to the captured images. Virtual elements include, but are not limited to, images, videos, and 3D models. The goal of AR technology is to overlay the virtual world on the real world for interactive viewing.

[0037] 4) Mixed Reality (MR): A simulated setting that integrates computer-generated sensory input (e.g., virtual objects) with sensory input from a physical setting, or its representation. In some MR settings, the computer-generated sensory input can adapt to changes in the sensory input from the physical setting. Additionally, some electronic systems used to render MR settings can monitor the orientation and / or position relative to the physical setting to enable virtual objects to interact with real objects (i.e., physical elements from the physical setting, or their representations). For example, the system can monitor motion so that virtual plants appear stationary relative to physical buildings.

[0038] 5) Extended Reality (XR) refers to all real and virtual combined environments and human-computer interactions generated by computer technology and wearable devices, including virtual reality (VR), augmented reality (AR) and mixed reality (MR).

[0039] 6) A virtual scene (also known as a virtual space) is a virtual scene displayed (or provided) when an application is running on an electronic device. The virtual scene can be a simulation of the real world, a semi-simulation and semi-fictitious virtual scene, or a purely fictitious virtual scene. The virtual scene can be any of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, or a three-dimensional virtual scene. The embodiments of the present application do not limit the dimensions of the virtual scene. For example, the virtual scene can include the sky, land, ocean, etc., and the land can include environmental elements such as deserts and cities. The user can control virtual objects to move in the virtual scene.

[0040] 7) Virtual objects are objects that interact in virtual scenes and are controlled by users or robot programs (for example, robot programs based on artificial intelligence). They can be still, move, and perform various behaviors in the virtual scene, such as various characters in games.

[0041] After introducing some concepts involved in the embodiments of the present application, a scene adjustment method, apparatus, device and storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0042] Figure 1 is a flow chart of a scene adjustment method provided in an embodiment of the present application. The scene adjustment method provided in an embodiment of the present application can be performed by a scene adjustment device. The scene adjustment device can be composed of hardware and / or software and can be integrated into an electronic device. In an embodiment of the present application, the electronic device can be an XR device, or it can be any other hardware device that can provide a virtual scene. Among them, the XR device can be a VR device, an AR device, or an MR device, etc., and this application does not specifically limit the type of electronic device.

[0043] As shown in FIG1 , the method includes the following steps:

[0044] S101: Acquire status information of a target object in a virtual scene.

[0045] S102: Adjusting environmental information within the virtual scene according to the state information of the target object.

[0046] Optionally, the user can select any type of virtual scene from multiple types of virtual scenes provided on the electronic device display interface and enter the virtual scene, thereby allowing the user to watch the performer's performance in the virtual scene as an audience.

[0047] That is to say, when the performance screen displayed in the virtual scene entered by the user is a live video stream screen, the virtual scene can be understood as a live performance room.

[0048] It should be understood that the performers and users entering the virtual scene can be real people or virtual objects, such as avatars. The virtual objects can be created based on real people or customized based on the performer's performance requirements or the audience's viewing needs, and this application does not impose any restrictions on this.

[0049] Among them, when customizing virtual objects, it can be implemented based on the role creation function provided by the electronic device. The specific creation process can be found in the existing solution, and it will not be described in detail here.

[0050] In addition, a performance stage and various decorative objects, such as various lights, display screens and props, can be set up in the above-mentioned virtual scene to enhance the performance effect of the performers in the virtual scene and serve as a foil for the performers' performance process. At the same time, the atmosphere between the performers and the audience in the entire virtual scene can also be adjusted to make the performance more contagious.

[0051] The above-mentioned various decorative objects can be set on the performance stage, as well as other areas outside the performance stage, such as the audience seating area, etc. This application does not impose any restrictions on this.

[0052] For example, if the virtual scene a user enters is a live music performance studio, the virtual scene can be equipped with a performance stage, and the performance stage can be decorated with various lights, display screens, props, and other decorative objects, as well as various musical instruments or equipment required for the performance. In addition, other areas of the virtual scene, such as the viewing area where the audience can watch the performance, can also be decorated with lights, props, and other decorative objects to serve as a foil for the performer's entire performance process. See Figure 2 for details.

[0053] Considering that after users enter a virtual scene, they can not only watch the performers' performance content but also interact with the performers in various ways. Therefore, this application can adjust the environmental information in the virtual scene based on the performers' performance content in the virtual scene and / or based on the interactive operations of the users as audience members to achieve more intelligent, precise and diversified performance effects, thereby improving the performers' performance effects and the users' viewing experience.

[0054] That is, the target object in this application includes the performer's performance content and / or user interactive operations.

[0055] Since the performance content of the performer corresponds to different status information, such as the status information that can be the music audio information and performance content type of the current performance, etc., and the interactive operations of the users as the audience also correspond to different status information, such as the status information that can be selected as operation instructions, interactive elements and interactive status, etc. Therefore, when the present application adjusts the environmental information in the virtual scene based on the performance content of the performer or the interactive operation of the user in the virtual scene, it can specifically dynamically adjust the environmental information in the virtual scene based on the status information of the performance content, and / or dynamically adjust the environmental information in the virtual scene based on the status information of the user interactive operation, so that the relatively static environmental information in the virtual scene becomes more diverse and interactive, thereby bringing more colorful performance effects to the performer's performance and improving the viewing experience of the audience users.

[0056] The scene adjustment method provided in this application obtains state information of a target object within a virtual scene and adjusts the environmental information within the virtual scene based on the target object's state information. This allows for real-time adjustment of the environmental information within the virtual scene where the performer resides, based on the acquired state information of the target object. This enriches the representation of the environmental information within the performance space, providing users with more diverse visual effects and artistic appeal, meeting their diverse viewing needs, enhancing their visual experience, and also increasing the appeal and enjoyment of musical performances.

[0057] Considering that the target object in this application can be the performer's performance content or the user's interactive operation as an audience, referring to FIG3 below, a specific description is given of adjusting the environment information in the virtual scene based on the state information of the performance content, with the target object being the performer's performance content.

[0058] As shown in FIG3 , the method may include the following steps:

[0059] S201: If the target object is performance content, obtain current music audio information and / or content type of the performance content in the virtual scene.

[0060] The current music audio information can be understood as the real-time audio of a piece of music currently being performed by a performer. For example, the real-time audio of a performer performing a piece of music on a DJ table or a disc jockey (DJ) booth on a stage. Another example is the real-time audio of a performer playing a piece of music on a musical instrument on the stage. Another example is the real-time audio of background music played while a performer dances on the stage, and so on.

[0061] The content type of the aforementioned performance content can be determined based on the performer's performance. For example, if the performance content is instrumental playing or singing, then the content type of the performance content can be music. For another example, if the performance content is street dance, square dance, modern dance, or other dance performances, then the content type of the performance content can be dance. For another example, if the performance content is various types of teaching, then the content type of the performance content can be knowledge teaching, etc.

[0062] It should be understood that the present application obtains the current music audio information and / or content type of the performance content in the virtual scene, which may be obtaining the current music audio information of the performance content in the virtual scene; or, it may be obtaining the content type of the performance content in the virtual scene; or, it may be obtaining the current music audio information and content type of the performance content in the virtual scene, and the present application does not impose any restrictions on this.

[0063] In some optional implementations, when a performer is performing in a virtual scene, the current music audio information and / or content type of the performance content can be obtained in real time using different methods.

[0064] Optionally, obtaining the current music audio information of the performance content can be achieved through an audio extraction tool or an audio extraction algorithm, and this application does not impose any restrictions on the method of obtaining the current music audio information. Among them, the audio extraction tool can be an audio extractor or audio extraction software.

[0065] Considering that when a virtual scene is used as a live broadcast room, the performer needs to set the content type of the performance content in the live broadcast room before performing a live performance in the live broadcast room. Therefore, this application can obtain the content type of the performance content by searching the virtual scene setting information for the content type of the performance content in the virtual scene; or by intercepting a performance video clip and identifying it through the performance video clip to determine the content type of the performance content, etc., and there is no limitation on this.

[0066] The aforementioned recognition of the captured performance video clips can be performed by inputting them into a type recognition model to determine the performance content type, or by analyzing and processing the performance video clips using other recognition algorithms to determine the performance content type. The type recognition model can be a pre-trained network model. The specific model training process can be referenced in existing training methods and will not be elaborated upon here.

[0067] S202: Determine target environment information based on current music audio information and / or content type of the performance content.

[0068] In this application, the target environment information may include at least one of the following: lighting effects, theme effects, environment maps, and prop materials.

[0069] The aforementioned lighting effects can be understood as the lighting effects within the virtual scene. Theme effects can be understood as the corresponding animation effects when the virtual scene presents any theme to the user. Ambient textures can be understood as the atmosphere textures presented in the stage background or other visible areas, such as a blue sky and white clouds texture at the top of the stage, or a large grassland texture on the stage floor. Prop materials can be understood as the prop materials presented in the virtual scene, such as prop color, prop transparency, and prop luminosity.

[0070] Optionally, determine the target environment information, which may include the following situations:

[0071] Case 1

[0072] Target environment information is determined based on current music audio information of the performance content.

[0073] Considering that music audio information includes rhythm and beat, the present application determines target environment information based on the current music audio information of the performance content, optionally first determining the rhythm and / or beat of the current music audio information, and then determining the target environment information based on the rhythm and / or beat of the current music audio information.

[0074] Among them, the rhythm and / or beat of the current music audio information can be understood as the rhythm of the current music audio information; or, it can also be the beat of the current music audio information; or, it can also be the rhythm and beat of the current music audio information.

[0075] Optionally, determining the rhythm and / or beat of the current music audio information can be achieved through the following implementations:

[0076] The first implementation method is

[0077] According to the spectrum analysis method, the current music audio information is analyzed and processed to determine the rhythm and / or beat of the current music audio information.

[0078] Among them, the spectrum analysis method specifically refers to a music analysis algorithm integrated within the Unity engine.

[0079] The spectrum analysis method within the Unity engine analyzes and processes the music audio. Specifically, based on the sound wave frequency that the human ear can hear, such as within the range of 20 to 20,000 vibrations per second, that is, between 20 Hz and 20,000, the continuous signal (i.e., sound wave) is reduced to a discrete signal (i.e., a series of sample data) by sampling. Furthermore, the discrete signal is converted using a fast Fourier transform algorithm, for example, by calling the fast Fourier transform algorithm through the Audio Source.GetSpectrumData command. The converted signal is then serialized into a preset numerical range through a serialization script to obtain the rhythm and / or beat of the music audio.

[0080] The above preset numerical range can be flexibly set according to actual needs, for example, it can be [0, 1], etc. This application does not impose any restrictions on this range.

[0081] The second implementation method,

[0082] According to the rhythm detection algorithm, the rhythm of the current music audio information is determined.

[0083] The rhythm detection algorithm may be a pre-trained rhythm detection model or a music rhythm extraction algorithm, etc., and there is no limitation on this.

[0084] The third way,

[0085] According to the beat detection algorithm, the beat of the current music audio information is determined.

[0086] It should be noted that the present application can also determine the rhythm of the current music audio information through a beat detection model or a music beat extraction algorithm, and the present application does not impose any specific restrictions on this.

[0087] The second and third methods can be implemented separately or in combination, depending on the determination of the rhythm and / or beat of the current music audio information. Specifically, when the rhythm of the current music audio information is determined, the second method is implemented separately; when the beat of the current music audio information is determined, the third method is implemented separately; and when both the rhythm and beat of the current music audio information are determined, the second and third methods are implemented in combination.

[0088] Generally speaking, rhythm refers to the organization of notes of varying lengths to create a regular pattern of dynamics, which can be understood as the proportional balance of a musical passage. Meter, on the other hand, refers to the regular recurrence of strong and weak beats of equal duration, specifically the total length of the notes in each measure of a musical score. Therefore, while determining the rhythm and / or meter of the current music audio information through the three methods described above, it is also possible to optionally determine the rhythmic dynamics and / or meter dynamics of the music audio.

[0089] After determining the rhythm and / or beat of the current music audio information, the present application can search for environmental information corresponding to the rhythm and / or beat of the current music audio information in a pre-established first mapping relationship based on the rhythm and / or beat of the current music audio information, and determine the found scene information as the target environmental information.

[0090] The pre-established first mapping relationship may be a mapping relationship between rhythm and environmental information, or a mapping relationship between beat and environmental information, or a mapping relationship between rhythm, beat and environmental information.

[0091] Case 2

[0092] Determine the target environment information based on the content type of the performance.

[0093] Optionally, to enrich the virtual scene's environmental information, a mapping relationship between content type and environmental information, i.e., a second mapping relationship, may be pre-established. Furthermore, based on the performance content's type, the pre-established second mapping relationship may be used to search for environmental information corresponding to the content type, and the found scene information may be determined as the target environmental information.

[0094] Case 3

[0095] Target environment information is determined based on current music audio information and content type of the performance content.

[0096] Specifically, the present application can determine the first environmental information based on the current music audio information of the performance content, and determine the second environmental information based on the content type of the performance content, and then fuse the first environmental information and the second environmental information to obtain the target environmental information.

[0097] The implementation principle for determining the first environmental information is the same or similar to that of the aforementioned scenario 1. The specific implementation process can be found in the aforementioned scenario 1, and will not be described in detail here. Furthermore, the implementation principle for determining the first environmental information is the same or similar to that of the aforementioned scenario 2. The specific implementation process can be found in the aforementioned scenario 2, and will not be described in detail here.

[0098] It should be understood that the aforementioned fusion of the first and second environmental information can be achieved by adding the same environmental parameters at the same time, or by adding the same environmental parameters at the same time and multiplying them by a fusion parameter. The fusion parameter can be flexibly set based on the fusion effect, such as 0.5 or 0.7, and is not specifically limited here.

[0099] For example, the first environment information at time t1 includes the first lighting effect, the first environment map, and the first prop material; the second environment information at time t1 includes the second lighting effect, the second environment map, and the second prop material. Then, the first and second environment information at time t1 are merged to form the first lighting effect + the second lighting effect, the first environment map + the second environment map, and the first prop material + the second prop material.

[0100] For example, the first environment information at time t1 includes the first lighting effect, the first environment map, and the first prop material; the second environment information at time t1 includes the second lighting effect, the second environment map, and the second prop material, and the fusion parameter a = 0.5. Then, the first environment information and the second environment information at time t1 are fused to obtain (first lighting effect + second lighting effect) * 0.5, (first environment map + second environment map) * 0.5, and (first prop material + second prop material) * 0.5.

[0101] Considering that the environmental parameters included in the first and second environmental information at the same moment may differ, for example, the first environmental information at time t2 includes the first lighting effect and the first theme special effect, while the second environmental information at time t2 includes the second lighting effect and the second environmental map. Therefore, only the identical environmental parameters at the same time are summed, while the different environmental parameters are directly reused to obtain the fused target environmental information.

[0102] Continuing with the first environmental information and the second environmental information at time t2 as an example, the first environmental information and the second environmental information at time t2 are specifically integrated, specifically the first lighting effect + the second lighting effect, the first theme special effect and the second environmental map.

[0103] It can be understood that the present application pre-configures various environmental information to the client by pre-establishing a first mapping relationship, or pre-establishing a second mapping relationship, so that when the client needs to adjust the environmental information of the virtual scene, it can directly call any pre-configured environmental information, thereby eliminating the need for real-time calculation and real-time rendering, which can greatly save the resource consumption of the electronic device and improve the display performance of the electronic device.

[0104] Moreover, the above-mentioned pre-configured environmental information can also support resource reuse. For example, when the rhythm and / or beat of the music audio information at a certain moment is the same as the rhythm and / or beat of the music audio at any previous historical moment, the environmental information used at any previous historical moment can be directly called from the database, thereby reducing the production cost of environmental information resources.

[0105] Among them, the database refers to a library that stores various environmental information pre-made by technicians. As mentioned above, the database can be stored on the electronic device (such as the client) side, or it can be stored on the electronic device communication connection server side, and this application does not impose any restrictions on this. The above-mentioned technicians pre-produce a variety of environmental information, including parameters such as environmental maps, lighting effects, theme effects and / or prop materials. The advantage of such a setting is that it can not only achieve resource reuse and resource sharing, but also when new environmental information needs to be produced, the technician can select the corresponding environmental data from the database to fine-tune the parameters, or recombine to obtain new environmental information, so as to reduce production costs and avoid problems such as duplicate production and waste of resources.

[0106] S203: Adjust the environment information in the virtual scene according to the target environment information.

[0107] The target environment information may include at least one of the following: lighting effects, theme effects, environment maps, and prop materials.

[0108] In some optional implementations, the present application adjusts the current environment information of the virtual scene in real time according to the target environment information, including at least one of the following:

[0109] According to the lighting effects in the target environment information, the lighting control components in virtual engines such as Unity are used to control the color, brightness, position, shape, motion trajectory and other parameters of the light through programming.

[0110] According to the prop material in the target environment information, the material control components in virtual engines such as Unity are used to control the material's color, transparency, reflectivity, luminosity and other parameters through programming.

[0111] According to the theme special effects in the target environment information, the special effects control components in virtual engines such as Unity are used to control the shape, size, color, motion trajectory and other parameters of the special effects through programming.

[0112] That is to say, by utilizing the target environmental information, the environmental information currently presented in the virtual scene can be adjusted in real time, so that the environmental information presented in the virtual scene can change accordingly with the rhythm and / or beat of the music audio, and / or the changes in content type, thereby bringing richer performance effects to the performance in the virtual scene, making the traditional relatively static environmental information more diverse and interactive, thereby improving the performance performance ability and user visual experience in the entire virtual scene.

[0113] The scene adjustment method provided by the present application obtains the status information of the target object in the virtual scene and adjusts the environmental information in the virtual scene according to the status information of the target object. As a result, the environmental information in the virtual scene where the performer is located can be adjusted in real time based on the acquired status information of the target object, thereby enriching the environmental information representation of the performance space, thereby providing users with more colorful visual effects and artistic appeal to meet the user's diverse viewing needs, enhance the user's visual experience, and also enhance the attractiveness and viewing experience of the music performance. In addition, by adjusting the environmental information in the virtual scene in real time based on the current music audio information and / or content type of the performance content in the virtual scene, a richer performance effect can be brought to the performance in the virtual scene, making the traditional relatively static environmental information more diverse and interactive, thereby improving the performance performance ability and user's visual experience in the entire virtual scene.

[0114] Based on the above embodiments, considering that the rhythm and / or beat of the music audio information have corresponding strength values, when determining the target environment information based on the current music audio information of the performance content, the present application can determine the target environment information based on the rhythm strength value and / or beat strength value of the current music audio information. Below, in conjunction with FIG4 , the present application specifically describes how to determine the target environment information based on the rhythm strength value and / or beat strength value of the current music audio information. Referring to FIG4 , the method may include the following steps:

[0115] S301: If the target object is the performance content, obtain current music audio information of the performance content in the virtual scene.

[0116] S302: Determine the tempo and / or beat of the current music audio information.

[0117] S303: Determine target environment information according to the rhythm strength value and / or beat strength value of the current music audio information.

[0118] Considering that when determining the rhythm and / or beat of the current music audio information, the rhythm strength value and / or beat strength value of the current music audio information can also be determined. Therefore, the present application can determine the target environment information based on the determined rhythm strength value and / or beat strength value.

[0119] Optionally, when determining the target environmental information based on the rhythm strength value and / or beat strength value of the current music audio information, the environmental information corresponding to the rhythm strength value and / or beat strength value of the current music audio information is specifically searched in a pre-established third mapping relationship, and the environmental information is used as the target environmental information.

[0120] Among them, the pre-established third mapping relationship can be a mapping relationship between rhythm strength value and environmental information, or a mapping relationship between beat strength value and environmental information, or a mapping relationship between rhythm strength value, beat strength value and environmental information.

[0121] That is to say, the present application determines the target environmental information based on the rhythm strength value and / or beat strength value of the current music audio information, including: determining the target environmental information in the mapping relationship between the rhythm strength value and the environmental information based on the rhythm strength value of the current music audio information; or, determining the target environmental information in the mapping relationship between the beat strength value and the environmental information based on the beat strength value of the current music audio information; or, determining the target environmental information in the mapping relationship between the rhythm strength value, the beat strength value and the environmental information based on the rhythm strength value and the beat strength value of the current music audio information.

[0122] In some optional implementations, the pre-established mapping relationship between the rhythm strength value and the environmental information may be as shown in Table 1 below:

[0123] Table 1

[0124] In some optional implementations, the pre-established mapping relationship between the beat strength value and the environmental information may be as shown in Table 2 below:

[0125] Table 2

[0126] In some optional implementations, the pre-established mapping relationship between the rhythm strength value, the beat strength value, and the environmental information may be as shown in Table 3 below:

[0127] Table 3

[0128] It should be noted that the preset numerical interval [0,1] corresponding to the rhythm strength value and / or beat strength value in Tables 1, 2, and 3 above can be flexibly adjusted according to actual needs, and the interval value for dividing the numerical interval [0,1] can be adjusted according to the scene adjustment accuracy. The above division method is only used as an exemplary explanation of this application and is not a specific limitation of this application. For example, in addition to 0.1, the interval value for dividing the numerical interval [0,1] can also be 0.05 as an interval value for segmentation, or 0.15 as an interval value for segmentation. This application does not impose any restrictions on it.

[0129] For example, assuming the rhythm strength value of the current music audio information is 0.2, then in the pre-established mapping relationship between beat strength values ​​and environmental information, such as in Table 2 above, the environmental information found based on the rhythm strength value of 0.2 includes: the lighting effect is adjusted from the first color to the second color; the theme effect is theme effect A; the environmental map is environment map X2, such as seaweed and small fish; and the prop material is the prop color, with the color value remaining unchanged at the second value. The searched environmental information can then be used as the target environmental information.

[0130] It is noteworthy that, when determining target environment information in the mapping relationship between rhythm strength value, beat strength value and environmental information according to the rhythm strength value and beat strength value of current music audio information, there may be a situation where the rhythm strength value and beat strength value are inconsistent. At this time, the rhythm strength value or the beat strength value can be used as a search condition to determine target environment information in the mapping relationship between rhythm strength value, beat strength value and environmental information. Alternatively, according to the preset priority of rhythm strength value and beat strength value, the strength value corresponding to the highest priority is used as a search condition to determine target environment information in the mapping relationship between rhythm strength value, beat strength value and environmental information. Exemplary, if the priority of rhythm strength value is higher than beat strength value, then when rhythm strength value and beat strength value are inconsistent, the rhythm strength value is used as a search condition to determine target environment information. For another example, if the priority of beat strength value is higher than rhythm strength value, then when rhythm strength value and beat strength value are inconsistent, the beat strength value is used as a search condition to determine target environment information.

[0131] Considering that when determining the target environment information in a pre-established third mapping relationship based on the rhythm strength value and / or beat strength value of the current music audio information, the target environment information may not be found in the pre-established third mapping relationship because the accuracy of the rhythm strength value and / or beat strength value is different from the accuracy in the pre-established third mapping relationship.

[0132] To this end, the present application can solve the problem of the accuracy of the rhythm strength value and / or the beat strength value by the following circumstances. When the accuracy is different from the accuracy in the pre-established third mapping relationship, the target environment information is searched in the pre-established third mapping relationship:

[0133] The first case

[0134] If the accuracy of the rhythm strength value and / or the beat strength value is 0.01, and the accuracy in the pre-established third mapping relationship is 0.1, the rhythm strength value and / or the beat strength value is adjusted in a rounded manner so that the accuracy of the adjusted rhythm strength value and / or the adjusted beat strength value is the same as the accuracy of the pre-established third mapping relationship, so that the target environment information corresponding to the rhythm strength value and / or the beat strength value can be found from the pre-established third mapping relationship.

[0135] For example, the rhythm strength value of a certain music audio is obtained to be 0.09, and the precision of the pre-established third mapping relationship is 0.1, such as the above Table 1. Because the last digit 9 in 0.09 is greater than 5, the unit to the left of 9 can be moved up one place, so that the adjusted rhythm strength value changes from 0.09 to 0.1. Furthermore, according to the adjusted rhythm strength value 0.1, it can be determined from Table 1 that the environmental information corresponding to the rhythm strength value 0.1 includes: the lighting effect is the light brightness adjusted from the first value to the second value; the theme effect is the theme effect A; the environmental map is the environmental map X1, such as seaweed, etc.; the prop material is the color value of the prop color adjusted from the first value to the second value. Then, the above-mentioned searched environmental information is used as the target environmental information.

[0136] The second situation

[0137] The strength value in the pre-established third mapping relationship is set as an interval, thereby determining the rhythm strength value interval corresponding to the rhythm strength value of the current music audio information and / or the beat strength value interval corresponding to the beat strength value of the current music audio information. Then, the target environment information is determined in the pre-established mapping relationship based on the rhythm strength value interval and / or the beat strength value interval.

[0138] Among them, the preset mapping relationship is the mapping relationship between rhythm strength value intervals and environmental information, the mapping relationship between beat strength value intervals and environmental information, or the mapping relationship between rhythm strength value intervals, beat strength value intervals and environmental information.

[0139] Considering that the implementation principle and implementation process of each of the above preset mapping relationships are the same or similar, the mapping relationship between the rhythm strength value interval and the environmental information is mainly used as an example for explanation below.

[0140] Assume that the mapping relationship between the above-mentioned rhythm strength value intervals and environmental information includes five rhythm strength value intervals, specifically: [0, 0.2), [0.2, 0.4), [0.4, 0.6), [0.6, 0.8), [0.8, 1], and the environmental information corresponding to each rhythm strength value interval is: the first group of environmental information, the second group of environmental information, the third group of environmental information, the fourth group of environmental information, and the fifth group of environmental information. For details, please refer to the following Table 4.

[0141] Table 4

[0142] If the rhythm strength value of the audio information of a certain piece of music at the current moment is 0.51, then the rhythm strength value interval corresponding to the rhythm strength value 0.51 is determined to be [0.4, 0.6). Furthermore, based on the rhythm strength value interval [0.4, 0.6), the target environmental information is determined to be the third set of environmental information in the mapping relationship between the rhythm strength value interval and the environmental information, and the third set of environmental information includes: the lighting effect is the first set of lights on and the second set of lights off; the theme special effect is theme special effect B; the environmental map is environmental map Y2; and the color value of the prop color remains unchanged at the fourth value.

[0143] S304: Adjust the environment information of the music performance stage according to the target environment information.

[0144] Optionally, after determining the target environmental information corresponding to the rhythm strength value and / or beat strength value of the current music audio information, the present application can adjust the current environmental information in the virtual scene based on the target environmental information, so that the environmental information presented in the virtual scene can be dynamically adjusted in real time following the rhythm strength value and / or beat strength value of the music audio information, thereby enriching the expression of the stage environmental information.

[0145] The scene adjustment method provided by the embodiment of the present application obtains the current music audio information of the performance content in the virtual scene and determines the rhythm and / or beat of the current music audio to determine the target environment information based on the rhythm strength value and / or beat strength value of the current music audio information, and then adjusts the environment information in the virtual scene based on the target environment information. In this way, the environmental information expression form of the performance space can be enriched, thereby providing users with more colorful visual effects and artistic appeal to meet the diverse viewing needs of users, enhance the user's visual experience, and also enhance the attractiveness and viewing value of the music performance. In addition, it can also bring richer performance effects to the performance in the virtual scene, making the traditional relatively static environment information more diverse and interactive, thereby improving the performance performance ability and user's visual experience in the entire virtual scene.

[0146] Considering that the target object in this application can be the performance content of the performer or the interactive operation of the user as the audience, the following is a detailed description of adjusting the environmental information in the virtual scene based on the state information of the user interactive operation, taking the target object as the user interactive operation, with reference to FIG5 .

[0147] As shown in FIG5 , the method includes the following steps:

[0148] S401: If the target object is a user interactive operation, obtain the operation instruction, interactive element or interactive state of the user interactive operation in the virtual scene.

[0149] S402: Determine target environment information according to the operation instruction, interactive element or interactive state of the user's interactive operation.

[0150] Considering that audience members may interact with the performance content within the virtual scene, such as interacting with performers and proactively triggering interactive operation instructions such as adjusting the environment information within the virtual scene, this application can obtain the operation instructions, interactive elements, or interactive status corresponding to the user's interactive operation. Then, based on the operation instructions, interactive elements, or interactive status of the user's interactive operation, the target environment information is determined.

[0151] The aforementioned operating instructions can be understood as triggering instructions for any environmental parameters by the user through the control panel provided in the virtual scene. The control panel can be a virtual screen located in the virtual scene, allowing the user as a viewer to perform various live broadcast operations through the control panel, such as adjusting the environmental parameters in the virtual scene.

[0152] Interactive elements can be understood as various elements that improve the interactive atmosphere, such as pen refill emojis, like emojis, fireworks emojis, text emojis, etc. There are no restrictions on them here.

[0153] The interactive state can be understood as the state of the user in the virtual scene, where the user's state in the virtual scene can be either spectator or performer. In other words, the user can switch their state from spectator to performer and perform live in the virtual scene.

[0154] In some optional implementations, determining target environment information based on an operation instruction, interactive element, or interactive state of a user interaction operation may include the following situations:

[0155] Scenario 1

[0156] The target environment information is determined in response to a user's operation instruction on any environment parameter on a control panel, wherein the control panel is located in a virtual scene.

[0157] As shown in Figure 6, assuming that the user triggers the environment adjustment control on the control panel, the control panel switches from the current display interface to the environment adjustment interface, where the environment adjustment interface may include various environment parameter items, such as lighting effects, theme effects, and environment maps. Furthermore, the user can adjust any environment parameter item according to his or her own adjustment needs for the environment information in the virtual scene. When the electronic device obtains the user's adjustment instruction for any environment parameter item on the environment adjustment interface, it counts which environment parameter items are adjusted and obtains the target environment information based on the adjusted environment parameters. It should be noted that in addition to being options, each environment parameter in Figure 6 can also be a progress bar set for each environment parameter, so that the user drags the progress bar to adjust the environment parameter.

[0158] Scenario 2

[0159] In response to the interactive operation triggered by the user, an interactive element corresponding to the interactive operation is determined, and the interactive element is added to the current environment information of the interactive scene to obtain target environment information.

[0160] Considering that users, as audience members, can interact with the performers or follow the performance content while watching the performers' performance, the user can interact by manipulating an interactive controller such as a handle, hand controller, or glove to manipulate a fluorescent stick or other type of interactive object; or the user can interact by holding an object such as a fluorescent stick with bare hands; or the user can interact by manipulating an interactive controller, etc. This application does not impose any restrictions on the interaction method.

[0161] When the electronic device detects an interactive operation triggered by a user, it first determines the interactive element corresponding to the interactive operation, and then adds the determined interactive element to the current environment information of the virtual scene to obtain target environment information.

[0162] For example, if the user triggers an interactive action by waving a light stick left and right with their bare hands, the interactive element corresponding to the interactive action can be determined from the pre-established mapping relationship between interactive actions and interactive elements. Then, the interactive element corresponding to the interactive action is added to the current environment information within the virtual scene to obtain the target environment information.

[0163] In an optional implementation of the present application, the pre-established mapping relationship between interactive operations and interactive elements may be as shown in Table 5 below:

[0164] Table 5

[0165] It should be noted that the mapping relationship between the above-mentioned interactive operations and interactive elements can be flexibly adjusted and updated according to the adjustment requirements of the environmental information in the virtual scene.

[0166] Scenario 3

[0167] In response to the identity switching operation triggered by the user, the user's identity is switched from audience to performer. Then, the target environment information is determined based on the performance content of the user whose interactive status is performer.

[0168] Considering that the audience and performers can switch their identities in the virtual scene, it can increase the interactive effect of the entire live broadcast room and improve the audience's enthusiasm for participation. Therefore, when a user wants to perform music as a performer, the user can send an identity switching operation through voice, eye tracking, gestures, or a controller to switch the user's audience identity to performer, and then perform in the virtual scene as a performer, and then determine the target environment information based on the user's performance content.

[0169] For example, a user can trigger an identity switching operation by clicking an identity switching control displayed on the live broadcast room through a controller. Then, the electronic device can switch the user's identity from audience to performer according to the identity switching operation triggered by the user.

[0170] When the user triggers the identity switch operation, a performer may be currently performing. In this case, the electronic device will display a prompt message to the user, such as "A performer is currently performing. Please wait." When it determines that the performer has completed their performance, the electronic device performs the identity switch operation, switching the user's identity from spectator to the latest performer, allowing the user to perform as a performer. This allows the user to more easily control the performance effect in the virtual scene, improving performance efficiency and quality.

[0171] S403: Adjust the environment information in the virtual scene according to the target environment information.

[0172] After determining the target environment information, the present application can update the environment information in the virtual scene in real time based on the target environment information.

[0173] In some optional implementations, when the target environmental information is to add interactive elements corresponding to the user's interactive operation to the current environmental information of the virtual scene, then when adjusting the environmental information within the virtual scene, the corresponding interactive elements can be displayed near the user-triggered interactive operation to respond to the user's interactive operation, thereby improving the visual effect of the user's interaction and the interactive atmosphere of the entire virtual scene.

[0174] Taking into account that there can be multiple audiences in a virtual scene, when any audience triggers an interactive operation in the virtual scene and adjusts the current environmental information in the virtual scene based on the target environmental information determined by the interactive elements of the interactive operation, the present application preferably only adjusts the environmental information seen by the user who triggers the interactive operation, so that the user who triggers the interactive operation can see the corresponding interactive response in his own perspective, while no processing is performed in the vision of other users, thereby achieving a single-person interactive response to the interactive operation triggered by a single person, avoiding the abrupt feeling of environmental information adjustment for other users during the performance due to the lack of triggering the interactive operation.

[0175] It should be noted that when a user performs as a performer, the present application will execute steps S201 to S203, or steps S301 to S304 in the aforementioned embodiments, so as to achieve real-time dynamic adjustment of the environmental information in the virtual scene following the performer's performance, thereby improving the performance ability of the virtual scene and the visual effect of the user.

[0176] In one optional implementation, when a user interacts within a virtual scene, the present application may optionally enhance the display based on the current environmental information of the virtual scene to adjust the environmental information, thereby further enhancing the user's interactive interest and enthusiasm. Enhanced display can be understood as increasing the brightness of lights, driving the playback of preset special effects, and / or turning on and off preset lighting groups based on the current environmental information.

[0177] The above current environment information is the environment information before adjustment or the environment information after adjustment, and no restriction is imposed on it here.

[0178] As an optional implementation, the enhanced display of the current environment information of the virtual scene may include at least one of the following situations:

[0179] The first case

[0180] If the motion trajectory of the interactive controller controlled by the user is a preset motion trajectory, the current environment information in the virtual scene is enhanced and displayed according to the motion trajectory.

[0181] Among them, the motion trajectory of the interactive controller can be determined based on the position data in the posture data determined by the inertial measurement unit data sent by the inertial measurement unit (IMU) in the interactive controller. The specific determination process is a conventional technology in this field and will not be described in detail here.

[0182] In this application, the preset motion trajectory can be set according to actual interaction needs, such as drawing a circle or a heart.

[0183] Optionally, the present application may compare the motion trajectory of the interactive controller with a preset motion trajectory. If the motion trajectory is the same as any preset motion trajectory, the current environmental information in the virtual scene is enhanced and displayed according to the enhanced display method associated with the preset motion trajectory.

[0184] The enhanced display mode associated with the preset motion trajectory can be a mapping relationship between the motion trajectory and the enhanced display mode pre-established by the technician, and this mapping relationship can also be stored in the database. Furthermore, the technician can periodically update the mapping relationship between the motion trajectory and the enhanced display mode to meet the needs of personalized virtual scene responses to interactive operations through environmental information.

[0185] Case 2

[0186] If the movement amplitude of the user manipulating the interactive controller reaches an amplitude threshold, the current environment information in the virtual scene is enhanced and displayed according to the movement amplitude.

[0187] The movement amplitude of the interactive controller can be determined based on inertial measurement unit data sent by an inertial measurement unit (IMU) in the interactive controller. Specifically, the movement distance of the user manipulating the interactive controller can be determined based on position data in the posture data determined by the IMU data, and the movement distance is determined as the movement amplitude of the interactive controller. The movement distance can be determined based on the position data of the interactive controller, and can be determined based on a two-point distance calculation formula, such as calculating the Euclidean distance, which is not specifically limited here.

[0188] In this application, the amplitude threshold can be set according to actual interaction needs, such as 20 cm or 40 cm, etc. This application does not impose any restrictions on it.

[0189] Optionally, the present application may compare the movement amplitude of the interactive controller with an amplitude threshold. If the movement amplitude is greater than the amplitude threshold, the current environment information in the virtual scene is enhanced and displayed according to a preset enhanced display mode. Otherwise, the current environment information in the virtual scene is enhanced and displayed without any processing.

[0190] There may be multiple preset enhanced display modes. When the motion amplitude is greater than the amplitude threshold, any mode is randomly selected from the multiple preset enhanced display modes for response display, thereby increasing the diversity and interest of the response.

[0191] Case 3

[0192] If the movement frequency of the user manipulating the interactive controller reaches a frequency threshold, the current environment information in the virtual scene is enhanced and displayed according to the movement frequency.

[0193] In this application, the frequency threshold can be set according to actual interaction needs. For example, if the number of times the interactive controller is shaken reaches 5 times within 1 minute, the frequency threshold is f = 1 / 5; or, if the number of times the interactive controller is shaken reaches 30 times within 5 minutes, the frequency threshold is f = 1 / 30, etc. This application does not impose any restrictions on it.

[0194] Optionally, the present application can capture real-time images of the user operating the interactive controller through a camera on a head-mounted display device in an electronic device, and then identify the real-time tracking image captured by the camera to determine the movement frequency of the interactive controller. The specific identification method can be referred to the existing implementation method, which will not be described in detail here.

[0195] Furthermore, the electronic device may compare the motion frequency of the interactive controller with a frequency threshold. If the motion frequency is greater than the frequency threshold, the current environment information in the virtual scene is enhanced and displayed according to a preset enhanced display method. Otherwise, the current environment information in the virtual scene is enhanced and displayed without any processing.

[0196] There may be multiple preset enhanced display modes. When the motion frequency is greater than a frequency threshold, any mode is randomly selected from the multiple preset enhanced display modes for response display, thereby increasing the diversity and interest of the response.

[0197] Case 4

[0198] If the motion trajectory of the user's hand is a preset motion trajectory, the current environmental information in the virtual scene is enhanced and displayed according to the motion trajectory.

[0199] The user's hand may be understood as the user's real hand, or a virtual hand model presented in the virtual space, and no specific limitation is imposed here.

[0200] Optionally, the present application can collect real-time images of the user's hands through the camera on the head-mounted display device in the electronic device, and then identify the real-time tracking image collected by the camera to determine the movement trajectory of the user's hands. The specific identification method can refer to the existing implementation method, which will not be described in detail here.

[0201] Furthermore, the motion trajectory of the user's hand can be compared with the preset motion trajectory. If the motion trajectory is the same as any preset motion trajectory, the current environmental information in the virtual scene is enhanced and displayed according to the enhanced display method associated with the preset motion trajectory.

[0202] The enhanced display mode associated with the preset motion trajectory can be a mapping relationship between the motion trajectory and the enhanced display mode pre-established by the technician, and this mapping relationship can also be stored in the database. Furthermore, the technician can periodically update the mapping relationship between the motion trajectory and the enhanced display mode to meet the personalized display requirements of different performance stages.

[0203] Case 5

[0204] If the movement amplitude of the user's hand reaches the amplitude threshold, the current environment information in the virtual scene is enhanced and displayed according to the movement amplitude.

[0205] In this application, the amplitude threshold can be set according to actual interaction needs, such as 20 cm or 40 cm, etc. This application does not impose any restrictions on it.

[0206] Optionally, the present application can capture real-time images of the user's hands through a camera on a head-mounted display device in an electronic device, and then identify the real-time tracking images captured by the camera to determine the movement amplitude of the user's hands. The specific identification method can be referred to the existing implementation method, which will not be described in detail here.

[0207] Furthermore, the user's hand movement amplitude can be compared with an amplitude threshold. If the movement amplitude is greater than the amplitude threshold, the current environment information in the virtual scene is enhanced and displayed according to a preset enhanced display method. Otherwise, the current environment information in the virtual scene is enhanced and displayed without any processing.

[0208] There may be multiple preset enhanced display modes. When the motion amplitude is greater than the amplitude threshold, any mode is randomly selected from the multiple preset enhanced display modes for response display, thereby increasing the diversity and interest of the response.

[0209] Case 6

[0210] If the movement frequency of the user's hand reaches a frequency threshold, the current environmental information in the virtual scene is enhanced and displayed according to the movement frequency.

[0211] In this application, the frequency threshold can be set according to actual interaction needs. For example, if the number of times the interactive controller is shaken reaches 5 times within 1 minute, the frequency threshold is f = 1 / 5; or, if the number of times the interactive controller is shaken reaches 30 times within 5 minutes, the frequency threshold is f = 1 / 30, etc. This application does not impose any restrictions on it.

[0212] Optionally, the present application can collect real-time images of the user's hands through the camera on the head-mounted display device in the electronic device, and then identify the real-time tracking images collected by the camera to determine the movement frequency of the user's hands. The specific identification method can refer to the existing implementation method, which will not be described in detail here.

[0213] Furthermore, the electronic device may compare the user's hand movement frequency with a frequency threshold. If the movement frequency is greater than the frequency threshold, the current environment information in the virtual scene is enhanced and displayed according to a preset enhanced display method. Otherwise, the current environment information in the virtual scene is enhanced and displayed without any processing.

[0214] There may be multiple preset enhanced display modes. When the motion frequency is greater than a frequency threshold, any mode is randomly selected from the multiple preset enhanced display modes for response display, thereby increasing the diversity and interest of the response.

[0215] Situation 7

[0216] If the user gesture is a preset gesture, the current environment information in the virtual scene is enhanced and displayed according to the gesture.

[0217] In this application, the preset gesture can be a pre-set gesture, and the gesture can be flexibly adjusted according to actual interaction needs, such as: a like gesture, a heart gesture, etc., and no specific restrictions are made here.

[0218] Optionally, the present application can collect real-time images of the user's hands through the camera on the head-mounted display device in the electronic device, and then identify the real-time tracking image collected by the camera to determine the gestures made by the user's hands. The specific identification method can be referred to the existing implementation method, which will not be described in detail here.

[0219] Furthermore, the electronic device can compare the gesture made by the user's hand with the preset gesture. If the gesture made by the user's hand is the same as any preset gesture, the current environmental information in the virtual scene will be enhanced and displayed according to the enhanced display method associated with the preset gesture.

[0220] The enhanced display mode associated with the preset gesture can be a mapping relationship between gestures and enhanced display modes pre-established by technicians, and this mapping relationship can also be stored in a database. Furthermore, technicians can periodically update the mapping relationship between gestures and enhanced display modes to meet the personalized display requirements of different performance stages.

[0221] That is, in response to the interactive operation triggered by the user, the present application performs enhanced display of the current environment information in the virtual scene, including at least one of the following:

[0222] If the motion trajectory of the interactive controller controlled by the user is a preset motion trajectory, the current environment information in the virtual scene is enhanced and displayed according to the motion trajectory;

[0223] If the movement amplitude of the user's manipulation of the interactive controller reaches an amplitude threshold, the current environment information in the virtual scene is enhanced and displayed according to the movement amplitude;

[0224] If the user's motion frequency of manipulating the interactive controller reaches a frequency threshold, the current environment information in the virtual scene is enhanced and displayed according to the motion frequency;

[0225] If the motion trajectory of the user's hand is a preset motion trajectory, the current environmental information in the virtual scene is enhanced and displayed according to the motion trajectory;

[0226] If the movement amplitude of the user's hand reaches the amplitude threshold, the current environmental information in the virtual scene is enhanced and displayed according to the movement amplitude;

[0227] If the user's hand movement frequency reaches a frequency threshold, the current environmental information in the virtual scene is enhanced and displayed according to the movement frequency;

[0228] If the user gesture is a preset gesture, the current environment information in the virtual scene is enhanced and displayed according to the gesture.

[0229] The scene adjustment method provided in the embodiment of the present application obtains the operation instructions, interactive elements or interactive status of the user's interactive operation in the virtual scene, determines the target environment information according to the operation instructions, interactive elements or interactive status of the user's interactive operation, and then adjusts the environment information in the virtual scene according to the target environment information. In this way, the environmental information expression form of the performance space can be enriched, thereby providing users with more colorful visual effects and artistic appeal to meet the user's diverse viewing needs, enhance the user's visual experience, and also enhance the attractiveness and viewing value of the music performance. In addition, by enhancing the display of the current environmental information in the virtual scene according to the interactive operation triggered by the user, the user's interactive interest and interactive enthusiasm can be improved, and the performance presentation effect in the entire virtual scene can be dynamically adjusted according to the user's interactive operation, further enriching the stage performance effect and enhancing the user's viewing experience.

[0230] A scene adjustment device according to an embodiment of the present application will be described below with reference to FIG7 FIG7 is a schematic block diagram of a scene adjustment device according to an embodiment of the present application.

[0231] As shown in FIG. 7 , the scene adjustment device 400 includes an information acquisition module 410 and a scene adjustment module 420 .

[0232] The information acquisition module 410 is used to obtain the status information of the target object in the virtual scene;

[0233] The scene adjustment module 420 is configured to adjust the environment information within the virtual scene according to the state information of the target object.

[0234] In an optional implementation of the embodiment of the present application, the target object includes: performance content and user interactive operations;

[0235] Accordingly, the information acquisition module 410 is specifically configured to:

[0236] If the target object is the performance content, obtaining current music audio information and / or content type of the performance content in the virtual scene;

[0237] If the target object is the user interactive operation, an operation instruction, an interactive element or an interactive state of the user interactive operation in the virtual scene is obtained.

[0238] In an optional implementation of the embodiment of the present application, when the target object is the performance content, the scene adjustment module 420 includes:

[0239] a first determining unit, configured to determine target environment information based on current music audio information and / or content type of the performance content;

[0240] An adjustment unit is used to adjust the environmental information in the virtual scene according to the target environmental information.

[0241] In an optional implementation of the embodiment of the present application, the first determining unit is specifically configured to:

[0242] Determining the tempo and / or beat of the current music audio information;

[0243] The target environment information is determined according to the rhythm and / or beat of the current music audio information.

[0244] In an optional implementation of the embodiment of the present application, the first determining unit is further configured to:

[0245] The target environment information is determined according to the rhythm strength value and / or beat strength value of the current music audio information.

[0246] In an optional implementation of the embodiment of the present application, the first determining unit is further configured to:

[0247] Determining the target environmental information in a mapping relationship between the rhythm strength value and the environmental information according to the rhythm strength value of the current music audio information;

[0248] or,

[0249] Determining the target environmental information in a mapping relationship between the beat strength value and the environmental information according to the beat strength value of the current music audio information;

[0250] or,

[0251] According to the rhythm strength value and beat strength value of the current music audio information, the target environmental information is determined in the mapping relationship between the rhythm strength value, the beat strength value and the environmental information.

[0252] In an optional implementation of the embodiment of the present application, the first determining unit is specifically configured to:

[0253] Determining a rhythm strength value interval corresponding to the rhythm strength value of the current music audio information, and / or a beat strength value interval corresponding to the beat strength value of the current music audio information;

[0254] Determining the target environment information in a preset mapping relationship according to the rhythm strength value interval and / or the beat strength value interval;

[0255] Among them, the preset mapping relationship is the mapping relationship between rhythm strength value intervals and environmental information, the mapping relationship between beat strength value intervals and environmental information, or the mapping relationship between rhythm strength value intervals, beat strength value intervals and environmental information.

[0256] In an optional implementation of the embodiment of the present application, the first determining unit is specifically configured to:

[0257] According to the content type of the performance content, the target environment information is determined in a mapping relationship between the content type and the environment information.

[0258] In an optional implementation of the embodiment of the present application, the first determining unit is specifically configured to:

[0259] Determining first environmental information corresponding to current music audio information of the performance content, and second environmental information corresponding to the content type of the performance content;

[0260] The first environment information and the second environment information are integrated to obtain the target environment information.

[0261] In an optional implementation of the embodiment of the present application, when the target object is the user interactive operation, the scene adjustment module 420 includes:

[0262] a second determining unit, configured to determine target environment information according to an operation instruction, interactive element, or interactive state of the user interactive operation;

[0263] The adjustment unit is further configured to adjust the environment information within the virtual scene according to the target environment information.

[0264] In an optional implementation of the embodiment of the present application, the second determining unit is specifically configured to:

[0265] The target environment information is determined in response to a user's operation instruction on any environment parameter on a control panel, wherein the control panel is located in the virtual scene.

[0266] In an optional implementation of the embodiment of the present application, the second determining unit is specifically configured to:

[0267] In response to an identity switching operation triggered by a user, switching the user's identity from a spectator to a performer;

[0268] The target environment information is determined according to the performance content of the user whose interactive state is a performer.

[0269] In an optional implementation of the embodiment of the present application, the scene adjustment module 420 further includes:

[0270] an element determining unit, configured to determine, in response to an interactive operation triggered by a user, an interactive element corresponding to the interactive operation;

[0271] Accordingly, the second determining unit is specifically configured to:

[0272] The interactive element is added to the current environment information of the interactive scene to obtain the target environment information.

[0273] In an optional implementation of an embodiment of the present application, the target environment information includes at least one of the following: lighting effects, theme special effects, environment maps, and prop materials.

[0274] The scene adjustment device provided in the embodiments of the present application obtains state information of a target object within the virtual scene and adjusts the environmental information within the virtual scene based on the target object's state information. This allows for real-time adjustment of the environmental information within the virtual scene where the performer resides, based on the acquired state information of the target object. This enriches the representation of the environmental information of the performance space, providing users with more diverse visual effects and artistic appeal, meeting their diverse viewing needs, enhancing their visual experience, and also increasing the appeal and enjoyment of musical performances.

[0275] It should be understood that the device embodiment and the aforementioned method embodiment may correspond to each other, and similar descriptions can refer to the method embodiment. To avoid repetition, no further description is given here. Specifically, the device 400 shown in FIG7 can execute the method embodiment corresponding to FIG1, and the aforementioned and other operations and / or functions of each module in the device 400 are respectively for implementing the corresponding processes in each method in FIG1. ​​For the sake of brevity, no further description is given here.

[0276] The above describes the device 400 of the embodiment of the present application from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the first aspect method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the first aspect method disclosed in conjunction with the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above-mentioned first aspect method embodiment in conjunction with its hardware.

[0277] FIG8 is a schematic block diagram of an electronic device provided in an embodiment of the present application. As shown in FIG8 , the electronic device 500 may include:

[0278] The memory 510 and the processor 520 are configured to store a computer program and transmit the program code to the processor 520. In other words, the processor 520 can call and run the computer program from the memory 510 to implement the scene adjustment method in the embodiment of the present application.

[0279] For example, the processor 520 may be configured to execute the above-mentioned scene adjustment method embodiment according to the instructions in the computer program.

[0280] Optionally, the scene adjustment method includes:

[0281] Obtaining status information of target objects in the virtual scene;

[0282] Adjusting environmental information within the virtual scene according to the state information of the target object.

[0283] Optionally, the target objects include: performance content and user interactive operations;

[0284] Accordingly, the step of obtaining the state information of the target object in the virtual scene includes:

[0285] If the target object is the performance content, obtaining current music audio information and / or content type of the performance content in the virtual scene;

[0286] If the target object is the user interactive operation, an operation instruction, an interactive element or an interactive state of the user interactive operation in the virtual scene is obtained.

[0287] Optionally, when the target object is the performance content, adjusting the environmental information in the virtual scene according to the state information of the target object includes:

[0288] determining target environment information based on current music audio information and / or content type of the performance content;

[0289] Adjust the environmental information within the virtual scene according to the target environmental information.

[0290] Optionally, determining target environment information based on current music audio information of the performance content includes:

[0291] Determining the tempo and / or beat of the current music audio information;

[0292] The target environment information is determined according to the rhythm and / or beat of the current music audio information.

[0293] Optionally, determining the target environment information according to the rhythm and / or beat of the current music audio information includes:

[0294] The target environment information is determined according to the rhythm strength value and / or beat strength value of the current music audio information.

[0295] Optionally, determining the target environment information according to the rhythm strength value and / or beat strength value of the current music audio information includes:

[0296] Determining the target environmental information in a mapping relationship between the rhythm strength value and the environmental information according to the rhythm strength value of the current music audio information;

[0297] or,

[0298] Determining the target environmental information in a mapping relationship between the beat strength value and the environmental information according to the beat strength value of the current music audio information;

[0299] or,

[0300] According to the rhythm strength value and beat strength value of the current music audio information, the target environmental information is determined in the mapping relationship between the rhythm strength value, the beat strength value and the environmental information.

[0301] Optionally, determining the target environment information according to the rhythm strength value and / or beat strength value of the current music audio information includes:

[0302] Determining a rhythm strength value interval corresponding to the rhythm strength value of the current music audio information, and / or a beat strength value interval corresponding to the beat strength value of the current music audio information;

[0303] Determining the target environment information in a preset mapping relationship according to the rhythm strength value interval and / or the beat strength value interval;

[0304] Among them, the preset mapping relationship is the mapping relationship between rhythm strength value intervals and environmental information, the mapping relationship between beat strength value intervals and environmental information, or the mapping relationship between rhythm strength value intervals, beat strength value intervals and environmental information.

[0305] Optionally, determining target environment information according to the content type of the performance content includes:

[0306] According to the content type of the performance content, the target environment information is determined in a mapping relationship between the content type and the environment information.

[0307] Optionally, determining the target environment information according to the current music audio information and content type of the performance content includes:

[0308] Determining first environmental information corresponding to current music audio information of the performance content, and second environmental information corresponding to the content type of the performance content;

[0309] The first environment information and the second environment information are integrated to obtain the target environment information.

[0310] Optionally, when the target object is the user interactive operation, adjusting the environmental information in the virtual scene according to the state information of the target object includes:

[0311] Determining target environment information according to an operation instruction, interactive element, or interactive state of the user interactive operation;

[0312] Adjust the environmental information within the virtual scene according to the target environmental information.

[0313] Optionally, determining target environment information according to the operation instruction of the user interactive operation includes:

[0314] The target environment information is determined in response to a user's operation instruction on any environment parameter on a control panel, wherein the control panel is located in the virtual scene.

[0315] Optionally, determining target environment information according to the interactive state of the user interactive operation includes:

[0316] In response to an identity switching operation triggered by a user, switching the user's identity from a spectator to a performer;

[0317] The target environment information is determined according to the performance content of the user whose interactive state is a performer.

[0318] Optionally, before determining the target environment information based on the interactive element of the user interactive operation, the method further includes:

[0319] In response to an interactive operation triggered by a user, determining an interactive element corresponding to the interactive operation;

[0320] Accordingly, determining target environment information according to the interactive elements of the user interactive operation includes:

[0321] The interactive element is added to the current environment information of the interactive scene to obtain the target environment information.

[0322] Optionally, the target environment information includes at least one of the following: lighting effects, theme effects, environment maps, and prop materials.

[0323] In some embodiments of the present application, the processor 520 may include but is not limited to:

[0324] General-purpose processor, Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0325] In some embodiments of the present application, the memory 510 includes but is not limited to:

[0326] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0327] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 510 and executed by the processor 520 to implement the scene adjustment method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0328] As shown in FIG8 , the electronic device 500 may further include:

[0329] The transceiver 530 may be connected to the processor 520 or the memory 510 .

[0330] The processor 520 may control the transceiver 530 to communicate with other devices. Specifically, it may send information or data to other devices or receive information or data sent by other devices. The transceiver 530 may include a transmitter and a receiver. The transceiver 530 may further include an antenna, which may be one or more.

[0331] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0332] The present application also provides a computer storage medium having a computer program stored thereon, which enables the computer to perform the method of the above method embodiment when the computer program is executed by the computer.

[0333] An embodiment of the present application further provides a computer program product comprising program instructions, which, when executed on an electronic device, enables the electronic device to execute the method of the above method embodiment.

[0334] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0335] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0336] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0337] Modules described as separate components may or may not be physically separate, and components displayed as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected based on actual needs to achieve the purpose of the present embodiment. For example, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module.

[0338] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A scene adjustment method, comprising: Obtaining status information of target objects in the virtual scene; According to the state information of the target object, the environmental information in the virtual scene is adjusted.

2. The method according to claim 1, wherein the target object comprises: Performance content and user interaction operations; Accordingly, the step of obtaining the state information of the target object in the virtual scene includes: If the target object is the performance content, obtaining current music audio information and / or content type of the performance content in the virtual scene; If the target object is the user interactive operation, an operation instruction, an interactive element or an interactive state of the user interactive operation in the virtual scene is obtained.

3. The method according to claim 2, wherein when the target object is the performance content, adjusting the environment information in the virtual scene according to the state information of the target object comprises: Determining target environment information according to current music audio information and / or content type of the performance content; According to the target environment information, the environment information in the virtual scene is adjusted.

4. The method according to claim 3, wherein the determining the target environment information according to the current music audio information of the performance content comprises: Determining the rhythm and / or beat of the current music audio information; The target environment information is determined according to the rhythm and / or beat of the current music audio information.

5. The method according to claim 4, wherein determining the target environment information according to the rhythm and / or beat of the current music audio information comprises: The target environment information is determined according to the rhythm strength value and / or the beat strength value of the current music audio information.

6. The method according to claim 5, wherein determining the target environment information according to the rhythm strength value and / or the beat strength value of the current music audio information comprises: Determining the target environment information in a mapping relationship between the rhythm strength value and the environment information according to the rhythm strength value of the current music audio information; or, According to the beat strength value of the current music audio information, determining the target environment information in the mapping relationship between the beat strength value and the environment information; or, According to the rhythm strength value and the beat strength value of the current music audio information, the target environmental information is determined in the mapping relationship between the rhythm strength value, the beat strength value and the environmental information.

7. The method according to claim 5, wherein determining the target environment information according to the rhythm strength value and / or the beat strength value of the current music audio information comprises: Determine the rhythm strength value interval corresponding to the rhythm strength value of the current music audio information, and / or the current music The beat strength value interval corresponding to the beat strength value of the audio information; Determining the target environment information in a preset mapping relationship according to the rhythm strength value interval and / or the beat strength value interval; Among them, the preset mapping relationship is a mapping relationship between rhythm strength value intervals and environmental information, a mapping relationship between beat strength value intervals and environmental information, or a mapping relationship between rhythm strength value intervals, beat strength value intervals and environmental information.

8. The method according to claim 3, wherein determining the target environment information according to the content type of the performance content comprises: The target environment information is determined in a mapping relationship between content type and environment information according to the content type of the performance content.

9. The method according to claim 3, wherein the determining the target environment information according to the current music audio information and content type of the performance content comprises: Determining first environmental information corresponding to current music audio information of the performance content, and second environmental information corresponding to a content type of the performance content; The first environment information and the second environment information are integrated to obtain the target environment information.

10. The method according to claim 2, wherein when the target object is the user interactive operation, adjusting the environment information in the virtual scene according to the state information of the target object comprises: Determining target environment information according to an operation instruction, an interactive element or an interactive state of the user interactive operation; According to the target environment information, the environment information in the virtual scene is adjusted.

11. The method according to claim 10, wherein determining the target environment information according to the operation instruction of the user interactive operation comprises: The target environment information is determined in response to an operation instruction of a user on a control panel for any environment parameter, wherein the control panel is located in the virtual scene.

12. The method according to claim 10, wherein determining target environment information according to the interactive state of the user interactive operation comprises: In response to an identity switching operation triggered by a user, switching the identity of the user from a spectator to a performer; The target environment information is determined according to the performance content of the user whose interactive state is a performer.

13. The method according to claim 10, wherein before determining the target environment information according to the interactive element of the user interactive operation, the method further comprises: In response to an interactive operation triggered by a user, determining an interactive element corresponding to the interactive operation; Accordingly, determining target environment information according to the interactive element of the user interactive operation includes: The interactive element is added to the current environment information of the interactive scene to obtain the target environment information.

14. The method according to any one of claims 3 to 13, wherein the target environment information comprises at least one of the following: lighting effects, theme special effects, environment maps, and prop materials.

15. A scene adjustment device, comprising: An information acquisition module is used to obtain the status information of the target object in the virtual scene; The scene adjustment module is used to adjust the environment information in the virtual scene according to the state information of the target object.

16. An electronic device comprising: A processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to execute the scene adjustment method as claimed in any one of claims 1 to 14. 17 . A computer-readable storage medium, used for storing a computer program, wherein the computer program enables a computer to execute the scene adjustment method according to claim 1 . 18 . A computer program product comprising program instructions, wherein when the program instructions are executed on an electronic device, the electronic device executes the scene adjustment method according to claim 1 .

19. A computer program, when the computer program is executed on an electronic device, enables the electronic device to execute the scene adjustment method according to any one of claims 1 to 14.

Images