Two-dimensional display method, device, electronic equipment, storage medium and program product of sound object

Abstract

Embodiments of the present application provide a two-dimensional display method and device for a sound object, electronic equipment, storage medium and program product. The method comprises: in response to a first trigger operation, controlling a movable object operated by the first trigger operation to move according to an operation trajectory of a user, and playing a sound object sound effect corresponding to both the movable object and the operation trajectory in a preset physical space; wherein the first trigger operation represents that the user moves the movable object on a two-dimensional interface; in response to a second trigger operation, controlling the moving speed of the movable object at the current moment, performing subsequent movement, and playing a sound object sound effect corresponding to both the movable object and the subsequent movement trajectory corresponding to the subsequent movement in the preset physical space; wherein the second trigger operation represents that the user no longer touches the movable object. The method is used to enable the user to not only visually see the movement of the object, but also aurally feel the source and changing effect of the sound.

Description

Technical Field

[0001] This application relates to the field of audio processing, and more particularly to a two-dimensional display method, apparatus, electronic device, storage medium, and program product for sound objects. Background Technology

[0002] With the rapid development of digital media technology, the production and consumption of audio and video content have permeated all aspects of people's daily lives. Sound object technology, as a crucial means of enhancing the audio experience, is becoming increasingly important. However, currently, most users rely primarily on headphones or speaker systems to perceive the spatial distribution of sound when enjoying sound object content. While this method can provide a certain level of sound object effect, it lacks intuitive visual presentation, and users cannot control the sound objects. Therefore, it fails to fully meet users' needs for an immersive experience.

[0003] Therefore, there is an urgent need for a technological solution that allows users not only to visually see the movement of objects, but also to auditorily perceive the source and changes of sound. Summary of the Invention

[0004] This application provides a two-dimensional display method, apparatus, electronic device, storage medium, and program product for sound objects, so as to enable users to not only see the movement of objects visually based on two-dimensional display, but also to feel the source and changes of sound auditorily.

[0005] In a first aspect, embodiments of this application provide a two-dimensional display method for a sound object, comprising: responding to a first trigger operation, controlling a movable object operated by the first trigger operation to move according to a user's operation trajectory, and playing sound object sound effects corresponding to both the movable object and the operation trajectory within a preset physical space; wherein, the first trigger operation represents the user moving a movable object on a two-dimensional interface; responding to a second trigger operation, controlling the movement speed of the movable object at the current moment to perform subsequent movement, and playing sound object sound effects corresponding to both the movable object and the subsequent movement trajectory corresponding to the subsequent movement within the preset physical space; wherein, the second trigger operation represents the user no longer touching the movable object.

[0006] In one possible implementation, the moving speed of the movable object at multiple times is obtained; based on the moving speed of the movable object at the current time and the moving speed at the multiple times, a speed to be used is determined; if the speed to be used is determined to be greater than a preset speed threshold, the movable object is controlled to move according to the speed to be used.

[0007] In one possible implementation, the movable object is controlled to move in a straight line at a constant speed according to the speed to be used; or, the movable object is controlled to move in a straight line at a deceleration speed according to the speed to be used and the deceleration of the scene corresponding to the at least one movable object; or, the subsequent movement trajectory of a preset shape is generated according to the operation trajectory and the speed to be used, and the movable object is controlled to move according to the subsequent movement trajectory of the preset shape with the speed to be used as the starting speed.

[0008] In one possible implementation, if it is detected that the movable object touches the first edge of the two-dimensional interface during the subsequent movement, the movable object is controlled to bounce off the first edge of the two-dimensional interface and then move.

[0009] In one possible implementation, the angle between the subsequent movement trajectory of the movable object and the normal of the edge is determined as the angle of incidence; the straight line containing the reflection angle of the angle of incidence is determined as the rebound movement trajectory line of the movable object; the movable object is controlled to move along the rebound movement trajectory line after rebounding from the edge of the two-dimensional interface.

[0010] In one possible implementation, if the movable object is detected to touch the second edge of the two-dimensional interface during the subsequent movement, a first volume is determined to be the volume of the movable object inside the second edge; wherein the first edge is the edge constituting a first virtual space, the second edge is the edge constituting a second virtual space, the second virtual space is located inside the first virtual space, and the second virtual space is a virtual object in the first virtual space; the first volume is determined to be the display volume of the movable object displayed inside the second edge.

[0011] In one possible implementation, the volume of the movable object is adjusted in response to a third trigger operation on the movable object.

[0012] In one possible implementation, the display size of the aperture surrounding the movable object is adjusted in real time based on the volume of the sound effect of the movable object at the current moment.

[0013] In one possible implementation, in response to a user's selection operation on a second interface, the trajectory indicated by the selection operation is determined to be a first preset movement trajectory; wherein, the second interface is a two-dimensional interface; multiple trajectories are displayed on the second interface; the selection operation indicates that a trajectory should be selected as the first preset movement trajectory; wherein, the first preset movement trajectory is used for a movable object to move on the two-dimensional interface based on the preset movement trajectory.

[0014] In a possible implementation manner, in response to an adjustment instruction issued by a user based on the first preset movement trajectory, the adjustment instruction carries time adjustment information and speed adjustment information; a second preset movement trajectory is generated based on the adjustment instruction.

[0015] In a possible implementation manner, according to the somatosensory information to be sent of each movable object displayed on the two-dimensional interface, the hardware devices arranged in the preset physical space are controlled to send associated somatosensory information.

[0016] In a possible implementation manner, according to the somatosensory information to be sent of each movable object displayed on the two-dimensional interface, conflict information between the somatosensory information to be sent of the movable objects of the same category is determined, and according to the conflict information, the associated somatosensory information of the hardware devices is determined from the somatosensory information to be sent of the movable objects of the same category; the hardware devices arranged in the preset physical space are controlled to send the associated somatosensory information.

[0017] In a second aspect, an embodiment of the present application provides a two-dimensional display device for a sound object, including: a first control unit, configured to, in response to a first trigger operation, control the movable object operated by the first trigger operation to move according to a user's operation trajectory, and play a sound object sound effect corresponding to both the movable object and the operation trajectory in the preset physical space; wherein, the first trigger operation represents that the user moves a movable object on a two-dimensional interface; a second control unit, configured to, in response to a second trigger operation, control the moving speed of the movable object at the current moment for subsequent movement, and play a sound object sound effect corresponding to both the movable object and a subsequent movement trajectory corresponding to the subsequent movement in the preset physical space; wherein, the second trigger operation represents that the user no longer touches the movable object.

[0018] In a third aspect, an embodiment of the present application provides a playback device for a sound object, including: a memory, a processor;

[0019] The memory stores computer execution instructions;

[0020] The processor executes the computer execution instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementation manners of the first aspect.

[0021] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementation manners of the first aspect.

[0022] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0023] In a sixth aspect, embodiments of this application provide a two-dimensional display method for sound objects applied in a sound production system, comprising: responding to a first trigger operation triggered by a user in the sound production system, controlling a movable object operated by the first trigger operation to move on the interface displayed by the sound production system according to the user's operation trajectory, and playing a stereoscopic generated sound effect corresponding to both the movable object and the operation trajectory within a preset physical space; wherein, the first trigger operation represents the user moving the movable object on the two-dimensional interface; responding to a second trigger operation triggered by the user in the sound production system, controlling the movement speed of the movable object at the current moment, performing subsequent movement on the interface displayed by the sound production system, and playing a sound object sound effect corresponding to both the movable object and the subsequent movement trajectory corresponding to the subsequent movement within the preset physical space; wherein, the second trigger operation represents the user no longer touching the movable object.

[0024] In a seventh aspect, embodiments of this application provide a two-dimensional display method for sound objects applied in an interactive audio system, comprising: responding to a first trigger operation triggered by a user in the interactive audio system, controlling a movable object operated by the first trigger operation to move on the interface displayed by the interactive audio system according to the user's operation trajectory, and playing stereoscopic generated sound effects corresponding to both the movable object and the operation trajectory within a preset physical space; wherein, the first trigger operation represents the user moving the movable object on the two-dimensional interface; responding to a second trigger operation triggered by the user in the interactive audio system, controlling the movement speed of the movable object at the current moment, performing subsequent movement on the interface displayed by the interactive audio system, and playing sound object sound effects corresponding to both the movable object and the subsequent movement trajectory corresponding to the subsequent movement within the preset physical space; wherein, the second trigger operation represents the user no longer touching the movable object.

[0025] The two-dimensional display method, apparatus, electronic device, storage medium, and program product for sound objects provided in this application calculate and play corresponding sound object sound effects in real time based on the user's operation trajectory when the user touches or moves a movable object. These sound effects change according to the user's actions and the object's position to simulate sound changes in a real environment. When the user stops touching the movable object, the system continues to calculate and play subsequent sound object sound effects based on the object's current movement speed and direction. This ensures that even when the user does not directly touch the object, the sound changes still match the object's movement. Based on the user's operation trajectory and the object's movement state, the system continuously adjusts the parameters of the sound object sound effects to simulate sound changes in a real environment, achieving the effect that the user can not only visually see the object's movement based on a two-dimensional display but also auditorily perceive the source and changes of the sound. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0027] Figure 1 A schematic diagram of a two-dimensional display of the sound object provided in this application;

[0028] Figure 2 A flowchart illustrating the two-dimensional display of the sound object provided in this application. Figure 1 ;

[0029] Figure 3 Flowchart of the two-dimensional display method for the sound object provided in this application Figure 2 ;

[0030] Figure 4 An edge diagram illustrating the two-dimensional display method for the sound object provided in this application;

[0031] Figure 5 A virtual space diagram illustrating the two-dimensional display method for sound objects provided in this application;

[0032] Figure 6 A schematic diagram of the trajectory template for the two-dimensional display method of the sound object provided in this application;

[0033] Figure 7 A schematic diagram of the structure of a two-dimensional device for the sound object provided in this application;

[0034] Figure 8 A schematic diagram of the structure of a two-dimensional display device for the sound object provided in this application.

[0035] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0036] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0037] Figure 1 A scene illustration for the two-dimensional display of the sound object provided in this application, such as... Figure 1 As shown, the specific application scenarios of this application are as follows:

[0038] With the continuous development of audio technology, sound objects (including mono, stereo, immersive, surround sound, spatial audio, 3D sound effects, etc.) are widely used in music, film, games, and virtual reality. Sound objects provide users with a more realistic and immersive experience through the collaborative work of single or multiple channels. However, despite their already impressive sound performance, sound objects still have some limitations, one of which is that users cannot visually perceive the movement of sound.

[0039] Traditional sound object systems primarily play sound through speakers, allowing users to receive sound information only through their ears, without any corresponding visual feedback. This one-way information transmission method limits the user experience, making it difficult for them to accurately determine the source and path of the sound.

[0040] While stereo can simulate the spatial location of sound to some extent, users may still feel confused in complex environments, such as when multiple people are speaking at the same time or in multi-channel audio scenarios, and find it difficult to accurately track the specific location of each sound.

[0041] Existing sound object systems typically lack interactivity with users. Users cannot directly participate in the sound scene through visual means, which limits the possibility of immersive experiences.

[0042] To provide a more realistic auditory experience, users want to be able to see the movement trajectory of sound and control and edit sound objects through interaction. This would not only enhance spatial awareness but also improve the overall immersion.

[0043] Based on the scenarios described above, it is clear that existing technologies lack intuitive two-dimensional visual presentation and do not allow users to control sound objects. These technical issues prevent the full fulfillment of users' needs for an immersive experience.

[0044] The two-dimensional display method for sound objects provided in this application calculates and plays corresponding sound effects in real time based on the user's touch or movement of the movable object. These sound effects change according to the user's actions and the object's position to simulate sound changes in a real environment. When the user stops touching the movable object, the system continues to calculate and play subsequent sound effects based on the object's current movement speed and direction. This ensures that even when the user does not directly touch the object, the sound changes still match the object's movement. Based on the user's operation trajectory and the object's movement state, the system continuously adjusts the parameters of the sound effects to simulate sound changes in a real environment, achieving the effect that allows the user to not only visually see the object's movement based on a two-dimensional display but also auditorily perceive the source and changes of the sound.

[0045] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0046] Figure 2 A flowchart illustrating the two-dimensional display of the sound object provided in this application. Figure 1 ,like Figure 2 As shown, the method includes:

[0047] S201. In response to a first trigger operation, control the movable object operated by the first trigger operation to move according to the user's operation trajectory, and play sound effects of sound objects corresponding to both the movable object and the operation trajectory within the preset physical space; wherein, the first trigger operation represents the user moving a movable object on a two-dimensional interface;

[0048] For example, the two-dimensional interface includes at least one movable object, which is either in a moving state or a non-moving state. That is, when there are multiple movable objects in the two-dimensional interface, there are three possibilities: all movable objects are in a moving state, all movable objects are in a stationary state, or some of the movable objects are in a moving state and some are in a stationary state.

[0049] For example, the movable object can be a geometric shape with a preset area or volume: circle, rectangle, ring, sector, star, etc. The movable object can also be a two-dimensional appearance shape set according to the corresponding sound object effect of the movable object.

[0050] Optionally, the appearance of these geometries can be customized according to user preferences or scenario requirements, including attributes such as color and texture.

[0051] When a sound object is associated with a specific animal or object, the appearance of the movable object can be set accordingly to the shape of that animal or object.

[0052] For example, if the sound object is a frog's croak, the appearance of the movable object can be set to the shape of a two-dimensional frog, including skin texture, color, and motion pattern.

[0053] For example, the preset physical space can be any physical space suitable for playing sound effects, such as: a movie theater, opera house, concert hall, museum, car in P gear, private cinema, themed restaurant, amusement park, escape room, a room in a home environment, etc.

[0054] For example, the first triggering operation may be a single click, long press, or swipe gesture on a touchscreen device, or on a non-touchscreen device it may be an operation such as pressing a button, clicking or dragging a mouse.

[0055] When a user performs their first trigger action, their action leaves a certain path or trace in space and time; this is the operation trajectory. For example, when a user slides their finger on a touchscreen, the path the finger moves from the starting position to the ending position is the operation trajectory; in mouse operation, the path the mouse pointer moves is the operation trajectory. Optionally, the movable object can be controlled based on information such as the direction, distance, and speed of the operation corresponding to the operation trajectory.

[0056] When a user performs the first trigger action, their action leaves a path or trace in space and time; this is called the operation trajectory. For example, when a user double-clicks the mouse on a computer device and drags the mouse, they draw a two-dimensional operation trajectory in space, such as a horizontal plane (x, y).

[0057] Optionally, after obtaining the two-dimensional operation trajectory, the user can also adjust the height based on the operation trajectory on the horizontal plane, that is, the position in the direction perpendicular to the horizontal plane.

[0058] Optionally, the height adjustment method includes: the user drags any one or more points on the two-dimensional operation trajectory along a direction perpendicular to the horizontal plane, and the resulting trajectory is defined as the operation trajectory. Alternatively, the user drags any one point on the two-dimensional operation trajectory along a direction perpendicular to the horizontal plane to obtain a height value (z), and the entire operation trajectory corresponds to the height value (z).

[0059] S202. In response to the second trigger operation, control the movement speed of the movable object at the current moment, perform subsequent movement, and play the sound object sound effect corresponding to both the movable object and the subsequent movement trajectory within the preset physical space; wherein, the second trigger operation indicates that the user no longer touches the movable object.

[0060] The second trigger indicates that the user has stopped touching the movable object. This means the user has ended direct control of the movable object, and the object will enter a state of autonomous movement. At this point, the system needs to control its subsequent movement based on the movable object's current state and preset rules.

[0061] For example, the second trigger operation can be based on a time interval, such as after the user stops touching, after a short delay, the system starts to control the movable object according to preset rules; or it can be based on specific conditions, such as the user releasing the movable object at a speed greater than a preset value, and then initiating the subsequent movement of the movable object.

[0062] After the user stops touching the object, the system calculates a suitable subsequent movement speed based on information such as the object's current speed, direction, and acceleration. This speed may be related to the speed at which the user stopped touching the object, or it may be adjusted according to the needs of the scenario.

[0063] In an optional embodiment, the movable object may also correspond to a mono sound effect or a stereo sound effect. The sound object may also be: mono, stereo, immersive sound, surround sound, spatial audio, three-dimensional sound effect, etc.

[0064] The two-dimensional display method for sound objects provided in this application calculates and plays corresponding sound object sound effects in real time based on the user's operation trajectory when the user touches or moves the movable object. These sound effects change according to the user's actions and the object's position to simulate sound changes in a real environment. When the user stops touching the movable object, the system continues to calculate and play subsequent sound object sound effects based on the object's current movement speed and direction. This ensures that even when the user does not directly touch the object, the sound changes still match the object's movement. Based on the user's operation trajectory and the object's movement state, the system continuously adjusts the parameters of the sound object sound effects to simulate sound changes in a real environment, achieving the effect that the user can not only visually see the object's movement but also auditorily perceive the source and changes of the sound.

[0065] Figure 3 Flowchart of the two-dimensional display method for the sound object provided in this application Figure 2 ,like Figure 3 As shown, in this embodiment... Figure 2 Based on the embodiments, a method for displaying sound objects in two dimensions is described in detail, which includes:

[0066] S301. In response to the first trigger operation, control the movable object operated by the first trigger operation to move according to the user's operation trajectory, and play the sound object sound effect corresponding to both the movable object and the operation trajectory in the preset physical space; wherein, the first trigger operation represents the user moving the movable object on the two-dimensional interface.

[0067] For example, when a two-dimensional interface is displayed on a computer screen, the triggered operations may include:

[0068] Touch input: For devices that support touchscreens (such as laptops and tablets), users can directly touch elements on the screen with their fingers to trigger operations. This method is intuitive and easy to use.

[0069] Keyboard shortcuts / hotkeys: Users can quickly execute certain commands or functions by pressing specific key combinations on the keyboard, which is very useful in improving operational efficiency.

[0070] Voice control: With the development of technology, more and more applications are beginning to support voice recognition technology, allowing users to interact by speaking.

[0071] When a two-dimensional interface displays a projection on a projection device based on a screen, floor, or wall, the triggered operations may include:

[0072] Gesture recognition: This method uses cameras or other sensors to capture a user's hand movements in space and translates them into corresponding commands. It does not rely on physical contact and is suitable for long-distance operation.

[0073] Posture sensing: In addition to simple gestures, technologies such as depth cameras can be used to detect changes in human posture, such as bending over or turning around, and use these as control signals.

[0074] Shadow Interaction: By analyzing the shadow patterns formed between a light source and an object, unique interactive experiences can be created. When a user changes their position or shape, their projected shadow will change accordingly, triggering different responses.

[0075] S302. In response to the second triggering operation, obtain the movement speed of the movable object at multiple times; determine the speed to be used based on the movement speed of the movable object at the current time and the movement speed at the multiple times.

[0076] For example, if it is determined that the speed to be used is greater than a preset speed threshold, the movable object is controlled to move in a straight line at a constant speed according to the speed to be used; or, the movable object is controlled to move in a straight line at a deceleration speed according to the speed to be used and the deceleration of the scene corresponding to the at least one movable object; or, the subsequent movement trajectory of a preset shape is generated according to the operation trajectory and the speed to be used, and the movable object is controlled to move according to the subsequent movement trajectory of the preset shape with the speed to be used as the starting speed.

[0077] For example, the second trigger operation might indicate that the user no longer touches the movable object. That is, the movable object needs to transition from a state directly controlled by the user to a state of autonomous movement according to system rules. The system needs to adjust the movable object's movement state based on certain parameters. By acquiring the movement speed at different times, the system determines the movable object's subsequent trajectory and state changes. Specifically, the system records the movable object's speed information at different points in time. These time points can be evenly distributed, such as recording the speed every 0.1 seconds; or they can be dynamically adjusted according to the movable object's movement state, such as recording speed data more frequently during acceleration or deceleration phases.

[0078] The system collects movement speed data at multiple points in time and then analyzes the motion trend of the movable object. By analyzing this data, the system can understand how the movable object changes from its initial speed to its current speed and predict its possible future speed changes. In other words, it infers its motion pattern by observing its position and speed at different points in time.

[0079] The current movement speed is the speed of a movable object at the instant the user stops touching it. It represents the immediate state of the movable object when the user's operation ends. For example, when the user stops touching a movable object in virtual reality space, the speed of the movable object at that moment is its current movement speed.

[0080] For example, the system compares and analyzes the current movement speed with the movement speeds recorded at multiple previous moments. If the current speed is close to a certain stable speed range from the past, the system may continue to maintain a similar speed strategy; if the current speed changes significantly from the previous speed (such as a sudden acceleration or deceleration), the system will consider this trend and adjust the speed to be used.

[0081] For example: when the system determines that the speed to be used is greater than the preset speed threshold:

[0082] Alternatively, the movable object may move at a constant speed in a straight line. This means that the movable object will maintain its currently determined speed (the speed to be used) and continue moving along a straight line. In this case, the system does not need to make any additional adjustments to the speed of the movable object (such as acceleration or deceleration).

[0083] Optionally, if the speed to be used exceeds a preset speed threshold, in addition to uniform linear movement, decelerated linear movement can also be selected. The system will determine the deceleration based on the scene information of the movable object. For example, preset scene information includes: gravity, zero gravity, damped, undamped; or gravity and undamped, gravity and zero damped, zero gravity and damped, zero gravity and undamped, etc.

[0084] Optionally, the movement of the movable object can be controlled by generating a subsequent movement trajectory of a preset shape based on the operation trajectory and the speed to be used. The system will combine the user's previous operation trajectory (even if the user has stopped touching) and the current speed to be used to determine the future movement path of the movable object.

[0085] For example, when the movable object is a ball, after the player throws the ball (the movable object) (the user stops touching, triggering a second trigger operation), the ball has an initial velocity (the velocity to be used). The system generates a parabolic trajectory (a preset shape) based on the angle at which the ball is launched (the trajectory) and its initial velocity. The player then controls the ball to move along this generated parabolic trajectory, starting from this initial velocity. This makes the motion of the movable object not just a simple uniform or decelerating linear motion, but a more complex curvilinear motion.

[0086] S303. If it is detected that the movable object touches the first edge of the two-dimensional interface during the subsequent movement, the movable object is controlled to bounce off the first edge of the two-dimensional interface and then move.

[0087] By continuously monitoring the position information of movable objects, it can be determined whether they have touched the edge of the two-dimensional interface. Optionally:

[0088] When the two-dimensional interface is displayed in full screen on the display device, the coordinates of the center point of the movable object can be compared with the coordinates of the screen boundary.

[0089] like Figure 4 As shown, Figure 4 This is an edge diagram of the two-dimensional display method for the sound object provided in this application. By continuously obtaining the center point coordinates (x, y) of the movable object, when the x coordinate is equal to the left or right boundary coordinate of the window, or the y coordinate is equal to the top or bottom boundary coordinate of the window, it can be determined that the icon touches the edge of the two-dimensional interface.

[0090] When the two-dimensional interface is not displayed in full screen on the display device, the coordinates of the center point of the movable object can be compared with the coordinates of the boundary of the two-dimensional interface.

[0091] Alternatively, the two-dimensional interface can also be a display interface projected onto the ground or a screen by a projection device.

[0092] The first step of S303 is to determine the angle between the subsequent movement trajectory of the movable object and the normal of the edge, which is the angle of incidence; to determine the straight line where the reflection angle of the incident angle is located, which is the rebound movement trajectory line of the movable object; and to control the movable object to move along the rebound movement trajectory line after rebounding from the edge of the two-dimensional interface.

[0093] When a movable object is detected to touch the edge of a 2D interface during its subsequent movement, the first step is to determine the angle between the subsequent movement trajectory of the movable object and the normal to the edge of the 2D interface. This angle is the angle of incidence.

[0094] The angle of incidence can be calculated by obtaining the direction vector of the movable object's trajectory and the direction vector of the edge normal of the 2D interface, and then using the formula for the angle between the vectors. For example, suppose the direction vector of the movable object's trajectory is... The direction vector of the first edge normal is The incident angle θ can then be calculated using formula (1):

[0095]

[0096] The system controls movable objects to bounce off the edges of the 2D interface along a predetermined bounce trajectory. This involves updating the position information of the movable object in each frame or time interval to ensure smooth movement along the bounce trajectory.

[0097] For example, the bounce direction is determined based on the position of the movable object touching the edge and the angle of impact. If the collision is perpendicular to the edge, the bounce direction will be directly perpendicular to the edge plane; if the collision is at an angle, the bounce direction is calculated based on the angle of incidence and the normal direction of the edge. When the movable object is associated with material information, energy loss during the collision process, such as friction and air resistance, can also be calculated. The bounce speed is usually less than the initial velocity. A bounce coefficient can be used to adjust the speed; the value of the bounce coefficient ranges from 0 to 1, and the specific value can be determined based on the properties of the movable object (such as mass and material) and the properties of the impact surface (such as roughness and hardness).

[0098] Based on the calculated rebound direction and speed, the system updates the trajectory of the movable object. For example, if the movable object was originally moving along the positive x-axis, after colliding with the right edge, it may move along the negative x-axis at a new speed based on the calculated rebound direction and speed.

[0099] In an alternative embodiment, lighting effects, such as highlight flashes and sparks, are also included at the moment the movable object collides with the edge.

[0100] The second step of S303: If it is detected that the movable object touches the second edge of the two-dimensional interface during the subsequent movement, then the first volume of the movable object entering the second edge is determined; wherein, the first edge is the edge constituting the first virtual space, the second edge is the edge constituting the second virtual space, and the second virtual space is located inside the first virtual space; if the two-dimensional interface has the second edge, then the space inside the second virtual space is the user-visible interface; the display volume of the movable object displayed inside the second edge is determined according to the first volume.

[0101] like Figure 5 As shown, Figure 5 This is a schematic diagram of the virtual space for the two-dimensional display method of the sound object provided in this application. Specifically, there exists a first virtual space and a second virtual space, wherein the first virtual space is formed by a first edge, and the second virtual space is formed by a second edge. The second virtual space is located inside the first virtual space. Movable objects are allowed to move freely within the first virtual space, but only the second virtual space is the user-visible spatial interface.

[0102] Optionally, the first virtual space and the second virtual space can be three-dimensional or two-dimensional. If the first virtual space and the second virtual space are two-dimensional, the two-dimensional display interface directly displays the two-dimensional image corresponding to the second virtual space. If the first virtual space and the second virtual space are three-dimensional, the two-dimensional display interface can display the projection of a movable object on any cross-section corresponding to the second virtual space.

[0103] Since the first virtual space is greater than the second virtual space, when a movable object moves within the first virtual space, the following three scenarios will occur:

[0104] All movable objects are located inside the second virtual space: then the user can see the complete movable objects and their movement trajectories.

[0105] All movable objects are located in the interlayer between the first virtual space and the second virtual space: users will not be able to see the movable objects, nor will they be able to see the movement trajectory of the movable objects.

[0106] A portion of the movable object is located within a layer between the first and second virtual spaces, while another portion is located within the second virtual space. The user can see the portion of the movable object located within the second virtual space, but cannot see the portion located within the layer between the first and second virtual spaces.

[0107] In practical applications, if a movable object moves within the first virtual space and then moves from the second virtual space into the space between the first and second virtual spaces, the user will see the movable object "penetrate" through the second edge corresponding to the second virtual space.

[0108] The first edge constitutes the edge of the first virtual space, while the second edge constitutes the edge of the second virtual space located within the first virtual space. Furthermore, if the two-dimensional interface has a second edge, then the space within the second virtual space is the user-visible interface. When a movable object is detected touching the second edge of the two-dimensional interface during subsequent movement, it is necessary to determine the first volume within the second edge that the movable object has entered.

[0109] The first volume can be determined in several ways, such as by calculating it based on geometry. If the second virtual space corresponding to the second edge is a regular geometric shape (such as a cuboid, sphere, etc.), the first volume can be calculated using the corresponding geometric formula based on the relative position, size, and shape of the movable object and the second edge.

[0110] In practical applications, such as movie theater scenarios, in addition to the main characters' dialogues and singing, there may also be narration (off-screen voice-over). During the user's viewing experience, the main characters' dialogues and singing can exist within a second virtual space, allowing the user to directly see the sound objects, while the off-screen voice-over exists in the interlayer between the first and second virtual spaces. That is, the user can hear the sound but cannot see its movement.

[0111] For example, in a scenario where music is played via an in-vehicle terminal, the space inside the car constitutes a second virtual space. Within this space, passengers can directly perceive the presence of the music and visually see that the sound object currently corresponds to a specific location within the car. The area outside this second virtual space represents the external environment. In this case, the size of the first virtual space can be set based on the farthest propagation distance of the sound.

[0112] S304. In response to a third trigger operation on the movable object, adjust the volume of the movable object.

[0113] Optionally, adjusting the volume of the movable object can be achieved by adjusting the adjustment control corresponding to the movable object, or by gestures, such as pinch or magnify gestures.

[0114] Optionally, users can influence the volume of a movable object through some form of interaction. This interaction can be varied, such as scrolling with a mouse wheel, using gestures, clicking and dragging a slider, swiping by touching the screen, or using specific keys on a keyboard, etc.

[0115] Taking control adjustment as an example, if the system determines that the user has performed a third trigger operation, it will immediately display the first control corresponding to the movable object in the two-dimensional interface. This first control is specifically designed for adjusting the volume of the movable object, and its appearance and display method can be adjusted according to the interface layout and user experience design. For example, the first control may appear as a small volume adjustment slider near the movable object, or it may appear as a floating volume adjustment panel in a fixed position on the interface, and its appearance and style will be adapted to the overall style of the two-dimensional interface.

[0116] The first control is specifically designed to adjust the volume of movable objects. When the user adjusts the volume using the first control, the movable object will adjust its audio output volume in real time according to the user's operation. If the user increases the volume using the first control, the stereo sound of the movable object will increase accordingly; conversely, if the volume is decreased, the audio sound will decrease. This volume adjustment function provides users with a more personalized audio experience, allowing them to control the audio intensity according to their needs and the environment.

[0117] Taking gesture adjustment or cursor adjustment as an example, the user selects a movable object and makes a pinch or expand gesture, or makes an action to shrink or enlarge the area of ​​the movable object, and then determines the volume corresponding to the current movable object based on the scaling ratio of the movable object.

[0118] For example, users control volume using gestures on touchscreen devices, employing two fingers to "pinch" or "spread" on the screen. A pinch typically involves bringing two fingers closer together, while a spread involves moving them further apart. As the user pinches or spreads, the selected object shrinks or enlarges accordingly. For instance, a volume slider might become longer (increase volume) or shorter (decrease volume) depending on the user's gesture. If the area of ​​the selected movable object represents volume, it might become larger (increase volume) or smaller (decrease volume) depending on the user's gesture.

[0119] Regarding volume adjustment, for example, in a playback scenario with four movable objects (A, B, C, and D), each corresponding to a sound effect, the user can adjust the volume of movable object A and movable object B to 50 dB, and the volume of movable object C and movable object D to 55 dB. Optionally, the user can also adjust the volume of the movable objects based on a percentage of the maximum playable volume of the playback device. For example, if the maximum playable volume of a playback device is 90 dB, the user can choose to adjust the volume of movable object A and movable object B to 50%, which is 45 dB, and the volume of movable object C and movable object D to 60%, which is 54 dB.

[0120] For example, a user selects a movable object and makes a pinch or expand gesture based on the touch screen device, or makes an action to shrink or enlarge the area of ​​the movable object, and then determines the volume corresponding to the movable object based on the scaling ratio of the movable object.

[0121] For example, the display size of the aperture surrounding the movable object is adjusted in real time according to the volume of the sound effect of the movable object at the current moment.

[0122] Optionally, the action of adjusting the volume is triggered in the following situations:

[0123] Users might click the grip area on the slider with the mouse pointer and then drag it left or right along the slider's track. This dragging action is the action of adjusting the volume.

[0124] The system determines whether a volume adjustment action is triggered based on preset rules and logic. These rules may include factors such as the direction of the operation, the magnitude of the operation, and the duration of the operation.

[0125] For example, when performing a mouse wheel scrolling operation, the system determines whether the scrolling direction is upward (usually indicating an increase in volume) or downward (usually indicating a decrease in volume), as well as the scrolling speed and duration. If the scrolling direction is upward, and the scrolling speed is fast and the duration is short, it may be interpreted as a rapid volume increase command; while if the scrolling direction is downward, and the scrolling is slow and lasts for a period of time, it may be an operation to slowly decrease the volume. Similarly, for touch screen swipe operations, the system also determines the user's intent based on the direction and distance of the swipe to ensure accurate recognition of the volume adjustment action.

[0126] It should be noted that multiple virtual playback scenes can be preset, and each scene includes various preset sound effects and movable objects, for example:

[0127] Forest scene: bird song sound effects, flowing water sound effects (gurgling water, splashing water), leaf sound effects, insect chirping sound effects.

[0128] Street scene: Car sound effects (engine roar, tire friction with the ground), horn sound effects, pedestrian sound effects (sounds of walking and talking on the street).

[0129] Seaside scene: sound effects of ocean waves (the sound of waves crashing against the shore, including the sound of surging waves and the sound of waves receding), and sound effects of seagulls (the sound of seagulls soaring and calling on the sea).

[0130] Meditation scenarios: voice guidance, meditation sound effects (wooden fish sound effects, xylophone sound effects, singing bowl sound effects, etc.).

[0131] Performance scenarios: guitar sound effects (including string sounds and plucking sounds), piano sound effects, violin sound effects (bow string friction sounds, high and low pitch transitions), etc.

[0132] Film and television sound effects: bullet sound effects (the sound of bullets being fired, flying, and hitting targets in gunfight movies), airplane sound effects (the sound of airplanes taking off, flying, and landing, including engine roars, whistling air, etc.), horse hoof sound effects (the sound of horses running in ancient wars or Western movies), etc.

[0133] Entertainment scenes: horror sound effects, door creaking sound effects, celebration sound effects, etc.

[0134] Users can choose preset playback scenes based on their preferences.

[0135] Optionally, in a meditation setting, the voice guidance can be the voice guidance for different meditation courses, thereby guiding the user's breathing rate or adjusting body posture, etc.

[0136] To better experience the meditation course, users may prefer a louder voice guidance. In this case, they can select the movable object corresponding to the voice guidance and increase its volume, while lowering the volume of the meditation sound effects. Alternatively, users may prefer a more immersive meditation music experience for relaxation. In this case, they can select the movable object corresponding to the meditation sound effects and increase its volume, while lowering the volume of the voice guidance.

[0137] Based on the different scenarios mentioned above, users can adjust the volume of movable objects with different sound effects according to the needs of different scenarios, so as to achieve a more immersive music experience.

[0138] Optionally, in the above scenarios, users can specify the playback duration of the scenario via a timer. For example, if a user needs to meditate for half an hour, after selecting the meditation scenario, the user can set a 30-minute countdown, and exit the meditation scenario when the countdown ends.

[0139] It should be noted that each first control is associated with the volume adjustment function of a movable object. This means there is a one-to-one correspondence; only a specific first control can adjust the volume of its corresponding movable object.

[0140] For example, in a two-dimensional virtual scene, there are multiple movable objects, each with its own sound characteristics and volume control requirements. When a user wants to adjust the volume of a specific movable object, they need to perform a volume adjustment operation on that object. This mapping relationship makes volume adjustment targeted and will not affect the volume settings of other movable objects.

[0141] This design allows users to easily and precisely adjust the volume of movable objects using the primary control, creating a sound environment that better suits their needs within the two-dimensional interface. Whether in virtual reality music playback, immersive audio experiences, or other application scenarios, the volume can be flexibly controlled according to the actual situation, enhancing the realism and fun of the interaction.

[0142] In one possible implementation, the display size of the aperture surrounding the movable object is adjusted in real time based on the volume of the sound effect of the movable object at the current moment.

[0143] S305. In response to the user's selection operation on the second interface, determine that the trajectory indicated by the selection operation is a first preset movement trajectory, wherein the first preset movement trajectory carries preset time information and preset speed information; wherein, the first preset movement trajectory is used for the movable object to move on the two-dimensional interface based on the first preset movement trajectory.

[0144] When users see multiple tracks displayed on the second interface, they will select one based on their needs and intentions. This selection can be done in various ways, such as clicking on a track with a mouse, swiping to select on a touchscreen, or specifying a track via voice command. Regardless of the method used, the user's goal is to choose the track that interests them from the many available options.

[0145] Optionally, the second interface is a pre-set interface that includes various trajectory templates. Figure 6 A schematic diagram of the trajectory template for the two-dimensional display method of the sound object provided in this application, as shown below. Figure 6 As shown:

[0146] The system will respond to the user's selection and determine the selected trajectory as the first preset movement trajectory. This first preset movement trajectory is not merely a path; it also carries preset time and speed information. The preset time information indicates the duration of the first preset movement trajectory.

[0147] S306. In response to an adjustment command issued by the user based on the first preset movement trajectory, wherein the adjustment command carries time adjustment information and speed adjustment information; generate a second preset movement trajectory based on the adjustment command.

[0148] The system modifies and optimizes the first preset movement trajectory based on the time and speed adjustment information in the adjustment command. For time adjustment, if the user requests to arrive at a certain location ahead of schedule, the system may find a more direct path or increase the speed of the movable object; if the arrival is delayed, it may appropriately reduce the speed or add some detours. For speed adjustment, the system adjusts the curvature and length of the corresponding sections of the trajectory according to the speed requirements specified by the user at different stages, ensuring that the movable object can move smoothly according to the new speed requirements.

[0149] S307. Based on the motion sensing information to be emitted from each movable object displayed on the two-dimensional interface, control the hardware device set in the preset physical space to emit linkage motion sensing information.

[0150] For example, based on the motion sensing information to be emitted by each movable object displayed on the two-dimensional interface, conflict information between the motion sensing information to be emitted by movable objects of the same category is determined. Based on the conflict information, the linkage motion sensing information of the hardware device is determined from the motion sensing information to be emitted by movable objects of the same category. The hardware device set in the preset physical space is controlled to emit the linkage motion sensing information.

[0151] For example, different movable objects correspond to different sound effects. Based on the different sound effect types, there are pre-defined associations with the hardware devices set up in the physical space. For example, the pre-defined hardware devices in the physical space include: audio equipment, aromatherapy devices, massage chair equipment, ambient lighting equipment, etc., as shown in Table 1.

[0152] Table 1

[0153]

[0154]

[0155] Each movable object corresponds to at least two association modes: the default mode and the user-defined mode.

[0156] For example: a movable pair in the shape of a campfire ball is preset, corresponding to campfire sound effects, corresponding to an air conditioner temperature of 25°C, releasing warm fragrance, adjusting the massage chair to a preset angle, and turning on the firelight jumping ambient light mode.

[0157] If both ice / snow spheres and campfire spheres exist simultaneously, the preset second massage chair angle corresponding to the ice / snow sphere is inconsistent with the preset first massage chair angle corresponding to the campfire sphere; the ice / snow sphere corresponds to an air conditioner temperature of 16℃ and a cold-type fragrance, while the campfire sphere corresponds to an air conditioner temperature of 25℃ and a warm-type fragrance, indicating inconsistent temperatures. These can be identified as conflicting information.

[0158] Regarding temperature perception information, a temperature conflict is identified if the temperature settings of different areas (represented by different movable objects) differ by more than a certain threshold (e.g., ±2℃). For massage chair angle perception information, an angle conflict is considered when the difference in massage chair angle settings associated with different movable objects exceeds a specific angle (e.g., ±10°). Regarding fragrance, a fragrance conflict is considered to exist if different areas expect different types of fragrance to be released, or if the concentration of the same fragrance differs significantly (e.g., more than 30%). Lighting conflict is determined based on the different needs of different areas for light brightness (difference exceeding 20%), color (color difference exceeding a certain standard), or mode (e.g., flicker frequency difference greater than 50%).

[0159] If a temperature conflict is detected, a global temperature balancing strategy is prioritized. The average temperature of all conflicting areas is calculated, and then the temperature control devices (such as air conditioners) in each area are adjusted to near this average temperature. For example, if the temperature in the campfire-shaped ball area is 25℃ and the temperature in the ice and snow-shaped ball area is 20℃, and the calculated average temperature is 22.5℃, then the air conditioning temperatures in both areas are adjusted to approximately 22.5℃.

[0160] When a conflict arises regarding the angle of the massage chair, consider the user's usage frequency and habits. If a user frequently uses a particular angle, then when a conflict occurs, prioritize adjusting the massage chair angle to that user's preferred angle. For example, if a user frequently uses the 45° angle, adjust the massage chair angle to 45° when an angle conflict occurs.

[0161] You can also set default massage chair angles for different functional areas, such as rest areas and entertainment areas (determined based on the attributes of the preset physical space). When a conflict occurs, the angles are adjusted according to the priority of the functional area. For example, the massage chair angle in the rest area will prioritize the specific angle that meets the relaxation needs when there is a conflict.

[0162] To address fragrance clashes, a zoned release strategy can be employed. Based on the intended use of different areas and user needs, fragrances can be divided into different diffusion zones. For example, a warm, woody fragrance can be released in the campfire-shaped sphere area, while a fresh, cool fragrance can be released in the snow-shaped sphere area, reducing interference between fragrances.

[0163] Alternatively, based on fragrance compatibility, choose a complex fragrance that can blend multiple scents to create a relatively harmonious overall atmosphere.

[0164] When lighting conflicts occur, adjustments should be made based on the scene's lighting requirements and atmosphere. In scenes requiring a bright environment (such as activity areas), prioritize higher brightness and more active lighting patterns (such as rapid flashing) to unify the lighting across the entire scene. For example, adjust the brightness of the lights in the campfire-shaped and snow-shaped sphere areas to a higher level, and unify the flashing frequency to a faster pattern.

[0165] In areas where a warm and soft atmosphere is needed (such as rest areas), reduce the overall brightness and use soft light colors and slow flickering or non-flickering modes to make the lighting in different areas harmonious and consistent under this atmosphere.

[0166] Optionally, a user preference database can be established to record each user's preference for different sensory information. For example, some users prefer higher temperatures and stronger fragrances, while others prefer lower temperatures and lighter fragrances.

[0167] When a conflict in sensory information is detected, the central control system queries the user preference database and adjusts the sensory information based on the user's location and historical preferences within the scene. For example, if a user who prefers high temperature and strong fragrance is near the campfire-shaped ball area, while a user who prefers low temperature and light fragrance is near the snow-shaped ball area, when a temperature and fragrance conflict occurs, the temperature and fragrance concentration in the campfire-shaped ball area are appropriately increased, while the relatively low temperature and fragrance concentration in the snow-shaped ball area are maintained.

[0168] After resolving the conflict, continuously monitor changes in haptic feedback. If a new conflict arises, promptly re-execute the conflict resolution strategy. For example, when a user moves within the scene, it may cause a new conflict to arise in the previously resolved haptic feedback, requiring the system to reassess and adjust accordingly. Establish user feedback channels, such as feedback buttons or on-site feedback terminals. Users can provide feedback at any time regarding the comfort level of their current haptic feedback, allowing for further optimization of the haptic feedback adjustment strategy. For example, if a user reports that the fragrance in a certain area is too strong, even if no fragrance conflict is currently detected, the system will appropriately reduce the fragrance concentration in that area.

[0169] The two-dimensional display method, apparatus, electronic device, storage medium, and program product for sound objects provided in this application calculate and play corresponding sound object sound effects in real time based on the user's operation trajectory when the user touches or moves a movable object. These sound effects change according to the user's actions and the object's position to simulate sound changes in a real environment. When the user stops touching the movable object, the system continues to calculate and play subsequent sound object sound effects based on the object's current movement speed and direction. This ensures that even when the user does not directly touch the object, the sound changes still match the object's movement. Based on the user's operation trajectory and the object's movement state, the system continuously adjusts the parameters of the sound object sound effects to simulate sound changes in a real environment, achieving the effect that the user can not only visually see the object's movement based on a two-dimensional display but also auditorily perceive the source and changes of the sound.

[0170] Figure 7 A schematic diagram of the structure of the two-dimensional device for the sound object provided in this application, as shown below. Figure 7 As shown, the two-dimensional device 70 for a sound object provided in this embodiment includes:

[0171] The first control unit 701 is configured to respond to a first trigger operation, control the movable object operated by the first trigger operation to move according to the user's operation trajectory, and play sound effects of sound objects corresponding to both the movable object and the operation trajectory within the preset physical space; wherein, the first trigger operation represents the user moving the movable object on the two-dimensional interface;

[0172] The second control unit 702 is used to respond to the second trigger operation, control the movement speed of the movable object at the current moment, perform subsequent movement, and play sound object sound effects corresponding to both the movable object and the subsequent movement trajectory within the preset physical space; wherein, the second trigger operation indicates that the user no longer touches the movable object.

[0173] In one possible implementation, the second control unit 702 is configured to: acquire the moving speed of the movable object at multiple times; determine a speed to be used based on the moving speed of the movable object at the current time and the moving speed at the multiple times; and if the speed to be used is determined to be greater than a preset speed threshold, control the movable object to move according to the speed to be used.

[0174] In one possible implementation, the second control unit 702 is further configured to: control the movable object to move in a uniform linear motion according to the speed to be used; or control the movable object to move in a decelerated linear motion according to the speed to be used and the deceleration of the scene corresponding to the at least one movable object; or generate the subsequent movement trajectory of a preset shape according to the operation trajectory and the speed to be used, and control the movable object to move according to the subsequent movement trajectory of the preset shape with the speed to be used as the starting speed.

[0175] In one possible implementation, the device further includes a detection unit, configured to, if the movable object is detected to have touched the edge of the two-dimensional interface during the subsequent movement, control the movable object to bounce off the edge of the two-dimensional interface and then move.

[0176] In one possible implementation, the detection unit is further configured to: determine the angle between the subsequent movement trajectory of the movable object and the normal of the edge, which is the angle of incidence; determine the straight line where the reflection angle of the angle of incidence is located, which is the rebound movement trajectory line of the movable object; and control the movable object to move along the rebound movement trajectory line after rebounding from the edge of the two-dimensional interface.

[0177] In one possible implementation, the device further includes a volume unit for adjusting the volume of the movable object in response to a third triggering operation on the movable object.

[0178] In one possible implementation, the device further includes an adjustment unit for adjusting the display size of the aperture surrounding the movable object in real time based on the volume of the sound effect of the movable object at the current moment.

[0179] In one possible implementation, the apparatus further includes: a template unit, configured to determine, in response to a user's selection operation on a second interface, that the trajectory indicated by the selection operation is a first preset movement trajectory; wherein the second interface is a two-dimensional interface; multiple trajectories are displayed on the second interface; the selection operation indicates that a trajectory should be selected as the first preset movement trajectory; wherein the first preset movement trajectory is used for a movable object to move on the two-dimensional interface based on the first preset movement trajectory.

[0180] In one possible implementation, the template unit is further configured to, in response to an adjustment command issued by a user based on the first preset movement trajectory, the adjustment command carrying time adjustment information and speed adjustment information, generate a second preset movement trajectory based on the adjustment command.

[0181] In one possible implementation, the device further includes: a motion sensing unit, used to control the hardware device set in the preset physical space to emit linkage motion sensing information based on the motion sensing information to be emitted from each movable object displayed on the two-dimensional interface.

[0182] In one possible implementation, the motion sensing unit is further configured to determine conflict information between the motion sensing information to be emitted by movable objects of the same category based on the motion sensing information to be emitted by each movable object displayed on the two-dimensional interface, and to determine the linkage motion sensing information of the hardware device from the motion sensing information to be emitted by movable objects of the same category based on the conflict information; and to control the hardware device set in the preset physical space to emit the linkage motion sensing information.

[0183] The two-dimensional display device for the sound object provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0184] This application provides a two-dimensional display method for sound objects applied in a sound production system, comprising: responding to a first trigger operation triggered by a user in the sound production system, controlling a movable object operated by the first trigger operation to move on the interface displayed by the sound production system according to the user's operation trajectory, and playing a stereoscopic generated sound effect corresponding to both the movable object and the operation trajectory within a preset physical space; wherein, the first trigger operation represents the user moving the movable object on the two-dimensional interface; responding to a second trigger operation triggered by the user in the sound production system, controlling the movement speed of the movable object at the current moment, performing subsequent movement on the interface displayed by the sound production system, and playing a sound object sound effect corresponding to both the movable object and the subsequent movement trajectory within the preset physical space; wherein, the second trigger operation represents the user no longer touching the movable object.

[0185] This application provides a two-dimensional display method for sound objects in an interactive audio system, comprising: responding to a first trigger operation triggered by a user in the interactive audio system, controlling a movable object operated by the first trigger operation to move on the interface displayed by the interactive audio system according to the user's operation trajectory, and playing stereoscopic generated sound effects corresponding to both the movable object and the operation trajectory within a preset physical space; wherein, the first trigger operation represents the user moving the movable object on the two-dimensional interface; responding to a second trigger operation triggered by the user in the interactive audio system, controlling the movement speed of the movable object at the current moment, performing subsequent movement on the interface displayed by the interactive audio system, and playing sound object sound effects corresponding to both the movable object and the subsequent movement trajectory within the preset physical space; wherein, the second trigger operation represents the user no longer touching the movable object.

[0186] Figure 8 A schematic diagram of the structure of a two-dimensional display device for the sound object provided in this application. Figure 8 As shown, the electronic device 80 provided in this embodiment includes at least one processor 801 and a memory 802. Optionally, the device 80 further includes a communication component 803. The processor 801, memory 802, and communication component 803 are connected via a bus 804.

[0187] In a specific implementation, at least one processor 801 executes computer execution instructions stored in memory 802, causing at least one processor 801 to perform the above-described method.

[0188] The specific implementation process of processor 801 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0189] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0190] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0191] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0192] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0193] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0194] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0195] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0196] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0197] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0198] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0199] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0200] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0201] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for displaying a two-dimensional sound object, characterized in that, The method is applied to a terminal device located within a preset physical space. The terminal device displays a two-dimensional interface, which includes at least one movable object. The movable object is in a moving state or a non-moving state. The method includes: In response to a first trigger operation, the movable object operated by the first trigger operation is controlled to move according to the user's operation trajectory, and sound effects of sound objects corresponding to both the movable object and the operation trajectory are played in the preset physical space; wherein, the first trigger operation represents the user moving the movable object on the two-dimensional interface; In response to the second trigger operation, the movement speed of the movable object at the current moment is controlled to perform subsequent movement, and sound effects of sound objects corresponding to both the movable object and the subsequent movement trajectory are played in the preset physical space; wherein, the second trigger operation indicates that the user no longer touches the movable object.

2. The method according to claim 1, characterized in that, Based on the moving speed of the movable object at the current moment, subsequent movement is performed, including: Obtain the movement speed of the movable object at multiple times; determine the speed to be used based on the movement speed of the movable object at the current time and the movement speed at the multiple times. If it is determined that the speed to be used is greater than a preset speed threshold, the movable object is controlled to move according to the speed to be used.

3. The method according to claim 2, characterized in that, Controlling the movable object to move according to the speed to be used includes: The movable object is controlled to move at a constant linear speed according to the speed to be used; Alternatively, the movable object can be controlled to decelerate and move in a straight line according to the speed to be used and the deceleration of the scene corresponding to the at least one movable object; Alternatively, based on the operation trajectory and the speed to be used, a subsequent movement trajectory of a preset shape is generated, and the movable object is controlled to move according to the subsequent movement trajectory of the preset shape, starting from the speed to be used.

4. The method according to claim 1, characterized in that, The method further includes: If it is detected that the movable object touches the first edge of the two-dimensional interface during the subsequent movement, the movable object is controlled to bounce off the first edge of the two-dimensional interface and then move.

5. The method according to claim 4, characterized in that, Controlling the movable object to bounce off the edge of the two-dimensional interface and then move includes: The angle between the subsequent movement trajectory of the movable object and the normal of the edge is determined as the angle of incidence; the straight line containing the reflection angle of the angle of incidence is determined as the rebound movement trajectory line of the movable object. The movable object is controlled to move along the bounce trajectory line after bouncing off the edge of the two-dimensional interface.

6. The method according to claim 4, characterized in that, The method further includes: If it is detected that the movable object touches the second edge of the two-dimensional interface during the subsequent movement, determine the first volume inside the second edge entered by the movable object; wherein, the first edge is the edge constituting the first virtual space, the second edge is the edge constituting the second virtual space, the second virtual space is located inside the first virtual space, and the second virtual space is a virtual object in the first virtual space. Determine the first volume as the display volume of the movable object displayed inside the second edge.

7. The method according to claim 1, characterized in that, The method further includes: In response to a third trigger operation on the movable object, adjust the volume of the movable object.

8. The method according to claim 1, characterized in that, The method further includes: According to the volume of the sound object sound effect of the movable object at the current moment, adjust the display size of the aperture around the movable object in real time.

9. The method according to any one of claims 1-8, characterized in that, The method further includes: In response to a selection operation by the user on the second interface, determine that the trajectory indicated by the selection operation is a first preset movement trajectory, and the first preset movement trajectory carries preset time information and preset speed information. Wherein, the first preset movement trajectory is used for the movable object to move on the two-dimensional interface based on the preset movement trajectory.

10. The method according to claim 9, characterized in that, The method further includes: In response to an adjustment instruction issued by the user based on the first preset movement trajectory, the adjustment instruction carries time adjustment information and speed adjustment information. Generate a second preset movement trajectory based on the adjustment instruction.

11. The method according to any one of claims 1-8, characterized in that, The method further includes: According to the somatosensory information to be sent by each movable object displayed on the two-dimensional interface, control the hardware devices set in the preset physical space to send associated somatosensory information.

12. The method according to claim 11, characterized in that, According to the somatosensory information to be sent by each movable object displayed on the two-dimensional interface, controlling the hardware devices set in the preset physical space to send associated somatosensory information includes: According to the somatosensory information to be sent by each movable object displayed on the two-dimensional interface, determine the conflict information between the somatosensory information to be sent by the movable objects of the same category, and according to the conflict information, determine the associated somatosensory information of the hardware device from the somatosensory information to be sent by the movable objects of the same category. Control the hardware devices set in the preset physical space to send the associated somatosensory information.

13. A two-dimensional display device for a sound object, characterized in that, The device is applied to a terminal device, the terminal device is located in a preset physical space, the terminal device displays a two-dimensional interface, the two-dimensional interface includes at least one movable object, the movable object is in a moving state or a non-moving state, and the device includes: A first control unit, configured to, in response to a first trigger operation, control the movable object operated by the first trigger operation to move according to the operation trajectory of the user, and play a sound object sound effect corresponding to both the movable object and the operation trajectory in the preset physical space; wherein, the first trigger operation represents that the user moves the movable object on the two-dimensional interface. The second control unit is used to respond to the second trigger operation, control the movement speed of the movable object at the current moment, perform subsequent movement, and play sound object sound effects corresponding to both the movable object and the subsequent movement trajectory within the preset physical space; wherein, the second trigger operation indicates that the user no longer touches the movable object.

14. A device for playing sound objects, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-12.

16. A computer program product comprising a computer program that, when executed by a processor, implements the method of any one of claims 1-12.

17. A method for displaying two-dimensional sound objects in a sound production system, characterized in that, The method includes: In response to a first trigger operation by the user in the sound production system, the movable object operated by the first trigger operation is controlled to move on the interface displayed by the sound production system according to the user's operation trajectory, and a stereo generated sound effect corresponding to both the movable object and the operation trajectory is played in a preset physical space; wherein, the first trigger operation represents the user moving the movable object on the two-dimensional interface; In response to a second trigger operation initiated by the user in the sound production system, the movement speed of the movable object at the current moment is controlled, and subsequent movement is performed on the interface displayed by the sound production system. The sound object sound effect corresponding to both the movable object and the subsequent movement trajectory is played within the preset physical space. The second trigger operation indicates that the user no longer touches the movable object.

18. A method for displaying two-dimensional sound objects in an interactive audio system, characterized in that, The method includes: In response to a first trigger operation by a user in an interactive audio system, the movable object operated by the first trigger operation is controlled to move on the interface displayed by the interactive audio system according to the user's operation trajectory, and a stereo generated sound effect corresponding to both the movable object and the operation trajectory is played in a preset physical space; wherein, the first trigger operation represents the user moving the movable object on the two-dimensional interface; In response to a second trigger operation initiated by the user in the interactive audio system, the movement speed of the movable object at the current moment is controlled, and subsequent movement is performed on the interface displayed by the interactive audio system. Sound effects of the sound objects corresponding to both the movable object and the subsequent movement trajectory are played within the preset physical space. The second trigger operation indicates that the user no longer touches the movable object.

Images