Interactive multi-user collaborative control method and control system
By adopting sound effects fusion and time window allocation mechanisms in virtual space, the sound overlap and confusion problems in multi-user collaborative control are solved, the interaction and communication efficiency of virtual space are improved, and seamless collaboration between multiple users is achieved. Real-time feedback.
Patent Information
- Application Number
- CN202411136924.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-08-19
AI Technical Summary
In virtual space, existing multi-user collaborative control technologies are difficult to achieve effective communication and collaborative control between multiple users, especially in scenarios where high collaborative and real-time feedback are required, such as online education and remote meetings, resulting in overlapping sounds and the confusion.
Through the intelligent sound effect fusion and time window allocation mechanism, the spectrum envelope matching method is used to fuse user sound effects and contextual sound effects, and the time window is allocated according to user priorities to ensure orderly transmission and feedback of sounds.
It realizes seamless collaboration between multiple users in the virtual space, improves communication and collaboration efficiency, enriches the interactive experience, avoids sound overlap and chaos, and enhances the interactivity and immersion of the virtual space.
Smart Images

Figure CN118672406B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of virtual interaction technology, and in particular to an interactive multi-user collaborative control method and control system. Background Art
[0002] With the rapid development of information technology, virtual space is increasingly used in various fields, especially in education, entertainment, medical treatment and industrial simulation, showing great potential and value. Virtual space abstracts and virtualizes physical space through software technology, providing users with a new, borderless interactive environment. In this environment, users can perform various operations, communicate and collaborate, which greatly enriches the user experience and improves work efficiency.
[0003] However, existing technologies still face many challenges in multi-user collaborative control in virtual space. Traditional single-user interaction methods can no longer meet the needs of multi-user simultaneous participation and real-time interaction. Especially in scenarios that require high collaboration and real-time feedback, such as online education, remote meetings, and multiplayer online games, how to achieve effective communication and collaborative control between multiple users has become an urgent problem to be solved.
[0004] Therefore, it is necessary to provide an interactive multi-user collaborative control method and control system to solve the above technical problems. Summary of the invention
[0005] In order to solve the above technical problems, the present invention provides an interactive multi-user collaborative control method and control system, which realizes seamless collaboration between multiple users in a virtual space through intelligent sound effect fusion and time window allocation mechanism.
[0006] The present invention provides an interactive multi-user collaborative control method, which is applied to a virtual space. The control method comprises the following steps:
[0007] In the virtual space, when it is detected that the first user generates a first operation for multiple second users, a situation template is obtained by matching from a preset situation library according to the current environmental parameters of the virtual space, wherein each situation template is correspondingly configured with a unique situation sound effect;
[0008] By using a spectrum envelope matching method, all user sound effects pre-configured by the second user and the user sound effects pre-configured by the first user are respectively merged with the situational sound effects of the situational template to obtain a first playback sound effect for each second user and a second playback sound effect for the first user;
[0009] Allocating a time window to each second user according to a priority preconfigured by the second user;
[0010] Playing the sounds from the plurality of second users to the first user in turn with the first playing sound effect according to the time window;
[0011] In response to receiving the second user's voice, the voice from the first user is played in turn to multiple second users with a second playback sound effect.
[0012] Preferably, the situation template is obtained by matching from a preset situation library according to the current environmental parameters of the virtual space, wherein each situation template is correspondingly configured with a unique situation sound effect, including:
[0013] monitoring environmental parameters in the virtual space in real time, wherein the environmental parameters include user density, background noise level, physical properties of the virtual space, and activity types;
[0014] According to real-time monitoring of the environmental parameters of the virtual space, the environmental parameters are mapped to the most similar environmental feature categories through a cluster analysis method, and a situation template that best matches the environmental feature category is selected from a preset situation library;
[0015] Extract uniquely configured situational sound effects from the matching situational template.
[0016] Preferably, the situation library is a database that brings together a variety of situation templates to match the situation sound effects in the situation templates according to the environmental parameters of the virtual space.
[0017] Preferably, the user sound effects pre-configured by the second user and the user sound effects pre-configured by the first user are both acquired through a preset hearing test session before entering the virtual space.
[0018] Preferably, the method of using the spectrum envelope matching method to fuse all the user sound effects pre-configured by the second user and the user sound effects pre-configured by the first user with the situational sound effects of the situational template to obtain the first playback sound effect for each second user and the second playback sound effect for the first user includes:
[0019] Preprocessing the user sound effect preconfigured by the first user, the user sound effect preconfigured by the second user, and the situational sound effect obtained from the situational template, and obtaining the corresponding first spectrum, second spectrum, and situational spectrum by Fourier transform;
[0020] Obtaining a first spectrum envelope, a second spectrum envelope, and a situation spectrum envelope of the first spectrum, the second spectrum, and the situation spectrum by taking the logarithm of the absolute value and performing smoothing processing;
[0021] The matching coefficient is calculated and the first spectrum envelope and the second spectrum envelope are adjusted according to the matching coefficient to make them close to the situation spectrum envelope, wherein the calculation formula of the matching coefficient is:
[0022]
[0023] in, is a small positive number, is the matching coefficient, is the first spectral envelope or the second spectral envelope, is the situational spectrum envelope;
[0024] The adjusted first spectrum envelope and the second spectrum envelope are inversely Fourier transformed, and the inverse Fourier transformed first spectrum envelope and the second spectrum envelope are linearly combined with the context spectrum envelope respectively to obtain the first playback sound effect for each second user and the second playback sound effect for the first user.
[0025] Preferably, allocating a time window to each second user according to a priority preconfigured by the second user includes:
[0026] Calculating a priority score for each second user, the priority including relationship between users, interaction frequency, and task importance;
[0027] Allocate a time window length for each second user according to the priority score, wherein the total length of the time window should be equal to one polling cycle;
[0028] Schedule the time window for each second user to ensure orderly playback within the polling cycle.
[0029] Preferably, the playing of the second play sound effect further includes storing the voice of the first user when the second user is not in a receivable state, and automatically playing it when the second user is in a receivable state.
[0030] The present invention also provides an interactive multi-user collaborative control system for an interactive multi-user collaborative control method, which is applied to a virtual space, and the control system comprises:
[0031] A situation template matching module, configured to, in the virtual space, when detecting that a first user generates a first operation for a plurality of second users, obtain a situation template by matching from a preset situation library according to current environmental parameters of the virtual space, wherein each situation template is correspondingly configured with a unique situation sound effect;
[0032] A sound effect fusion module, used to fuse all the user sound effects pre-configured by the second user and the user sound effects pre-configured by the first user with the situational sound effects of the situational template by using a spectrum envelope matching method, to obtain a first playback sound effect for each second user and a second playback sound effect for the first user;
[0033] A time allocation module, configured to allocate a time window to each second user according to a priority preconfigured by the second user;
[0034] a sound carousel module, configured to play the sounds from the plurality of second users to the first user in turn with the first play sound effect according to the time window;
[0035] The sound feedback module is used to play the sound from the first user to multiple second users in turn with the second play sound effect in response to receiving the sound of the second user.
[0036] Compared with the related art, the interactive multi-user collaborative control method and control system provided by the present invention have the following beneficial effects:
[0037] When the present invention detects that the first user has performed operations on multiple second users, it can match the most suitable situation template from the preset situation library according to the current environmental parameters of the virtual space, and generate corresponding situation sound effects accordingly; then, using the spectrum envelope matching technology, the user sound effects pre-configured by the first user and all second users are merged with the situation sound effects to generate their own unique playback sound effects; then, according to the pre-configured priority of the second user, a specific time window is allocated to each second user. In this way, it can be ensured that the voice of each second user can be heard by the first user within a limited time, and the problems of sound overlap and confusion can be avoided. At the same time, according to the setting of the time window, the voice of the first user will also be played to all second users in turn in the form of the second playback sound effect, realizing two-way transmission and real-time feedback of sound.
[0038] In summary, the present invention effectively solves the problem of multi-user collaborative control in virtual space through intelligent sound effect fusion and time window allocation mechanism, which not only improves the communication and collaboration efficiency between users, but also greatly enriches the interactive experience of virtual space, and provides strong technical support for the application of virtual space in various fields. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 A flowchart of an interactive multi-user collaborative control method provided by the present invention;
[0040] Figure 2 A module structure diagram of an interactive multi-user collaborative control system provided by the present invention. DETAILED DESCRIPTION
[0041] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without conflict.
[0042] It should also be noted that, for ease of description, only the parts related to the present invention, but not all of the contents, are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to methods, functions, procedures, subroutines, subprograms, etc.
[0043] Embodiment 1
[0044] The invention provides an interactive multi-user collaborative control method, which is applied to virtual space. Figure 1 As shown, the control method comprises the following steps:
[0045] S1: In the virtual space, when it is detected that a first user generates a first operation against multiple second users, a situation template is matched from a preset situation library according to current environmental parameters of the virtual space, wherein each situation template is correspondingly configured with a unique situation sound effect.
[0046] In this embodiment, environmental parameters in the virtual space are monitored in real time, including but not limited to user density, background noise level, physical properties of the virtual space (such as terrain, structure) and activity types (such as meetings, games), and these parameters are obtained through sensors or APIs of the virtual space.
[0047] When it is detected that the first user generates a first operation for multiple second users, a situation template that best matches the current environment parameters is searched from a preset situation library, wherein the situation library stores multiple situation templates, each situation template corresponds to a specific set of environment parameters and a matching situation sound effect.
[0048] The selection of situational templates is based on real-time analysis of environmental parameters and a preset clustering algorithm, which maps the current environmental parameters to the closest environmental feature category, thereby selecting the most suitable situational template.
[0049] Ultimately, through real-time monitoring and dynamic matching, it is possible to ensure that the sound effects are coordinated with the environment of the virtual space, thereby improving immersion and user experience. The precise matching of situational sound effects enhances the realism and interactivity of the virtual environment, making user operations and environmental feedback more natural and coherent.
[0050] S2: using a spectrum envelope matching method, the user sound effects pre-configured by all second users and the user sound effects pre-configured by the first user are respectively merged with the situational sound effects of the situational template to obtain a first playback sound effect for each second user and a second playback sound effect for the first user.
[0051] In this embodiment, a spectrum envelope matching method is used to merge all user sound effects (including volume, pitch, and timbre) pre-configured by the second user and the user sound effects pre-configured by the first user with the situational sound effects obtained from S1.
[0052] Spectral envelope matching includes calculating the spectral envelopes of user sound effects and situational sound effects, and adjusting the frequency response of the user sound effects to match the spectral envelope of the situational sound effects, thereby ensuring the natural transition and smooth conversion of the sound effects in different situations, reducing the sense of abruptness, and improving the naturalness and realism of the sound effects.
[0053] The fused sound effects, namely the first playback sound effect and the second playback sound effect, are respectively used to play the second user's voice to the first user, and to play the first user's voice to the second user.
[0054] Through the above-mentioned personalized sound effects fusion, a customized sound experience can be provided to each user, while ensuring the harmonious unity of the sound effects and the situation, thereby enhancing the quality of interaction and immersive experience between users.
[0055] S3: Allocate a time window to each second user according to the priority preconfigured by the second user.
[0056] In this embodiment, a time window is allocated to each second user according to the priority preconfigured by the second user, and a user with a higher priority will obtain an earlier time window to ensure that his voice is heard by the first user first.
[0057] Among them, the priority is based on the relationship between users, interaction frequency and task importance.
[0058] The priority and time window allocation mechanism ensures the orderliness and efficiency of information transmission, avoids the confusion caused by multiple users speaking at the same time, improves the effectiveness of communication and user satisfaction, and promotes equal participation of all users in the virtual space through fair time window allocation, enhancing the interactivity and collaboration of the community.
[0059] S4: Playing the sounds from the plurality of second users to the first user in turn with the first play sound effect according to the time window.
[0060] In this embodiment, according to the time window allocated to the second user, the sounds from the plurality of second users are played to the first user in turn with the first play sound effect.
[0061] The polling mechanism of the time window ensures that the first user can hear the voice of each second user in turn without missing any information, and the playback mechanism also takes into account the fusion effect of the sound effects to ensure that the voice heard by the first user is both personalized and consistent with the current situation.
[0062] The polling playback mechanism ensures the continuity and integrity of information. Even in a multi-user environment, the first user can clearly distinguish and understand the voice of each second user, enhancing the transparency and efficiency of communication. Through orderly sound playback, the risk of information overload is reduced, enabling the first user to better concentrate, thereby improving the quality and effectiveness of interaction in the virtual space.
[0063] S5: In response to receiving the second user's voice, play the voice from the first user in turn to multiple second users with a second play sound effect.
[0064] In this embodiment, after the second user receives the voice of the first user, the voice from the first user is played to multiple second users in turn using the second play sound effect.
[0065] The playback mechanism also follows the arrangement of time windows, ensuring that each second user has the opportunity to hear the first user's voice without missing information due to conflicts in time windows.
[0066] The playback process also takes into account the integration of sound effects, ensuring that the sound heard by the second user is consistent with the current context while maintaining the clarity and recognizability of the first user's voice.
[0067] Through responsive playback, a two-way flow of information is achieved, which enhances the interactivity and participation of users in the virtual space and promotes deeper communication and collaboration. The orderly playback mechanism and sound effect fusion ensure the accurate transmission of information, reduce misunderstandings and communication barriers, and improve the cooperation efficiency and satisfaction among users in the virtual space.
[0068] Specifically, step S1 includes the following steps:
[0069] S101: Monitoring environmental parameters in the virtual space in real time, wherein the environmental parameters include user density, background noise level, physical properties of the virtual space, and activity types.
[0070] In this embodiment, environmental parameters in the virtual space are monitored in real time through a sensor network or a virtual environment API in the virtual space, including but not limited to user density, background noise level, physical properties of the virtual space (such as terrain, building layout) and activity types (such as lectures, concerts, game competitions).
[0071] These parameters are monitored continuously in order to capture dynamic changes in the environment; for example, an increase in the number of users may lead to an increase in user density, while different types of activities may result in different background noise levels.
[0072] S102: According to real-time monitoring of the environmental parameters of the virtual space, the environmental parameters are mapped to the most similar environmental feature categories through a cluster analysis method, and a situation template that best matches the environmental feature category is selected from a preset situation library.
[0073] In this embodiment, a pre-trained clustering analysis model is used to map the environmental parameters monitored in real time to the closest environmental feature categories. The clustering analysis is built based on historical data and can identify common environmental feature combinations, such as "high-density gatherings", "quiet study areas", etc.
[0074] According to the mapping results, the situation template that best matches the current environmental feature category is selected from the preset situation library. The situation library contains various predefined situation templates, each of which is associated with a specific environmental feature category and is configured with a unique situational sound effect.
[0075] The clustering analysis method can intelligently identify the characteristics of the current environment and thus select the most suitable situational template, enhancing the coordination between the sound effects and the environment in the virtual space and improving the sense of immersion.
[0076] S103: extracting a uniquely configured situational sound effect from the matched situational template.
[0077] In this embodiment, once the best-matching situational template is determined, uniquely configured situational sound effects are extracted from the situational template. These sound effects are designed for specific environments and can enhance the atmosphere of the virtual space, such as the faint echo in a conference room, the natural wind sound of outdoor activities, etc. The extracted situational sound effects are then applied to the entire virtual space, covering the auditory experience of all users, and also serving as the basis for the next step of sound effect fusion.
[0078] Accurately extracted situational sound effects can further enhance the realism and immersion of the virtual space, allowing users to experience an auditory experience similar to the actual environment, thereby increasing the overall attractiveness of the virtual space.
[0079] Specifically, the situation library is a database that collects a variety of situation templates to match the situation sound effects in the situation templates according to the environmental parameters of the virtual space.
[0080] In this embodiment, the situation library is a database that integrates multiple situation templates, where each situation template corresponds to a set of specific environmental parameters and matching situational sound effects. The construction of the situation library is based on in-depth research on different types of virtual space environments, covering a wide range of situation categories, such as indoor meetings, outdoor activities, game arenas, etc.
[0081] The introduction of the situational library has greatly enriched the auditory experience of the virtual space. By precisely matching the environmental parameters, it can provide users with an immersive feeling and enhance the authenticity and immersion of the virtual space. The dynamic adjustment mechanism ensures the flexibility and adaptability of the situational sound effects. Even when the environmental parameters change rapidly, the consistency and high quality of the sound effects can be maintained, thereby improving the user's comfort and satisfaction in the virtual space.
[0082] Specifically, the user sound effects pre-configured by the second user and the user sound effects pre-configured by the first user are both obtained through a preset hearing test session before entering the virtual space.
[0083] In this embodiment, the user sound effect configuration refers to the personalized sound effect parameters obtained by the user through a preset hearing test before entering the virtual space. These parameters reflect the user's hearing characteristics, including volume sensitivity, frequency response preference, dynamic range, etc.
[0084] The hearing test includes a series of standard tests, such as pure tone hearing threshold test and speech recognition test, to evaluate the user's hearing response at different frequencies and volumes.
[0085] The test results are used to generate user sound effect profiles, which contain personalized parameters for sound effect fusion. After the user enters the virtual space, these parameters are integrated with the situational sound effects to provide the best auditory experience, ensuring that each user can enjoy the most suitable sound effect settings in the virtual space, avoiding information omissions or overstimulation due to hearing differences, and improving the accuracy of information transmission and user comfort.
[0086] Specifically, step S2 includes the following steps:
[0087] S201: pre-processing the user sound effects pre-configured by the first user, the user sound effects pre-configured by the second user, and the situational sound effects of the situational template, and using Fourier transform to obtain corresponding first spectrum, second spectrum, and situational spectrum.
[0088] In this embodiment, the pre-configured sound effects of the first user and the second user, as well as the situational sound effects extracted from the situational library, are first preprocessed, including steps such as noise removal, volume standardization, and equalization, to ensure that all audio signals are comparable in subsequent processing. The discrete Fourier transform (DFT) or fast Fourier transform (FFT) algorithm is used to convert the audio signal in the time domain into spectrum data in the frequency domain, so that the frequency composition of the audio, namely the first spectrum, the second spectrum, and the situational spectrum, can be easily analyzed.
[0089] S202: Obtain a first spectrum envelope, a second spectrum envelope, and a situational spectrum envelope of the first spectrum, the second spectrum, and the situational spectrum by taking the logarithm of the absolute value and performing smoothing processing.
[0090] In this embodiment, in order to more intuitively express the amplitude variation trend of the spectrum, the obtained first spectrum, second spectrum and context spectrum are further processed. First, the amplitude of the spectrum is logarithmized, and then a smoothing filter is applied to extract the spectrum envelope, i.e., the first spectrum envelope, the second spectrum envelope and the context spectrum envelope. The extraction of the envelope helps to highlight the main frequency components of the audio signal while reducing the influence of high-frequency noise.
[0091] S203: Calculate a matching coefficient and adjust the first spectrum envelope and the second spectrum envelope according to the matching coefficient to make them close to the situation spectrum envelope, wherein the calculation formula of the matching coefficient is:
[0092]
[0093] in, is a small positive number, is the matching coefficient, is the first spectral envelope or the second spectral envelope, is the situational spectrum envelope.
[0094] In this embodiment, a matching coefficient is calculated to adjust the spectrum envelopes of the first user and the second user so that they are as close as possible to the situational spectrum envelope. The adjustment process is to multiply the calculated matching coefficient so that the spectrum envelopes of the users are closer to the situational spectrum envelope, thereby achieving a better sound fusion effect.
[0095] S204: Perform inverse Fourier transform on the adjusted first spectrum envelope and second spectrum envelope, and linearly combine the inverse Fourier transformed first spectrum envelope and second spectrum envelope with the context spectrum envelope respectively to obtain a first playback sound effect for each second user and a second playback sound effect for the first user.
[0096] In this embodiment, an inverse Fourier transform (IDFT or IFFT) is performed on the adjusted first spectrum envelope and the second spectrum envelope to convert them back into time domain signals, and the inverse transformed first spectrum envelope and the second spectrum envelope are linearly combined with the context spectrum envelope. The linear combination is achieved by weighted averaging, where the weights are fixed. Ultimately, a customized playback sound effect is generated for each user, namely, a first playback sound effect (for the second user) and a second playback sound effect (for the first user), ensuring that the sound effect received by each user is in line with personal preferences and coordinated with the current context.
[0097] Specifically, step S3 includes the following steps:
[0098] S301: Calculate a priority score for each second user, where the priority includes the relationship between users, interaction frequency, and task importance.
[0099] In this embodiment, the priority score of each second user is calculated by analyzing the relationship between users (such as social network links, team member relationships), interaction frequency (such as the number of message exchanges in the past period of time) and task importance (such as ongoing critical tasks or events).
[0100] These indicators can be quantified and weighted to form a comprehensive priority score. For example, for relationships, it is based on the strength of the connection between users (such as the number of common friends and interaction records); for interaction frequency, it is based on the communication records in the recent period; for task importance, it is based on the importance level of the tasks or activities currently being carried out in the virtual space.
[0101] S302: Allocate a time window length to each second user according to the priority score, wherein the total length of the time window should be equal to one polling cycle.
[0102] In this embodiment, the length of the time window is calculated and allocated according to the priority score of each second user. The length of the time window is proportional to the priority score. The higher the priority, the earlier the allocated time window.
[0103] A fixed polling cycle length is set, which is equal to the sum of all time window lengths. This ensures that every second user has a chance to be heard within a complete polling cycle.
[0104] S303: Arrange a time window for each second user to ensure orderly playback within the polling cycle.
[0105] In this embodiment, once a time window is allocated to each second user, it is necessary to arrange these time windows to ensure that they are played in order within the polling cycle, with users with high priority arranged in front and users with low priority arranged in the back. In this way, it can be ensured that within the polling cycle, the voice of each second user can be played within the predetermined time window, avoiding overlapping or missing information, and ensuring the orderliness and efficiency of information transmission.
[0106] Specifically, the playing of the second playback sound effect also includes storing the voice of the first user when the second user is not in a receivable state, and automatically playing it when the second user is in a receivable state.
[0107] In this embodiment, the status of each second user is continuously monitored, including but not limited to their online status, attention level, and whether they participate in other interactive activities. This can be achieved by analyzing the user's activity pattern and location data in the virtual space or directly asking the user about his or her current receivable status.
[0108] When the first user's sound needs to be played to the second user, it is first checked whether the second user is in a receivable state. If the second user is not in a receivable state (such as the user is offline, busy with other tasks, or in an area in the virtual space where the sound cannot be heard), the second playback sound effect will not be played immediately, but will be stored.
[0109] Stored sounds can be tagged for later retrieval and playback. Storage mechanisms include cache, temporary files, or dedicated sound storage services, depending on the system architecture and resource management strategy.
[0110] When the second user re-enters the receivable state, the corresponding second playback sound effect is retrieved from the storage for playback, which means that even if the user missed the first user's voice before, there is still a chance to hear this information.
[0111] The working principle of an interactive multi-user collaborative control method provided by the present invention is as follows: when it is detected that the first user generates an operation for multiple second users, the most suitable situation template can be matched from the preset situation library according to the current environmental parameters of the virtual space, and the corresponding situation sound effect can be generated accordingly; then, the spectrum envelope matching technology is used to merge the user sound effects pre-configured by the first user and all second users with the situation sound effects to generate their own unique playback sound effects; then, according to the pre-configured priority of the second user, a specific time window is allocated to each second user. In this way, it can be ensured that the voice of each second user can be heard by the first user within a limited time, and the problem of sound overlap and confusion can be avoided. At the same time, according to the setting of the time window, the voice of the first user will also be played to all second users in turn in the form of the second playback sound effect, realizing two-way transmission and real-time feedback of sound.
[0112] In summary, the present invention effectively solves the problem of multi-user collaborative control in virtual space through intelligent sound effect fusion and time window allocation mechanism, which not only improves the communication and collaboration efficiency between users, but also greatly enriches the interactive experience of virtual space, and provides strong technical support for the application of virtual space in various fields.
[0113] Embodiment 2
[0114] The present invention also provides an interactive multi-user collaborative control system for an interactive multi-user collaborative control method, which is applied to a virtual space, referring to Figure 2 As shown, the control system includes:
[0115] The situation template matching module 100 is used to match the situation template from the preset situation library according to the current environmental parameters of the virtual space when it is detected that the first user generates a first operation against multiple second users in the virtual space, wherein each situation template is configured with a unique situation sound effect.
[0116] The sound effect fusion module 200 is used to use the spectrum envelope matching method to fuse all the user sound effects pre-configured by the second users and the user sound effects pre-configured by the first user with the situational sound effects of the situational template to obtain the first playback sound effect for each second user and the second playback sound effect for the first user.
[0117] The time allocation module 300 is used to allocate a time window to each second user according to the priority preconfigured by the second user.
[0118] The sound carousel module 400 is used to play the sounds from multiple second users to the first user in turn with the first play sound effect according to the time window.
[0119] The sound feedback module 500 is used to play the sound from the first user to multiple second users in turn with the second play sound effect in response to receiving the sound of the second user.
[0120] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0121] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, the storage medium including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically-erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.
[0122] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
Claims
1. An interactive multi-user collaborative control method, applied to virtual space, It is characterized in that The control method comprises the following steps: In the virtual space, when it is detected that the first user generates a first operation for multiple second users, a situation template is obtained by matching from a preset situation library according to the current environmental parameters of the virtual space, wherein each situation template is correspondingly configured with a unique situation sound effect; By using a spectrum envelope matching method, all user sound effects pre-configured by the second user and the user sound effects pre-configured by the first user are respectively merged with the situational sound effects of the situational template to obtain a first playback sound effect for each second user and a second playback sound effect for the first user; Allocating a time window to each second user according to a priority preconfigured by the second user; Playing the sounds from the plurality of second users to the first user in turn with the first playing sound effect according to the time window; In response to receiving the second user's voice, the voice from the first user is played in turn to multiple second users with a second playback sound effect.
2. An interactive multi-user collaborative control method according to claim 1, It is characterized in that The scenario template is obtained by matching from a preset scenario library according to the current environment parameters of the virtual space, wherein each scenario template is configured with a unique scenario sound effect, including: monitoring environmental parameters in the virtual space in real time, wherein the environmental parameters include user density, background noise level, physical properties of the virtual space, and activity types; According to real-time monitoring of the environmental parameters of the virtual space, the environmental parameters are mapped to the most similar environmental feature categories through a cluster analysis method, and a situation template that best matches the environmental feature category is selected from a preset situation library; Extract uniquely configured situational sound effects from the matching situational template.
3. An interactive multi-user collaborative control method according to claim 2, It is characterized in that The situation library is a database that collects a variety of situation templates to match the situation sound effects in the situation templates according to the environmental parameters of the virtual space.
4. The interactive multi-user collaborative control method according to claim 3, It is characterized in that The user sound effects pre-configured by the second user and the user sound effects pre-configured by the first user are both obtained through a preset hearing test session before entering the virtual space.
5. An interactive multi-user collaborative control method according to claim 4, It is characterized in that The method of using the spectrum envelope matching method to fuse all the user sound effects pre-configured by the second user and the user sound effects pre-configured by the first user with the situational sound effects of the situational template to obtain the first playback sound effect for each second user and the second playback sound effect for the first user includes: Preprocessing the user sound effect preconfigured by the first user, the user sound effect preconfigured by the second user, and the situational sound effect obtained from the situational template, and obtaining the corresponding first spectrum, second spectrum, and situational spectrum by Fourier transform; Obtaining a first spectrum envelope, a second spectrum envelope, and a situation spectrum envelope of the first spectrum, the second spectrum, and the situation spectrum by taking the logarithm of the absolute value and performing smoothing processing; The matching coefficient is calculated and the first spectrum envelope and the second spectrum envelope are adjusted according to the matching coefficient to make them close to the situation spectrum envelope, wherein the calculation formula of the matching coefficient is: in, is a small positive number, is the matching coefficient, is the first spectral envelope or the second spectral envelope, is the situational spectrum envelope; The adjusted first spectrum envelope and the second spectrum envelope are inversely Fourier transformed, and the first spectrum envelope and the second spectrum envelope after inverse Fourier transformation are linearly combined with the context spectrum envelope respectively to obtain the first playback sound effect for each second user and the second playback sound effect for the first user.
6. An interactive multi-user collaborative control method according to claim 5, It is characterized in that Allocating a time window to each second user according to the priority preconfigured by the second user includes: Calculating a priority score for each second user, the priority including relationship between users, interaction frequency, and task importance; Allocate a time window length for each second user according to the priority score, wherein the total length of the time window should be equal to one polling cycle; Schedule the time window for each second user to ensure orderly playback within the polling cycle.
7. An interactive multi-user collaborative control method according to claim 6, It is characterized in that The playing of the second play sound effect also includes storing the voice of the first user when the second user is not in a receivable state, and automatically playing it when the second user is in a receivable state.
8. An interactive multi-user collaborative control system, used to execute an interactive multi-user collaborative control method as claimed in any one of claims 1 to 7, applied to a virtual space, It is characterized in that The control system comprises: A situation template matching module, configured to, in the virtual space, when detecting that a first user generates a first operation for a plurality of second users, obtain a situation template by matching from a preset situation library according to current environmental parameters of the virtual space, wherein each situation template is correspondingly configured with a unique situation sound effect; A sound effect fusion module, used to fuse all user sound effects pre-configured by the second user and the user sound effects pre-configured by the first user with the situational sound effects of the situational template by using a spectrum envelope matching method, to obtain a first playback sound effect for each second user and a second playback sound effect for the first user; A time allocation module, configured to allocate a time window to each second user according to a priority preconfigured by the second user; a sound carousel module, configured to play the sounds from the plurality of second users to the first user in turn with the first play sound effect according to the time window; The sound feedback module is used to play the sound from the first user to multiple second users in turn with the second play sound effect in response to receiving the sound of the second user.
