Voice production control method, head-mounted display device, and computer storage medium

Abstract

The application discloses a sound production control method, a head-mounted display device and a computer storage medium. The sound production control method comprises the following steps: receiving a plurality of sound scanning signals of sound, and determining a low-frequency sound production sound according to each sound scanning signal; determining a correction curve corresponding to each sound scanning signal based on a pre-stored target frequency response, and collecting the correction curves to obtain a correction curve set; determining sound position information corresponding to each sound scanning signal according to an acquired instruction transceiving delay value, and performing sound production control according to the sound position information, the correction curve set and the low-frequency sound production sound. The application improves the sound production effect of the head-mounted display device at low frequencies.

Description

Technical Field

[0001] The present application relates to the field of sound processing technology, and in particular to a sound control method, a head-mounted display device, and a computer storage medium. Background Art

[0002] Currently, the number of head-mounted display devices such as VR (Virtual Reality) devices and AR (Augmented Reality) devices on the market is gradually increasing. Due to their immersive experience, head-mounted display devices are widely used in games and immersive movies. However, due to the trend of lightweight head-mounted display devices, there are not many options for sound units for head-mounted display devices, resulting in poor performance in low frequencies. For example, due to the limitations of the lightweight head-mounted display device and its own sound mode, the head-mounted display device can only produce sound through its own speaker unit, which makes it impossible to achieve diversification of sound units and results in poor sound effects in low frequencies.

[0003] Therefore, in the application process of the above solution, there is a defect that sound can only be produced through its own speaker unit, which results in poor sound effect of the head-mounted display device at low frequencies. Summary of the Invention

[0004] The main purpose of this application is to provide a sound control method, a head-mounted display device and a computer storage medium, aiming to solve the technical problem that the head-mounted display device has poor sound effect at low frequencies.

[0005] To achieve the above objectives, the present application provides a sound control method, which is applied to a head-mounted display device. The sound control method comprises the following steps:

[0006] receiving an audio sweep frequency signal of at least one audio device, and determining a bass sounding audio device according to each of the audio sweep frequency signals;

[0007] determining a correction curve corresponding to the audio frequency sweep signal based on a pre-stored target frequency response, and aggregating the correction curves corresponding to the audio frequency sweep signals to obtain a correction curve set;

[0008] The sound position information corresponding to the sound sweep signal is determined according to the collected instruction sending and receiving delay value, and sound control is performed according to the sound position information, the correction curve set and the bass sound.

[0009] Optionally, the step of determining the bass sound according to each of the audio sweep frequency signals includes:

[0010] sequentially determining a bass performance curve corresponding to the audio frequency sweep signal, and detecting whether the bass performance curve matches a pre-stored optimal bass performance curve, wherein the performance curve is obtained by performing a Fourier transform on the audio frequency sweep signal, and the bass performance curve is a curve in the performance curve having a frequency value less than a preset frequency value;

[0011] If they match, the sound corresponding to the sound sweep signal that matches the optimal bass performance curve is determined as the bass sounding sound.

[0012] Optionally, the step of determining a correction curve corresponding to the audio sweep frequency signal based on a pre-stored target frequency response includes:

[0013] Determining a mid-high frequency performance curve corresponding to the audio frequency sweep signal, and determining an audio frequency response corresponding to the mid-high frequency performance curve, wherein the mid-high frequency performance curve is a curve in the performance curve having a frequency value greater than or equal to a preset frequency value;

[0014] Determine the difference between the audio frequency response and a pre-stored target frequency response as a correction dotted line;

[0015] An acoustic sound signal corresponding to the correction dotted line is received, and sound control is performed according to the acoustic sound signal and the correction dotted line to obtain a correction curve.

[0016] Optionally, the step of performing sound control according to the acoustic sound signal and the corrected dotted line to obtain a correction curve includes:

[0017] Determining a mid-high frequency performance curve in the sound signal, and performing frequency response calibration on the mid-high frequency performance curve based on the corrected dotted line to obtain a sound frequency response;

[0018] Detecting whether a difference between the sound frequency response and the target frequency response is less than a preset value;

[0019] If it is less than the preset value, the correction dotted line is determined to be a correction curve.

[0020] Optionally, before the step of determining the sound position information corresponding to the sound sweep signal according to the collected instruction sending and receiving delay value, the step includes:

[0021] Determining that the audio frequency sweep signal corresponds to high-frequency sound pressure information, and determining that the maximum sound pressure direction in the high-frequency sound pressure information is the audio angular position;

[0022] Obtaining a first delay value, wherein the first delay value is a time delay value between the time of the play instruction at the initial position and the time of receiving the instruction;

[0023] Obtaining a second delay value, wherein the second delay value is a time delay value between the time of playing the instruction at the end position and the time of receiving the instruction;

[0024] Using the difference between the first delay value and the second delay value as the instruction sending and receiving delay value;

[0025] The step of determining the sound position information corresponding to the sound sweep signal according to the collected instruction receiving and sending delay value includes:

[0026] The sound distance corresponding to the collected command receiving and sending delay value is determined, and the sound distance corresponding to the sound angle position is used as the sound position information corresponding to the sound sweep signal.

[0027] Optionally, the step of performing sound control according to the sound position information, the correction curve set, and the bass sound includes:

[0028] Determine the sound production requirement corresponding to the audio to be played, and detect whether the sound production requirement is a mid-high pitch sound production requirement;

[0029] If the sound production requirement is a mid-high frequency sound production requirement, receiving the mid-high frequency sound produced by the speaker, determining a target correction curve for the mid-high frequency sound in the correction curve set, and performing frequency response calibration on the mid-high frequency sound based on the speaker position information and the target correction curve to produce the mid-high frequency sound;

[0030] If the sound production requirement is not a mid-high pitch sound production requirement, the bass sound producing speaker is controlled to produce bass sound.

[0031] Optionally, before the step of receiving audio sweep signals of a plurality of audio devices, the step includes:

[0032] Establish Bluetooth connections with multiple speakers;

[0033] sending audio frequency sweep signal request instructions to the audio system in sequence based on the Bluetooth connection;

[0034] The step of receiving the audio sweep frequency signals of multiple audio systems includes:

[0035] The audio frequency sweep signals generated by a plurality of audio systems based on the audio frequency sweep signal request instruction are received.

[0036] Optionally, after the step of receiving audio sweep signals of a plurality of audio devices, the method further comprises:

[0037] Determining the personalized sound output according to each of the audio sweep frequency signals;

[0038] Determining a personalized correction curve corresponding to the personalized sound emission sound based on a pre-stored personalized target frequency response;

[0039] The personalized sound position information corresponding to the personalized sound emitting sound is determined according to the collected personalized instruction sending and receiving delay value, and the sound emission control is performed according to the personalized sound position information and the personalized correction curve.

[0040] The present application also provides a head-mounted display device, which is a physical device. The head-mounted display device includes: a memory, a processor, and a program of the sound control method stored in the memory and executable on the processor. When the program of the sound control method is executed by the processor, the steps of the sound control method described above can be implemented.

[0041] The present application also provides a computer storage medium on which a program for implementing the sound control method is stored. The program for implementing the sound control method is executed by a processor to implement the steps of the above-mentioned sound control method.

[0042] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned sound control method when executed by a processor.

[0043] The technical solution of the present application is to receive audio sweep frequency signals from a plurality of audio devices, and determine a bass sound according to each of the audio sweep frequency signals; determine a correction curve corresponding to the audio sweep frequency signal based on a pre-stored target frequency response, and aggregate the correction curves corresponding to the audio sweep frequency signals to obtain a correction curve set; determine audio position information corresponding to the audio sweep frequency signal according to a collected instruction receiving and sending delay value, and perform sound control according to the audio position information, the correction curve set, and the bass sound, so that the head-mounted display device produces mid- and high-pitched sounds according to the audio position information and the correction curve set and produces bass sounds according to the bass sound, thereby achieving the purpose of improving the sound effect of the head-mounted display device at low frequencies.

[0044] Currently, stereo systems are established through Bluetooth and speakers. For head-mounted display devices such as VR products, establishing a stereo system with speakers is, on the one hand, compared to traditional stereo speakers, the acoustic performance of the speakers does not change with the body's movement. However, VR products, thanks to the presence of IMUs (Inertial Measurement Units) and cameras, can effectively capture the body's movement and position. Therefore, real-time calibration will provide people with a more realistic sound environment. On the other hand, because the sound units of speakers are more diverse, including tweeters, midrange units, and woofers.

[0045] In addition, since the user determines the bass sound through the audio sweep signal on the one hand, and controls the head-mounted display device according to the bass sound to achieve bass sound, and on the other hand determines the correction curve corresponding to the audio sweep signal based on the pre-stored target frequency response, and then determines the audio correction curve set, and also determines the audio position information corresponding to the audio sweep signal, and finally controls the head-mounted display device according to the audio position information and the correction curve set to achieve mid- and high-pitched sound, that is, the head-mounted display device can achieve bass sound through the bass sound, and can also correct the mid- and high-pitched sound of the audio through the correction curve set in the head-mounted display device to achieve mid- and high-pitched sound. The present invention can eliminate the bass sound defect of the current head-mounted display device to the greatest extent, and then realize personalized sound selection by jointly producing sound through the speaker and the internal speaker unit of the head-mounted display device, and improve the low-frequency sound effect of the head-mounted display device through the bass speaker sound. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0048] Figure 1 This is a flow chart of the first embodiment of the sound control method of the present application;

[0049] Figure 2 This is a flow chart of the second embodiment of the sound control method of the present application;

[0050] Figure 3 A schematic diagram of a scene constructed by a head-mounted display device and a speaker in one embodiment of the present application;

[0051] Figure 4 This is a schematic diagram of a sound calibration process for a head-mounted display device according to an embodiment of the present application;

[0052] Figure 5 This is a schematic diagram of a personalized audio sound calibration process for a head-mounted display device in one embodiment of the present application;

[0053] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the head-mounted display device in the embodiment of the present application.

[0054] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0055] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0056] In this embodiment, the head-mounted display device of the present application can be, for example, a mixed reality (Mixed Reality) - MR device (such as MR glasses or MR helmet), an augmented reality (Augmented Reality) - AR device (such as AR glasses or AR helmet), a virtual reality (Virtual Reality) - VR device (such as VR glasses or VR helmet), an extended reality (Extended Reality) - XR device or some combination thereof.

[0057] Currently, due to the inherent trend toward lightweight head-mounted display devices, such as VR products, there are limited speaker options. This limited installation space limits the choice of speaker size, which can result in poor low-frequency performance for VR products. For example, when gaming on a VR device, bass sounds can be subpar, impacting the user's gaming experience.

[0058] Example 1

[0059] Based on this, please refer to Figure 1 , a flow chart of a first embodiment of a sound control method, this embodiment provides a sound control method, the sound control method is applied to a head-mounted display device, the sound control method comprises the following steps:

[0060] Step S100, receiving audio sweep frequency signals of a plurality of audio systems, and determining a bass sound according to each of the audio sweep frequency signals;

[0061] In this embodiment, a system is established in which a VR device is connected to multiple speakers via Bluetooth, and then the VR device of the system is calibrated to achieve sound control. During the calibration process, by receiving the audio sweep signals of multiple speakers, the multiple speakers refer to the speakers of the system. The number of speakers in the system will receive the audio sweep signals of as many speakers as there are speakers. The audio sweep signal refers to the signal sent by the speaker that continuously changes from low frequency to high frequency, or from high frequency to low frequency. Because of the influence of the small size of the speaker of the VR device itself, the bass sound will be determined according to the audio sweep signal. The bass sound refers to the speaker specifically responsible for bass sound in the VR device of this embodiment. Then, when the VR device needs to make bass sound, the bass sound is controlled by Bluetooth to make sound, thereby avoiding the phenomenon that the existing VR device has poor sound effect in the low frequency band.

[0062] Step S200, determining a correction curve corresponding to the audio sweep frequency signal based on a pre-stored target frequency response, and aggregating the correction curves corresponding to the audio sweep frequency signals to obtain a correction curve set;

[0063] In this embodiment, after determining the bass sound, the mid- and high-frequency sounds are calibrated. Correction curves corresponding to the audio sweep signals are sequentially determined using a pre-stored target frequency response. After the correction curves corresponding to all audio sweep signals are aggregated to form a correction curve set, this correction curve set is written to the VR device's equalizer for subsequent use when processing the audio mid- and high-frequency sounds. The target frequency response refers to the frequency response of the VR device's internal speakers. The correction curves are used to correct the frequency response of the audio mid- and high-frequency sounds. The correction curve set refers to the correction curves for all audio sweep signals corresponding to the audio mid- and high-frequency sounds. Because the mid- and high-frequency sounds are produced jointly by the audio system and the VR device's internal speaker units, correction curves are required to ensure that the mid- and high-frequency sounds are produced jointly with the VR device's internal speaker units. Furthermore, mid- and high-frequency sounds exhibit phase differences that are even greater than intensity differences, resulting in strong directional characteristics. Therefore, the position and angle of the mid- and high-frequency sounds must be determined to adjust the audio mid- and high-frequency sounds and improve the VR device's realistic simulation performance.

[0064] Step S300 , determining the sound position information corresponding to the sound sweep signal according to the collected instruction sending and receiving delay value, and performing sound control according to the sound position information, the correction curve set and the bass sound.

[0065] In this embodiment, because the phase difference in the mid- and high-frequency sounds mentioned above is even greater than the intensity difference, and therefore the mid- and high-frequency sounds are highly directional, it is necessary to determine the position information of the sound corresponding to the audio sweep signal based on the collected command transmission and reception delay value. In other words, the position information of the sound corresponding to the audio sweep signal is determined by the command transmission and reception delay value. The command transmission and reception delay value refers to the difference between the command transmission and reception delays at the starting position and when it is infinitely close to the end position. The distance of the sound from the starting position can then be calculated based on the characteristics of the sound. The angle between the sound and the starting position can also be determined based on the high-frequency sound, that is, the azimuth information of the sound at the starting position. Ultimately, the mid- and high-frequency sound generation of the VR device is controlled based on the sound position information and the correction curve set, achieving joint sound generation of the sound and the VR device, and increasing the diversity of the VR device's sound generation units. The bass sound generation of the VR device is controlled by the bass generation sound, thereby avoiding the problem of poor bass sound generation in VR devices in the prior art.

[0066] The technical solution of the present application is to receive audio sweep frequency signals from a plurality of audio devices, and determine a bass sound according to each of the audio sweep frequency signals; determine a correction curve corresponding to the audio sweep frequency signal based on a pre-stored target frequency response, and aggregate the correction curves corresponding to the audio sweep frequency signals to obtain a correction curve set; determine audio position information corresponding to the audio sweep frequency signal according to a collected instruction receiving and sending delay value, and perform sound control according to the audio position information, the correction curve set, and the bass sound, so that the head-mounted display device produces mid- and high-pitched sounds according to the audio position information and the correction curve set and produces bass sounds according to the bass sound, thereby achieving the purpose of improving the sound effect of the head-mounted display device at low frequencies.

[0067] Currently, stereo systems are established through Bluetooth and speakers. For head-mounted display devices such as VR products, establishing a stereo system with speakers is, on the one hand, compared to traditional stereo speakers, the acoustic performance of the speakers does not change with the body's movement. However, VR products, thanks to the presence of IMUs (Inertial Measurement Units) and cameras, can effectively capture the body's movement and position. Therefore, real-time calibration will provide people with a more realistic sound environment. On the other hand, because the sound units of speakers are more diverse, including tweeters, midrange units, and woofers.

[0068] In addition, since the user determines the bass sound through the audio sweep signal on the one hand, and controls the head-mounted display device according to the bass sound to achieve bass sound, and on the other hand determines the correction curve corresponding to the audio sweep signal based on the pre-stored target frequency response, and then determines the audio correction curve set, and also determines the audio position information corresponding to the audio sweep signal, and finally controls the head-mounted display device according to the audio position information and the correction curve set to achieve mid- and high-pitched sound, that is, the head-mounted display device can achieve bass sound through the bass sound, and can also correct the mid- and high-pitched sound of the audio through the correction curve set in the head-mounted display device to achieve mid- and high-pitched sound. The present invention can eliminate the bass sound defect of the current head-mounted display device to the greatest extent, and then realize personalized sound selection by jointly producing sound through the speaker and the internal speaker unit of the head-mounted display device, and improve the low-frequency sound effect of the head-mounted display device through the bass speaker sound.

[0069] In one practicable manner, the step of determining the bass sound according to each of the audio sweep frequency signals includes:

[0070] Step A10, sequentially determining a bass performance curve corresponding to the audio frequency sweep signal, and detecting whether the bass performance curve matches a pre-stored optimal bass performance curve, wherein the performance curve is obtained by performing a Fourier transform on the audio frequency sweep signal, and the bass performance curve is a curve in the performance curve having a frequency value less than a preset frequency value;

[0071] In step A20 , if there is a match, the sound source corresponding to the sound source frequency sweep signal that matches the optimal bass performance curve is determined as the bass sound source.

[0072] In this embodiment, because the VR device has a poor bass sound effect, it is connected to the speaker via Bluetooth, and then the speaker is controlled to produce bass sound to avoid the problem of poor bass sound effect of the VR device.

[0073] After receiving a speaker sweep signal, the system sequentially performs a Fourier transform on the sweep signal to obtain a bass performance curve (the curve with a frequency value less than a preset frequency value is used as the bass performance curve). The system then checks whether the bass performance curve matches a pre-stored optimal bass performance curve. If the two match, the speaker corresponding to the sweep signal matching the optimal bass performance curve is determined as the bass-generating speaker. In other words, the system determines the speaker corresponding to the sweep signal matching the optimal bass performance curve from among multiple sweep signals and selects this speaker as the bass-generating speaker. This speaker is then subsequently controlled to generate bass sound. A bass performance curve refers to the sound curve of a speaker when generating bass sound. An optimal bass performance curve refers to a bass curve that meets the required bass effect. Matching means that the bass performance curve meets the requirements of the optimal bass performance curve. For example, if the optimal bass performance curve is curve A and the bass performance curve is curve B, if curve B produces a stronger bass effect than curve A, then the bass performance curve is determined to match the optimal bass performance curve. Bass effect can be determined by factors such as the smoothness and stability of the curve.

[0074] After the step of detecting whether the bass performance curve matches the pre-stored optimal bass performance curve, the method further comprises:

[0075] Step A30, determining a pre-stored device bass performance curve, and detecting whether the pre-stored bass performance curve matches the device bass performance curve;

[0076] In step A40 , if there is a match, the sound corresponding to the sound sweep signal that matches the bass performance curve of the device is determined as the bass sound.

[0077] When there is no audio sweep signal that matches the optimal bass performance curve, the pre-stored device bass performance curve will be determined to detect whether the bass performance curve matches the device bass performance curve. When the two do not match, the determination of the bass sound is terminated and the VR device continues to be used for bass sound. When the two match, the audio corresponding to the audio sweep signal that matches the device bass performance curve is determined as the bass sound. The device bass performance curve refers to the bass performance curve of the VR device itself. By determining the bass sound that matches the device bass performance curve for bass sound, the bass sound effect of the VR device can be effectively achieved.

[0078] Furthermore, in a possible implementation, the step of determining a correction curve corresponding to the audio frequency sweep signal based on a pre-stored target frequency response includes:

[0079] Step B10, determining a mid-high frequency performance curve in the audio sweep signal, and determining an audio frequency response corresponding to the mid-high frequency performance curve, wherein the mid-high frequency performance curve is a curve in the performance curve having a frequency value greater than or equal to a preset frequency value;

[0080] Step B20, determining the difference between the audio frequency response and the pre-stored target frequency response as a correction dotted line;

[0081] Step B30: receiving an audible sound signal corresponding to the corrected dotted line, and performing sound control according to the audible sound signal and the corrected dotted line to obtain a correction curve.

[0082] In this embodiment, after the bass sound of the VR device is calibrated, the mid-high frequency of the VR device will be calibrated. Because of the sound requirements of the mid-high frequency (the mid-high frequency performance curve is a curve in which the frequency value in the performance curve is greater than or equal to the preset frequency value (which can be 300hz)), that is, the mid-high frequency sound of the VR device is jointly produced by the speaker unit of the VR device and the mid-high frequency of the audio, so in order to prevent the inconsistency of the sound of the two, it is necessary to correct the mid-high frequency of the audio, and then realize the mid-high frequency sound together with the speaker unit of the VR device. By performing Fourier transform on the audio sweep signal to obtain the mid-high frequency performance curve, and determining the audio frequency response corresponding to the mid-high frequency performance curve, the difference between the audio frequency response and the pre-stored target frequency response will be used as the correction dotted line, that is, to determine the difference between the frequency response of the mid-high frequency performance curve and the target frequency response of the mid-high frequency sound produced by the speaker unit inside the VR device. The mid-treble performance curve refers to the sound curve corresponding to the mid-treble in the audio sweep signal, the audio frequency response refers to the frequency response of the mid-treble corresponding to the mid-treble performance curve, the target frequency response refers to the frequency response of the mid-treble produced by the speaker unit inside the VR device, and the correction dotted line refers to the correction curve for calibration. Only when the correction dotted line passes the calibration will it be determined as the correction curve. After the correction dotted line is determined, it will be calibrated to obtain the correction curve.

[0083] After the step of determining the difference between the audio frequency response and the pre-stored target frequency response as the correction dotted line, the following steps are included:

[0084] Step B21, determining the target sound corresponding to the modified dotted line, and generating a sound demand instruction based on the modified dotted line;

[0085] Step B22: sending the voice request instruction to the target speaker.

[0086] In this embodiment, the correction dotted line is calibrated by determining the target sound corresponding to the correction dotted line, which will determine the sound sweep signal of the correction dotted line, and then determine the target sound corresponding to the correction dotted line. Because the correction dotted line needs to be calibrated, after obtaining the correction dotted line, a sound demand instruction will be generated, and then the target sound will be controlled to make a sound through the sound demand instruction, and the sound sound signal that is caused to make a sound and sent by the target sound based on the sound demand instruction of the correction dotted line will be received. Among them, the target sound refers to the sound corresponding to the correction dotted line, the sound demand instruction refers to the instruction to control the target sound to make a sound through Bluetooth, and the sound sound signal refers to the sound signal that the sound makes based on the sound demand instruction and is then transmitted to the VR device. Furthermore, this step can also be based on the generation of a mid-high pitch sound demand instruction based on the correction dotted line, so as to achieve the target sound to emit a mid-high pitch signal, and the mid-high pitch sound demand instruction refers to the instruction to control the sound to emit a mid-high pitch. Because the actual correction curve only calibrates the mid-high frequencies, the demand for mid-high frequencies can be directly sent to the speaker. After receiving the speaker sound signal or the mid-high frequency signal, the sound control will be performed according to the speaker sound signal or the mid-high frequency signal and the correction dotted line to obtain the correction curve, that is, to further verify whether the correction dotted line obtained before is qualified. The verification is performed through actual sound, thereby ensuring the accuracy of the correction curve.

[0087] In another possible implementation, the step of performing sound control according to the acoustic sound signal and the correction dotted line to obtain a correction curve includes:

[0088] Step C10, determining a mid-high frequency performance curve in the audio signal, and performing frequency response calibration on the mid-high frequency performance curve based on the corrected dotted line to obtain an audio frequency response;

[0089] Step C20, detecting whether the difference between the sound frequency response and the target frequency response is less than a preset value;

[0090] Step C30: If the value is less than the preset value, the correction dotted line is determined to be a correction curve.

[0091] In this embodiment, when calibrating the corrected dashed line, a mid-treble performance curve is determined for the audio signal or the mid-treble signal. The mid-treble performance curve is determined in the same manner as the aforementioned mid-treble performance curve, requiring Fourier transform. The obtained mid-treble performance curve is then subjected to frequency response calibration based on the corrected dashed line to obtain the audible frequency response. The audible frequency response refers to the audible frequency response of the audio signal or the mid-treble signal after calibration using the corrected dashed line. The difference between the audible frequency response and the target frequency response is compared with a preset value. If the difference is less than the preset value, the corrected dashed line is determined to meet the requirements and is then determined as the corrected curve. The difference between the audible frequency response and the target frequency response refers to the difference between the two across the entire frequency response. The frequency response refers to the frequency response, and the preset value is a user-defined optimal frequency response difference. Conversely, if the difference is not less than the preset value, the difference between the audible frequency response and the pre-stored target frequency response is used as the corrected dashed line, and the step of receiving the audio signal corresponding to the corrected dashed line is performed. By calibrating the correction dotted line, the accuracy of the determined correction curve can be guaranteed, thereby achieving the accuracy of the treble 1 control in the VR device.

[0092] Further, refer to Figure 4 , Figure 4 Schematic diagram of the speaker calibration process for a head-mounted display device. The number of speakers is determined, and the speaker's frequency sweep signals are compared to determine which one has better bass sound. This speaker is then designated as the bass speaker. The speaker module within the VR product then generates sound, and the calibrated microphone (MIC) receives the sound, thereby determining the target frequency response of the VR product. Each speaker is calibrated in turn. Taking speaker 1 as an example, the target frequency response is subtracted from the mid- and high-frequency response in speaker 1's frequency sweep signal to obtain a corrected dashed curve for speaker 1, which is stored in the equalizer (EQ). Speaker 1 is then controlled to sound again, and the corrected dashed line in the EQ is applied to analyze the frequency response of this re-sounding sound. The difference between the obtained frequency response and the target frequency response across the entire frequency range is tested to see if it is within 3dB. If so, calibration of the correction curve for speaker 1 is complete. Otherwise, the difference between the frequency response of the re-sounding speaker 1 and the target frequency response is determined again, and then the correction dotted line of the speaker 1 is obtained. Until the difference between the sound frequency response and the target frequency response in the entire frequency band is within 3dB, the correction curve calibration of the speaker 1 is completed, and the correction curves of the other speakers are calibrated. Then, the correction curves of all speakers in the mid-high frequency can be determined, and then the mid-high frequency of the speaker can be corrected when using VR equipment to achieve accuracy and authenticity of the mid-high frequency sound.

[0093] In one practicable manner, before the step of determining the sound position information corresponding to the sound sweep signal according to the collected instruction sending and receiving delay value, the step includes:

[0094] Step D10, determining that the audio frequency sweep signal corresponds to high-frequency sound pressure information, and determining the maximum sound pressure direction in the high-frequency sound pressure information as the audio angle position;

[0095] Step D20, obtaining a first delay value, wherein the first delay value is a time delay value between the time of the initial position playing instruction and the time of receiving the instruction;

[0096] Step D30, obtaining a second delay value, wherein the second delay value is a time delay value between the time of the play instruction at the end position and the time of receiving the instruction;

[0097] Step D40: Using the difference between the first delay value and the second delay value as a command sending and receiving delay value.

[0098] In this embodiment, because the sound waves of the left and right ears are prone to phase differences that are even stronger than the intensity differences when receiving mid- and high-frequency sounds, the high-frequency sounds have strong directionality. Therefore, it is necessary to calibrate the end point position to ensure the directionality of the mid- and high-frequency sounds and improve the user's experience of the mid- and high-frequency sounds when using VR devices. By determining the high-frequency sound pressure information corresponding to the sound sweep signal, the high-frequency sound pressure information refers to the sound pressure in different directions when the sound is emitting high-frequency sound. When the sound is emitting high-frequency sound, the direction of the maximum sound pressure in the high-frequency sound pressure information is determined to be the sound angle position. The sound angle position refers to the direction of the sound in the wearer's direction. The distance between the wearer's position and the sound is determined based on the sound angle position. By determining the first delay value between the first play instruction and the first receive instruction at the initial position, and at the same time determining the second delay value between the second play instruction and the second receive instruction at the end point, the difference between the first delay value and the second delay value is finally used as the instruction sending and receiving delay value. The initial position refers to the position where the wearer wears the VR device, and generally the wearer starts using the VR device at a fixed position. The end position refers to the end position where the wearer wears the device and is infinitely close to the speaker. The first play instruction and the first receive instruction refer to the instruction for controlling the speaker playback sent by the wearer at the initial position and the instruction for receiving the speaker playback sound. The second play instruction and the second receive instruction refer to the instruction for controlling the speaker playback sent by the wearer at the end position and the instruction for receiving the speaker playback sound. The first delay value refers to the time delay value between the first play instruction and the first receive instruction. The second delay value refers to the time delay value between the second play instruction and the second receive instruction. The difference between the two delay values ​​is then determined as the instruction sending and receiving delay value. By determining the delay values ​​of the end position and the initial position, it can be ensured that the delay time of internal instruction sending does not affect the distance determination, thereby improving the accuracy of distance determination.

[0099] The step of determining the sound position information corresponding to the sound sweep signal according to the collected instruction receiving and sending delay value includes:

[0100] Step D50 , determining the sound distance corresponding to the collected command receiving and sending delay value, and using the sound distance corresponding to the sound angle position as the sound position information corresponding to the sound sweep signal.

[0101] In this embodiment, the sound distance corresponding to the command transmission and reception delay value is determined. The determination step is to accurately determine the sound distance based on the command transmission and reception delay value and the propagation speed of sound. The sound distance refers to the distance from the initial position to the end position. The sound distance is determined by the command transmission and reception delay value between the two positions, thereby avoiding the phenomenon of inaccurate distance determination caused by the actual command transmission and reception, internal command processing, and internal delay. Ultimately, the sound distance corresponding to the sound angle position can be determined as the sound position information corresponding to the sound sweep signal, that is, the angle and distance initial position of the sound in the entire area are determined, thereby realizing the connection between the sound emission and the distance, thereby improving the authenticity of the sound emitted by the VR device. On the other hand, when the VR device is equipped with a camera, the sound position information will be directly determined by the camera, thereby realizing the combination of the sound position information with the wearer's movement, thereby improving the authenticity of the sound emitted by the VR device.

[0102] In a possible implementation, the step of performing sound control based on the sound position information, the correction curve set, and the bass sound includes:

[0103] Step E10, determining the sound production requirement corresponding to the audio to be played, and detecting whether the sound production requirement is a mid-high pitch sound production requirement;

[0104] Step E20: If the sound production requirement is a mid-high frequency sound production requirement, receiving a mid-high frequency sound produced by the speaker, determining a target correction curve for the mid-high frequency sound in the correction curve set, and performing frequency response calibration on the mid-high frequency sound based on the speaker position information and the target correction curve to produce mid-high frequency sound.

[0105] Step E30: If the sound production requirement is not a mid-high pitch sound production requirement, the bass sound producing speaker is controlled to produce bass sound.

[0106] In this embodiment, when sound control is performed, the sound requirement is determined based on the audio to be played. That is, the decibel level required by the wearer at the location is determined based on the audio location information. The audio to be played refers to the audio to be played. The sound requirement refers to the type of sound required by the control speaker, including bass and mid-high tones, and decibel requirements are only set for mid-high tones. The steps before determining the sound requirement based on the audio location information include:

[0107] Step E01: collecting angle information and correcting the sound position information based on the angle information;

[0108] Step E02: updating the sound position information according to the corrected sound position information.

[0109] In this embodiment, since the user's wearing position can be fixed each time, but the actual angle may vary, the IMU can be used to determine angle information, and the sound position information can be corrected based on this angle information. The angle information refers to the angle between the VR device and the calibration. This means that the sound position information during calibration may be directly in front, in which case the default angle information is 0 degrees. If the angle information collected during wear is 30 degrees to the right, the sound position information is corrected to 30 degrees to the left of the front. The corrected sound position information is then used to update the sound position information, thereby ensuring the accuracy of the sound position information during each use and improving the accuracy of subsequent mid- and high-frequency sound control. As an extreme case, the user can start wearing the headset on a circle centered on the speaker. The sound position information is then updated based on the speaker's angle position, based on the collected angle information and the high-frequency sound pressure information obtained from the pre-calibration, with the maximum sound pressure direction as the speaker.

[0110] The system detects whether the sound demand is for mid- and high-frequency sound generation. The mid- and high-frequency sound generation requirement refers to whether mid- and high-frequency sound generation is required. If mid- and high-frequency sound generation is not required, the system directly controls the bass-generating speaker via Bluetooth to generate bass sound. Because bass is not highly directional, the placement of the bass-generating speaker does not require determining the speaker's position information. However, if the bass-generating speaker generates mid- and high-frequency sound generation, determining the speaker's position information is required. Conversely, if mid- and high-frequency sound generation is required, the system controls the speaker to generate mid- and high-frequency sound generation, and receives mid- and high-frequency frequency response information from at least one speaker. For example, using speaker 2 as an example, the mid- and high-frequency frequency response refers to the frequency response corresponding to the mid- and high-frequency sound generation. A target correction curve for the mid- and high-frequency frequency response within the correction curve set is determined, specifically the target correction curve for speaker 2 within the correction curve set. The target correction curve refers to the correction curve previously obtained by speaker 2 during calibration. Frequency response calibration of the mid- and high-frequency frequency response is then performed based on the target correction curve to generate mid- and high-frequency sound generation. That is to say, the mid-high tones emitted by the speakers are calibrated with the corresponding correction curve for frequency response (correcting the mid-high tones), and then the mid-high tones are emitted by the speaker unit inside the VR device. The mid-high tones sound refers to the mid-high tones sound of the speakers combined with the mid-high tones sound of the internal speaker unit. On the one hand, the mid-high tones sound of the speakers are combined with the mid-high tones sound of the internal speaker unit to achieve the diversification of the sound units. On the other hand, the bass sound of the speakers can be directly emitted through the bass sound to improve the bass sound effect of the VR device. After the mid-high tones sound is calibrated based on the target correction curve to perform the mid-high tones sound, the process includes:

[0111] Step E21: Collect real-time movement information and determine the mid-high frequency control instruction based on the movement information and the speaker position information;

[0112] Step E22: Control at least one of the speakers to generate a specified mid-high frequency sound based on the mid-high frequency control instruction;

[0113] Step E23: Perform frequency response calibration on the specified mid-high frequency sound based on the target correction curve to achieve mid-high frequency sound production.

[0114] In this embodiment, during the actual use of the VR device, by collecting real-time movement information and determining the mid-high frequency control instruction based on the movement information and the speaker position information, the movement information refers to information such as the distance and angle of the wearer's movement, and the mid-high frequency control instruction refers to an instruction for controlling the mid-high frequency to produce sound. That is, according to the distance and angle of the user's movement, the sound production standard of the mid-high frequency is determined. For example, if the mid-high frequency sound of the speaker is emitted D meters in front, and the user moves forward in a straight line by C meters (C < D), the sound production requirement of the mid-high frequency of the speaker at the user's moved position will be determined, and then the mid-high frequency control instruction will be generated to control the speaker to achieve mid-high frequency sound production. Then, the step of controlling at least one of the speakers to generate a specified mid-high frequency sound based on the mid-high frequency control instruction will be executed. Its processing flow is the same as the step of receiving the mid-high frequency sound sent by at least one of the speakers, except that the former controls the generation of the specified mid-high frequency sound according to the mid-high frequency control instruction, and the latter generates the mid-high frequency sound according to the mid-high frequency sound production requirement. That is, the specified mid-high frequency sounds at different positions are different, so control instructions are needed to control the generation of different specified mid-high frequency sounds at different positions. The specified mid-high frequency sound refers to the mid-high frequency sound that needs to be generated corresponding to the control instruction. By generating different specified mid-high frequency sounds at different positions, the authenticity of the mid-high frequency sound production of the speaker is realized.

[0115] In a possible implementation manner, before the step of receiving the speaker sweep signal of multiple speakers, it includes:

[0116] Step F10: Establish a Bluetooth connection with the Bluetooth of multiple speakers;

[0117] Step F20: Based on the Bluetooth connection, sequentially send a speaker sweep signal requirement instruction to the speakers.

[0118] In this embodiment, in a system established by a head-mounted display device and a speaker connected via Bluetooth, the entire system can be controlled by activating the Bluetooth of the head-mounted display device and establishing a Bluetooth connection with the speaker. Through the Bluetooth connection with the head-mounted display device, the speaker sequentially sends audio sweep signal request instructions. The audio sweep signal request instructions are instructions to the head-mounted display device requesting the speaker to generate and transmit an audio sweep signal. Upon receiving the audio sweep signal, the speaker generates the audio sweep signal and transmits it to the head-mounted display device. The head-mounted display device then performs calibration based on the received audio sweep signal. The audio sweep signal is used to determine the bass sound, thereby achieving bass sound. The audio sweep signal is also used to determine the correction curve for mid- and high-pitched sounds and the speaker's position information, thereby achieving mid-, high-, and low-pitched sound.

[0119] The step of receiving an audio sweep signal of at least one audio device comprises:

[0120] Step F30 : receiving an audio frequency sweep signal generated by at least one audio device based on the audio frequency sweep signal request instruction.

[0121] In this embodiment, an audio sweep signal request instruction is sent to a speaker, which then responds with an audio sweep signal. The head-mounted display device then receives the audio sweep signal generated by at least one speaker based on the audio sweep signal request instruction. The at least one speaker refers to a speaker connected to the head-mounted display device via Bluetooth, i.e., a speaker in the system comprising the speaker and the head-mounted display device. By sequentially sending the audio sweep signal request instruction to each speaker in the system, the system receives the audio sweep signal sent by each speaker. This allows for calibration of the bass and mid- and high-frequency sound of the head-mounted display device based on the audio sweep signal, thereby improving the bass sound quality of the head-mounted display device.

[0122] Example 2

[0123] Based on the first embodiment of the present application, in another embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 2 , a flow chart of a second embodiment of the sound control method, after the step of obtaining all input audio sweep frequency signals, the method further includes:

[0124] Step S310, determining a personalized sound according to each of the audio sweep frequency signals;

[0125] In this embodiment, because the mid- and high-pitched sounds are combined with the sound waves, personalized sound needs can be achieved through the sound waves. The personalized sound sound is determined by the sound wave sweep signal, and its implementation process is consistent with the process of determining the bass sound sound according to each of the sound wave sweep signals. The step of determining the personalized sound sound according to each of the sound wave sweep signals includes:

[0126] Step G10, sequentially determining mid- and high-frequency performance curves in the audio sweep signal, and detecting whether the mid- and high-frequency performance curves match a pre-stored optimal personalized sound performance curve, wherein the mid- and high-frequency performance curves are obtained by performing Fourier transform on the audio sweep signal;

[0127] If a match is found, step G20 determines that the sound source corresponding to the audio frequency sweep signal that matches the optimal personalized sound performance curve is designated as the personalized sound source. The focus is on testing whether the intensity difference and phase difference of the mid- and treble-frequency performance curve, after Fourier transformation, meet the intensity and phase difference requirements of the optimal personalized sound performance curve. However, the personalized sound source tests whether the mid- and treble-frequency performance curve matches the pre-stored optimal personalized sound performance curve. The optimal personalized sound performance curve refers to a mid- and treble-frequency curve that meets the required mid- and treble-frequency performance. Specifically, the personalized sound source at that frequency is determined. Steps A30 and A40 are also included to determine if the sound source at a specific frequency matches the device's sound quality at that frequency and is therefore designated as the personalized sound source. A personalized sound source is a sound source that produces personalized mid- and treble-frequency sounds. The mid- and treble-frequency sound source can be customized based on user needs, thereby improving the functionality of the VR device and enhancing user experience and choice.

[0128] Step S320, determining a personalized correction curve corresponding to the personalized sound based on a pre-stored personalized target frequency response;

[0129] Step S330 , determining personalized audio position information corresponding to the personalized sound emitting sound according to the collected personalized instruction sending and receiving delay value, and performing sound control according to the personalized audio position information and the personalized correction curve.

[0130] In this embodiment, after determining the personalized sound emitting speaker, a personalized correction curve corresponding to the personalized sound emitting speaker is determined based on a pre-stored personalized target frequency response. This implementation is identical to step S200 of the first embodiment, except that the personalized target frequency response is used as a subtrahend to obtain the personalized correction curve, while step S200 uses the target frequency response as a subtrahend to obtain the correction curve. The personalized target frequency response refers to the customized frequency response of the VR device's internal speakers. For example, if the target frequency response is 20dB, the user can customize the target frequency response to 10dB or 5dB. The personalized sound location information corresponding to the personalized sound emitting speaker is then determined based on the collected personalized command transmission and reception delay value. Sound control is then performed based on this personalized sound location information and the personalized correction curve. The personalized sound location information refers to the location of the sound emitting the personalized sound. Because personalized sound emitting is a mid- to high-pitched sound, the location must be determined to ensure the authenticity of sound control. The personalized command transmission and reception delay value refers to the delay between the transmission and reception of commands by the personalized sound emitting speaker. Then, by changing the personalized correction curve through personalized target frequency response, personalized sound of mid-high pitch can be achieved, improving the sound diversity of VR equipment.

[0131] Further, refer to Figure 5 , Figure 5 Schematic diagram of the personalized audio calibration process for head-mounted display devices. When the wearer selects personalized audio, the DSP (digital signal processor) will process and analyze the frequency characteristics of the frequency response, and select the appropriate audio equipment based on the frequency characteristics, and recalibrate the selected audio equipment again. The test audio is played in sequence through multiple audio devices. After the calibration MIC receives it, the intensity difference and phase difference are determined according to the Fourier transform technology to determine whether the intensity difference and phase difference meet the requirements. If it meets the requirements, it is determined that the appropriate audio device is selected as the personalized sound. Otherwise, the appropriate audio device is re-determined based on the test audio until the intensity difference and phase difference meet the requirements. This step can be the step of obtaining all the input audio sweep signals and then performing personalized audio calibration and control. The determination of the personalized sound can achieve personalized sound for VR devices, expanding the sound selectivity of VR devices.

[0132] In another possible implementation, the step of performing sound control according to the personalized sound position information and the personalized correction curve includes:

[0133] Step K10: if a personalized demand instruction is received, determining a target personalized sound sound corresponding to the personalized demand instruction;

[0134] Step K20: performing personalized sound production based on the target personalized sound production sound.

[0135] In this embodiment, when a user inputs a personalized demand instruction during use, the VR device will control the target personalized sound sound according to the target personalized sound sound corresponding to the personalized demand instruction, and correct it using the target personalized sound sound correction curve corresponding to the target personalized sound sound. The personalized demand instruction refers to the instruction input by the user to select personalized sound, and the target personalized sound sound refers to the speaker corresponding to the user's personalized sound. For example, if the personalized demand instruction is a heavy high-pitched sound, the speaker 3 corresponding to the heavy high-pitched sound determined during the previous calibration will be determined, and the sound will be emitted through this speaker 3. Personalized sound control enables personalized sound of the VR device, expanding the functionality of the VR device.

[0136] To help understand the technical concept of this application, a specific embodiment is listed below:

[0137] The head-mounted display device involved in this specific embodiment is a VR device, which generates a virtual world in a three-dimensional space through computer simulation. With the support of playback plug-ins such as Java or Quicktime, ActiveX, Flash, etc., the experimenter can also zoom in, zoom out, rotate, etc. on the image, allowing the user to experience the unparalleled realism, stereoscopic feeling, and immersive feeling of general images and three-dimensional shapes. This embodiment establishes a stereo system with the VR device through Bluetooth and speakers, thereby supporting more diverse sound units, including tweeters, midrange units, and woofers. Its usage scenario is shown in the figure below. Figure 3 As shown, a schematic diagram of the scene constructed by the head-mounted display device and the speakers. The VR device establishes a Bluetooth connection with the speakers in the system (the system includes the VR device and multiple speakers) via Bluetooth, and then the Bluetooth control of the VR device can realize the sound (the bass is sounded by a specific bass sounding speaker, and the mid-high tones are sounded by the speaker unit of the VR device and the mid-high tones of the speakers). This can overcome the poor bass effect when the VR device is sounding, and can also expand the user's requirement for personalized sound selection through the sound. During calibration, a sound sweep signal is sent to all speakers in the system through Bluetooth, and then the bass sound and the correction curve of each speaker are determined based on the sound sweep signal. And when correcting, the position of all speakers is determined, for example, the distance X of speaker 1 to the VR device in the figure, and the wearer can determine whether the calibration MIC needs to change the sound according to the wearer's walking through distance detection. For example, if the wearer wearing the VR device moves closer to Speaker 1, the mid-high tones of the calibrated MIC will become louder (higher decibel value), and conversely, if the wearer moves away from Speaker 1, the mid-high tones of the calibrated MIC will become smaller (lower decibel value). There is also the angle issue, whether the mid-high tones are emitted from the back or the front.

[0138] The IMU in the VR device determines the wearer's position and angle changes while wearing the VR device. Bass doesn't suffer from these changes. The principle of sound source localization states that when a sound source reaches the human ear, there are differences in sound pressure level, time, and phase. These differences are processed by the human brain, allowing the user to perceive the direction of the sound source. This is sound localization. The human ear's ability to localize sound sources also depends on frequency. Frequency localization is poor for frequencies below 300Hz (bass) due to its strong diffraction. Because its wavelength is much larger than the distance between the ears, the phase and intensity differences perceived by the human ear are minimal, resulting in limited localization. Therefore, a home theater typically requires only a single subwoofer, which can be placed anywhere, without requiring specific angles or positions. However, the ability to localize sound above 300Hz (mid- and high-pitched sounds) gradually improves. As frequency increases, the wavelength shortens. By the time a sound reaches the human ear, the distance between the ears is already significant compared to its wavelength. At this point, the phase difference between the sound waves received by the left and right ears is likely to be even greater than the intensity difference, resulting in a strong directional effect for high-pitched sounds. Therefore, for the mid- and high-pitched sounds of the speakers, it is necessary to determine the position and angle relationship, so as to determine how the decibel value of the mid- and high-pitched sounds needs to change when the wearer moves or after moving, so as to improve the authenticity of the VR device. It is also possible to use specific bass sounds to produce the effect of bass sounds through the VR device.

[0139] It should be noted that the many details described in this specific embodiment are only helpful for understanding the technical concept of this application and do not constitute a limitation of this application. More simple transformations based on the technical concept of this application should all be within the scope of protection of this application.

[0140] Example 3

[0141] An embodiment of the present invention provides a head-mounted display device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the sound control method of the above-mentioned embodiment 1.

[0142] Reference below Figure 6 , Figure 6This is a schematic diagram of the device structure of the hardware operating environment involved in the head-mounted display device, which shows a schematic diagram of the structure of the head-mounted display device suitable for implementing the embodiments of the present disclosure. The head-mounted display device in the embodiments of the present disclosure may include, but is not limited to, mixed reality (Mixed Reality) - MR devices (such as MR glasses or MR helmets), augmented reality (Augmented Reality) - AR devices (such as AR glasses or AR helmets), virtual reality (Virtual Reality) - VR devices (such as VR glasses or VR helmets), extended reality (Extended Reality) - XR devices, or some combination thereof, etc. Figure 6 The head-mounted display device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0143] like Figure 6 As shown, the head-mounted display device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM 1002) or a program loaded from a storage device into a random access memory (RAM 1004). The RAM 1004 also stores various programs and data required for the operation of the head-mounted display device. The processing device 1001, ROM 1002, and RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface is also connected to the bus 1005.

[0144] Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, hard disk, etc.; and communication devices 1009. The communication devices 1009 can allow the head-mounted display device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a head-mounted display device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have instead.

[0145] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0146] The head-mounted display device provided by the present invention, using the sound control method of the first or second embodiment above, can enhance the low-frequency sound effects of the VR device through bass sound, and can also achieve personalized sound selection by combining the sound with the internal speaker unit of the VR device. Compared with the prior art, the beneficial effects of the head-mounted display device provided by the embodiment of the present invention are the same as those of the sound control method provided by the first embodiment above, and the other technical features of the head-mounted display device are the same as those disclosed in the method of the previous embodiment, and are not further described here.

[0147] It should be understood that various parts of the present disclosure can be implemented with hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in an appropriate manner.

[0148] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0149] Example 4

[0150] An embodiment of the present invention provides a computer storage medium, which is a computer storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the sound control method in the above-mentioned embodiment 1.

[0151] The computer storage medium provided in the embodiment of the present invention can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer storage medium can be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0152] The computer storage medium may be included in the head-mounted display device, or may exist independently without being assembled into the head-mounted display device.

[0153] The computer storage medium carries one or more programs. When executed by a head-mounted display device, the one or more programs cause the head-mounted display device to: receive a sound sweep frequency signal from at least one sound source, and determine a bass sound source based on each of the sound sweep frequency signals; determine a correction curve corresponding to the sound sweep frequency signal based on a pre-stored target frequency response, and aggregate the correction curves corresponding to each of the sound sweep frequency signals to form a correction curve set; determine sound location information corresponding to the sound sweep frequency signal based on a collected instruction transmission and reception delay value, and perform sound control based on the sound location information, the correction curve set, and the bass sound source.

[0154] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0155] The flow charts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0156] The modules involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0157] The computer storage medium provided by the present invention stores computer-readable program instructions for executing the aforementioned sound control method. This can enhance the low-frequency sound effects of a VR device through bass sound, and can also achieve personalized sound selection by combining the sound with the VR device's internal speaker unit. Compared to the prior art, the beneficial effects of the computer storage medium provided by the embodiments of the present invention are the same as those of the sound control method provided by the aforementioned first or second embodiments, and are not further elaborated here.

[0158] Example 5

[0159] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the steps of the above-mentioned sound control method when executed by a processor.

[0160] The computer program product provided in this application can improve the low-frequency sound effects of VR devices through bass sound, and can also achieve personalized sound selection by combining the sound with the internal speaker unit of the VR device. Compared with the existing technology, the beneficial effects of the computer program product provided by the embodiment of the present invention are the same as the beneficial effects of the sound control method provided by the above-mentioned embodiment 1 or embodiment 2, and will not be repeated here.

[0161] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A method for controlling sound emission, characterized in that: The sound control method is applied to a head-mounted display device, and the steps of the sound control method include: receiving audio sweep frequency signals of a plurality of audio systems, and determining a bass sounding audio system according to each of the audio sweep frequency signals; Determining a correction curve corresponding to the audio sweep signal based on a pre-stored target frequency response, and aggregating the correction curves corresponding to the audio sweep signals to obtain a correction curve set, wherein the correction curve is a sound curve for correcting the frequency response of the mid- and high-pitched sounds of the audio; determining the sound position information corresponding to the sound sweep signal according to the collected instruction sending and receiving delay value, and performing sound control according to the sound position information, the correction curve set, and the bass sound; The step of performing sound control according to the sound position information, the correction curve set and the bass sound comprises: Determine the sound production requirement corresponding to the audio to be played, and detect whether the sound production requirement is a mid-high pitch sound production requirement; If the sound production requirement is a mid-high frequency sound production requirement, receiving the mid-high frequency sound produced by the speaker, determining a target correction curve for the mid-high frequency sound in the correction curve set, and performing frequency response calibration on the mid-high frequency sound based on the speaker position information and the target correction curve to produce the mid-high frequency sound; If the sound production requirement is not a mid-high pitch sound production requirement, the bass sound producing speaker is controlled to produce bass sound.

2. The sound control method according to claim 1, wherein: The step of determining the bass sound according to each of the audio sweep frequency signals comprises: sequentially determining a bass performance curve corresponding to the audio frequency sweep signal, and detecting whether the bass performance curve matches a pre-stored optimal bass performance curve, wherein the performance curve is obtained by performing a Fourier transform on the audio frequency sweep signal, and the bass performance curve is a curve in the performance curve having a frequency value less than a preset frequency value; If they match, the sound corresponding to the sound sweep signal that matches the optimal bass performance curve is determined as the bass sounding sound.

3. The sound control method according to claim 2, wherein: The step of determining the correction curve corresponding to the audio frequency sweep signal based on the pre-stored target frequency response includes: Determining a mid-high frequency performance curve corresponding to the audio frequency sweep signal, and determining an audio frequency response corresponding to the mid-high frequency performance curve, wherein the mid-high frequency performance curve is a curve in the performance curve having a frequency value greater than or equal to a preset frequency value; Determine the difference between the audio frequency response and a pre-stored target frequency response as a correction dotted line; An acoustic sound signal corresponding to the correction dotted line is received, and sound control is performed according to the acoustic sound signal and the correction dotted line to obtain a correction curve.

4. The sound control method according to claim 3, wherein: The step of performing sound control according to the sound sound signal and the correction dotted line to obtain a correction curve includes: Determining a mid-high frequency performance curve in the sound signal, and performing frequency response calibration on the mid-high frequency performance curve based on the corrected dotted line to obtain a sound frequency response; Detecting whether a difference between the sound frequency response and the target frequency response is less than a preset value; If it is less than the preset value, the correction dotted line is determined to be a correction curve.

5. The sound control method according to claim 1, wherein: Before the step of determining the sound position information corresponding to the sound sweep signal according to the collected instruction receiving and sending delay value, the method further comprises: Determining that the audio frequency sweep signal corresponds to high-frequency sound pressure information, and determining that the maximum sound pressure direction in the high-frequency sound pressure information is the audio angular position; Obtaining a first delay value, wherein the first delay value is a time delay value between the time of the play instruction at the initial position and the time of receiving the instruction; Obtaining a second delay value, wherein the second delay value is a time delay value between the time of playing the instruction at the end position and the time of receiving the instruction; Using the difference between the first delay value and the second delay value as the instruction sending and receiving delay value; The step of determining the sound position information corresponding to the sound sweep signal according to the collected instruction receiving and sending delay value includes: The sound distance corresponding to the collected command receiving and sending delay value is determined, and the sound distance corresponding to the sound angle position is used as the sound position information corresponding to the sound sweep signal.

6. The sound control method according to claim 1, wherein: The step of receiving the audio sweep frequency signals of the plurality of audio devices includes: Establish Bluetooth connections with multiple speakers; sending audio frequency sweep signal request instructions to the audio system in sequence based on the Bluetooth connection; The step of receiving the audio sweep frequency signals of multiple audio systems includes: The audio frequency sweep signals generated by a plurality of audio systems based on the audio frequency sweep signal request instruction are received.

7. The sound control method according to claim 1, wherein: After the step of receiving the audio sweep frequency signals of a plurality of audio devices, the method further includes: Determining the personalized sound output according to each of the audio sweep frequency signals; Determining a personalized correction curve corresponding to the personalized sound emission sound based on a pre-stored personalized target frequency response; The personalized sound position information corresponding to the personalized sound emitting sound is determined according to the collected personalized instruction sending and receiving delay value, and the sound emission control is performed according to the personalized sound position information and the personalized correction curve.

8. A head-mounted display device, characterized in that: The head-mounted display device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the sound control method described in any one of claims 1 to 7.

9. A computer storage medium, characterized in that The computer storage medium stores a program for implementing the sound control method, and the program for implementing the sound control method is executed by a processor to implement the steps of the sound control method according to any one of claims 1 to 7.

Images