Sound source adjustment method and device based on audience position, equipment and medium

Abstract

The embodiment of the invention provides a sound source adjustment method and device based on audience positions, equipment and a medium, and the method comprises the steps: obtaining sound source image position information and audience position information in a screen image display process, and recognizing a connection line between a sound source and audiences according to the information. And then, determining a target ultrasonic transmitting unit according to the identified connecting line and intersection point. Finally, these processing policies are executed to ensure that the audio beam can accurately point to the viewer's location. According to the embodiment of the invention, the operation process for realizing sound and picture synchronization is simplified, and more efficient and immersive watching experience is provided for audiences.

Description

Technical Field

[0001] The present invention relates to the technical field of sound source adjustment based on audience position, and in particular to a sound source adjustment method, device, equipment and medium based on audience position. Background Art

[0002] With the development of technology, the types and functions of audio-visual products are increasing, and the audience's requirements for the effects of audio-visual products are also increasing. However, when the audience deviates from the sweet spot of the picture, the sound field produced by the speakers will deteriorate sharply, and there will be an obvious sense of spatial separation between the display screen, causing the sound and picture to be out of sync, affecting the viewing experience. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a method, device, equipment and medium for adjusting a sound source based on the audience position, which overcome the above problems or at least partially solve the above problems.

[0004] In order to solve the above problem, an embodiment of the present invention discloses a method for adjusting a sound source based on an audience position, comprising:

[0005] During the display of the picture on the screen, determining the position information of the sound source image and determining the position information of the audience;

[0006] Determine a connecting line corresponding to the position information of the sound source image and the position information of the audience, and extend the connecting line to an intersection on the screen;

[0007] According to the intersection point, determining a plurality of target ultrasonic emitting units from a plurality of ultrasonic emitting units arranged along the screen;

[0008] The audio beams output by the plurality of target ultrasonic emitting units are controlled respectively according to the position information of the audience.

[0009] Optionally, determining the location information of the sound source image includes:

[0010] Determine the type of sound source;

[0011] When the sound source type is human, determining the position of the center point of the human face and mouth in the sound image, and using the position of the center point of the human face and mouth as the position information of the sound source image;

[0012] When the sound source type is non-human, the lateral center point position of the sound image is determined, and the lateral center point position is used as the position information of the sound source image.

[0013] Optionally, the plurality of ultrasonic emitting units are arranged along a lateral edge of the screen; and the step of determining a plurality of target ultrasonic emitting units from the plurality of ultrasonic emitting units arranged along the screen according to the intersection point comprises:

[0014] Determine a target area in a deployment area of ​​the plurality of ultrasonic transmitting units along a vertical direction of the intersection;

[0015] The ultrasonic wave transmitting unit located in the target area is determined as the target ultrasonic wave transmitting unit.

[0016] Optionally, the controlling the audio beams output by the plurality of target ultrasonic emitting units respectively according to the position information of the audience includes:

[0017] According to the position information of the audience, the directions of the audio beams output by the multiple target ultrasonic emitting units are respectively controlled to be toward the audience.

[0018] Optionally, the controlling the audio beams output by the plurality of target ultrasonic emitting units respectively according to the position information of the audience further includes:

[0019] Determining the distances of the plurality of target ultrasonic emitting units relative to the audience according to the position information of the audience;

[0020] The widths of the audio beams output by the multiple target ultrasonic emitting units are controlled according to the distances of the multiple target ultrasonic emitting units relative to the audience.

[0021] Optionally, the method further comprises:

[0022] When the position information of the audience remains unchanged and the position information of the sound source image changes, detecting the change angle of the direction of the audio beams output by the multiple target ultrasonic emitting units;

[0023] When the change angle is greater than a preset angle threshold, the target ultrasonic emitting unit is re-determined.

[0024] Optionally, the method further comprises:

[0025] When the position information of the audience changes and the position information of the sound source image remains unchanged, determining whether the position information of the audience exceeds the coverage range of the audio beams output by the multiple target ultrasonic emitting units;

[0026] When the position information of the audience exceeds the coverage of the audio beams output by the multiple target ultrasonic emitting units, the target ultrasonic emitting units are re-determined.

[0027] Correspondingly, the embodiment of the present invention further discloses a sound source adjustment device based on the audience position, comprising:

[0028] A position information determination module is used to determine the position information of the sound source image and the position information of the audience during the process of displaying the picture on the screen;

[0029] An intersection determination module, used to determine a connecting line corresponding to the position information of the sound source image and the position information of the audience, extending the connecting line to the intersection on the screen;

[0030] a target ultrasonic emitting unit determining module, configured to determine a plurality of target ultrasonic emitting units from a plurality of ultrasonic emitting units arranged along the screen according to the intersection point;

[0031] The audio beam control module is used to control the audio beams output by the multiple target ultrasonic emitting units respectively according to the position information of the audience.

[0032] Optionally, the intersection determination module includes:

[0033] A sound source type determination submodule, used to determine the sound source type;

[0034] A first position information determination submodule, for determining the position of the center point of the human face and mouth in the sound image when the sound source type is human, and using the center point of the human face and mouth as the position information of the sound source image;

[0035] The second position information determination submodule is used to determine the lateral center point position of the sound image when the sound source type is non-human, and use the lateral center point position as the position information of the sound source image.

[0036] Optionally, the plurality of ultrasonic emitting units are arranged along a lateral edge of the screen; and the target ultrasonic emitting unit determination module comprises:

[0037] A target area determination submodule, used to determine a target area in a deployment area of ​​the plurality of ultrasonic transmitting units along a vertical direction of the intersection;

[0038] The target ultrasonic emitting unit submodule is used to determine the ultrasonic emitting unit located in the target area as the target ultrasonic emitting unit.

[0039] Optionally, the audio beam control module includes:

[0040] The audio beam direction control submodule is used to control the directions of the audio beams output by the multiple target ultrasonic emitting units to be toward the audience according to the position information of the audience.

[0041] Optionally, the audio beam control module further includes:

[0042] A distance determination submodule, used to determine the distances of the plurality of target ultrasonic emitting units relative to the audience according to the position information of the audience;

[0043] The width determination submodule is used to control the widths of the audio beams output by the multiple target ultrasonic emitting units according to the distances of the multiple target ultrasonic emitting units relative to the audience.

[0044] Optionally, the method further comprises:

[0045] An angle detection module, used for detecting the change angle of the direction of the audio beams output by the multiple target ultrasonic emitting units when the position information of the audience remains unchanged and the position information of the sound source image changes;

[0046] The first redetermining submodule is used to redetermine the target ultrasonic transmitting unit when the change angle is greater than a preset angle threshold.

[0047] Optionally, the method further comprises:

[0048] A coverage range determination module, used for determining whether the position information of the audience exceeds the coverage range of the audio beams output by the multiple target ultrasonic emitting units when the position information of the audience changes and the position information of the sound source image remains unchanged;

[0049] The second redetermining submodule is used to redetermine the target ultrasonic emitting unit when the position information of the audience exceeds the coverage of the audio beams output by the multiple target ultrasonic emitting units.

[0050] The embodiments of the present invention include the following advantages:

[0051] In an embodiment of the present invention, firstly, the position information of the sound source image in the picture displayed on the screen and the position information of the audience are obtained; then, by analyzing these position information, the connecting line corresponding to the position information of the sound source image and the position information of the audience is determined, and extended to the intersection on the screen; then, according to the identified intersection, multiple target ultrasonic emitting units are determined from multiple ultrasonic emitting units arranged along the screen; finally, according to the position information of the audience, the audio beams output by the multiple target ultrasonic emitting units are controlled respectively. In this way, the embodiment of the present invention can comprehensively, quickly and accurately identify and process the relationship between the sound source image and the audience position, enhance the intelligence of the audio output, realize the synchronization of sound and picture, and ensure that the audience obtains the best auditory experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1is a flow chart of the steps of a method for adjusting a sound source based on an audience position provided by an embodiment of the present invention;

[0053] Figure 2 is a flowchart of another method for adjusting a sound source based on an audience position provided by an embodiment of the present invention;

[0054] Figure 3 is a flowchart of specific steps of a method for adjusting a sound source based on audience positions provided by an embodiment of the present invention;

[0055] Figure 4 is a flowchart of specific steps of another method for adjusting a sound source based on audience positions provided by an embodiment of the present invention;

[0056] Figure 5 It is a structural block diagram of a sound source adjustment device based on audience position provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] With the development of technology, the types and functions of audio-visual products are increasing, and the audience's requirements for the effects of audio-visual products are also increasing. However, when the audience deviates from the sweet spot of the picture, the sound field produced by the speaker will deteriorate sharply, and there will be a clear sense of spatial separation between the display screen, causing the sound and picture to be out of sync, affecting the viewing experience.

[0059] Traditional countermeasures seem to be inadequate. These problems not only involve the expansion of audio-visual equipment, but also the complexity of sound field management. Traditional audio-visual systems fail to make full use of information and digital technologies to monitor and optimize sound field effects in real time, thus failing to effectively improve the audience's auditory experience, bringing inconvenience to people's daily lives.

[0060] One of the core concepts of the embodiments of the present invention is that the position information of the sound source image in the picture displayed on the screen and the position information of the audience can be obtained in real time. At the same time, the abnormal sound field area can be accurately identified and processed by controlling the speaker or linking other devices. By accurately analyzing these position information, the relationship between the sound source image and the audience position can be accurately identified. Based on these identification results, the audio and video system will display the corresponding processing strategy for rapid response. In this way, abnormal sound field conditions can be quickly and accurately identified and processed, and the audio and video synchronization effect is intelligently achieved to ensure that the audience has the best auditory experience.

[0061] Reference Figure 1, shows a flow chart of the steps of a sound source adjustment method based on audience position provided by an embodiment of the present invention, and the method may specifically include the following steps:

[0062] Step 101, during the process of displaying the picture on the screen, determining the position information of the sound source image and determining the position information of the audience;

[0063] The method of the embodiment of the present invention can be applied to an intelligent audio-visual system or a terminal device. The screen of the intelligent audio-visual system or the terminal device can be fixed on a wall, and the display screen used to play the picture can be a flat screen, a curved screen, or a naked-eye 3D screen.

[0064] In some examples, the audience position may be determined using optical (camera), ultrasonic, electromagnetic field, etc.

[0065] In some examples, for a flat-panel display, the lines connecting viewers at different positions to the sound source image point to the same intersection on the screen, so the intersection position for multiple viewers in a flat-panel display scenario is unique; for the sound source image in front of / behind the screen (i.e., a naked-eye 3D display), the lines connecting viewers at different positions to the sound source image are different.

[0066] In some instances, the sound source image position is equivalent to the sound source expected by the audience. The human auditory system has the highest localization accuracy for the sound source in the horizontal direction in front, and the localization accuracy decreases rapidly from the middle to the sides. However, it is insensitive to the movement of the sound source in height. Sometimes the sound source movement will not be perceived until it has an angular displacement of 60° at the subject. For screens with a vertical viewing angle much less than 60°, only the sound and image are considered in the horizontal direction, and the sound and image differences caused by different sound source image positions in the vertical direction can be ignored.

[0067] The method of the embodiment of the present invention can select parametric array speakers at different positions to directional sound for different audiences, and has the best effect on non-planar displays such as naked-eye 3D displays.

[0068] Step 102, determining a connecting line corresponding to the position information of the sound source image and the position information of the audience, and extending the connecting line to an intersection on the screen;

[0069] In the embodiment of the present invention, a series of calculations and analyses can be performed based on the location information of the sound source image and the location information of the audience to determine the connection line between the two and extend it to the intersection on the screen. This step is a key link in achieving accurate sound source positioning and audience interaction.

[0070] In some examples, the position of the sound source image and the position of the audience can first be obtained through step 101. Next, the position information of the sound source image and the position information of the audience can be matched and calculated. Through geometric principles, the connection line between the sound source and the audience can be determined and extended to the intersection on the screen. This intersection represents the projection position of the sound source on the screen, thereby realizing the visualization of the sound source.

[0071] In some examples, the line connecting the sound source and the audience can be calculated through geometric calculation and projection technology: first, the three-dimensional coordinates of the sound source and the audience are calculated through sensor data; then, the equation of the line between the sound source and the audience is calculated using geometric principles; finally, the line is extended to the intersection on the screen to calculate the projection position.

[0072] In some examples, environmental factors such as the propagation path and reflection of the sound can also be taken into account to further improve the accuracy of positioning. Through these steps, precise interaction between the sound source and the audience can be achieved, providing the audience with a more immersive experience.

[0073] Step 103, determining a plurality of target ultrasonic emitting units from a plurality of ultrasonic emitting units arranged along the screen according to the intersection point;

[0074] In the embodiment of the present invention, the ultrasonic emitting units to be activated can be further determined according to the calculated intersection positions. These ultrasonic emitting units are usually arranged along the edge of the screen or in a specific area to enhance the localization effect of the sound source and the auditory experience of the audience.

[0075] In some examples, the ultrasonic transmitting units closest to the intersection can be calculated based on the position of the intersection, and these transmitting units will be marked as target ultrasonic transmitting units and activated to transmit ultrasonic signals of specific frequency and intensity.

[0076] In some examples, the ultrasonic emitting units may be symmetrically distributed on both sides of the vertical position of the intersection, and the specific position may be a square or circular area of ​​a certain size preset above the intersection.

[0077] The parametric array speaker system is usually composed of multiple ultrasonic transmitting units. The ultrasonic transmitting unit is a key component of the parametric array speaker and is responsible for transmitting high-frequency ultrasonic signals. These units are usually composed of ultrasonic transducers, drive circuits, control units and housings. These units are arranged in a specific geometric layout to form a parametric array. By controlling the transmission signal of each ultrasonic transmitting unit and utilizing nonlinear acoustic effects, high-frequency ultrasonic signals can be modulated into audible sound waves, thereby realizing functions such as directional audio transmission and noise control. By combining ultrasonic transmitting units into a parametric array speaker, high-precision and directional sound wave transmission can be achieved, improving the performance and effects of various application scenarios.

[0078] Step 104: Control the audio beams output by the multiple target ultrasonic emitting units respectively according to the position information of the audience.

[0079] In the embodiment of the present invention, the audio beams output by the multiple target ultrasonic emitting units can be further controlled according to the position information of the audience. For example, the direction of the audio beam is controlled to be toward the audience, and the coverage range of the width of the audio beam is controlled to be the audience's head, etc. This step is intended to achieve a personalized audio experience and ensure that each audience can obtain the best hearing effect.

[0080] In some examples, the best hearing position for each audience member can be calculated based on the audience's three-dimensional coordinates and movement trajectory. Then, the output parameters of the target ultrasonic transmitting unit, such as frequency, intensity, and phase, can be adjusted to ensure that the audio beam can accurately point to the position of each audience member.

[0081] In the embodiment of the present invention, the playback strategy can be dynamically adjusted according to the actual position of the audience and the position of the sound source image, so as to better adapt to different environments and user needs. By obtaining the sound source image position information and the audience position information during the screen display process, real-time monitoring, precise positioning and optimization of the interactive experience of the sound source and the audience are achieved, thereby improving the efficiency of the system and the user experience, and helping users quickly solve the problem of asynchrony between sound and image.

[0082] Reference Figure 2 , shows a flowchart of another method for adjusting a sound source based on an audience position provided by an embodiment of the present invention, wherein the method may specifically include the following steps:

[0083] Step 201, during the process of displaying the picture on the screen, determining the position information of the sound source image;

[0084] In some embodiments, step 201 may include the following sub-steps:

[0085] Sub-step S21, when the sound source type is human, determining the center point position of the human face and mouth in the sound image, and using the center point position of the human face and mouth as the position information of the sound source image;

[0086] In the embodiment of the present invention, when the sound source type is human, it means that the sound source image contains human face information. At this time, it is necessary to determine the position of the center point of the human face mouth in the sound image, and use the position as the position information of the sound source image.

[0087] In some examples, the method can be applied to an intelligent audio-visual system, which can identify and locate faces in sound images through computer vision technology. Specifically, the system uses a camera to capture sound images and uses a face detection algorithm to identify the face area in the image. Subsequently, the system further analyzes the face area and locates the center of the mouth of the face. This step usually involves facial key point detection technology, which calculates the precise position of the center of the mouth by identifying key points of the face such as the eyes, nose, and mouth.

[0088] In some examples, the intelligent audio-visual system can also combine deep learning models to improve the accuracy of face detection and mouth center point positioning. For example, the intelligent audio-visual system can use pre-trained face detection models and key point detection models to process sound images, thereby achieving high-precision face and mouth center point positioning. In this way, the system can accurately determine the location information of the sound source image, providing a basis for subsequent sound source positioning and audience interaction.

[0089] Sub-step S22, when the sound source type is non-human, determining the lateral center point position of the sound image, and using the lateral center point position as the position information of the sound source image.

[0090] In the embodiment of the present invention, when the sound source type is non-human, it means that the sound source image does not contain face information. At this time, it is necessary to determine the horizontal center point position of the sound image and use the position as the position information of the sound source image.

[0091] In some examples, the intelligent audio and video system can identify and locate the horizontal center point of the sound image through image processing technology. Specifically, the system uses a camera to capture the sound image and identifies the sound source area in the image through an image processing algorithm. Subsequently, the system further analyzes the sound source area and calculates the horizontal center point of the image. This step usually involves image segmentation and edge detection technology, which calculates the precise position of the horizontal center point by identifying the boundaries of the sound source area.

[0092] In some examples, the system can also combine machine learning models to improve the accuracy of sound source area identification and horizontal center point positioning. For example, the system can use pre-trained image segmentation models and edge detection models to process sound images to achieve high-precision sound source area and horizontal center point positioning. In this way, the system can accurately determine the location information of the sound source image, providing a basis for subsequent sound source positioning and audience interaction.

[0093] Step 202, determining the location information of the audience;

[0094] In the embodiment of the present invention, the intelligent audio-visual system can accurately identify and locate the position information of the audience through various technical means, providing a basis for subsequent sound source positioning and audience interaction.

[0095] In some examples, the system can capture the audience's location information in real time through multiple sensors such as cameras, ultrasonic sensors, and infrared sensors. Specifically, the system uses cameras to capture images of the audience and identifies the audience's location through computer vision technology. At the same time, the system can also use ultrasonic sensors and infrared sensors to determine the audience's three-dimensional coordinates and movement trajectory by measuring the reflection time of sound waves and infrared radiation.

[0096] In some examples, the method for identifying the audience's position may be: cameras are distributed around or on the top of the screen to periodically capture images of the audience; the captured images are transmitted to the system's central processing unit (CPU) for preliminary processing and filtering; the system first determines a reference position, which is usually based on the center point of the screen or a preset comfortable viewing area; the system compares the position information of each audience with the reference position, and identifies audiences whose positions are significantly deviated from or close to the reference position; if the position information of a certain audience continuously deviates from the reference value, that is, it is in a non-sweet spot, the system will mark it as an abnormal audience position.

[0097] In some examples, cameras are distributed around or on top of the screen to periodically capture images of viewers. The image data is transmitted to the system's central processing unit (CPU) for preliminary processing and filtering to remove noise and outliers. The system then determines a reference position based on the center point of the screen or a preset comfortable viewing area. The system compares the position information of each viewer with the reference position and identifies viewers whose positions are significantly deviated from or close to the reference position. If a viewer's position information continues to deviate from the reference value, the system will mark it as an abnormal viewer position and trigger the corresponding processing strategy.

[0098] By capturing and analyzing the audience's location information in real time through a variety of sensors, the system can accurately identify the audience's location and compare it with the benchmark position, identify the audience whose position deviates significantly from or is close to the benchmark position, and mark them as abnormal audience positions, thereby triggering corresponding processing strategies.

[0099] Step 203, determining a connecting line corresponding to the position information of the sound source image and the position information of the audience, and extending the connecting line to an intersection on the screen;

[0100] In the embodiment of the present invention, the intelligent audio-visual system accurately calculates the line between the sound source image and the viewer's position, and extends it to the intersection point on the screen as a basis for subsequent judgment.

[0101] In some examples, the system can calculate the connection between the sound source image and the audience by combining the location information of the sound source image and the audience's location information. Specifically, the system can capture the location information of the sound source image using a microphone array and capture the audience's location information through a camera or a sensor network. Subsequently, the system can match and calculate the location information of the sound source image and the audience's location information, and determine the connection between the sound source and the audience through geometric principles.

[0102] In some examples, the method for calculating the connection line can be: the position information of the sound source image is captured by a microphone array, and the position information of the audience is captured by a camera or a sensor network; the captured data is transmitted to the system's central processing unit (CPU) for preliminary processing and filtering; the system first determines a reference position, which is usually based on the center point of the screen or a preset comfortable viewing area; the system matches and calculates the position information of the sound source image and the position information of the audience, and determines the connection line between the sound source and the audience through geometric principles; the system extends the connection line to the intersection on the screen and calculates the projection position.

[0103] In some instances, the location information of the sound source image is captured by a microphone array, and the location information of the audience is captured by a camera or sensor network. These data are transmitted to the system's central processing unit (CPU) for preliminary processing and filtering to remove noise and outliers. The system then determines a reference position based on the center point of the screen or a preset comfortable viewing area. The system matches and calculates the location information of the sound source image and the location information of the audience, and determines the connection line between the sound source and the audience through geometric principles. The system extends the connection line to the intersection on the screen and calculates the projection position.

[0104] By accurately calculating the line between the sound source image and the audience position and extending it to the intersection on the screen, the system can achieve accurate sound source positioning and audience interaction. By combining the location information of the sound source image and the audience's location information, calculating the line between the two and extending it to the intersection on the screen, the system can identify the projection position of the sound source on the screen, thereby achieving a visual presentation of the sound source.

[0105] Step 204, determining a target area in the deployment area of ​​the plurality of ultrasonic transmitting units along a vertical direction of the intersection;

[0106] In the embodiment of the present invention, the intelligent audio and video system determines the target area in the deployment area of ​​the plurality of ultrasonic transmitting units in the vertical direction along the intersection.

[0107] In some examples, the system can determine the target area by combining the location information of the intersection and the deployment area of ​​the ultrasonic emitting unit. Specifically, the system uses the location information of the intersection to calculate the range of the target area in the vertical direction. Then, the system matches the deployment area of ​​the ultrasonic emitting unit with the target area to determine the ultrasonic emitting unit located in the target area.

[0108] In some examples, the method for determining the target area may be: the position information of the intersection is calculated by connecting the sound source image and the audience position; the system calculates the range of the target area in the vertical direction based on the position information of the intersection; the deployment area of ​​the ultrasonic transmitting unit is determined by a preset geometric layout; the system matches the deployment area of ​​the ultrasonic transmitting unit with the target area to determine the ultrasonic transmitting unit located in the target area.

[0109] In some examples, the position information of the intersection is calculated by connecting the sound source image and the viewer position. The system calculates the range of the target area in the vertical direction based on the position information of the intersection. The deployment area of ​​the ultrasonic transmitting unit is determined by a preset geometric layout, usually arranged along the edge of the screen or a specific area. The system matches the deployment area of ​​the ultrasonic transmitting unit with the target area and determines the ultrasonic transmitting unit located in the target area.

[0110] In some examples, the ultrasonic emitting units can be deployed symmetrically on both sides of the vertical position of the intersection, and the shape of the parametric array speaker composed of the ultrasonic sound-emitting units can be planned as a regular shape such as a square or circle of a certain size preset above the intersection. For example, the parametric array speaker can be a circle or square with a width equal to the width of a normal human head of 18 cm or set to the observed outline size of the audience's head. This patent does not limit the spacing of the ultrasonic emitting units.

[0111] Step 205, determining the ultrasonic transmitting unit located in the target area as the target ultrasonic transmitting unit;

[0112] In the embodiment of the present invention, the intelligent audio and video system achieves accurate sound source positioning and audience interaction by determining the ultrasonic emitting units located in the target area as target ultrasonic emitting units.

[0113] In some examples, the system can determine the target ultrasonic emitting unit by combining the location information of the target area and the deployment area of ​​the ultrasonic emitting unit. Specifically, the system uses the location information of the target area to identify the ultrasonic emitting units located in the target area. Subsequently, the system marks these ultrasonic emitting units as target ultrasonic emitting units and performs corresponding control and management.

[0114] Step 206, controlling the audio beams output by the plurality of target ultrasonic wave transmitting units respectively according to the position information of the audience, includes:

[0115] Sub-step S23, according to the position information of the audience, respectively controlling the directions of the audio beams output by the multiple target ultrasonic emitting units to be toward the audience.

[0116] In the embodiment of the present invention, the intelligent audio-visual system controls the directions of the audio beams output by the plurality of target ultrasonic emitting units respectively according to the position information of the audience, so that the audio beams are directed toward the audience.

[0117] In some examples, the system can control the direction of the audio beam by combining the viewer's location information and the location information of the target ultrasonic emitting unit. Specifically, the system uses the viewer's location information to calculate the optimal direction of the audio beam output by each target ultrasonic emitting unit. The system then adjusts the output parameters of the target ultrasonic emitting unit to ensure that the audio beam can accurately point to the viewer's location.

[0118] By controlling the direction of the audio beams output by multiple target ultrasonic emitting units according to the audience's position information, the system can achieve accurate sound source positioning and audience interaction. By combining the audience's position information with the position information of the target ultrasonic emitting unit to control the direction of the audio beam, the system can ensure that the audio beam can accurately point to the audience's position, thereby enhancing the audience's auditory experience.

[0119] Sub-step S24, determining the distances of the multiple target ultrasonic emitting units relative to the audience according to the position information of the audience; and controlling the widths of the audio beams output by the multiple target ultrasonic emitting units according to the distances of the multiple target ultrasonic emitting units relative to the audience.

[0120] In an embodiment of the present invention, the intelligent audio and video system determines the distances of multiple target ultrasonic transmitting units relative to the audience based on the audience's position information, and controls the widths of the audio beams of different ultrasonic transmitting units respectively, thereby ensuring that the audio beam can cover the audience's head.

[0121] In some examples, the system can calculate the distance of each target ultrasonic emitting unit relative to the audience by combining the audience's location information and the location information of the target ultrasonic emitting unit. Specifically, the system uses the audience's location information to calculate the distance between each target ultrasonic emitting unit and the audience. Then, the system adjusts the width of the audio beam output by the target ultrasonic emitting unit based on these distances to ensure that the audio beam can cover the audience's position.

[0122] By determining the distances of multiple target ultrasonic emitting units relative to the audience based on the audience's location information and controlling the width of the audio beam respectively, the system can achieve accurate sound source positioning and audience interaction. By combining the audience's location information with the location information of the target ultrasonic emitting unit, calculating the distance of each target ultrasonic emitting unit relative to the audience and adjusting the width of the audio beam, the system can ensure that the audio beam can cover the audience's position, thereby enhancing the audience's auditory experience.

[0123] Step 207, when the position information of the audience remains unchanged and the position information of the sound source image changes, detecting the change angle of the direction of the audio beams output by the multiple target ultrasonic emitting units; when the change angle is greater than a preset angle threshold, re-determining the target ultrasonic emitting unit;

[0124] In the embodiment of the present invention, the intelligent audio and video system dynamically adjusts the target ultrasonic emitting units by detecting the direction change angles of the audio beams output by the multiple target ultrasonic emitting units.

[0125] In some examples, a method for detecting the angle of change in the direction of an audio beam may be: the position information of the audience is captured through a camera or a sensor network; the position information of the sound source image is captured through a microphone array; the system calculates the optimal direction of the audio beam output by each target ultrasonic transmitting unit based on the position information of the audience; the system monitors changes in the position information of the sound source image and calculates the angle of change in the direction of the audio beam; when the change angle is greater than a preset angle threshold, the system re-determines the target ultrasonic transmitting unit.

[0126] Reference Figure 3 As shown, the direction of the audio beam is defined as the azimuth angle θ of the audience. The azimuth angle is the angle between the line connecting the audience position and the sound source image position and the vertical line of the screen. When the audience position remains unchanged and the sound source image position changes, a sound-image separation angle will be generated. The sound-image separation angle is equal to the difference in azimuth before and after the sound source image position changes, that is, △θ=θ1-θ2. If the sound-image separation angle is too large, the audience will feel a sense of sound-image separation; according to relevant literature, audiences without special training can perceive sound-image deviations of more than 20°. Therefore, the preset angle threshold can be determined as 20°, that is, when the audience's azimuth angle changes by more than 20°, the system re-determines the intersection point based on the changed sound source image position, thereby determining the position of the target ultrasonic transmitting unit, the angle and coverage of the audio beam, etc.

[0127] Step 208, when the position information of the audience changes and the position information of the sound source image remains unchanged, determine whether the position information of the audience exceeds the coverage range of the audio beams output by the multiple target ultrasonic transmitting units; when the position information of the audience exceeds the coverage range of the audio beams output by the multiple target ultrasonic transmitting units, re-determine the target ultrasonic transmitting unit.

[0128] In an embodiment of the present invention, the intelligent audio-visual system detects changes in the viewer's position information and determines whether it exceeds the coverage of the audio beams output by multiple target ultrasonic emitting units. If so, the target ultrasonic emitting units are dynamically adjusted.

[0129] In some examples, a method for detecting that the audience's position information exceeds the coverage of the audio beam may be: the audience's position information is captured by a camera or a sensor network; the position information of the target ultrasonic transmitting unit is determined by a preset geometric layout; the system calculates the coverage of the audio beam output by each target ultrasonic transmitting unit based on the audience's position information and the distance between the audience and the target ultrasonic transmitting unit; the system monitors changes in the audience's position information and determines whether the audience exceeds the coverage of the audio beam; when the audience's position information exceeds the coverage of the audio beam, the system re-determines the target ultrasonic transmitting unit.

[0130] In some examples, the radiation angle α of the audio beam is related to factors such as the size of the parametric array speaker and the distance from the audience.

[0131] Reference Figure 4 As shown, for example, the constraint condition for the radiation angle α of the audio beam can be α≥2*arctan(d / 2D), where d is the width of the audience's head, which can be 18 cm or an actual measured value, and D is the distance between the audience's head and the center of the parametric array speaker.

[0132] When the sound source image position remains unchanged and the audience position changes, the sound-image separation angle △θ may be very small, but if the audience leaves the audio beam coverage of the original parametric array speaker, they will not hear the sound and will not have the sound-image co-location experience. At this time, the lateral displacement △x of the audience in front of the screen and the coverage of the audio beam are used as the basis for judgment. Because the radiation angle α of the audio beam is very small, the lateral width of the beam at the audience position is taken as tan(1 / 2α)*2D; when △x>tan(1 / 2α)*D, that is, the audience leaves the beam coverage of the original parametric array speaker, the position of the parametric array speaker should be reselected and the new audience azimuth angle θ2 should be used as the beam direction for sound.

[0133] In the embodiment of the present invention, the playback strategy can be dynamically adjusted according to the actual position of the audience and the position of the sound source image, so as to better adapt to different environments and user needs. By obtaining the sound source image position information and the audience position information during the screen display process, and setting a preset threshold to dynamically adjust the target ultrasonic emission unit, real-time monitoring, precise positioning and optimization of the interactive experience of the sound source and the audience are achieved, thereby improving the efficiency of the system and the user experience, and helping users quickly solve the problem of audio and video asynchrony.

[0134] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0135] Reference Figure 5 , shows a structural block diagram of a sound source adjustment device based on audience position provided by an embodiment of the present invention, which may specifically include the following modules:

[0136] The position information determination module 301 is used to determine the position information of the sound source image and the position information of the audience during the process of displaying the picture on the screen;

[0137] An intersection determination module 302 is used to determine a line corresponding to the position information of the sound source image and the position information of the audience, and extend the line to the intersection on the screen;

[0138] A target ultrasonic emitting unit determining module 303 is used to determine a plurality of target ultrasonic emitting units from a plurality of ultrasonic emitting units arranged along the screen according to the intersection point;

[0139] The audio beam control module 304 is used to control the audio beams output by the multiple target ultrasonic emitting units respectively according to the position information of the audience.

[0140] In an embodiment of the present invention, the intersection determination module includes:

[0141] A sound source type determination submodule, used to determine the sound source type;

[0142] A first position information determination submodule, for determining the position of the center point of the human face and mouth in the sound image when the sound source type is human, and using the center point of the human face and mouth as the position information of the sound source image;

[0143] The second position information determination submodule is used to determine the lateral center point position of the sound image when the sound source type is non-human, and use the lateral center point position as the position information of the sound source image.

[0144] Optionally, the plurality of ultrasonic emitting units are arranged along a lateral edge of the screen; and the target ultrasonic emitting unit determination module comprises:

[0145] A target area determination submodule, used to determine a target area in a deployment area of ​​the plurality of ultrasonic transmitting units along a vertical direction of the intersection;

[0146] The target ultrasonic emitting unit submodule is used to determine the ultrasonic emitting unit located in the target area as the target ultrasonic emitting unit.

[0147] In an embodiment of the present invention, the audio beam control module includes:

[0148] The audio beam direction control submodule is used to control the directions of the audio beams output by the multiple target ultrasonic emitting units to be toward the audience according to the position information of the audience.

[0149] In an embodiment of the present invention, the audio beam control module further includes:

[0150] A distance determination submodule, used to determine the distances of the plurality of target ultrasonic emitting units relative to the audience according to the position information of the audience;

[0151] The width determination submodule is used to control the width of the audio beams output by the multiple target ultrasonic emitting units according to the distances of the multiple target ultrasonic emitting units relative to the audience.

[0152] In an embodiment of the present invention, the method further includes:

[0153] An angle detection module, used for detecting the change angle of the direction of the audio beams output by the multiple target ultrasonic emitting units when the position information of the audience remains unchanged and the position information of the sound source image changes;

[0154] The first redetermining submodule is used to redetermine the target ultrasonic transmitting unit when the change angle is greater than a preset angle threshold.

[0155] In an embodiment of the present invention, the method further includes:

[0156] A coverage range determination module, used for determining whether the position information of the audience exceeds the coverage range of the audio beams output by the multiple target ultrasonic emitting units when the position information of the audience changes and the position information of the sound source image remains unchanged;

[0157] The second redetermining submodule is used to redetermine the target ultrasonic emitting unit when the position information of the audience exceeds the coverage of the audio beams output by the multiple target ultrasonic emitting units.

[0158] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0159] An embodiment of the present invention further provides an electronic device, including:

[0160] It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned sound source adjustment method embodiment based on the audience position are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0161] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned embodiment of the sound source adjustment method based on the audience position is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0162] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0163] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0164] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0165] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0167] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0168] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0169] The above is a detailed introduction to the sound source adjustment method, device, equipment and medium based on the audience position provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A sound source adjustment method based on audience position, characterized in that: include: During the display of the picture on the screen, determining the position information of the sound source image and determining the position information of the audience; Determine a connecting line corresponding to the position information of the sound source image and the position information of the audience, and extend the connecting line to an intersection on the screen; According to the intersection point, determining a plurality of target ultrasonic emitting units from a plurality of ultrasonic emitting units arranged along the screen; The audio beams output by the plurality of target ultrasonic emitting units are controlled respectively according to the position information of the audience.

2. The method according to claim 1, characterized in that The step of determining the location information of the sound source image includes: Determine the type of sound source; When the sound source type is human, determining the position of the center point of the human face and mouth in the sound image, and using the position of the center point of the human face and mouth as the position information of the sound source image; When the sound source type is non-human, the lateral center point position of the sound image is determined, and the lateral center point position is used as the position information of the sound source image.

3. The method according to claim 1, characterized in that The plurality of ultrasonic emitting units are arranged along the lateral edge of the screen; and the determining of a plurality of target ultrasonic emitting units from the plurality of ultrasonic emitting units arranged along the screen according to the intersection point comprises: Determine a target area in a deployment area of ​​the plurality of ultrasonic transmitting units along a vertical direction of the intersection; The ultrasonic wave transmitting unit located in the target area is determined as the target ultrasonic wave transmitting unit.

4. The method according to claim 1, characterized in that: The step of controlling the audio beams output by the plurality of target ultrasonic wave transmitting units respectively according to the position information of the audience comprises: According to the position information of the audience, the directions of the audio beams output by the multiple target ultrasonic emitting units are respectively controlled to be toward the audience.

5. The method according to claim 4, characterized in that The step of controlling the audio beams output by the plurality of target ultrasonic wave transmitting units respectively according to the position information of the audience further comprises: Determining the distances of the plurality of target ultrasonic emitting units relative to the audience according to the position information of the audience; The widths of the audio beams output by the multiple target ultrasonic emitting units are controlled according to the distances of the multiple target ultrasonic emitting units relative to the audience.

6. The method according to claim 1, characterized in that Also includes: When the position information of the audience remains unchanged and the position information of the sound source image changes, detecting the change angle of the direction of the audio beams output by the multiple target ultrasonic emitting units; When the change angle is greater than a preset angle threshold, the target ultrasonic emitting unit is re-determined.

7. The method according to claim 1, characterized in that Also includes: When the position information of the audience changes and the position information of the sound source image remains unchanged, determining whether the position information of the audience exceeds the coverage range of the audio beams output by the multiple target ultrasonic emitting units; When the position information of the audience exceeds the coverage of the audio beams output by the multiple target ultrasonic emitting units, the target ultrasonic emitting units are re-determined.

8. A sound source adjustment device based on audience position, characterized in that: The device comprises: A position information determination module is used to determine the position information of the sound source image and the position information of the audience during the process of displaying the picture on the screen; An intersection determination module, used to determine a connecting line corresponding to the position information of the sound source image and the position information of the audience, extending the connecting line to the intersection on the screen; a target ultrasonic emitting unit determining module, configured to determine a plurality of target ultrasonic emitting units from a plurality of ultrasonic emitting units arranged along the screen according to the intersection point; The audio beam control module is used to control the audio beams output by the multiple target ultrasonic emitting units respectively according to the position information of the audience.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the sound source adjustment method based on the audience position as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the sound source adjustment method based on the audience position as described in any one of claims 1 to 7 are implemented.

Images