Sound field adaptive cloud control method and system based on deep learning

Abstract

The invention relates to the field of sound equipment control, and discloses a sound field adaptive cloud control method and system based on deep learning, and the method comprises the steps: obtaining three-dimensional point cloud data corresponding to a sound field target space and physical layout parameters of a sound equipment array in the sound field target space; obtaining working parameter information when the sound array works in the sound field target space, and obtaining position information of the microphone array and impulse response data corresponding to the working parameter information detected by the microphone array in the sound field target space; and through a cloud sound field control model, according to the three-dimensional point cloud data of the sound field target space, the physical layout parameters of the sound array, the working parameter information of the sound array, the impulse response data detected by the microphone array and the position information of the aligned microphone array, determining the control parameter information of the sound array. According to the sound field adaptive cloud control method based on deep learning, the sound experience of a user can be improved.

Description

Technical Field

[0001] The present application relates to the field of audio control technology, and more specifically, to a sound field adaptive cloud control method and system based on deep learning. Background Art

[0002] In the field of smart home, sound field control technology is the core support for achieving high-quality audio experience and is widely used in scenarios such as smart speakers, home theaters, and distributed background music systems. As users' demand for intelligent and personalized home environments increases, different functional areas (including living rooms, bedrooms, and kitchens) have higher requirements for the dynamic adjustment of the sound field. For example, when watching movies in the living room, an immersive surround sound field is required to enhance the audio-visual experience, a soft and uniform white noise sound field is required for the sleep-aiding scene in the bedroom, and directional sound coverage is required in noisy environments such as the kitchen or bathroom to ensure the clarity of voice interaction. Traditional smart home audio devices mostly use sound field modes with fixed parameters, which cannot be adaptively adjusted according to real-time environmental parameters (such as changes in room layout, furniture reflection characteristics, and interference from multiple sound sources) and the dynamic position of the user. As a result, the sound field uniformity, noise reduction effect, and sound quality are difficult to meet the diverse needs of complex home scenarios.

[0003] Existing smart home sound field control technologies have significant limitations. The sound field adjustment method based on a fixed speaker array relies on a preset acoustic model. When people move, doors and windows open and close, or new obstacles are added in the home environment, the sound field distribution will change due to reflection and diffraction effects. Traditional technologies lack real-time environmental perception and dynamic modeling capabilities, resulting in a decrease in the accuracy of sound field control. In addition, existing technologies are insufficient in learning users' personalized listening preferences and are unable to provide customized sound field solutions based on user age, auditory characteristics, and real-time activity status (such as reading, fitness, and meeting guests). The above problems have led to technical bottlenecks in the intelligence level of smart home sound field control and the improvement of user experience. It is necessary to improve the existing sound field control methods to meet users' higher audio experience. Summary of the Invention

[0004] The purpose of this application is to provide a sound field adaptive cloud control method and system based on deep learning, which solves the technical problems that the existing sound field adjustment method relies on preset acoustic models and has poor sound field control effect, and achieves the technical effect of improving the user's audio experience through the sound field adaptive cloud control method based on deep learning.

[0005] An embodiment of the present application provides a deep learning-based adaptive cloud control method for a sound field, the method comprising: obtaining three-dimensional point cloud data corresponding to a sound field target space and physical layout parameters of an acoustic array within the sound field target space; obtaining operating parameter information of the acoustic array when operating within the sound field target space, and obtaining position information of a microphone array and impulse response data corresponding to the operating parameter information detected by the microphone array within the sound field target space; aligning the position information of the microphone array with the three-dimensional point cloud data to obtain position information of the aligned microphone array; wherein the three-dimensional point cloud data includes a normal vector corresponding to each detection point; determining control parameter information of the acoustic array through a cloud-based sound field control model based on the three-dimensional point cloud data of the sound field target space, the physical layout parameters of the acoustic array, the operating parameter information of the acoustic array, the impulse response data detected by the microphone array, and the position information of the aligned microphone array, and adjusting the operating state of the acoustic array according to the control parameter information.

[0006] In one possible implementation, the method also includes: obtaining the material reflection intensity corresponding to each detection point in the three-dimensional point cloud data, and obtaining the material sound absorption coefficient corresponding to different material reflection intensities; determining, through a local space detection unit, the target space matrix corresponding to the sound field target space based on the three-dimensional point cloud data corresponding to the sound field target space and the material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data; determining, through a cloud-based sound field control model, the control parameter information of the sound array based on the target space matrix, the physical layout parameters of the sound array, the working parameter information of the sound array, the impulse response data detected by the microphone array, and the position information of the aligned microphone array, and adjusting the working state of the sound array according to the control parameter information.

[0007] In another possible implementation, the method also includes: clustering the material reflection intensity corresponding to each detection point in the three-dimensional point cloud data to obtain different reflection areas corresponding to different material reflection intensity ranges, and determining the reflection intensity difference between different reflection areas; determining multiple reflection area groups corresponding to reflection intensity differences that are less than a preset reflection intensity difference; obtaining material information input by the user for each reflection area in the multiple reflection area groups, obtaining the material sound absorption coefficient corresponding to different material information, and updating the material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data according to the material sound absorption coefficient corresponding to the different material information.

[0008] In another possible implementation, a target space matrix corresponding to the sound field target space is determined through a local space detection unit based on the three-dimensional point cloud data corresponding to the sound field target space and the material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data, including: determining the angle between the normal vector corresponding to each detection point and the sound main axis direction of each speaker in the sound array, and determining that the angle between the normal vector corresponding to each detection point and the sound main axis direction of each speaker in the sound array is the minimum value, as the sound field main axis angle corresponding to each detection point; determining the sound field main axis angle range within which the sound field main axis angle corresponding to each detection point is located, and obtaining the reflection weight coefficient corresponding to each sound field main axis angle range to obtain the reflection weight coefficient corresponding to each detection point; determining the target space matrix corresponding to the sound field target space through a local space detection unit based on the three-dimensional point cloud data corresponding to the sound field target space, the material sound absorption coefficient and the reflection weight coefficient corresponding to each detection point in the three-dimensional point cloud data.

[0009] In another possible implementation, the target space matrix corresponding to the sound field target space is determined by a local space detection unit based on the three-dimensional point cloud data corresponding to the sound field target space and the material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data. It also includes: determining the sum of the reflection area areas of the reflection areas where the detection points corresponding to each sound field main axis angle range are located, and determining the natural logarithm of the ratio of the area of each reflection area to the minimum reflection area area as the area priority value corresponding to each detection point in each reflection area; wherein the minimum reflection area areas corresponding to different sound field main axis angle ranges are different; and determining the target space matrix corresponding to the sound field target space by a local space detection unit based on the three-dimensional point cloud data corresponding to the sound field target space and the material sound absorption coefficient, reflection weight coefficient and area priority value corresponding to each detection point in the three-dimensional point cloud data.

[0010] In another possible implementation, the method also includes: obtaining sound pressure values in different detection areas within the sound field target space, and determining the sound pressure change value between any two adjacent detection areas based on the sound pressure values of the different detection areas; when the sound pressure change value between two target adjacent detection areas is greater than or equal to a preset sound pressure change value, adding a preset reflection weight coefficient to the reflection weight coefficient of the two target adjacent areas to obtain adjusted reflection weight coefficients corresponding to the two target adjacent areas; and updating the target space matrix corresponding to the sound field target space based on the adjusted reflection weight coefficients corresponding to the two target adjacent areas.

[0011] In another possible implementation, the preset reflection weight coefficient is determined by the following method: obtaining the target sound field main axis angles corresponding to two adjacent target detection areas, and determining the average of the ratios of the two target sound field main axis angles and the basic sound field main axis angle as the preset reflection weight coefficient; wherein the basic sound field main axis angle is 15° to 30°.

[0012] In another possible implementation, the method further includes: after adjusting the operating state of the sound array according to the control parameter information, obtaining impulse response data detected by the adjusted microphone array, and obtaining a difference between the impulse response data detected by the adjusted microphone array and the impulse response data detected by the microphone array before the adjustment as an impulse response data change value; determining, by a control detection unit, a control adjustment confidence level of the sound array based on the target space matrix, physical layout parameters of the sound array, operating parameter information of the sound array, impulse response data detected by the microphone array before the adjustment, the impulse response data change value, and position information of the aligned microphone array; when the control adjustment confidence level is greater than or equal to a preset control adjustment confidence level, adjusting the operating state of the sound array again according to the control parameter information based on user feedback; when the control adjustment confidence level is less than the preset control adjustment confidence level, reacquiring three-dimensional point cloud data corresponding to the sound field target space, and updating the target space matrix corresponding to the sound field target space based on the reacquired three-dimensional point cloud data corresponding to the sound field target space.

[0013] In another possible implementation, the working state of the sound array is adjusted again according to the control parameter information based on the user's feedback, including: obtaining reverberation feedback information corresponding to different positions of the user in the target sound field space; through the cloud-based sound field control model, according to the three-dimensional point cloud data of the target sound field space, the physical layout parameters of the sound array, the working parameter information of the sound array, the impulse response data detected by the microphone array, the position information of the aligned microphone array and the reverberation feedback information, the control parameter information of the sound array is determined, and the working state of the sound array is adjusted according to the control parameter information.

[0014] An embodiment of the present application also provides a deep learning-based sound field adaptive cloud control system, comprising a unit for executing any of the methods described above.

[0015] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0016] The present application provides a deep learning-based adaptive cloud control method for a sound field, the method comprising: obtaining three-dimensional point cloud data corresponding to a target sound field space and physical layout parameters of an acoustic array within the target sound field space; obtaining operating parameter information of the acoustic array when operating within the target sound field space, and obtaining position information of a microphone array and impulse response data corresponding to the operating parameter information detected by the microphone array within the target sound field space; aligning the position information of the microphone array with the three-dimensional point cloud data to obtain position information of the aligned microphone array; wherein the three-dimensional point cloud data includes a normal vector corresponding to each detection point; and determining control parameter information of the acoustic array based on the three-dimensional point cloud data of the target sound field space, the physical layout parameters of the acoustic array, the operating parameter information of the acoustic array, the impulse response data detected by the microphone array, and the position information of the aligned microphone array using a cloud-based sound field control model, and adjusting the operating state of the acoustic array according to the control parameter information. The deep learning-based adaptive cloud control method for a sound field in the embodiment of the present application can optimize the sound field in the cloud by combining the three-dimensional point cloud data corresponding to the target sound field space, thereby improving the sound field optimization effect within the target sound field space. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A schematic diagram of the flow of the first deep learning-based sound field adaptive cloud control method provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of a planar area of a sound field target space detected by the first deep learning-based sound field adaptive cloud control method provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of the workflow of the first deep learning-based sound field adaptive cloud control method provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of a second method for adaptive cloud control of a sound field based on deep learning provided in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of a flow chart of a third deep learning-based sound field adaptive cloud control method provided in an embodiment of the present application;

[0023] Figure 6A schematic diagram of the logical structure of a deep learning-based sound field adaptive cloud control system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0025] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0026] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0027] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0028] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0029] Existing smart home sound field control technology lacks real-time environmental perception and dynamic modeling capabilities, resulting in a decrease in sound field control accuracy. Therefore, it is necessary to improve the existing sound field control method to provide users with a higher audio experience.

[0030] Based on the above reasons, an embodiment of the present application provides a deep learning-based adaptive cloud control method for a sound field, the method comprising: obtaining three-dimensional point cloud data corresponding to a sound field target space and physical layout parameters of an acoustic array within the sound field target space; obtaining operating parameter information of the acoustic array when operating within the sound field target space, and obtaining position information of a microphone array and impulse response data corresponding to the operating parameter information detected by the microphone array within the sound field target space; aligning the position information of the microphone array with the three-dimensional point cloud data to obtain position information of the aligned microphone array; wherein the three-dimensional point cloud data includes a normal vector corresponding to each detection point; and determining control parameter information of the acoustic array based on the three-dimensional point cloud data of the sound field target space, the physical layout parameters of the acoustic array, the operating parameter information of the acoustic array, the impulse response data detected by the microphone array, and the position information of the aligned microphone array through a cloud-based sound field control model, and adjusting the operating state of the acoustic array according to the control parameter information. The deep learning-based adaptive cloud control method for a sound field in the embodiment of the present application can optimize the sound field in the cloud by combining the three-dimensional point cloud data corresponding to the sound field target space, thereby improving the sound field optimization effect within the sound field target space.

[0031] In some scenarios, a sound field adaptive cloud control method based on deep learning in an embodiment of the present application can be applied to a home audio and video system, which can optimize and control the sound field in the home audio and video system and improve the use effect of the home audio and video system.

[0032] The following is a detailed description of a deep learning-based adaptive cloud control method for sound fields provided in an embodiment of the present application with reference to specific examples.

[0033] Figure 1 The flowchart of the first sound field adaptive cloud control method based on deep learning provided in the embodiment of the present application is as follows: Figure 1 As shown, the above method includes S110 to S120, and S110 to S120 are described in detail below.

[0034] S110: Obtain three-dimensional point cloud data corresponding to the target sound field space and physical layout parameters of the speaker array within the target sound field space. Obtain operating parameter information of the speaker array when operating within the target sound field space, and obtain position information of the microphone array and impulse response data corresponding to the operating parameter information detected by the microphone array within the target sound field space. Align the microphone array position information with the three-dimensional point cloud data to obtain aligned microphone array position information. The three-dimensional point cloud data includes a normal vector corresponding to each detection point.

[0035] In this implementation, when performing adaptive cloud control of the sound field, three-dimensional point cloud data corresponding to the sound field target space can be obtained through a three-dimensional laser scanner or a depth camera. The three-dimensional point cloud data can represent the geometric structure and surface features of each detection point in the space. Figure 2 A schematic diagram of a plane area of a sound field target space detected by the first deep learning-based sound field adaptive cloud control method provided in an embodiment of the present application, as shown in FIG. Figure 2 As shown, the sound field target space can be the room where the home audio and video equipment is arranged, and the three-dimensional point cloud data can reflect Figure 2 The three-dimensional spatial data corresponding to the plane area in the image can be used to optimize the sound field based on the three-dimensional spatial data.

[0036] In this implementation, the physical layout parameters of the speaker array can also be obtained. The microphone array position information includes physical attribute parameters such as the number of speaker units, spatial distribution positions, and installation angles, as well as operating parameter information such as the power range and frequency response characteristics of the speaker array during operation. For example, the physical layout parameters of the speaker array can be obtained by combining depth image data with image recognition to determine the placement of each speaker 101 in the speaker array.

[0037] In this implementation, the microphone array arranged in the target space of the sound field can also be used to collect the sound field characteristic data in real time, specifically including the position coordinates of the microphone unit and the detected impulse response data. The impulse response data can reflect the reflection and absorption characteristics in the sound wave propagation path. For example, the impulse response data can be obtained by Figure 2 The circled area in FIG. 1 represents the detection result of the microphone 102 .

[0038] During data processing, the position information of the microphone array and the spatial coordinate system of the three-dimensional point cloud data can be aligned and matched through a coordinate system conversion algorithm to ensure that the acoustic detection data and the spatial geometric data have a unified coordinate reference.

[0039] It's important to note that 3D point cloud data includes a normal vector corresponding to each detection point. These normal vectors can be used to analyze the reflection direction of sound waves on spatial interfaces, providing geometric constraints for subsequent sound field modeling. For example, when a detection point is located on a wall, its normal vector can be used to calculate the theoretical value of the sound wave reflection angle, thereby improving the accuracy of sound field prediction.

[0040] S120. Determine control parameter information of the sound array through a cloud-based sound field control model based on the three-dimensional point cloud data of the sound field target space, the physical layout parameters of the sound array, the operating parameter information of the sound array, the impulse response data detected by the microphone array, and the position information of the aligned microphone array, and adjust the operating state of the sound array according to the control parameter information.

[0041] In this implementation, a deep learning model deployed in the cloud can integrate the three-dimensional geometric characteristics of the sound field target space, the physical configuration parameters of the sound system and real-time acoustic feedback data to generate an optimized control parameter set.

[0042] Figure 3 The first method of sound field adaptive cloud control based on deep learning is provided in the embodiment of the present application. Figure 3 As shown, when optimizing sound control parameters, the cloud-based sound field control model can analyze the spatial structural features in the 3D point cloud data, combine it with the physical layout parameters of the sound array to establish a sound field propagation model, and dynamically calibrate the model based on the impulse response data detected by the microphone array. For example, if sound wave interference is detected in a specific area, the cloud-based sound field control model can automatically adjust the phase difference parameters of adjacent sound units to effectively control the sound field energy distribution and improve the sound field control effect.

[0043] For example, the control parameter information may include core parameters such as the power distribution ratio, frequency balancing parameters, and signal delay time of the audio unit. The parameter instructions are sent to the execution unit of the audio array through the wireless communication module to achieve real-time control of the working status.

[0044] Exemplarily, the cloud-based sound field control model can be a deep learning model trained based on the three-dimensional point cloud data of the sound field target space as sample data, the physical layout parameters of the sound array, the working parameter information of the sound array, the impulse response data detected by the microphone array, the position information of the aligned microphone array, and the control parameter information of the sample sound array.

[0045] The beneficial effect of this implementation is that, through multi-dimensional data fusion and spatial alignment processing, the accuracy and reliability of sound field modeling can be improved, providing accurate input conditions for control parameter optimization. Intelligent analysis based on cloud-based deep learning models can dynamically adapt to changes in the sound field environment, achieve adaptive adjustment of acoustic characteristics, and effectively improve the spatial coverage uniformity and sound quality of the sound system.

[0046] The beneficial effect of the above implementation method is that, through the coordinated optimization of physical layout parameters and operating parameters, fine control of sound field energy distribution can be achieved while ensuring system stability, meeting the acoustic requirements of different application scenarios.

[0047] In some implementations, the above method further includes S130 to S140, which are described in detail below.

[0048] S130 , obtaining the material reflection intensity corresponding to each detection point in the three-dimensional point cloud data, and obtaining the material sound absorption coefficient corresponding to different material reflection intensities.

[0049] In this implementation, the material reflection intensity corresponding to each detection point in the three-dimensional point cloud data can also be obtained, and the material sound absorption coefficient corresponding to different material reflection intensities can be obtained, and the sound field can be further optimized based on the material sound absorption coefficient. When obtaining the material reflection intensity corresponding to each detection point, the three-dimensional point cloud data of the sound field target space can be obtained through a three-dimensional laser scanner or a depth camera. The three-dimensional point cloud data not only contains the geometric coordinates and normal vector information of each detection point, but also can be analyzed through the material reflection intensity analysis module to analyze the reflection characteristics of different surface materials. The reflection characteristics include the material reflection intensity corresponding to each detection point, and the material sound absorption coefficient corresponding to different material reflection intensities can be obtained.

[0050] For example, the difference in reflection intensity between metal and wood materials can be distinguished by a multispectral sensor, and then the corresponding material sound absorption coefficient can be matched through a local database or a cloud material library to form an acoustic characteristic mapping relationship.

[0051] S140: Determine, via the local spatial detection unit, a target spatial matrix corresponding to the target sound field space based on the three-dimensional point cloud data corresponding to the target sound field space and the material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data. Determine, via the cloud-based sound field control model, control parameter information for the sound array based on the target spatial matrix, the physical layout parameters of the sound array, operating parameter information for the sound array, impulse response data detected by the microphone array, and position information of the aligned microphone array, and adjust the operating state of the sound array in accordance with the control parameter information.

[0052] In this implementation, the local spatial detection unit can determine the target spatial matrix corresponding to the target space of the sound field based on the 3D point cloud data corresponding to the target space of the sound field and the sound absorption coefficient of the material corresponding to each detection point in the 3D point cloud data, and then optimize the sound field using the target spatial matrix. During the computational processing, the local spatial detection unit can perform a weighted fusion of the 3D point cloud data and the acoustic parameters of the materials, and through matrix operations, generate a target spatial matrix that contains spatial structural characteristics and sound absorption properties. The target spatial matrix can characterize the propagation attenuation patterns and reflection distribution characteristics of sound waves within the space.

[0053] During the calculation and processing process, the local space detection unit is a locally deployed computing component. When performing cloud-based sound field optimization, it only needs to send the target space matrix to the cloud-based sound field control model deployed in the cloud, and optimize the sound field through the cloud-based sound field control model, thereby reducing the amount of data transmission during the sound field optimization process.

[0054] During the data processing phase, a coordinate alignment algorithm can be used to precisely match the microphone array's positional information with the spatial coordinate system of the 3D point cloud data. For example, when a wall surface is detected to be made of a soft-pack material with a high sound absorption coefficient, the corresponding sound absorption coefficient value at the corresponding detection point in the target spatial matrix is larger, accurately reflecting the sound energy attenuation characteristics of that area during sound field modeling. This multidimensional data fusion approach can effectively improve the accuracy of sound field propagation path predictions and provide complete spatial acoustic feature input for cloud-based model parameter optimization.

[0055] In this implementation, a cloud-based sound field control model deployed in the cloud can integrate the acoustic characteristics of the target space matrix, the physical layout parameters of the audio system, and real-time acoustic feedback data to generate a dynamically optimized control parameter set. The cloud-based sound field control model can analyze the sound wave reflection patterns of different material areas and establish a three-dimensional sound field propagation model based on the installation angle and power parameters of the audio array.

[0056] For example, when it is detected that the high-frequency sound wave reflection is too strong in the glass wall area, the model can automatically adjust the high-frequency attenuation parameters of adjacent audio units, while optimizing the power distribution ratio in the mid- and low-frequency bands to achieve refined control of the sound field energy distribution.

[0057] The beneficial effect of this implementation is that, by integrating multidimensional data on material reflection intensity and acoustic absorption coefficient, the physical accuracy of sound field modeling can be significantly improved, providing reliable data support for analyzing the acoustic characteristics of complex spatial environments. The dynamic adjustment mechanism of sound field control parameters based on material characteristics can achieve noise suppression and sound energy balance while ensuring accurate sound quality, meeting the acoustic needs of different users in different home decoration scenarios.

[0058] The beneficial effect of the above implementation method is that the target space matrix generated by local preprocessing effectively reduces the data transmission volume of cloud computing, realizes the coordinated optimization of edge computing and cloud intelligence, and improves the system response efficiency.

[0059] In some implementations, the above method further includes S130 to S140, which are described in detail below.

[0060] S130: Clustering the material reflection intensity corresponding to each detection point in the three-dimensional point cloud data to obtain different reflection areas corresponding to different material reflection intensity ranges, and determining the reflection intensity difference between the different reflection areas. Determine multiple reflection area groups corresponding to reflection intensity differences less than a preset reflection intensity difference.

[0061] In this implementation, when processing three-dimensional point cloud data, cluster analysis can be performed on the material reflection intensity data corresponding to each detection point, classifying detection points with similar reflection intensity characteristics into the same reflection area. Specifically, by clustering the material reflection intensity corresponding to each detection point in the three-dimensional point cloud data, different reflection areas corresponding to different material reflection intensity ranges are obtained. Based on the distribution characteristics of the reflection intensity values, the boundaries of the reflection areas corresponding to different material reflection intensity ranges are automatically divided. After obtaining the different reflection areas, the reflection intensity difference between the different reflection areas can be determined. The reflection intensity difference represents the difference in reflection intensity between the different reflection areas.

[0062] When dividing the reflective area using three-dimensional point cloud data, there may be differences in optical and acoustic reflection characteristics. For example, a black wooden cabinet and a black sofa have similar optical reflection characteristics but quite different acoustic reflection characteristics. Therefore, areas with similar optical reflection characteristics can be further manually differentiated to optimize the sound field control effect.

[0063] After obtaining the reflection intensity difference, multiple reflection area groups corresponding to the reflection intensity difference being less than the preset reflection intensity difference can be determined. For the reflection area groups whose reflection intensity difference is less than the preset threshold, it can be determined that the corresponding materials have similar acoustic properties, thereby forming a material grouping that can be processed uniformly. Subsequently, the materials in the reflection area group can be manually labeled to improve the optimization effect of the sound field.

[0064] S140. Obtain the material information input by the user for each reflection area in the multiple reflection area groups, obtain the material sound absorption coefficient corresponding to the different material information, and update the material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data according to the material sound absorption coefficient corresponding to the different material information.

[0065] In this implementation, to obtain user-defined material annotation information for reflective area groups, after dividing the reflective areas, a visual interface displays the spatial distribution of multiple reflective area groups, allowing users to intuitively select the reflective areas they wish to annotate. For example, when a user selects a reflective area, they can use a drop-down menu or text input to annotate the area with the corresponding material type, such as wood, metal, or glass. The system can automatically access a pre-stored material database to obtain the standard sound absorption coefficient parameters corresponding to different material types.

[0066] After the user completes the annotation of the material information within the reflection area group, the material information input by the user for each reflection area in multiple reflection area groups can be obtained, and the material sound absorption coefficient corresponding to different material information can be obtained. The material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data can be updated according to the material sound absorption coefficient corresponding to the different material information, so as to realize batch update of the material sound absorption coefficient of each detection point in the three-dimensional point cloud data according to the sound absorption characteristics corresponding to different material types.

[0067] For example, for a reflective area group labeled "black wooden cabinets," the frequency-dependent absorption coefficient curve for the wood material in the database can be automatically loaded, replacing the default parameter values based on the estimated reflection intensity. This update ensures that the material parameters used in the acoustic simulation remain highly consistent with the actual material properties.

[0068] The beneficial effect of the above implementation method is that, through the intelligent clustering and difference analysis of the reflection intensity data, it can effectively identify material areas with similar acoustic characteristics, providing a reliable data basis for the subsequent refined parameter configuration; by combining the material type information input by the user, the standard acoustic parameters of the material can be accurately applied, avoiding the incorrect labeling of the acoustic reflection coefficient of materials with similar optical reflection characteristics, and can significantly improve the accuracy of the three-dimensional space sound field simulation.

[0069] The beneficial effect of the above implementation is that, by updating material parameters in groups, data processing efficiency is guaranteed, and accurate description of complex material distribution is achieved, providing effective technical support for sound field optimization.

[0070] In some implementations, in the above-mentioned S140, the target space matrix corresponding to the sound field target space is determined through the local space detection unit based on the three-dimensional point cloud data corresponding to the sound field target space and the material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data, including S141 to S142. S141 to S142 are described in detail below.

[0071] S141. Determine the angle between the normal vector corresponding to each detection point and the main axis direction of the sound of each speaker in the speaker array, and determine that the angle between the normal vector corresponding to each detection point and the main axis direction of the sound of each speaker in the speaker array is the minimum value, which is used as the main axis angle of the sound field corresponding to each detection point.

[0072] In this implementation, in order to further improve the sound field optimization effect, each detection point can be optimized with respect to the sound field reflection direction to improve the sound field optimization effect. When performing sound field optimization, the angle between the normal vector corresponding to each detection point and the sound main axis direction of each speaker in the sound array can be determined, and the angle between the normal vector corresponding to each detection point and the sound main axis direction of each speaker in the sound array can be determined to be the minimum value, which is used as the sound field main axis angle corresponding to each detection point. The sound field main axis angle represents the angle that produces the most direct reflection between the detection point and each speaker in the sound array.

[0073] For example, when determining the main axis angle of the sound field, the angle between the surface normal vector corresponding to each detection point and the main axis direction of the sound of each audio unit in the audio array can be calculated. Then, by traversing the sound directions of all audio units, the main axis direction with the smallest angle with the normal vector of the detection point can be screened out, and the corresponding angle value can be used as the main axis angle parameter of the sound field at the detection point. The main axis angle of the sound field can characterize the main propagation direction characteristics of the sound wave when it is reflected on the surface of the detection point.

[0074] S142: Determine the sound field principal axis angle range within which the sound field principal axis angle corresponding to each detection point lies, and obtain the reflection weight coefficient corresponding to each sound field principal axis angle range to obtain the reflection weight coefficient corresponding to each detection point. The target space matrix corresponding to the sound field target space is determined by the local space detection unit based on the three-dimensional point cloud data corresponding to the sound field target space, the material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data, and the reflection weight coefficient.

[0075] After obtaining the sound field principal axis angle corresponding to each detection point, the sound field principal axis angle range within which the sound field principal axis angle corresponding to each detection point is located can be determined, and the reflection weight coefficient corresponding to each sound field principal axis angle range can be obtained to obtain the reflection weight coefficient corresponding to each detection point. The reflection weight coefficient corresponding to each detection point represents the contribution of the detection point to the sound field optimization that needs to be considered during sound field optimization.

[0076] For example, when calculating the angle range of the main axis of the sound field, the angle interval of 0 degrees to 180 degrees can be divided into multiple continuous sub-intervals, and each sub-interval corresponds to a preset reflection weight coefficient. For example, when the main axis angle of the sound field at the detection point is in the range of 0 degrees to 30 degrees, a higher reflection weight coefficient can be assigned, reflecting that the sound wave has a stronger specular reflection characteristic in this area; when the angle is in the range of 150 degrees to 180 degrees, a lower reflection weight coefficient can be assigned, indicating that the sound wave in this area is mainly diffusely reflected. By establishing a mapping relationship between the angle range and the reflection weight, the degree of influence of different spatial regions on the reflection of sound waves can be effectively quantified.

[0077] After obtaining the reflection weight coefficient corresponding to each sound field main axis angle range, and using the reflection weight coefficient corresponding to the sound field main axis angle range corresponding to each detection point as the reflection weight coefficient corresponding to each detection point, the target space matrix corresponding to the sound field target space can be determined through the local space detection unit based on the three-dimensional point cloud data corresponding to the sound field target space, the material sound absorption coefficient and the reflection weight coefficient corresponding to each detection point in the three-dimensional point cloud data.

[0078] For example, after obtaining the material sound absorption coefficient and reflection weight coefficient for each detection point, the local spatial detection unit can be used to perform spatial gridding on the three-dimensional point cloud data, dividing the three-dimensional space into uniform voxel units. Based on the spatial distribution density of the detection points within each voxel, the comprehensive acoustic parameters of the voxel unit are weighted and calculated. At the same time, by integrating the material absorption coefficient, reflection weight coefficient, and spatial geometric relationships, a target space matrix can be constructed to characterize the acoustic propagation characteristics of the target space of the sound field. The target space matrix can accurately describe the energy attenuation, reflection characteristics, and diffraction effects of sound waves as they propagate in space.

[0079] The beneficial effect of this implementation is that, by precisely calculating the angle between the principal axes of the sound field and establishing a reflection weight model, it is possible to accurately quantify the influence of different spatial regions on the direction of sound wave propagation, significantly improving the physical accuracy of sound field modeling. By matrixing the material absorption characteristics with the geometric reflection characteristics, a multi-dimensional acoustic parameter space can be constructed, providing a data foundation for precise sound field simulation and audio system optimization.

[0080] The beneficial effect of the above implementation method is that, through the distributed computing of the local space detection unit, it not only ensures the efficiency of large-scale point cloud data processing, but also realizes the refined description of complex spatial acoustic characteristics, providing effective technical support for the spatial adaptive adjustment of the intelligent audio system.

[0081] In some implementations, in the above-mentioned S140, the target space matrix corresponding to the sound field target space is determined through the local space detection unit based on the three-dimensional point cloud data corresponding to the sound field target space and the material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data. It also includes S143 to S144. S143 to S144 are described in detail below.

[0082] S143. Determine the sum of the reflection area areas of the reflection areas where the detection points are located, corresponding to each sound field principal axis angle range, and determine the natural logarithm of the ratio of the area of each reflection area to the area of the minimum reflection area as the area priority value corresponding to each detection point in each reflection area. The minimum reflection area area corresponding to different sound field principal axis angle ranges is different.

[0083] When optimizing the acoustic modeling of the target space of the sound field, the spatial distribution characteristics of the reflection area can be quantitatively analyzed. For the reflection area corresponding to each sound field main axis angle range, the sum of the areas of the reflection areas of all detection points within the angle range can be calculated. By comparing the sum of the areas of the reflection areas corresponding to different angle ranges, the distribution characteristics of the reflection areas with significant spatial proportions in the sound field can be identified. At the same time, according to the preset minimum area benchmark values corresponding to different angle ranges, a priority evaluation model for the area proportions of the regions can be established.

[0084] In this implementation, in order to improve the optimization effect of the sound field, the sound field can be further optimized in combination with the reflection area to improve the sound field optimization effect of the reflection area with a smaller sound field main axis angle and a larger reflection area. During the optimization, the sum of the reflection area areas of the reflection areas where the detection points corresponding to each sound field main axis angle range are located can be determined, and the natural logarithm of the ratio of the area of each reflection area to the area of the minimum reflection area can be determined as the area priority value corresponding to each detection point in each reflection area. The area priority value represents the priority of the area ratio of each reflection area.

[0085] It should be noted that the minimum reflection area corresponding to different sound field principal axis angle ranges is different, and thus the contribution of different sound field principal axis angle ranges to the reflection area can be calculated differently.

[0086] For example, when calculating the area priority value, the ratio of the minimum reflection area corresponding to the angle range of the main axis of the sound field to which the sum of the areas of the reflection areas belongs can be calculated for the reflection area where each detection point is located. For example, when the sum of the areas of a certain reflection area reaches three times the minimum reference area of the angle range, the area priority value representing the spatial influence of the area can be obtained by calculating the natural logarithm. This calculation method can effectively distinguish the differences in the contributions of reflection areas of different spatial scales to the propagation of the sound field, and ensure that large-area reflection areas receive appropriate weight distribution in the acoustic model.

[0087] S144. Determine, through the local space detection unit, a target space matrix corresponding to the sound field target space based on the three-dimensional point cloud data corresponding to the sound field target space, the material sound absorption coefficient, the reflection weight coefficient, and the area priority value corresponding to each detection point in the three-dimensional point cloud data.

[0088] In this implementation method, the local space detection unit can be further used to determine the target space matrix corresponding to the sound field target space based on the three-dimensional point cloud data corresponding to the sound field target space, the material sound absorption coefficient, the reflection weight coefficient and the area priority value corresponding to each detection point in the three-dimensional point cloud data. The target space matrix combines the area priority value to improve the optimization effect of the spatial sound field.

[0089] After obtaining the material's sound absorption coefficient, reflection weight coefficient, and area priority value, the local spatial detection unit can be used to fuse the multi-dimensional acoustic parameters. For example, by dividing the 3D point cloud data into a spatial grid, a target space matrix can be constructed that reflects the comprehensive acoustic characteristics of the target space in the sound field. The target space matrix can accurately characterize the coupled relationship between energy attenuation, reflection distribution, and spatial coverage during sound wave propagation.

[0090] The beneficial effect of the above implementation method is that by introducing the calculation of the area priority value of the reflection area, it is possible to effectively quantify the differences in the contributions of spatial regions with different areas and different sound field main axis angle ranges to the sound field propagation, thereby avoiding excessive interference of local small-area reflection areas on the overall acoustic model.

[0091] The beneficial effect of the above implementation method is that by integrating the three-dimensional parameter system of material absorption, reflection weight and area priority value, a mathematical model that is more in line with the actual acoustic propagation laws can be constructed, which can significantly improve the accuracy and reliability of sound field simulation.

[0092] Figure 4 A flow chart of the second sound field adaptive cloud control method based on deep learning provided in the embodiment of the present application is shown as follows: Figure 4 As shown, the above method further includes S210 to S220, and S210 to S220 are described in detail below.

[0093] S210 , obtaining sound pressure values in different detection areas within the target sound field space, and determining a sound pressure change value between any two adjacent detection areas based on the sound pressure values of the different detection areas.

[0094] In this implementation, when optimizing the acoustic model of a target sound field space, dynamic parameter adjustments can be made by real-time monitoring of the sound pressure distribution characteristics within the space. For example, for different detection areas within the target sound field space, a distributed sound pressure sensor array can be used to obtain real-time sound pressure values for each area. By calculating the sound pressure gradient change between any two adjacent detection areas, areas with sudden changes in sound field energy distribution can be identified, providing data support for the refinement of the acoustic model.

[0095] In this implementation, the sound pressure values in different detection areas within the target space of the sound field can be obtained through detection. According to the sound pressure values of the different detection areas, the sound pressure change value between any two adjacent detection areas can be determined. Subsequently, the sound field can be optimized according to the sound pressure change value between any two adjacent detection areas.

[0096] For example, in a spatial scenario of a home audio-visual system, the difference in sound pressure at the boundary between two areas can be detected and compared. For example, if the sound pressure difference between the front and back of a sofa exceeds a preset threshold, the reflection weight coefficients of the adjacent areas can be increased to enhance the sound wave reflection characteristic parameters of the boundary area between the front and back of the sofa, prompting the acoustic model to more accurately reflect the energy concentration characteristics of the actual sound field.

[0097] S220: When the sound pressure change value between two adjacent target detection areas is greater than or equal to a preset sound pressure change value, add the preset reflection weight coefficient to the reflection weight coefficients of the two adjacent target areas to obtain adjusted reflection weight coefficients corresponding to the two adjacent target areas. Update the target space matrix corresponding to the sound field target space based on the adjusted reflection weight coefficients corresponding to the two adjacent target areas.

[0098] In this implementation, when it is detected that the sound pressure change value between two adjacent detection areas of the target is greater than or equal to the preset sound pressure change value, the preset reflection weight coefficient can be added to the reflection weight coefficient of the two adjacent target areas to obtain the adjusted reflection weight coefficient corresponding to the two adjacent target areas, and then the target space matrix is updated according to the adjusted adjusted reflection weight coefficient.

[0099] During the updating, the target space matrix corresponding to the sound field target space may be updated according to the adjusted reflection weight coefficients corresponding to the two target adjacent areas.

[0100] For example, for adjacent areas with significant sound pressure variations, a preset reflection weighting factor of 0.15-0.3 can be added to the original reflection weighting factor. The specific value of the preset reflection weighting factor is dynamically configured based on the actual use of the sound field space. The rapid computing power of the local space detection unit enables real-time parameter updates of the target space matrix, ensuring that the acoustic model remains synchronized with the sound field characteristics of the physical space.

[0101] The beneficial effect of the above implementation method is that, through dynamic detection of sound pressure change values and threshold determination, it is possible to effectively identify areas with abnormal sound field energy distribution, thereby improving the accuracy of acoustic modeling in capturing the characteristics of the actual spatial sound field.

[0102] The aforementioned implementation also offers the beneficial effect of significantly improving the acoustic model's simulation fidelity in areas of sudden energy changes through the dynamic enhancement of reflection weight coefficients, providing a reliable basis for adaptive equalization of the sound system. The real-time update capability of the matrix parameters ensures the timeliness of the acoustic model while enabling rapid response to changes in complex sound field environments.

[0103] In some implementations, in S220 above, the preset reflection weight coefficient is determined by obtaining the target sound field principal axis angles corresponding to two adjacent target detection areas, and determining the average of the ratios of the two target sound field principal axis angles to the base sound field principal axis angle as the preset reflection weight coefficient, where the base sound field principal axis angle is 15° to 30°.

[0104] In this implementation, when dynamically adjusting the sound field reflection weight coefficient, an adaptive adjustment mechanism can be established through the comparative relationship between the sound field principal axis angle and the reference parameters. For adjacent detection areas with significant sound pressure changes, their corresponding sound field principal axis angle parameters can be extracted respectively. By calculating the relative ratio of these angles to the reference angle range, a dynamic adjustment coefficient reflecting the differences in acoustic characteristics between areas is generated.

[0105] In this implementation, the target sound field main axis angles corresponding to two adjacent target detection areas can be obtained, and the average of the ratios of the two target sound field main axis angles and the basic sound field main axis angle can be determined as the preset reflection weight coefficient.

[0106] Exemplarily, the angle of the main axis of the basic sound field may be 15° to 30°.

[0107] To perform the calculation, first obtain the sound field principal axis angle values corresponding to the two adjacent detection areas to be adjusted. For example, if the sound field principal axis angle of the left detection area is 25 degrees and that of the right detection area is 18 degrees, the median value of the reference angle range can be selected as the basic parameter for comparison and calculation.

[0108] For example, when the reference angle is set to 22.5 degrees, the main axis angle of the sound field in the front area of the sofa is 28 degrees, and the main axis angle of the sound field in the back area of the sofa is 16 degrees. Then, the ratios of (28 / 22.5) and (16 / 22.5) can be calculated respectively and the arithmetic average can be taken to obtain the reflection weight adjustment coefficient reflecting the acoustic characteristics of the boundary area, so that the reflection weight adjustment coefficient can ensure that the weight coefficient maintains a positive correlation with the actual sound wave propagation direction characteristics.

[0109] The beneficial effect of the above implementation method is that, by calculating the ratio mean of the angle between the main axes of the sound field, a dynamic adjustment mechanism that conforms to the law of sound wave propagation can be established, effectively improving the scientific nature of the setting of the reflection weight coefficient.

[0110] The beneficial effect of the above implementation method is that, by introducing the angle of the main axis of the basic sound field, it can ensure that the weight adjustment of different spatial areas has a unified reference standard, thereby enhancing the interpretability and consistency of the acoustic model parameters.

[0111] Figure 5A flow chart of the third sound field adaptive cloud control method based on deep learning provided in the embodiment of the present application is shown as follows: Figure 5 As shown, the above method further includes S310 to S320, and S310 to S320 are described in detail below.

[0112] S310. After adjusting the working state of the sound array according to the control parameter information, obtain impulse response data detected by the adjusted microphone array, and obtain a difference between the impulse response data detected by the adjusted microphone array and the impulse response data detected by the microphone array before the adjustment as an impulse response data change value.

[0113] In this implementation, after the working state of the audio array is adjusted according to the control parameter information, in order to evaluate the optimization effect of the audio, dynamic adjustment verification can be achieved through feedback analysis of multi-dimensional sensor data when performing closed-loop control optimization of the audio array. Specifically, impulse response data detected by the adjusted microphone array can be obtained, and the difference between the impulse response data detected by the adjusted microphone array and the impulse response data detected by the microphone array before adjustment can be obtained as the impulse response data change value.

[0114] For example, after completing the adjustment of the sound parameters, the spatial impulse response data can be collected through a high-precision microphone array. By comparing the changes in acoustic characteristics before and after the adjustment, and verifying the degree of matching between the acoustic model and the actual sound field response through a feedback mechanism, the reliability of the system control can be guaranteed.

[0115] For example, after adjusting the parameters of a sound array in a home audio-visual environment, an impulse response can be collected using a circular microphone array. For example, by comparing the change in the energy ratio of direct sound to reverberant sound received by the microphones before and after the adjustment, the change in impulse response data can be calculated to represent the effect of the sound field manipulation. This change in impulse response data can reflect the actual impact of the acoustic parameter adjustment on the spatial sound field characteristics.

[0116] S320. Determine, through a control detection unit, a control adjustment confidence level for the sound array based on the target space matrix, the physical layout parameters of the sound array, the operating parameter information of the sound array, the impulse response data detected by the microphone array before adjustment, the impulse response data change value, and the position information of the aligned microphone array. When the control adjustment confidence level is greater than or equal to a preset control adjustment confidence level, adjust the operating state of the sound array again according to the control parameter information based on user feedback. When the control adjustment confidence level is less than the preset control adjustment confidence level, reacquire the three-dimensional point cloud data corresponding to the sound field target space, and update the target space matrix corresponding to the sound field target space based on the reacquired three-dimensional point cloud data corresponding to the sound field target space.

[0117] In this implementation, the control detection unit can be controlled to determine the control adjustment confidence of the sound array based on the target space matrix, the physical layout parameters of the sound array, the operating parameter information of the sound array, the impulse response data detected by the microphone array before adjustment, the change value of the impulse response data, and the position information of the aligned microphone array. The control adjustment confidence represents the degree of optimization effect of the sound.

[0118] For example, during the control adjustment confidence calculation process, data such as the acoustic propagation model parameters of the target spatial matrix, the position coordinates and power parameters of the speaker units, and the impulse response change can be input into the neural network model. A preset confidence assessment algorithm is then used to output a confidence score for the control effect. If the score reaches a preset threshold, or if it falls short, after obtaining the control adjustment confidence, if the control adjustment confidence is greater than or equal to the preset control adjustment confidence, it indicates that the current adjustment has effectively improved the sound field characteristics, and the operating state of the speaker array can be further adjusted based on user feedback and the control parameter information.

[0119] After obtaining the control adjustment confidence, when the control adjustment confidence is less than the preset control adjustment confidence, the spatial data re-collection process is triggered to ensure that the acoustic model and the physical space are updated synchronously, and the three-dimensional point cloud data corresponding to the sound field target space is re-acquired. The target space matrix corresponding to the sound field target space is updated according to the re-acquired three-dimensional point cloud data corresponding to the sound field target space, thereby improving the sound field optimization effect of the audio.

[0120] The beneficial effect of the above implementation method is that, through the dynamic comparative analysis of impulse response data, a closed-loop verification mechanism for the sound field control effect can be established, which significantly improves the accuracy and reliability of the sound parameter adjustment.

[0121] The beneficial effect of the above implementation method is that, by integrating the confidence assessment model of multi-dimensional parameters, it is possible to intelligently judge the effectiveness of the control strategy and avoid control failure problems caused by environmental changes.

[0122] In some implementations, in the above S320 , the operating state of the audio array is adjusted again according to the control parameter information based on the user feedback, including S321 to S322 , which are described in detail below.

[0123] S321. Obtain reverberation feedback information corresponding to different positions of the user in the target sound field space.

[0124] When intelligently optimizing and adjusting the speaker array, precise control can be achieved by integrating subjective user feedback with objective acoustic data. At different locations in the target sound field space, user terminal devices can collect perceptual feedback on the auditory experience and optimize the sound field, forming a closed-loop control system for human-machine collaboration.

[0125] In this implementation, reverberation feedback information corresponding to different user locations within the target sound field space can be obtained. For example, user ratings of the reverberation characteristics of each area can be obtained through a mobile application interface, and the sound field can be optimized based on the ratings of the reverberation characteristics of each area.

[0126] For example, in a home audio-visual scene, the user's reverberation perception feedback can be collected through the interactive terminal. When the user marks "insufficient reverberation" in the sofa area and "too long reverberation" in the balcony area, the system can convert these subjective evaluations into reverberation time correction parameters for different spatial locations.

[0127] S322. Using the cloud-based sound field control model, determine the control parameter information of the sound array based on the three-dimensional point cloud data of the sound field target space, the physical layout parameters of the sound array, the operating parameter information of the sound array, the impulse response data detected by the microphone array, the position information of the aligned microphone array, and the reverberation feedback information, and adjust the operating state of the sound array according to the control parameter information.

[0128] In order to obtain the reverberation time correction parameters for different spatial positions, the cloud-based sound field control model can be used to determine the control parameter information of the sound array based on the three-dimensional point cloud data of the sound field target space, the physical layout parameters of the sound array, the working parameter information of the sound array, the impulse response data detected by the microphone array, the position information of the aligned microphone array, and the reverberation feedback information. The working state of the sound array can be adjusted according to the control parameter information to achieve the purpose of continuously optimizing the sound field based on the reverberation feedback information.

[0129] During the parameter optimization stage, multi-source data can be fused and processed, and the spatial acoustic characteristics represented by three-dimensional point cloud data, the physical layout parameters of the audio equipment, the impulse response waveform data collected in real time, and the reverberation evaluation information fed back by users can be input into the cloud-based sound field control model. The cloud-based sound field control model can generate an optimized control parameter set that takes into account both objective acoustic indicators and subjective listening experience, such as the directional angle adjustment amount of the audio unit, the equalizer frequency response curve correction parameters, etc.

[0130] The beneficial effect of this implementation is that, by integrating subjective user feedback data, it can overcome the limitations of traditional purely physical parameter optimization and further achieve personalized sound field adjustment that incorporates the user's auditory perception characteristics. By establishing a mapping model between user feedback and physical parameters, it can continuously accumulate acoustic optimization experience data for different scenarios, forming an intelligent control system with self-learning capabilities.

[0131] An embodiment of the present application also provides a deep learning-based sound field adaptive cloud control system, comprising a unit for executing any of the methods described above.

[0132] Figure 6 A logical structure diagram of a sound field adaptive cloud control system based on deep learning provided in an embodiment of the present application is shown as follows: Figure 6 As shown, the system 1 of this embodiment includes a processing unit 11, a storage unit 12, and a transceiver unit 13. The processing unit 11 is used to process data, the storage unit 12 is used to store data, and the transceiver unit 13 is used to send and receive data. The processing unit 11, the storage unit 12, and the transceiver unit 13 cooperate with each other to implement the above method. The beneficial effects of the embodiment of the present application have been described in the above method and will not be repeated here.

[0133] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0134] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0135] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process of the above-mentioned method embodiment by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can at least include: any entity or device capable of carrying computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, mobile hard drive, magnetic disk, or optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0136] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0137] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0138] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0139] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0140] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A sound field adaptive cloud control method based on deep learning, characterized in that: The method comprises: Obtaining three-dimensional point cloud data corresponding to the sound field target space and physical layout parameters of the sound array in the sound field target space; obtaining operating parameter information of the sound array when operating in the sound field target space, and obtaining position information of the microphone array and impulse response data corresponding to the operating parameter information detected by the microphone array in the sound field target space; aligning the position information of the microphone array with the three-dimensional point cloud data to obtain the aligned position information of the microphone array; wherein the three-dimensional point cloud data includes a normal vector corresponding to each detection point; Through the cloud-based sound field control model, the control parameter information of the sound array is determined based on the three-dimensional point cloud data of the sound field target space, the physical layout parameters of the sound array, the working parameter information of the sound array, the impulse response data detected by the microphone array, and the position information of the aligned microphone array, and the working state of the sound array is adjusted according to the control parameter information.

2. The method according to claim 1, wherein The method further comprises: Obtain the material reflection intensity corresponding to each detection point in the 3D point cloud data, and obtain the material sound absorption coefficient corresponding to different material reflection intensities; Through the local space detection unit, the target space matrix corresponding to the sound field target space is determined based on the three-dimensional point cloud data corresponding to the sound field target space and the material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data; through the cloud-based sound field control model, the control parameter information of the sound array is determined based on the target space matrix, the physical layout parameters of the sound array, the working parameter information of the sound array, the impulse response data detected by the microphone array and the position information of the aligned microphone array, and the working state of the sound array is adjusted according to the control parameter information.

3. The method according to claim 2, wherein The method further comprises: Clustering the material reflection intensity corresponding to each detection point in the three-dimensional point cloud data to obtain different reflection areas corresponding to different material reflection intensity ranges, and determining the reflection intensity difference between different reflection areas; determining multiple reflection area groups corresponding to reflection intensity differences less than a preset reflection intensity difference; Obtain material information input by the user for each reflection area in multiple reflection area groups, obtain material sound absorption coefficients corresponding to different material information, and update the material sound absorption coefficients corresponding to each detection point in the three-dimensional point cloud data according to the material sound absorption coefficients corresponding to the different material information.

4. The method according to claim 3, wherein The target space matrix corresponding to the sound field target space is determined by the local space detection unit based on the three-dimensional point cloud data corresponding to the sound field target space and the material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data, including: Determine the angle between the normal vector corresponding to each detection point and the main axis direction of the sound of each speaker in the sound array, and determine that the angle between the normal vector corresponding to each detection point and the main axis direction of the sound of each speaker in the sound array is the minimum value, and use it as the main axis angle of the sound field corresponding to each detection point; Determine the sound field principal axis angle range within which the sound field principal axis angle corresponding to each detection point is located, obtain the reflection weight coefficient corresponding to each sound field principal axis angle range, and obtain the reflection weight coefficient corresponding to each detection point; through the local space detection unit, determine the target space matrix corresponding to the sound field target space based on the three-dimensional point cloud data corresponding to the sound field target space, the material sound absorption coefficient and the reflection weight coefficient corresponding to each detection point in the three-dimensional point cloud data.

5. The method according to claim 4, wherein Determining a target space matrix corresponding to the sound field target space through a local space detection unit based on the three-dimensional point cloud data corresponding to the sound field target space and the material sound absorption coefficient corresponding to each detection point in the three-dimensional point cloud data, further comprising: Determine the sum of the reflection area areas of the reflection areas where the detection points are located corresponding to each sound field principal axis angle range, and determine the natural logarithm of the ratio of the area of each reflection area to the area of the minimum reflection area as the area priority value corresponding to each detection point in each reflection area; wherein the minimum reflection area area corresponding to different sound field principal axis angle ranges is different; Through the local space detection unit, the target space matrix corresponding to the sound field target space is determined based on the three-dimensional point cloud data corresponding to the sound field target space, the material sound absorption coefficient, the reflection weight coefficient and the area priority value corresponding to each detection point in the three-dimensional point cloud data.

6. The method according to claim 5, wherein The method further comprises: Acquire the sound pressure values in different detection areas within the target sound field space, and determine the sound pressure change value between any two adjacent detection areas based on the sound pressure values of the different detection areas; When the sound pressure change value between two adjacent detection areas of the target is greater than or equal to the preset sound pressure change value, the preset reflection weight coefficient is added to the reflection weight coefficient of the two adjacent target areas to obtain the adjusted reflection weight coefficients corresponding to the two adjacent target areas; the target space matrix corresponding to the sound field target space is updated according to the adjusted reflection weight coefficients corresponding to the two adjacent target areas.

7. The method according to claim 6, wherein The preset reflection weight coefficient is determined by the following method: Obtain the target sound field main axis angles corresponding to the two adjacent detection areas of the target, and determine the average of the ratios of the two target sound field main axis angles and the basic sound field main axis angle as the preset reflection weight coefficient; wherein the basic sound field main axis angle is 15° to 30°.

8. The method according to claim 7, wherein The method further comprises: After adjusting the working state of the sound array according to the control parameter information, obtaining impulse response data detected by the adjusted microphone array, and obtaining a difference between the impulse response data detected by the adjusted microphone array and the impulse response data detected by the microphone array before the adjustment as an impulse response data change value; The control adjustment confidence of the sound array is determined by a control detection unit based on the target space matrix, the physical layout parameters of the sound array, the working parameter information of the sound array, the impulse response data detected by the microphone array before adjustment, the change value of the impulse response data, and the position information of the aligned microphone array; when the control adjustment confidence is greater than or equal to the preset control adjustment confidence, the working state of the sound array is adjusted again according to the control parameter information based on user feedback; when the control adjustment confidence is less than the preset control adjustment confidence, the three-dimensional point cloud data corresponding to the sound field target space is re-acquired, and the target space matrix corresponding to the sound field target space is updated according to the re-acquired three-dimensional point cloud data corresponding to the sound field target space.

9. The method according to claim 8, wherein Based on user feedback, the working status of the audio array is adjusted again according to the control parameter information, including: Obtain reverberation feedback information corresponding to different positions of the user in the target sound field space; Through the cloud-based sound field control model, the control parameter information of the sound array is determined based on the three-dimensional point cloud data of the sound field target space, the physical layout parameters of the sound array, the working parameter information of the sound array, the impulse response data detected by the microphone array, the position information of the aligned microphone array and the reverberation feedback information, and the working state of the sound array is adjusted according to the control parameter information.

10. A sound field adaptive cloud control system based on deep learning, characterized in that: Comprising means for performing the method according to any one of claims 1 to 9.

Images