Environment perception method and device, computer equipment and storage medium
By collecting environmental and point cloud data from audio equipment, extracting multi-dimensional features, and using a neural network model to identify scenes, the problem of single perception dimension of audio equipment is solved, achieving accurate scene recognition and adaptive adjustment, and improving user experience.
Patent Information
- Application Number
- CN202511753869.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-26
- Publication Date
- 2026-04-24
AI Technical Summary
Existing audio equipment has limited environmental perception capabilities, resulting in low scene recognition accuracy, which affects sound clarity and compatibility, and reduces user experience.
By collecting environmental data and point cloud data, spatial features and human activity features are extracted, and a pre-set neural network model is used to determine the scene type of the target environment. Multi-dimensional features are then integrated for accurate identification.
It improves the scene recognition accuracy of audio equipment in the target environment, realizes intelligent adaptive adjustment, and enhances the user experience.
Smart Images
Figure HDA0005707622730000011 
Figure HDA0005707622730000021 
Figure HDA0005707622730000022
Abstract
Description
Technical Field
[0001] This application relates to the field of audio equipment, and more particularly to an environmental sensing method, apparatus, computer device, and storage medium. Background Technology
[0002] With the development of the internet, audio equipment has been widely used in various scenarios such as conference rooms, living rooms, classrooms, and stages. Users can use audio equipment to amplify sound, enhance stage atmosphere, and perform other functions. As its popularity increases, users' demands for audio equipment are constantly evolving, with "dynamic environmental perception and adaptive adjustment" becoming a key requirement. Audio equipment captures changes in the surrounding environment in real time and automatically optimizes sound playback parameters to adapt to the auditory needs of different scenarios.
[0003] Currently, traditional audio equipment has a relatively simple environmental perception capability, mainly relying on a single sensor. Specifically, traditional audio equipment uses a microphone to capture ambient noise, then analyzes the noise data to determine the environmental conditions in which the audio equipment is located.
[0004] However, this perception method based solely on noise has significant limitations. When audio equipment is in a noisy environment, noise data alone cannot accurately distinguish between indoor and outdoor scenes. This ambiguity in environmental recognition directly leads to inaccurate adjustment of playback parameters, ultimately affecting the clarity, stereo effect, and compatibility of the sound, severely reducing the user experience. Summary of the Invention
[0005] This application provides an environmental perception method, apparatus, computer device, and storage medium, aiming to solve the technical problem of low scene recognition accuracy caused by the single perception dimension of audio equipment in the prior art.
[0006] In a first aspect, embodiments of this application provide an environmental perception method, the method being applied to an audio device, comprising:
[0007] Collect environmental data and point cloud data of the target environment;
[0008] Extract spatial features from the point cloud data;
[0009] Extract human activity features from the point cloud data;
[0010] The scene type of the target environment is determined based on the environmental data, the spatial features, and the personnel activity features.
[0011] In one embodiment, the environmental parameters include temperature and humidity, light intensity, and ambient noise in decibels.
[0012] In one embodiment, the spatial features include spatial scale and spatial shape, and the extraction of spatial features from the point cloud data includes:
[0013] The point cloud data is subjected to noise reduction processing;
[0014] Valid scene point cloud data are selected from the noise-reduced point cloud data.
[0015] Extract geometric features from the effective scene point cloud data;
[0016] The spatial scale and spatial shape are determined based on the geometric features.
[0017] In one embodiment, the personnel activity features include basic behavior types and interaction behavior types of personnel, and the extraction of personnel activity features from the point cloud data includes:
[0018] The point cloud data is subjected to noise reduction processing;
[0019] Human point cloud data is filtered from the point cloud data, which includes the human point cloud data of the target person.
[0020] Extract the position change features, posture features, and morphological features of the target person from the human body point cloud data;
[0021] The basic behavior type and interaction behavior type of the target person are determined based on the position change characteristics, posture characteristics and morphological characteristics of the target person.
[0022] In one embodiment, determining the scene type of the target environment based on the environmental data, the spatial features, and the personnel activity features includes:
[0023] The environmental data, spatial features, and human activity features are input into a preset neural network model to obtain the scene type of the target environment.
[0024] In one embodiment, the construction steps of the preset neural network model include:
[0025] Acquire training data, which includes environmental data and point cloud data for multiple environments;
[0026] Build a basic neural network model;
[0027] The loss function of the basic neural network model is set to the cross-entropy loss function;
[0028] The activation function of the basic neural network model is set to the ReLU function;
[0029] The evaluation metrics for the basic neural network model are set, including: precision, recall, and F1 score.
[0030] Based on the loss function, the activation function, and the evaluation metric, the training data is used to train, validate, and test the basic neural network model to obtain the preset neural network model.
[0031] In one embodiment, if the target environment is an outdoor scene, the method further includes:
[0032] When the audio device is playing music, the echo cancellation function and dynamic equalization function are activated.
[0033] Secondly, embodiments of this application also provide an environmental sensing device, which includes a unit for performing the above-described method.
[0034] Thirdly, embodiments of this application also provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.
[0035] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described method.
[0036] This application provides an environmental perception method, apparatus, computer device, and storage medium. The method, applied to audio equipment, includes: collecting environmental data and point cloud data of a target environment; extracting spatial features from the point cloud data; extracting human activity features from the point cloud data; and determining the scene type of the target environment based on the environmental data, the spatial features, and the human activity features. Thus, this application's technical solution collects environmental data and point cloud data of the target environment. Next, spatial features and human activity features are extracted from the point cloud data. Finally, the scene type of the target environment is determined based on the environmental data, spatial features, and human activity features. Therefore, this application's technical solution integrates physical environment data and point cloud data, constructing multi-dimensional features from three dimensions: environmental data, spatial structure, and human dynamic activities, achieving accurate identification of the target environment scene type. This design provides underlying technical support for the intelligent adaptive adjustment of audio equipment, effectively solving the technical problems of low scene recognition accuracy and weak adaptive capability caused by the single perception dimension in existing technical solutions, ultimately significantly improving the user experience. Attached Figure Description
[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0039] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.
[0040] Figure 1 A flowchart illustrating an environmental perception method provided in an embodiment of this application;
[0041] Figure 2 A schematic block diagram of an environmental sensing device provided in an embodiment of this application;
[0042] Figure 3 A computer device provided in an embodiment of this application. Detailed Implementation
[0043] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0044] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.
[0045] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0046] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0047] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0048] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."
[0049] To address the technical problem of low scene recognition accuracy caused by the single perception dimension of existing audio equipment, this application provides an environmental perception device that can improve the accuracy of scene recognition.
[0050] Figure 1 This is a flowchart illustrating an environmental perception method provided in an embodiment of this application. In one embodiment, the method is applied to an audio device, and the method includes steps S101-S104.
[0051] S101. Collect environmental data and point cloud data of the target environment.
[0052] In one embodiment, environmental data includes, but is not limited to, temperature and humidity, light intensity, and ambient noise levels in decibels. In this embodiment, a temperature and humidity sensor, a light sensor, and a microphone are integrated into the audio equipment to collect data on the target environment's temperature, humidity, light intensity, and ambient noise levels in decibels.
[0053] Point cloud data includes single-frame point cloud data and point cloud sequence data. The point cloud sequence data includes point cloud data from multiple related frames. It should be noted that both single-frame and point cloud sequence data are three-dimensional point cloud data.
[0054] It should be noted that in this embodiment, a LiDAR is integrated into the audio device to acquire point cloud data of the surrounding environment. Alternatively, in this embodiment, the audio device can be linked with a dedicated scanning device for acquiring point cloud data. The scanning device acquires point cloud data of the surrounding environment, which is then sent to the audio device for processing.
[0055] S102. Extract spatial features from point cloud data.
[0056] In one embodiment, the spatial features include, but are not limited to, spatial scale and spatial shape, and S102 specifically includes the following steps: S1021-S1024.
[0057] S1021. Perform noise reduction processing on the point cloud data.
[0058] S1022. Select valid scene point cloud data from the point cloud data after noise reduction processing.
[0059] S1023. Extract geometric features from effective scene point cloud data.
[0060] S1024. Determine the spatial scale and shape based on geometric characteristics.
[0061] It should be noted that S1021-S1024 will be explained in detail below.
[0062] In this embodiment, the point cloud data is first subjected to noise reduction and downsampling. Next, the RANSAC plane fitting algorithm is used to filter valid scene point cloud data from the noise-reduced point cloud data. Valid scene point cloud data includes, but is not limited to, ground, walls, and static obstacles. Furthermore, geometric features are extracted from the valid scene point cloud data, and the spatial scale and shape are calculated based on these geometric features.
[0063] S103. Extract human activity features from point cloud data.
[0064] In one embodiment, the characteristics of personnel activity include basic behavior types and interactive behavior types of personnel, and S103 above specifically includes the following steps: S1031-S1033.
[0065] S1031. Filter human body point cloud data from point cloud data.
[0066] Among them, human body point cloud data includes human body point cloud data of the target person.
[0067] S1032. Extract the position change features, posture features and morphological features of the target person from the human body point cloud data of the target person.
[0068] S1033. Determine the basic behavior type and interaction behavior type of the target personnel based on their positional change characteristics, posture characteristics, and morphological characteristics.
[0069] It should be noted that S1031-S1033 will be explained in detail below.
[0070] In this embodiment, the point cloud data is first denoised. Then, human point cloud data is filtered from the denoised point cloud data. The human point cloud data includes multiple associated data frames, each containing a cluster of human point clouds. Next, the position change features, posture features, and morphological features of the target person are extracted from the human point cloud clusters. The position change features characterize the target person's current movement speed or position change relationship. The posture features characterize the target person's current posture, such as bending over or standing. The morphological features characterize the target person's shape. In this embodiment, the basic behavior type or interaction behavior type of the target person is determined based on their position change features, posture features, and morphological features. For example, the basic behavior type could be sitting and reading, and the interaction behavior type could be conversing.
[0071] S104. Determine the scene type of the target environment based on environmental data, spatial characteristics, and human activity characteristics.
[0072] In one embodiment, S104 specifically includes the following steps: S1041.
[0073] S1041. Input environmental data, spatial features, and personnel activity features into a preset neural network model to obtain the scene type of the target environment.
[0074] The steps for constructing the pre-defined neural network model include:
[0075] a. Obtain training data.
[0076] The training data includes environmental data and point cloud data from multiple environments. These environments include, but are not limited to, conference rooms, living rooms, kitchens, bedrooms, plazas, and parks.
[0077] In this embodiment, noise reduction processing is required for the point cloud data of each environment. Then, spatial features and human activity features are extracted from the noise-reduced point cloud data. For specific extraction methods, please refer to the above embodiments. This application will not elaborate further here.
[0078] b. Construct a basic neural network model.
[0079] c. Set the loss function of the basic neural network model to the cross-entropy loss function.
[0080] d. Set the activation function of the basic neural network model to the ReLU function;
[0081] e. Set the evaluation metrics for the basic neural network model.
[0082] Evaluation metrics include precision, recall, and F1 score.
[0083] f. Based on the loss function, activation function, and evaluation metrics, the basic neural network model is trained, validated, and tested using training data to obtain the preset neural network model.
[0084] This application provides an environmental perception method. The method is applied to audio equipment and includes: collecting environmental data and point cloud data of a target environment; extracting spatial features from the point cloud data; extracting human activity features from the point cloud data; and determining the scene type of the target environment based on the environmental data, the spatial features, and the human activity features. Therefore, this application's technical solution collects environmental data and point cloud data of the target environment. Next, spatial features and human activity features are extracted from the point cloud data. Finally, the scene type of the target environment is determined based on the environmental data, spatial features, and human activity features. Thus, this application's technical solution integrates physical environment data and point cloud data, constructing multi-dimensional features from three dimensions: environmental data, spatial structure, and human dynamic activities, achieving accurate identification of the target environment scene type. This design provides underlying technical support for the intelligent adaptive adjustment of audio equipment, effectively solving the technical problems of low scene recognition accuracy and weak adaptive capability caused by the single perception dimension in existing technical solutions, ultimately significantly improving the user experience.
[0085] In one embodiment, if the target environment is an outdoor scene, the method further includes: S105.
[0086] S105. If the audio equipment is playing music, activate the echo cancellation function and dynamic equalization function.
[0087] When audio equipment is in an outdoor setting, sound will be reflected multiple times by different objects such as buildings and the ground, creating echoes. Therefore, it is necessary to activate the echo cancellation function.
[0088] In addition, the noise sources in outdoor scenes are more complex. Compared with indoor scenes, audio equipment needs to activate the dynamic echo equalization function to achieve dynamic equalization of the sound field, eliminate reverberation, and improve clarity.
[0089] See Figure 2 , Figure 2This is a schematic block diagram of an environmental sensing device provided in an embodiment of this application. Corresponding to the above-described environmental sensing method, this application also provides an environmental sensing device. This environmental sensing device includes a unit for executing the above-described environmental sensing method, and can be configured in a terminal such as a desktop computer, tablet computer, or laptop computer. Specifically, the environmental sensing device includes:
[0090] Acquisition unit 201 is used to acquire environmental data and point cloud data of the target environment;
[0091] The first extraction unit 202 is used to extract spatial features from the point cloud data;
[0092] The second extraction unit 203 is used to extract human activity features from the point cloud data;
[0093] The determining unit 204 is used to determine the scene type of the target environment based on the environmental data, the spatial features, and the personnel activity features.
[0094] In one embodiment, the environmental data includes temperature and humidity, light intensity, and ambient noise in decibels.
[0095] In one embodiment, the spatial features include spatial scale and spatial shape, and the first extraction unit 202 is specifically used for:
[0096] The point cloud data is subjected to noise reduction processing;
[0097] Valid scene point cloud data are selected from the noise-reduced point cloud data.
[0098] Extract geometric features from the effective scene point cloud data;
[0099] The spatial scale and spatial shape are determined based on the geometric features.
[0100] In one embodiment, the personnel activity characteristics include basic behavior types and interactive behavior types of personnel, and the second extraction unit 203 is specifically used to filter human point cloud data from the point cloud data, wherein the human point cloud data includes human point cloud data of the target personnel.
[0101] Extract the position change features, posture features, and morphological features of the target person from the human body point cloud data;
[0102] The basic behavior type and interaction behavior type of the target person are determined based on the position change characteristics, posture characteristics and morphological characteristics of the target person.
[0103] In one embodiment, the determining unit 204 is specifically used for:
[0104] The environmental data, spatial features, and human activity features are input into a preset neural network model to obtain the scene type of the target environment.
[0105] In one embodiment, the determining unit 204 is further specifically used for:
[0106] Acquire training data, which includes environmental data and point cloud data for multiple environments;
[0107] Build a basic neural network model;
[0108] The loss function of the basic neural network model is set to the cross-entropy loss function;
[0109] The activation function of the basic neural network model is set to the ReLU function;
[0110] The evaluation metrics for the basic neural network model are set, including precision, recall, and F1 score.
[0111] Based on the loss function, the activation function, and the evaluation metric, the training data is used to train, validate, and test the basic neural network model to obtain the preset neural network model.
[0112] In one embodiment, the device further includes a startup unit 205, used to activate the echo cancellation function and the dynamic equalization function when the audio device is in music playback mode.
[0113] like Figure 3 As shown, this application provides a computer device including a processor 31, a communication interface 32, a memory 33, and a communication bus 34. The processor 31, the communication interface 32, and the memory 33 communicate with each other through the communication bus 34. The memory 33 is used to store computer programs.
[0114] In one embodiment of this application, when the processor 31 executes the program stored in the memory 33, it implements the environment-aware control method provided in any of the foregoing method embodiments.
[0115] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.
[0116] Therefore, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the environment perception method provided in any of the foregoing method embodiments.
[0117] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code. The computer-readable storage medium can be non-volatile or volatile.
[0118] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.
[0119] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.
[0120] The steps in the methods of this application embodiment can be adjusted, merged, or deleted according to actual needs. The units in the apparatus of this application embodiment can be merged, divided, or deleted according to actual needs. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.
[0122] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0123] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Since these modifications and variations fall within the scope of the claims and their equivalents, this application also intends to include these modifications and variations.
[0124] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. An environmental perception method, characterized in that, The method is applied to audio equipment, and the method includes: Collect environmental data and point cloud data of the target environment; Extract spatial features from the point cloud data; Extract human activity features from the point cloud data; The scene type of the target environment is determined based on the environmental data, the spatial features, and the personnel activity features.
2. The method according to claim 1, characterized in that, The environmental data includes temperature and humidity, light intensity, and ambient noise levels in decibels.
3. The method according to claim 1, characterized in that, The spatial features include spatial scale and spatial shape, and the extraction of spatial features from the point cloud data includes: The point cloud data is subjected to noise reduction processing; Valid scene point cloud data are selected from the noise-reduced point cloud data. Extract geometric features from the effective scene point cloud data; The spatial scale and spatial shape are determined based on the geometric features.
4. The method according to claim 1, characterized in that, The personnel activity features include basic behavior types and interaction behavior types. Extracting personnel activity features from the point cloud data includes: The point cloud data is subjected to noise reduction processing; Human point cloud data is filtered from the point cloud data, which includes the human point cloud data of the target person. Extract the position change features, posture features, and morphological features of the target person from the human body point cloud data; The basic behavior type and interaction behavior type of the target person are determined based on the position change characteristics, posture characteristics and morphological characteristics of the target person.
5. The method according to claim 1, characterized in that, Determining the scene type of the target environment based on the environmental data, the spatial features, and the personnel activity features includes: The environmental data, spatial features, and human activity features are input into a preset neural network model to obtain the scene type of the target environment.
6. The method according to claim 5, characterized in that, The steps for constructing the preset neural network model include: Acquire training data, which includes environmental data and point cloud data for multiple environments; Build a basic neural network model; The loss function of the basic neural network model is set to the cross-entropy loss function; The activation function of the basic neural network model is set to the ReLU function; The evaluation metrics for the basic neural network model are set, including: precision, recall, and F1 score. Based on the loss function, the activation function, and the evaluation metric, the basic neural network model is trained, validated, and tested using the training data to obtain the preset neural network model.
7. The method according to claims 1 to 6, characterized in that, If the target environment is an outdoor scene, the method further includes: When the audio device is playing music, the echo cancellation function and dynamic equalization function are activated.
8. An environmental sensing device, characterized in that the device is applied to an audio device, the device comprising: The acquisition unit is used to acquire physical environment data and point cloud data of the target environment; The first extraction unit is used to extract spatial features from the point cloud data; The second extraction unit is used to extract human activity features from the point cloud data; The determining unit is used to determine the scene type of the target environment based on the environmental data, the spatial features, and the personnel activity features.
9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1 to 7.