Method and system for enhancing scenic spot sound effects

Abstract

The present invention provides a method and system for enhancing the sound effects of scenic spots, involving artificial intelligence technology, including: identifying a sound type dataset A based on sounds collected at the scenic spot; identifying the types of animals and characteristic landscapes that appear at the scenic spot in different time periods based on the scenic spot's location information; identifying the scenic spot type based on images collected at the scenic spot; obtaining all sound types corresponding to the scenic spot type based on the scenic spot type, and filtering out a sound type dataset C from all sound types using the animal type and characteristic landscape as filtering criteria; comparing sound type dataset A with sound type dataset C, and selecting a dataset based on the comparison results; optimizing the selected dataset based on data on major consumer groups in the previous year, and finally generating an audio file for playback. The present invention can intelligently identify landscape or scene features and scene sounds, achieve scene soundscape matching, and dynamically adjust them to achieve the purpose of sound effect enhancement.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for enhancing sound effects at scenic spots. Background Art

[0002] Sound plays a vital role in the tourism experience. It can not only create a unique atmosphere and enhance tourists' emotional resonance with the scenery or scenes, but also guide tourists' attention and enhance the immersion and memory points of the tour. The clever use of sound elements can greatly enrich and enhance the overall quality of the tourism experience. In the process of realizing the present invention, the applicant found that: in the current scenery or scene creation technology, there are significant deficiencies in the coordination of "people-scenery-sound". On the one hand, there are situations where the soundscape is missing and inharmonious; on the other hand, the existing sound effect technology is single, and the existing sound effect technology is often limited to simple background music playback, lacks deep integration with the scenery or scene characteristics, and cannot be dynamically adjusted according to the emotional preferences of tourists; on the other hand, the existing technology has a low degree of intelligence, and traditional soundscape creation methods mostly rely on manual operations, lack intelligent recognition and adaptability, and are difficult to respond to tourist needs and scenery changes in real time. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art or related art, and discloses a method and system for enhancing the sound effects of scenic spots. Through intelligent sound effect adjustment measures, the automatic identification, adjustment and optimization of soundscapes are realized, thereby reducing manual operation costs, improving management efficiency, and providing tourists with a richer, more three-dimensional and immersive sightseeing experience.

[0004] A first aspect of the present invention discloses a method for enhancing sound effects at a scenic spot, comprising:

[0005] Sound type recognition: Use Mel-frequency cepstral coefficients to extract audio features from the scene sounds of the scenic spot and generate the scenic spot sound features. The scenic spot sound features are input into the trained classifier model to identify the various sound types contained in the scene sounds.

[0006] Volume recognition: extract the volume corresponding to various sound types; delete the sound types with volume greater than the noise standard to obtain the sound type dataset A1 and the corresponding volume dataset A2. The datasets A1 and A2 together constitute the sound dataset A; Geographic and time recognition: input the geographical location information and date and time information of the collected scenic spots into the trained first decision tree model to obtain the animal types and characteristic landscapes of the scenic spots at the collection time and collection area; Scenic spot type recognition: use the trained CNN model to recognize the scenic spot environment image to obtain the scenic spot type; Soundscape analysis: input the scenic spot type into the trained second decision tree model to obtain all the sound types corresponding to the scenic spot type as sound Type dataset B1; use animal type and characteristic landscape as screening conditions to filter the sound type dataset B1 to obtain sound dataset C, which contains sound type dataset C1; compare sound type dataset A1 and sound type dataset C1 based on the difflib module, and determine A1=C1 or A1≠C1; soundscape adjustment: output sound dataset A when A1=C1, and output sound dataset C when A1≠C1, adjust and optimize sound dataset A or sound dataset C according to the preference data of scenic spot consumers, and finally use pydub library, pyaudio library or soundfile library to generate the audio file corresponding to the dataset for playback.

[0007] The method for enhancing the sound effects of scenic spots disclosed in the present invention preferably further includes: soundscape acquisition: using a sound receiving device to acquire the sound of the scenic spot, and converting it into audio data and storing it in a memory; using a positioning device to acquire the geographical location information and date and time information of the scenic spot and storing it in a memory; using an image acquisition device to acquire an image of the scenic spot, and storing the acquired scenic spot environment image in a memory.

[0008] According to the scenic spot sound effect enhancement method disclosed in the present invention, preferably, the trained classifier model is obtained by the following steps: preparing a training set and a test set based on audio feature data sets of multiple known sound types; using the training set to train an SVM classifier and using the test set to perform performance evaluation and optimization, wherein the sound types include at least: environmental sounds and animal sounds, environmental sounds include wind sounds, rain sounds, thunder sounds, and flowing water sounds, and animal sounds include animal calls and human voices.

[0009] In this technical solution, the sound type recognition process based on the SVM classifier can be replaced by a decision tree classification method or a convolutional neural network recognition method.

[0010] According to the scenic spot sound enhancement method disclosed in the present invention, preferably, the trained first decision tree model is obtained by the following steps: preparing a training set and a test set based on the animal types and characteristic landscape data sets in a known area during the day and at night; using the training set to train the decision tree model to form classification rules, and using the test set to evaluate and optimize the performance of the decision tree model.

[0011] In this technical solution, the geographic and temporal recognition process based on the decision tree model can be replaced by an SVM classification method or a convolutional neural network recognition method.

[0012] According to the scenic spot sound effect enhancement method disclosed in the present invention, preferably, the trained second decision tree model is obtained by the following steps: preparing a training set and a test set based on known scenic spot types and corresponding sound type data sets; using the training set to train the decision tree model to form classification rules, and using the test set to evaluate and optimize the performance of the decision tree model.

[0013] In this technical solution, the sound type acquisition process based on the decision tree model can be replaced by an SVM classification method or a convolutional neural network recognition method.

[0014] According to the scenic spot sound enhancement method disclosed in the present invention, preferably, the trained CNN model is obtained by the following steps: preparing a training set and a test set according to the scenic spot type and the corresponding sound type data set of the tourist attraction; constructing a CNN model, training the model using the training set, and evaluating and optimizing the performance of the decision tree model using the test set.

[0015] In this technical solution, the scenic spot type recognition process based on the CNN model can be replaced by the SVM classification method or the decision tree classification method.

[0016] According to the scenic spot sound effect enhancement method disclosed in the present invention, preferably, the preference data of scenic spot consumers are obtained through the following steps: preparing a training set and a test set based on known scenic spot consumer group characteristics and corresponding sound attribute preferences; using the training set to train a decision tree model to form classification rules, and using the test set to evaluate and optimize the performance of the decision tree model; inputting the characteristics of the main consumer group or target consumer group of the scenic spot in the previous year into the trained decision tree model to obtain the sound attributes under the preferences of the consumer group, wherein the consumer group characteristics include: age, gender and income; and the sound attribute preferences include: volume, pitch, timbre, rhythm, continuity, purity, direction and distance, diversity, content and emotion.

[0017] In this technical solution, the sound attribute preference acquisition process based on the decision tree model can be replaced by the SVM classification method or the convolutional neural network recognition method.

[0018] The second aspect of the present invention further discloses a scenic spot sound effect enhancement system, comprising: a memory for storing program instructions; a processor for calling the program instructions stored in the memory to implement the scenic spot sound effect enhancement method provided by any of the above technical solutions.

[0019] The beneficial effects of the present invention include at least: through multi-dimensional soundscape data collection, automatic recognition, matching analysis, adjustment and optimization of existing soundscapes are completed, and more targeted sound effects are generated according to the preferences of consumer groups, thereby providing tourists with a richer, three-dimensional and immersive sightseeing experience; through intelligent recognition of the characteristics of the scenery or scenes of the scenic spots and the scene sound data, the degree of soundscape matching can be accurately analyzed and the corresponding sound elements can be supplemented to enhance the consistency of the soundscape; the sound effects can be dynamically adjusted according to the preferences of tourists to meet the personalized needs of tourists; through the introduction of advanced intelligent technology, the automatic recognition, adjustment and optimization of soundscapes are realized, the manual operation costs are reduced, the management efficiency is improved, and a new solution is provided for the creation of sound effects for cultural and tourism scenery or scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The figure shows a principle diagram of implementing a method for enhancing the sound effects of a scenic spot according to an embodiment of the present invention.

[0021] Figure 2 FIG. 4 is a schematic block diagram of a scenic spot sound effect enhancement system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0024] Explanation of terms:

[0025] 1. Mel-Frequency Cepstral Coefficients (MFCC): A feature widely used in speech and audio signal processing. It is based on the characteristics of human auditory perception, specifically the Mel scale, which is proportional to the logarithm of the frequency and matches the nonlinear frequency perception of the human auditory system.

[0026] 2. SVM classification method: The basic principle is to find an optimal hyperplane to separate sample points of different categories as much as possible in the feature space. This hyperplane is called the decision boundary, and it maximizes the distance between different categories, also known as the inter-class margin. The goal of SVM is to find a hyperplane that maximizes the distance between the sample points closest to the hyperplane.

[0027] 3. Decision tree classification algorithm: This algorithm performs classification by constructing a tree-like structure consisting of nodes and directed edges. Each internal node represents a test condition for a feature or attribute, each branch represents a test result, and each leaf node represents a category.

[0028] 4. Convolutional Neural Networks (CNN): Deep neural networks are primarily used to process data with a grid structure, such as images. By simulating how neurons in the human visual cortex process visual information, CNNs can automatically extract high-level feature representations from raw images, enabling tasks such as image recognition, classification, and detection.

[0029] 5. Boolean Indexing: A highly efficient method in the Pandas library for conditionally selecting data. Based on NumPy’s Boolean indexing functionality, it allows users to directly filter rows (or columns) in a DataFrame or Series using conditional expressions.

[0030] 6. difflib module: A module in the Python standard library that provides functions for comparing differences in sequences (usually strings or lists).

[0031] like Figure 1 As shown, according to one embodiment of the present invention, a method for enhancing the sound effects of a scenic spot is disclosed, comprising:

[0032] Information Collection (S101):

[0033] (1) Scenic spot sound information collection: Use professional sound receiving instruments such as microphones and other professional sound recording instruments to collect the sound of the scenic spot scene environment, and convert it into audio data and store it in the memory.

[0034] (2) Geographical time information collection: Use GPS positioning tools to collect the area where the scenic spot scene is located and the date and time information used. Geographical positioning is used to obtain regional information, and date and time information is used to determine the time period of the area at the time of collection (before and after sunrise, daytime, before and after dusk, and night). The final output geographic positioning information and date and time information are stored in the memory.

[0035] (3) Scene environment information collection: Use professional instruments such as cameras and video recorders to collect images of the scenic scene environment where the sound effects need to be optimized, and store the collected scene environment images in the memory.

[0036] Information identification (S102):

[0037] (1) Based on the collected scenic spot sound information, the geographic time information is used as a filtering condition to remove the noise and obtain the scenic spot sound dataset A. The removed noise may contain valid sound types, such as loud human voices, which can be retrieved from dataset C in the subsequent steps and the volume can be reset to achieve optimization. The specific methods are as follows: First, use the Mel-frequency cepstral coefficients to extract the features (timbre, pitch, etc.) of the scene sound audio of S101 (1); Second, collect known audio features (feature set) - sound types such as wind, rain, birds, insects, and people (target set) and divide the data set into a training set and a test set; Third, use the SVM classification method to select a suitable SVM kernel function and use the training set to train the SVM classifier and use the test set to evaluate and optimize the performance; Fourth, input the features extracted from the Mel-frequency cepstral coefficients into the classifier and output the various sound types existing in the scene; Fifth, calculate the root mean square (RMS) value of the audio signal to extract the volume information of each sound type; Sixth, based on the national standard "GB3096-2008 Sound Environment Quality Standard", use the logistic regression algorithm to screen the volume information of each sound type under the geographical area and time conditions, and remove the sound types that exceed the noise standard; Sixth, after removing the noise, obtain the scenic spot sound data set A, including the sound type data set A1 and the sound volume data set A2.

[0038] (2) Based on the collected geographic and temporal information, obtain the main and typical animal types and characteristic landscape information in the area during the day and night. The main and typical animal types are selected in the following ways: first, common creatures that can make sounds in the area; second, creatures that are relatively typical and characteristic of the area increase uniqueness and appeal; third, human content including local dialects under this concept. Plants generally do not make sounds, and the sounds of plants that can make sounds are generally inaudible to the human ear, so plants are not considered. Characteristic landscapes, such as wheat fields in the countryside, can be used as a screening condition to add the sound of wind blowing through the wheat waves. The role of date and time information: According to the habits of animals, there will be different types of active animals in the early morning, daytime, dusk, and night. The day and night on different dates in different places are inconsistent. This information is collected to determine the time period of the scene in which the information was collected. The specific methods are as follows: 1. Collect the main and typical animal types and characteristic landscapes (target set) in the known area and time (feature set) during the day / night in the area and divide the data set into training set and test set; 2. Use the training set to train the decision tree model to form classification rules, and use the test set to evaluate and optimize the performance of the decision tree model; 3. Input the geographic location and date and time data of S101 (2) into the trained decision tree model to obtain the scenic spot scene collection time and the main and typical animal types and characteristic landscapes in the collection area.

[0039] (3) Classify different types of scenic spots based on the collected scene environment information. The specific methods are as follows: 1. Based on the "Classification of Tourist Attractions" group standard (T / CTAA 0001-2019), the scenic spot scene types are divided into 26 basic types; 2. Collect the dataset of 26 types of scenic spot scenes (target set) with known typical identification objects (feature set) and divide it into training set and test set; 3. Construct a CNN model, train the model using the training set, and use the test set to evaluate and optimize the performance of the decision tree model; 4. Input the scenic spot scene image obtained in step S101 (3) into the trained CNN model to obtain the scenic spot type corresponding to the scenic spot scene.

[0040] Soundscape Analysis (S103):

[0041] (1) Obtain a sound dataset of this type based on the scenic spot scene type. The specific method is as follows: 1. Collect a dataset of known scenic spot scene types (feature sets) and scene sound types (target sets) and divide it into a training set and a test set; 2. Use the training set to train a decision tree model to form classification rules, and use the test set to evaluate and optimize the performance of the decision tree model; 3. Input the scenic spot type identified in step S102 (3) into the trained decision tree model to obtain all the sound type data in this type of scenic spot scene, and obtain a scenic spot sound dataset B, including a sound type dataset B1.

[0042] (2) Using the main and typical animal types and characteristic landscape information of the region as screening conditions, a filtered sound dataset was obtained. The specific method is as follows: Introduce the Pandas library and use the Boolean indexing tool in the Pandas library to filter dataset B using "animal type" and "characteristic landscape" as conditions. After filtering, the main and typical sound dataset C of the scenic spot under the collection area and time conditions is obtained, including the sound type dataset C1. In addition to the Boolean indexing tool in the Pandas library, the query method can also be used to filter dataset B.

[0043] (3) Comparison of dataset A and dataset C. The specific method is as follows: introduce the difflib module to compare dataset A1 and dataset C1, and the output content is dataset A1 = dataset C1 or dataset A1 ≠ dataset C1.

[0044] Soundscape Adjustment (S104):

[0045] When Dataset A1 = Dataset C1, the output is Sound Dataset A; when Dataset A1 ≠ Dataset C1, the output is Sound Dataset C. This step is equivalent to removing interference from the captured scenic spots. For example, if construction site sounds are present in a captured rural scene, Dataset A1 ≠ Dataset C1, resulting in Dataset C being free of the interference from the construction site sounds. Only the sound types need to be compared, and subsequent settings will need to be made based on consumer group preferences. Outputting Dataset C is equivalent to resetting the sound volume, while outputting Dataset A is equivalent to optimizing the sound volume.

[0046] (1) Introduce consumer preference variables to adjust and optimize sound features. The specific methods are as follows: 1. Collect known consumer group characteristics such as age, gender, income, type, etc. (feature set) - group sound characteristic preferences such as volume, pitch, timbre, rhythm, continuity, purity, direction and distance, diversity, content and emotion (recognizable and convertible human voice information) (target set) and divide the data set into training set and test set; 2. Use the training set to train the decision tree model to form classification rules, and use the test set to evaluate and optimize the performance of the decision tree model; 3. Input the characteristics of the main consumer group or target consumer group of the scenic spot in the previous year into the trained decision tree model to obtain the sound characteristics under the preferences of the consumer group.

[0047] (2) Sound adjustment, optimization, and output. The specific methods are as follows: 1. Adjust and optimize the volume dataset A2 based on the group preference sound features obtained in S104 (1) and set the tone dataset A3, timbre dataset A4, etc. corresponding to the sound type dataset A1, or set the volume dataset C2, tone dataset C3, timbre dataset C4, etc. corresponding to the sound type dataset C1, and finally obtain a dataset that matches the soundscape; 2. Introduce libraries such as pydub, pyaudio, or soundfile to read data, and use the corresponding functions in the library to generate audio files; 3. Play the audio through a device such as a speaker.

[0048] like Figure 2 As shown, according to another embodiment of the present invention, a scenic spot sound effect enhancement system 200 is also disclosed, including: a memory 201 for storing program instructions; a processor 202 for calling the program instructions stored in the memory to implement the scenic spot sound effect enhancement method as described in the above embodiment.

[0049] In summary, the present invention utilizes specialized equipment such as microphones, GPS positioning, and cameras to collect scenic spot sound information, geographic time information, and scene environment information. Next, a support vector machine (SVM) classification method is used to remove noise from the scenic spot sound information, using geographic time as a filtering criterion. A decision tree classification algorithm is then used to obtain information on the main and typical animal species and distinctive landscape features within the collection time and range. A convolutional neural network is then used to classify the scenic spot scenes into different types based on the collected scene environment information. Next, a decision tree classification algorithm is used to obtain a sound dataset B (sound type dataset B1) for each scenic spot scene type. A Boolean indexing tool is used to filter the main and typical animal species and distinctive landscape features of the region to obtain a filtered sound dataset C (sound type dataset C1). Datasets A1 and C1 are compared to determine soundscape compatibility. Finally, a decision tree classification algorithm is used to incorporate data on the scenic spot's main consumer groups or target groups from the previous year to adjust and configure sound features, thereby obtaining optimized scene sounds that are output through audio equipment, such as speakers. This method can be used to identify and analyze the soundscape matching of existing scenic spots using artificial intelligence algorithms and generate scene sound effects with higher matching and perfection, solving the problem of "people-scenery-sound" disharmony in the existing technology, providing targeted improvement solutions for the soundscape coordination matching of existing scenic spots, and providing a feasible option for improving scene sound effects.

[0050] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for enhancing the sound effect of a scenic spot, characterized in that: include: Sound type recognition: Use Mel-frequency cepstral coefficients to extract audio features of the scene sounds of the scenic spot and generate the sound features of the scenic spot; Inputting the scenic spot sound features into a trained classifier model to identify multiple sound types contained in the scene sound; Volume recognition: Extract the volume corresponding to various sound types; delete the sound types with volume greater than the noise standard to obtain the sound type dataset A1 and the corresponding volume dataset A2. Dataset A1 and dataset A2 together constitute the sound dataset A; Geographic and temporal identification: The geographical location information and date and time information of the collected scenic spots are input into the trained first decision tree model to obtain the animal types and characteristic landscapes of the scenic spots at the time of collection and in the collection area; Scenic spot type recognition: Use the trained CNN model to identify scenic spot environment images to obtain the type of scenic spot; Soundscape analysis: Input the scenic spot type into the trained second decision tree model to obtain all sound types corresponding to the scenic spot type as a sound type dataset B1; filter the sound type dataset B1 using the animal type and the characteristic landscape as screening conditions to obtain a sound dataset C, which includes a sound type dataset C1; compare the sound type dataset A1 with the sound type dataset C1 based on the difflib module to determine whether A1=C1 or A1≠C1; Soundscape adjustment: When A1=C1, output sound dataset A; when A1≠C1, output sound dataset C. Adjust and optimize the selected dataset based on the preference data of scenic spot consumers. Finally, use the pydub library, pyaudio library, or soundfile library to generate the audio file corresponding to the dataset for playback.

2. The scenic spot sound effect enhancement method according to claim 1, characterized in that: Also includes: Soundscape collection: Use radio equipment to collect sounds from scenic spots and convert them into audio data and store them in memory; Using a positioning device to collect the geographical location information and date and time information of the scenic spot and store them in a memory; The image acquisition device is used to acquire images of the scenic spot, and the acquired scenic spot environment images are stored in a memory.

3. The scenic spot sound effect enhancement method according to claim 1, characterized in that: The trained classifier model is obtained by the following steps: A training set and a test set are created based on audio feature datasets of multiple known sound types; the training set is used to train an SVM classifier, and the test set is used to evaluate and optimize performance, wherein the sound types include at least environmental sounds and animal sounds, the environmental sounds include wind, rain, thunder, and running water, and the animal sounds include animal calls and human voices.

4. The method for enhancing scenic spot sound effects according to claim 1, wherein: The trained first decision tree model is obtained by the following steps: A training set and a test set are created based on the animal types and characteristic landscape datasets in a known area during the day and at night. The training set is used to train a decision tree model to form classification rules, and the test set is used to evaluate and optimize the performance of the decision tree model.

5. The scenic spot sound effect enhancement method according to claim 1, characterized in that: The trained second decision tree model is obtained by the following steps: A training set and a test set are created based on known scenic spot types and corresponding sound type data sets; the training set is used to train the decision tree model to form classification rules, and the test set is used to evaluate and optimize the performance of the decision tree model.

6. The scenic spot sound effect enhancement method according to claim 1, characterized in that: The trained CNN model is obtained by the following steps: Create training and test sets based on the types of scenic spots in tourist attractions and the corresponding sound type data sets; build a CNN model, use the training set to train the model, and use the test set to evaluate and optimize the performance of the decision tree model.

7. The method for enhancing scenic spot sound effects according to claim 1, characterized in that: The preference data of scenic spot consumers are obtained through the following steps: A training set and a test set were created based on the known characteristics of the scenic spot's consumer groups and their corresponding sound attribute preferences. The training set was used to train a decision tree model to form classification rules, and the test set was used to evaluate and optimize the performance of the decision tree model. The characteristics of the scenic spot's main consumer group or target consumer group in the previous year were input into the trained decision tree model to obtain the sound attributes preferred by the consumer group. The consumer group characteristics included age, gender, and income, and the sound attribute preferences included volume, pitch, timbre, rhythm, continuity, purity, direction and distance, diversity, content, and emotion.

8. A scenic spot sound enhancement system, characterized in that: include: a memory for storing program instructions; A processor is configured to call the program instructions stored in the memory to implement the scenic spot sound effect enhancement method according to any one of claims 1 to 7.

Images