Method, device, electronic device and storage medium for predicting acoustic environment annoyance

Abstract

The embodiments of the present application provide a method, device, electronic device and storage medium for predicting the annoyance of an acoustic environment. The method includes: obtaining a panoramic environment image and geographic location information corresponding to a sampling area; inputting the panoramic environment image and geographic location information into a pre-trained annoyance prediction model, extracting features of the panoramic environment image through the annoyance prediction model to obtain image features of the panoramic environment image, and obtaining the predicted annoyance value of the acoustic environment of the sampling area for residents based on the image features and the above-mentioned geographic location. It can be seen that in the present technical solution, the prediction of the annoyance of an acoustic environment can be achieved based on the panoramic environment image, geographic location information and the annoyance prediction model, without consuming a lot of manpower costs, and the prediction method is simple and convenient, which greatly improves the prediction efficiency of the annoyance of an acoustic environment, and has high accuracy.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of acoustic environment technology, and in particular to a method, device, electronic device and storage medium for predicting annoyance level of an acoustic environment. Background Art

[0002] With the acceleration of urbanization, acoustic environment problems are becoming increasingly serious, such as traffic noise, industrial noise, construction noise, morning market and other life noise, which seriously affect the quality of life of residents. Therefore, in order to improve the quality of life of residents, acoustic environment issues play an important role in urban planning, traffic management and other fields. Therefore, when conducting urban planning, traffic management or noise control, it is necessary to collect the impact of the acoustic environment on residents in relevant areas, such as a street, a town or a community. At present, most of them use the annoyance index to characterize the impact of the acoustic environment on residents.

[0003] In the prior art, the annoyance level of the acoustic environment is mostly collected through visits or questionnaires, which has the problems of low efficiency and high labor costs. In addition, the annoyance level can also be predicted through some algorithms or models, such as predicting the annoyance level based on the noise information or traffic flow information in a certain environment, but this method still needs to collect or measure the noise information or traffic flow information in the environment through visits or questionnaires, which also has the problems of low efficiency and high labor costs.

[0004] Therefore, it is necessary to propose a method for predicting the annoyance degree of an acoustic environment to solve the problems of low efficiency and high labor cost in the prior art in predicting the annoyance degree of an acoustic environment. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the embodiments of the present application provide a method, device, electronic device and storage medium for detecting the annoyance level of an acoustic environment.

[0006] According to a first aspect of an embodiment of the present application, a method for detecting annoyance level of an acoustic environment is provided, the method comprising:

[0007] Obtaining panoramic environment images and geographic location information corresponding to the sampling area;

[0008] The panoramic environment image and the geographic location information are input into a pre-trained annoyance prediction model, and features of the panoramic environment image are extracted through the annoyance prediction model to obtain image features of the panoramic environment image, and based on the image features and the geographic location information, a predicted annoyance value of the sound environment of the sampling area on residents is obtained.

[0009] According to a second aspect of an embodiment of the present application, a device for detecting annoyance level of an acoustic environment is provided, the device comprising:

[0010] A first acquisition module is used to acquire a panoramic environment image and geographic location information corresponding to a sampling area;

[0011] An input module, used for inputting the panoramic environment image and the geographical location information into a pre-trained annoyance prediction model;

[0012] The prediction module is used to extract features of the panoramic environment image through the annoyance prediction model to obtain image features of the panoramic environment image, and obtain the annoyance prediction value of the acoustic environment of the sampling area to residents based on the image features and the geographical location information.

[0013] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.

[0014] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and the program is executed by a processor to implement the method as described in the first aspect or as described above.

[0015] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising instructions, which, when executed on a computer, cause the computer to execute the method according to the first aspect.

[0016] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0017] In the embodiment of the present application, when it is necessary to know the annoyance degree caused by the sound environment of a certain area to the residents, the panoramic environment image and geographical location information corresponding to the sampling area are obtained, and the panoramic environment image and geographical location information are input into a pre-trained annoyance prediction model. The panoramic environment image is feature extracted by the annoyance prediction model, and the predicted value of the annoyance degree caused by the sound environment of the sampling area to the residents is obtained based on the obtained image features and the above-mentioned geographical location information; through the embodiment of the present application, the prediction of the annoyance degree of the sound environment can be achieved in an end-to-end manner only based on the panoramic environment image and geographical location information, and there is no need to obtain the annoyance degree of the area through visits or questionnaires, nor is there a need to spend a lot of manpower costs to collect the noise source information of the sampling area. The annoyance prediction method is simple and convenient, saving a lot of labor costs, and by adopting the model prediction method, it is more objective, thereby improving the accuracy of the annoyance prediction of the area.

[0018] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments of the present application. It should be understood that for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0020] Figure 1 One of the flow charts of the method for predicting the acoustic environment annoyance degree provided in the embodiment of the present application;

[0021] Figure 2 A schematic diagram of the model structure of the annoyance prediction model in the acoustic environment annoyance prediction method provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of the structure of a feature extraction network in the acoustic environment annoyance prediction method provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of the structure of a feature mapping network in the acoustic environment annoyance prediction method provided in an embodiment of the present application;

[0024] Figure 5 The second flowchart of the method for predicting the annoyance level of an acoustic environment provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of the module composition of the acoustic environment annoyance prediction device provided in an embodiment of the present application;

[0026] Figure 7 An exemplary structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solution of the embodiment of the present application is described below in conjunction with the drawings in the embodiment of the present application.

[0028] The terms used in the following embodiments of the present application are for the purpose of describing specific embodiments, and are not intended to be used as limitations on the technical solutions of the present application. As used in the specification and appended claims of the present application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to include plural expressions as well, unless there is a clear indication to the contrary in the context.

[0029] It should also be understood that although the terms first, second, etc. may be used to describe a certain type of object in the following embodiments, the object should not be limited to these terms. These terms are used to distinguish the specific implementation objects of this type of object.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] The annoyance of the acoustic environment refers to the subjective annoyance degree of the human body caused by the acoustic environment, which is used to evaluate the impact of the acoustic environment on residents. At present, the annoyance degree of the acoustic environment in the relevant area is usually referred to when conducting urban planning, traffic management or noise control. However, in the current existing technology, when obtaining the annoyance degree of the acoustic environment, one way is to use a questionnaire survey. The annoyance degree obtained in this way is greatly affected by the subjective feelings of the user, and visiting a large number of residents to fill out questionnaires requires a lot of manpower, which has high manpower costs and low efficiency; in addition, the annoyance degree in the corresponding environment can also be calculated based on the noise source information or traffic flow information in the environment and other influencing factors that can generate noise, but this method also requires the collection of influencing factors that can generate noise in the relevant area. At present, it is still achieved by visiting a large number of residents to fill out questionnaires or surveying, which still requires a lot of manpower costs and is inefficient.

[0032] Based on this, the present application embodiment provides a method for predicting the annoyance of an acoustic environment, such as Figure 1 As shown, the method includes the following steps 101 and 102:

[0033] Step 101, obtaining a panoramic environment image and geographical location information corresponding to a sampling area.

[0034] Step 102, input the panoramic environment image and geographic location information into a pre-trained annoyance prediction model, extract features of the panoramic environment image through the annoyance prediction model to obtain image features of the panoramic environment image, and obtain an annoyance prediction value of the acoustic environment of the sampling area for residents based on the image features and the geographic location information.

[0035] Among them, the above-mentioned sampling area can be a street, a community or a village, etc. The specific size of the above-mentioned sampling area can be set according to actual needs. Here, only several possible range areas are listed as examples, which is not a limitation of the present application scheme.

[0036] In specific implementation, the execution subject of the present application solution may be a server or a terminal device such as a computer, tablet, or mobile phone. For example, the trained annoyance prediction model may be deployed or stored on a server or terminal device, and the annoyance prediction model may be called when the annoyance of the acoustic environment needs to be predicted.

[0037] Optionally, in a specific implementation, the above-mentioned image features may be visual features in the panoramic environment image, including but not limited to color features, shape features, texture features, edge features, etc. in the panoramic environment image.

[0038] In addition, it should be noted that the annoyance degree in the embodiment of the present application represents the degree of annoyance caused by the sound environment to the residents. The annoyance degree can be represented in the form of a single numerical value, such as 1, 2, 3, 4, 5, etc. In a specific embodiment, the annoyance degree of the sound environment can be divided into eleven different levels of 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10. The larger the numerical value, the higher the annoyance degree of the sound environment to the residents. Correspondingly, in this case, the above-mentioned annoyance degree prediction model outputs the predicted value of the annoyance degree of the sound environment as the above-mentioned single numerical value. For example, the annoyance degree prediction model outputs a numerical value of 1, and the degree of annoyance caused by the sound environment of the sampling area to the situation is 1. Of course, in the specific implementation, two-digit numerical values ​​or other characters such as letters can also be used to represent the predicted value of the annoyance degree of the sound environment. In some specific embodiments, the annoyance degree prediction model can also directly output words such as high annoyance, low annoyance, or medium annoyance as the predicted value of the annoyance degree of the sound environment in the sampling area. The specific output of the annoyance degree prediction model can be set according to actual needs, and the embodiments of the present application are no longer listed one by one.

[0039] The method for predicting the annoyance level of an acoustic environment provided in the embodiment of the present application can obtain the predicted value of the annoyance level caused by the acoustic environment of the sampling area to residents by simply inputting the panoramic environment image and geographic location information of the sampling area into a pre-trained annoyance level prediction model. There is no need to obtain the annoyance level of the area through visits or questionnaires, nor is there a need to spend a lot of manpower costs to collect noise source information in the sampling area. The annoyance level prediction method is simple and convenient, saving a lot of labor costs, and by adopting a model prediction method, it is more objective, thereby improving the accuracy of the annoyance level prediction of the area.

[0040] In specific implementation, in the above step 101, obtaining the panoramic environment image and geographical location information corresponding to the sampling area can be achieved in a variety of ways. For example, for the above panoramic environment image, the panoramic environment image of the sampling area can be obtained from an image acquisition device, which can be a camera, a mobile phone, a drone or other device with image acquisition function; or, the panoramic environment image corresponding to the sampling area can be directly obtained from a map server.

[0041] Of course, in some specific implementations, it is also possible to obtain environmental images of the sampling area at multiple different angles from an image acquisition device, and stitch the environmental images at multiple different angles to obtain a panoramic environmental image corresponding to the sampling area; or it is also possible to obtain multiple local environmental images of the sampling area from an image acquisition device, and stitch the local environmental images to obtain a panoramic environmental image of the sampling area.

[0042] Optionally, in a specific implementation, the collected panoramic environment images can also be stored on a terminal device. When it is necessary to predict the acoustic environment annoyance of a certain sampling area, the panoramic environment images corresponding to the sampling area can be searched from the panoramic environment images stored on the terminal device and the acoustic environment annoyance can be predicted. For example, the panoramic environment images of each sampling area can be stored in a folder of a specified path, and a mapping relationship table of the identification information of each sampling area and the storage path of the corresponding panoramic environment image can be established. In this way, the storage path information of the panoramic environment image corresponding to a certain sampling area can be determined based on the mapping relationship table, and the corresponding panoramic environment image can be found.

[0043] For the above-mentioned geographic location information, the geographic location information of the sampling area can be obtained from the Global Positioning System (GPS), or the geographic location information corresponding to the sampling area can be directly obtained from the map server, or, in some specific implementations, if an image acquisition device with a positioning function is used to acquire images of the sampling area, the geographic location information of the sampling area can also be obtained from the image acquisition device. The embodiments of the present application only exemplarily list several implementation methods that can obtain geographic location information, and other solutions that can obtain geographic location information can be applied to the embodiments of the present application, which will not be repeated here.

[0044] In this embodiment of the present application, the above-mentioned geographical location information can be represented by longitude and latitude values.

[0045] In a specific implementation, after obtaining the prediction of the annoyance level caused by the sound environment of the sampling area to the residents, the annoyance level prediction value corresponding to the sampling area can be displayed to the user through the terminal device; or, the annoyance level prediction value corresponding to the sampling area can be stored; or, the sound environment annoyance level prediction value corresponding to the sampling area can be sent to the terminal device specified by the user; of course, in a specific implementation, other methods can be used for display according to actual needs, and the embodiments of the present application will not be listed one by one.

[0046] In a specific implementation, after extracting features from the panoramic environment image through the annoyance prediction model to obtain image features of the panoramic environment image, and obtaining the annoyance prediction value of the acoustic environment of the sampling area for residents based on the image features and the above-mentioned geographic location information, the method provided in the embodiment of the present application further includes the following steps:

[0047] The geographical location information of the sampling area and the predicted value of the annoyance level are stored correspondingly.

[0048] In a specific implementation, the annoyance prediction value of the sampling area may be stored in the following format:

[0049] {Geographic location information, annoyance prediction value}.

[0050] The geographical location information of the sampling area can be represented by longitude and latitude.

[0051] In a specific implementation, the geographical location information, identification information and the annoyance prediction value of the sampling area may also be stored correspondingly.

[0052] Among them, the identification information of the sampling area can be the number information of the sampling area or the area name information, such as XX Street, XX Community, etc. Of course, the above identification information can also be other. This is just an illustrative explanation here by way of example, and it is not a limitation of the embodiments of the present application.

[0053] In specific implementation, Figure 1 The method shown can obtain the predicted value of the annoyance of the sound environment in a single sampling area to residents; in some specific scenarios, such as urban planning, traffic planning, etc., in these scenarios, it may be necessary to know the distribution of the annoyance of the sound environment in a larger area. In this case, it is necessary to predict the annoyance of the sound environment in multiple sampling areas within the larger area, and generate a distribution map of the annoyance of the sound environment corresponding to the larger area based on the predicted values ​​of the annoyance of the sound environment corresponding to the multiple sampling areas. The larger area can be a city, a community, or a township, etc.

[0054] Therefore, in some specific implementations, the method provided in the embodiments of the present application further includes the following steps:

[0055] Obtain annoyance prediction values ​​and geographic location information of each sampling area in the multiple sampling areas included in the area to be predicted; obtain map information corresponding to the area to be predicted; and generate a sound environment annoyance distribution map of the area to be predicted based on the geographic location information, annoyance prediction values ​​of each sampling area and the map information of the area to be predicted.

[0056] The above-mentioned area to be predicted is an area larger than the sampling area. For example, the above-mentioned area to be predicted may be a community, an urban area, a township or a city, etc. Of course, the specific area range of the area to be predicted can be set according to the actual application scenario, and the embodiment of the present application is not limited to this.

[0057] For each of the multiple sampling areas contained in the area to be predicted, Figure 1 The method shown in FIG. 1 is used to obtain the predicted value of the acoustic environment annoyance corresponding to each sampling area. Figure 1 After obtaining the acoustic environment annoyance prediction corresponding to each sampling area, the shown acoustic environment annoyance prediction method can store the geographical location information and the acoustic environment annoyance prediction value of each sampling area accordingly. When it is necessary to generate an acoustic environment annoyance distribution map of the area to be predicted, it is necessary to obtain each geographical location information and its corresponding acoustic environment annoyance prediction value.

[0058] For example, in a specific implementation, a possible representation form of the acquired acoustic environment annoyance prediction value and geographical location information of each sampling area is as follows:

[0059] Geographic location 1, acoustic environment annoyance prediction value 1;

[0060] Geographic location 2, acoustic environment annoyance prediction value 2;

[0061] Geographic location 3, acoustic environment annoyance prediction value 3; ...

[0062] Geographic location N, predicted value of acoustic environment annoyance N.

[0063] In specific implementation, map information of the area to be predicted can be obtained from a map server. Of course, it can also be obtained through other methods, which will not be listed one by one in the embodiments of the present application.

[0064] In an embodiment of the present application, a sound environment annoyance distribution map of the area to be predicted is generated, that is, the sound environment annoyance distribution of the area to be predicted is displayed in the form of a map, so that the user can intuitively feel the overall distribution of the sound environment annoyance of the area to be predicted.

[0065] After obtaining the geographical location information of each sampling area in the area to be predicted and its corresponding acoustic environment annoyance prediction value, and obtaining the map information of the area to be predicted, a distribution map of the acoustic environment annoyance of the area to be predicted is generated according to the geographical location information, the annoyance prediction value and the map information of the area to be predicted of each sampling area, which specifically includes the following process:

[0066] According to the geographical location information and annoyance prediction values ​​of each sampling area, the predicted area is interpolated to obtain the annoyance prediction values ​​of the areas corresponding to multiple interpolation points in the predicted area; according to the geographical location information of each sampling area and the area corresponding to the difference point, the corresponding annoyance prediction value is mapped with the map information to obtain the sound environment annoyance distribution map.

[0067] In an embodiment of the present application, an inverse distance weighted (IDW) algorithm can be used to interpolate the area to be predicted. In a specific implementation, the position of the interpolation point to be interpolated is first determined, and the distance between the interpolation point and the existing geographical location information is calculated based on the position of the interpolation point and the geographical location information corresponding to the existing sampling area, and the corresponding weight is calculated by the distance. Finally, the annoyance prediction value of each interpolation point is calculated. Specifically, the annoyance prediction value of each interpolation point can be calculated by weighted average method.

[0068] Of course, in specific implementation, other interpolation algorithms may also be used to perform interpolation processing on the area to be predicted, and the embodiments of the present application will not be listed one by one for explanation.

[0069] After obtaining the annoyance prediction value corresponding to each interpolation point, the position of each interpolation point and sampling area on the map of the area to be predicted is determined based on the geographical location information corresponding to each interpolation point and sampling area, and the corresponding annoyance prediction value is marked at the corresponding position, thereby achieving the sound environment annoyance distribution map of the area to be predicted.

[0070] In an embodiment of the present application, the predicted values ​​of the sound environment annoyance for more geographical locations can be obtained by interpolating the predicted area. On the one hand, the amount of data on the sound environment annoyance distribution map is increased, so that the sound environment annoyance distribution map has a more detailed display of the sound environment annoyance. In addition, there is no need to additionally collect panoramic environmental images of these areas, which greatly reduces the workload.

[0071] In a specific embodiment, the annoyance prediction model includes a feature extraction network and a feature mapping network arranged in cascade;

[0072] In the above step 102, the feature extraction of the panoramic environment image is performed through the annoyance prediction model to obtain the image features of the panoramic environment image, and the annoyance prediction value of the acoustic environment of the sampling area to the residents is obtained based on the image features and the above geographical location information, which specifically includes the following process:

[0073] The feature extraction network in the annoyance prediction model is used to extract features from the panoramic environment image to obtain the above-mentioned image features; the feature mapping network is used to perform feature mapping processing on the above-mentioned image features and the above-mentioned geographic location information, and the above-mentioned image features and geographic location information are mapped to feature values ​​of a single dimension as the annoyance prediction value.

[0074] In a specific embodiment, Figure 2 As shown, the feature extraction network can be composed of a plurality of convolutional layers cascaded, and the feature mapping network can be composed of one or more fully connected layers cascaded; wherein the specific number of convolutional layers and fully connected layers can be set according to actual needs, Figure 2 This is just an illustrative description and is not a limitation on the specific structure of the model.

[0075] In a specific embodiment, the feature extraction network includes a first convolution layer, two second convolution layers and four residual networks in a cascade arrangement, wherein the convolution kernel of the first convolution layer is larger than the convolution kernel of the second convolution layer;

[0076] The feature extraction network in the annoyance prediction model extracts features from the panoramic environment image to obtain image features, which specifically includes the following process:

[0077] The panoramic environment image is feature extracted by the first convolution layer to obtain low-level features of the panoramic environment image; the low-level features are feature extracted by the second convolution layer to obtain high-level features of the panoramic environment image; the high-level features are feature extracted by the residual network to obtain deep-level features of the panoramic environment image.

[0078] Among them, the above-mentioned deep-level features are image features that need to be input into the feature mapping network.

[0079] In the specific implementation, the panoramic environment image of the sampling area is input into the trained annoyance prediction model. First, the panoramic environment image enters the first convolution layer, and the panoramic environment image is feature extracted through the first convolution layer. The first convolution layer is mainly used to extract initial low-level features from the panoramic image, such as edge features and texture features of the panoramic environment image. In a specific implementation, the first convolution layer uses a convolution layer with a convolution kernel of 5x5 and a stride of 2. The setting of a larger convolution kernel and a stride of 2 can help to quickly reduce the spatial dimension of the image and reduce the amount of calculation; in the embodiment of the present application, the setting of a larger convolution kernel in the first convolution layer can perform initial feature extraction on the panoramic environment image, so that the local information in the panoramic environment image can be more widely captured, thereby helping to improve the initial perception ability of the annoyance prediction model.

[0080] After processing the panoramic environment image through the first convolution layer, the low-level features corresponding to the panoramic environment image can be obtained, and the low-level features are input into the second convolution layer. The second convolution layer further extracts the low-level features, mainly extracting high-level features in the image, such as extracting more detailed information in the image, thereby improving the accuracy of the prediction of the acoustic environment annoyance. In a specific embodiment, the second convolution layer uses a convolution layer with a convolution kernel of 3x3 and a stride of 1. Through such a setting, the spatial resolution of the image can be maintained, and the computational efficiency and feature extraction capabilities can be balanced, thereby helping the model learn more detailed image features.

[0081] After further feature extraction of low-level features through the second convolutional layer, high-level features corresponding to the panoramic environment image can be obtained, and the high-level features are input into the residual network, and the high-level features are further feature extracted through the residual network to obtain deep-level features of the panoramic environment image. In the embodiment of the present application, each residual network is composed of two convolutional layers with a convolution kernel of 3x3. Through such a design, more complex and deeper features can be learned, and further feature extraction of high-level features is achieved; in addition, the residual connection in the residual network can alleviate the gradient vanishing problem, that is, by introducing the residual network in the feature extraction network, the network can have a deeper stacked convolution layer.

[0082] In addition, in order to reduce the amount of calculation, reduce the spatial dimension of the features, and retain important feature information, in an embodiment of the present application, a pooling layer is provided behind each residual network to downsample the features output by the residual network, thereby reducing the amount of calculation and the amount of parameters. In addition, through the processing of the pooling layer, the translation invariance of the features can also be improved, which helps to capture key information at different scales.

[0083] Figure 3A possible structural diagram of a feature extraction network provided in an embodiment of the present application is shown. Figure 3 As shown in the figure, in the feature extraction network, the panoramic environment image passes through a 5x5 convolution layer, two 3x3 convolution layers, a residual network composed of two 3x3 convolution layers, a 2x2 pooling layer, a residual network composed of two 3x3 convolution layers, a 2x2 pooling layer, a residual network composed of two 3x3 convolution layers, a 2x2 pooling layer, a residual network composed of two 3x3 convolution layers, a 2x2 pooling layer, a residual network composed of two 3x3 convolution layers, and a 2x2 pooling layer to finally output the image features of the panoramic environment image.

[0084] Optionally, in a specific implementation, the feature mapping network includes a first fully connected layer, a second fully connected layer, and a third fully connected layer arranged in cascade;

[0085] Specifically, the feature mapping network is used to perform feature mapping processing on the image features and geographic location information, and the image features and geographic location information are mapped to feature values ​​of a single dimension as the annoyance prediction value, which specifically includes the following process:

[0086] The image features and geographic location information are processed by dimensionality reduction through the first fully connected layer to obtain features of the first dimension; the features of the first dimension are processed by dimensionality reduction through the second fully connected layer to obtain features of the second dimension; the features of the second dimension are mapped to feature values ​​of a single dimension through the third fully connected layer as predicted values ​​of the annoyance level.

[0087] In a specific embodiment, the first fully connected layer flattens the above-mentioned geographic location information (which can be two values ​​of longitude and latitude) and the image features output by the feature extraction network (that is, the deep features mentioned above) and converts them into lower-dimensional feature vectors. In this way, a large-scale dimensionality reduction of the image features can be achieved, thereby reducing the computational complexity, while retaining key feature information without affecting the prediction of the annoyance. For example, the size of the feature input to the first fully connected layer can be 1024 pixels, a longitude value and a dimension value. After the dimensionality reduction is performed by the first fully connected layer, the size of the output feature can be 512.

[0088] The first fully connected layer inputs the output features of the first dimension after dimensionality reduction into the second fully connected layer, and the features of the first dimension are further reduced by the second fully connected layer to obtain a more compact and efficient feature vector.

[0089] For example, continuing with the above example, the size of the image feature input to the second fully connected layer is 512. After further dimensionality reduction by the second fully connected layer, the size of the output feature is 256.

[0090] In the embodiment of the present application, by gradually reducing the dimension of the feature vector through the first fully connected layer and the second fully connected layer, the overfitting phenomenon can be avoided, thereby enhancing the generalization ability of the annoyance prediction model, thereby improving the accuracy of the annoyance prediction.

[0091] The image features of the second dimension output by the second fully connected layer are output to the third fully connected layer, and the third fully connected layer directly performs regression prediction, that is, mapping the image features to a single numerical value, which is the predicted value of the acoustic environment annoyance corresponding to the sampling area.

[0092] Figure 4 A possible structural diagram of a feature mapping network provided in an embodiment of the present application is as follows: Figure 4 As shown, the geographic location information and the image features extracted by the feature extraction network are input into the first fully connected layer, and then pass through the second fully connected layer and the third fully connected layer in sequence to output the predicted value of the acoustic environment annoyance.

[0093] In addition, it should be noted that in the specific implementation, each convolutional layer and fully connected layer uses the ReLU activation function, and the ReLU activation function is applied after each convolutional layer and fully connected layer. This can introduce nonlinearity, thereby improving the expressiveness of the annoyance prediction model; in addition, the ReLU activation function is simple and computationally efficient, which can avoid the gradient vanishing problem and thus help the training of deep networks.

[0094] To facilitate understanding of the method provided in the embodiment of the present application, the following will take obtaining a sound environment annoyance distribution map of a target area as an example, and introduce the method provided in the embodiment of the present application in combination with a specific embodiment. Figure 5 The second method flow chart of the method for predicting the acoustic environment annoyance degree provided in the embodiment of the present application is as follows: Figure 5 As shown, the method comprises the following steps:

[0095] Step 201, obtaining the panoramic environment image and geographical location information corresponding to each sub-area in the target area;

[0096] Step 202, for each sub-area, inputting the corresponding panoramic environment image and geographical location information into a pre-trained annoyance prediction model;

[0097] Step 203, extracting features from the panoramic environment image through a feature extraction network in the annoyance prediction model to obtain image features of the panoramic environment image;

[0098] Step 204, mapping the above-mentioned image features and geographic location information through a feature mapping network in the annoyance prediction model, mapping the image features and geographic location information to a single-dimensional value as an annoyance prediction value;

[0099] Step 205, obtaining geographical location information and corresponding annoyance prediction values ​​corresponding to each sub-area;

[0100] Step 206, interpolating the target area according to the geographical location information and the annoyance prediction value of each sub-area, to obtain the annoyance prediction values ​​corresponding to the multiple interpolation points in the target area;

[0101] Step 207, obtaining map information corresponding to the target area;

[0102] Step 208 : Map the corresponding annoyance prediction value with the map information according to the geographical location information of the area corresponding to each sub-area and the interpolation point, and obtain a sound environment annoyance distribution map of the target area.

[0103] In the specific implementation, in order to realize the prediction of the annoyance degree of the acoustic environment based on the panoramic environment image through the annoyance degree prediction model, it is necessary to train the annoyance degree prediction model in advance. Therefore, the method provided in the embodiment of the present application also includes a training process of the annoyance degree prediction model. In the specific implementation, the above annoyance degree prediction model is obtained by training through the following process:

[0104] Acquire training sample data; wherein the training sample data includes environmental sample images, sample geographic location information and corresponding annoyance degree annotation values ​​of multiple sampling sample areas; construct an initial neural network model; input the environmental sample images and sample geographic location information into the initial neural network model, extract features of the environmental sample images through the initial neural network model, obtain sample image features of the environmental sample images, and obtain an annoyance degree prediction value of the acoustic environment of the sampling sample area for residents based on the sample image features and the above-mentioned sample geographic location information; calculate the value of the loss function according to the annoyance degree annotation value and the annoyance degree prediction value output by the initial neural network model; adjust the parameters of the initial neural network model according to the value of the loss function until the value of the loss function reaches the target requirement.

[0105] Among them, the model structure of the constructed initial neural network model is the same as the model structure of the annoyance prediction model in the embodiment of the present application.

[0106] In specific implementation, mean squared error (MSE) can be used as the loss function for model training. In a specific implementation, the above target requirement can be that the loss value is less than 0.01. Of course, it can also be other values. This is just an exemplary description of the above target requirement, and it is not the present of the present application scheme.

[0107] Of course, other loss functions may also be used during training, and the embodiments of the present application will not list them one by one.

[0108] In addition, it should be noted that after obtaining the sample data, the sample data can be divided into two parts, one part of which is used as training sample data for training the annoyance prediction model, and the other part of the sample data can be used as test sample data for testing and evaluating the trained annoyance prediction model. Optionally, in a specific implementation, the sample data used for training the annoyance prediction model accounts for approximately 80% of the total sample data, and the test sample data used for model evaluation accounts for approximately 20% of the total sample data. The total sample data is no less than 5,000 copies. In this way, approximately 4,000 samples of data are used for model training, and 1,000 samples of data are used for model evaluation. Of course, this is only an exemplary description of a possible sample data and its distribution. In specific implementation, more sample data may be required for model training and model evaluation. This is only an exemplary description of the sample data situation and does not constitute a limitation on the present application scheme.

[0109] In specific implementation, the determination coefficient and / or root mean square error (RMSE) may be used to perform model evaluation on the trained annoyance prediction model.

[0110] Optionally, in a specific implementation, the annoyance level label value corresponding to each sampling sample area can be collected by means of a questionnaire survey. For example, a small program for collecting acoustic environment annoyance questionnaires can be created or paper questionnaires can be used to collect residents' acoustic environment annoyance information.

[0111] Of course, whether it is through the mini program or paper questionnaire, when collecting the degree of annoyance of the acoustic environment, it is also necessary to collect some other basic information of residents, such as the residents' gender, age, education level, occupation, health status, income, household registration, noise sensitivity, and noise interference complaint history. After obtaining a large number of questionnaires, the questionnaire samples need to be preprocessed, such as data cleaning, to remove the questionnaire samples that lack key information, or are obviously false or not filled out in accordance with the specifications.

[0112] In addition, after the above data cleaning process is performed on the questionnaire sample, the cleaned questionnaire sample is screened according to the information such as gender, age, occupation, education level, income, household registration, noise sensitivity, etc. in the questionnaire to obtain the questionnaire sample for model training. For example, when screening, the male-female ratio can be kept at 1:1; for age, the above questionnaire samples can be proportioned according to the proportion requirements of each age group. For example, in the final screened questionnaire, residents aged 16-30 account for 30%, residents aged 30-60 account for 55%, and residents over 60 account for 15%. Of course, the proportion of each age group can also fluctuate up and down, such as the fluctuation of no more than 5%; for occupation, the final screened questionnaire can cover residents from all walks of life, rather than focusing on a certain industry; for education level, the final screened questionnaire can cover residents of various academic qualifications; the same is true for household registration and income. Of course, in the specific implementation, it can be screened according to actual needs and the information filled in the questionnaire. This is just an exemplary explanation of how to screen the above questionnaire samples, and it is not a limitation of this application scheme.

[0113] The annoyance value in the questionnaire sample finally obtained above and the panoramic environment image of the area corresponding to the questionnaire sample are stored correspondingly, for example, using the following structure for storage:

[0114] {Panoramic environment image, geographic location information, annoyance value}

[0115] That is, the above-mentioned training sample data is obtained.

[0116] In addition, it should be noted that the various panoramic environment images in the training sample data should have the same resolution and clear images to avoid the situation where the images are too bright or too dark.

[0117] Of course, in addition to this, the acoustic environment annoyance value of each sampling sample area can also be obtained by other means as the annoyance label value, such as calculating based on the noise source of the sampling sample area, etc. The embodiments of the present application are not listed one by one.

[0118] The method for predicting the annoyance of an acoustic environment provided in the embodiment of the present application, when it is necessary to know the annoyance degree caused by the acoustic environment of a certain area to residents, obtains the panoramic environment image and geographical location information corresponding to the sampling area, inputs the panoramic environment image and geographical location information into a pre-trained annoyance prediction model, extracts features of the panoramic environment image through the annoyance prediction model, and obtains the predicted value of the annoyance degree caused by the acoustic environment of the sampling area to residents based on the obtained image features and the above-mentioned geographical location information; through the embodiment of the present application, the prediction of the annoyance degree of the acoustic environment can be achieved in an end-to-end manner only based on the panoramic environment image and geographical location information, and there is no need to obtain the annoyance degree of the area through visits or questionnaires, nor is there a need to spend a lot of manpower costs to collect noise source information in the sampling area. The annoyance prediction method is simple and convenient, saves a lot of labor costs, and by adopting a model prediction method, it is more objective, thereby improving the accuracy of the annoyance prediction of the area.

[0119] Based on Figure 1-Figure 5 The present application also provides a device for predicting the annoyance level of an acoustic environment, for performing Figure 1 The method for predicting the annoyance level of the acoustic environment shown in FIG. Figure 6 A schematic diagram of the module composition of the device for predicting the acoustic environment annoyance degree provided in an embodiment of the present application is shown. Figure 6 As shown, the acoustic environment annoyance prediction device comprises:

[0120] The first acquisition module 301 is used to acquire the panoramic environment image and geographical location information corresponding to the sampling area;

[0121] An input module 302, used to input the panoramic environment image and geographic location information into a pre-trained annoyance prediction model;

[0122] The prediction module 303 is used to extract features from the panoramic environment image through an annoyance prediction model to obtain image features of the panoramic environment image, and obtain an annoyance prediction value of the acoustic environment in the sampling area for residents based on the image features and the geographical location information.

[0123] In a specific implementation, the above-mentioned acoustic environment annoyance prediction device further includes:

[0124] The second acquisition module is used to obtain the annoyance prediction value and geographical location information of each sampling area in the multiple sampling areas included in the area to be predicted;

[0125] The third acquisition module is used to obtain map information corresponding to the area to be predicted;

[0126] The generation module is used to generate a sound environment annoyance distribution map of the area to be predicted based on the geographical location information, annoyance prediction value and map information of the area to be predicted of each sampling area.

[0127] In a specific implementation, the above generation module is specifically used to:

[0128] According to the geographical location information and the annoyance prediction values ​​of each sampling area, the predicted area is interpolated to obtain the annoyance prediction values ​​of the areas corresponding to multiple interpolation points in the predicted area; according to the geographical location information of each sampling area and the areas corresponding to the interpolation points, the corresponding annoyance prediction values ​​are mapped with the map information to obtain the above-mentioned sound environment annoyance distribution map.

[0129] In a specific implementation, the annoyance prediction model includes a feature extraction network and a feature mapping network arranged in cascade;

[0130] The above prediction module is specifically used for:

[0131] The feature extraction network in the annoyance prediction model is used to extract features from the panoramic environment image to obtain image features; the image features and the geographic location information are subjected to feature mapping processing through the feature mapping network to map the image features and the geographic location information to feature values ​​of a single dimension as the annoyance prediction value.

[0132] In a specific implementation, the feature extraction network includes a first convolution layer, two second convolution layers, and four residual networks in a cascade arrangement, and the convolution kernel of the first convolution layer is larger than the convolution kernel of the second convolution layer;

[0133] The above prediction module is further specifically used for:

[0134] The first convolutional layer is used to extract the features of the panoramic environment image to obtain low-level features of the panoramic environment image; the second convolutional layer is used to extract the above low-level features to obtain high-level features of the panoramic environment image; the residual network is used to extract the above high-level features to obtain deep-level features of the panoramic environment image.

[0135] In a specific embodiment, the feature mapping network includes a first fully connected layer, a second fully connected layer, and a third fully connected layer arranged in cascade;

[0136] The above prediction module is further specifically used for:

[0137] The image features and geographic location information are processed by the first fully connected layer to obtain features of the first dimension. The features of the first dimension are processed by the second fully connected layer to obtain features of the second dimension. The features of the second dimension are mapped to feature values ​​of a single dimension through the third fully connected layer as the annoyance prediction value.

[0138] In a specific implementation, the above-mentioned acoustic environment annoyance prediction device further includes:

[0139] The storage module is used to store the geographical location information, identification information and annoyance prediction value of the sampling area correspondingly.

[0140] In a specific implementation, the above-mentioned acoustic environment annoyance prediction device further includes:

[0141] A fourth acquisition module is used to acquire training sample data; wherein the training sample data includes environmental sample images of multiple sampling sample areas, sample geographical location information and corresponding annoyance degree annotation values;

[0142] Building module, used to build the initial neural network model;

[0143] A training module is used to input the environmental sample image and the sample geographic location information into the initial neural network model, perform feature extraction on the environmental sample image through the initial neural network model to obtain the sample image features of the environmental sample image, and obtain the predicted value of annoyance caused by the sound environment of the sampling sample area to residents based on the sample image features and the sample geographic location information; calculate the value of the loss function according to the above-mentioned annoyance label value and the annoyance prediction value output by the initial neural network model; adjust the parameters of the initial neural network model according to the value of the loss function until the value of the loss function reaches the target requirement.

[0144] The device for predicting the annoyance degree of an acoustic environment provided in the embodiment of the present application is similar to the device for predicting the annoyance degree of an acoustic environment provided in the embodiment of the present application Figure 1-Figure 5 The method for predicting the annoyance level of an acoustic environment provided by the illustrated embodiment is based on the same inventive concept and has the same beneficial effects as the method adopted, operated or implemented by the method.

[0145] The implementation process of the functions and effects of each module in the acoustic environment annoyance prediction device provided in the embodiment of the present application is specifically described in detail in the implementation process of the corresponding steps in the above method embodiment, which will not be repeated here.

[0146] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components illustrated as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application. A person of ordinary skill in the art may understand and implement it without creative work.

[0147] Understandably, Figure 6 The division of the various modules is only a division of logical functions. In actual implementation, the functions of the above modules can be integrated into the hardware entity implementation of the device for predicting the acoustic environment annoyance level. For example, the functions of the first acquisition module 301, the input module 302 and the prediction module 303 can be integrated into the processor implementation of the hardware entity.

[0148] Figure 7 The hardware structure diagram of an electronic device provided in an embodiment of the present application includes: a processor 401, a transceiver 403 and a memory 402, and these components are connected and communicated through a communication bus 404. The memory 402 is used to store a computer program, which can be executed by the processor. When the processor executes the computer program, the aforementioned Figure 1-Figure 5 Any implementation in the illustrated embodiments provides a method for predicting the annoyance level of an acoustic environment.

[0149] The electronic device provided in the embodiment of the present application and the method for predicting the acoustic environment annoyance degree provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0150] The present application also provides a computer-readable storage medium corresponding to the method for predicting the acoustic environment annoyance degree provided in the above embodiment, on which a computer program is stored. When the computer program is executed by a processor, the above Figure 1-Figure 5 The method for predicting the acoustic environment annoyance level provided in any implementation manner of the corresponding embodiment.

[0151] The computer-readable storage medium provided in the above-mentioned embodiment of the present application and the method for predicting the acoustic environment annoyance degree provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0152] The present application also provides a computer program product corresponding to the method for predicting the acoustic environment annoyance degree provided in the above embodiment. The computer program product includes a computer program. The computer program is executed by a processor to implement the above Figure 1-Figure 5 The method for predicting the acoustic environment annoyance level provided in any implementation manner of the corresponding embodiment.

[0153] The computer program product provided in the above-mentioned embodiment of the present application and the method for predicting the acoustic environment annoyance degree provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0154] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0155] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0156] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0157] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0158] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a smart phone or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0159] Although the optional embodiments of the present application have been described, those skilled in the art, once informed of the basic creative concept, may make additional changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiment and all changes and modifications falling within the scope of the present application.

[0160] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation methods of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the scope of protection of the present application.

Claims

1. A method for predicting the annoyance level of an acoustic environment, characterized in that: The method comprises: Obtaining panoramic environment images and geographic location information corresponding to the sampling area; Inputting the panoramic environment image and the geographical location information into a pre-trained annoyance prediction model, wherein the annoyance prediction model includes a feature extraction network and a feature mapping network arranged in cascade; Extracting features from the panoramic environment image through a feature extraction network in the annoyance prediction model to obtain image features; The feature mapping network performs feature mapping processing on the image features and the geographic location information, and maps the image features and the geographic location information to feature values ​​of a single dimension as an annoyance prediction value.

2. The method according to claim 1, characterized in that The method further comprises: Obtain annoyance prediction value and geographical location information of each sampling area in multiple sampling areas included in the area to be predicted; Obtaining map information corresponding to the area to be predicted; A sound environment annoyance distribution map of the area to be predicted is generated according to the geographical location information, the annoyance prediction value of each sampling area and the map information of the area to be predicted.

3. The method according to claim 2, characterized in that The step of generating a sound environment annoyance distribution map of the area to be predicted based on the geographical location information, the annoyance prediction value and the map information of the area to be predicted of each sampling area comprises: According to the geographical location information and the annoyance prediction value of each of the sampling areas, interpolation processing is performed on the area to be predicted to obtain annoyance prediction values ​​of the areas corresponding to multiple interpolation points in the area to be predicted; According to the geographical location information of each sampling area and the area corresponding to the interpolation point, the corresponding annoyance prediction value is mapped with the map information to obtain the acoustic environment annoyance distribution map.

4. The method according to any one of claims 1 to 3, characterized in that: The feature extraction network includes a first convolutional layer, two second convolutional layers and four residual networks in a cascade arrangement, wherein the convolution kernel of the first convolutional layer is larger than the convolution kernel of the second convolutional layer; The step of extracting features from the panoramic environment image through a feature extraction network in the annoyance prediction model to obtain the image features includes: Extracting features of the panoramic environment image through the first convolutional layer to obtain low-level features of the panoramic environment image; Extracting the low-level features through the second convolutional layer to obtain high-level features of the panoramic environment image; The high-level features are extracted through the residual network to obtain deep-level features of the panoramic environment image.

5. The method according to claim 4, characterized in that The feature mapping network includes a first fully connected layer, a second fully connected layer and a third fully connected layer arranged in cascade; The performing feature mapping processing on the image features and the geographic location information through the feature mapping network to map the image features and the geographic location information to feature values ​​of a single dimension as the annoyance prediction value includes: Performing dimensionality reduction processing on the image features and the geographic location information through the first fully connected layer to obtain features of a first dimension; Performing dimensionality reduction processing on the features of the first dimension through the second fully connected layer to obtain features of the second dimension; The features of the second dimension are mapped to feature values ​​of a single dimension through the third fully connected layer as the annoyance prediction value.

6. The method according to claim 1 or 2, characterized in that: After extracting features from the panoramic environment image using the annoyance prediction model to obtain image features of the panoramic environment image, and obtaining an annoyance prediction value of the acoustic environment of the sampling area on residents based on the image features and the geographic location information, the method further includes: The geographical location information of the sampling area and the annoyance prediction value are stored correspondingly.

7. The method according to claim 1 or 2, characterized in that: The annoyance prediction model is trained through the following process: Acquire training sample data; wherein the training sample data includes environmental sample images, sample geographic location information and corresponding annoyance degree annotation values ​​of multiple sampling sample areas; Build an initial neural network model; Inputting the environmental sample image and the sample geographic location information into the initial neural network model, extracting features of the environmental sample image through the initial neural network model to obtain sample image features of the environmental sample image, and obtaining a predicted value of annoyance caused by the acoustic environment of the sampling sample area to residents based on the sample image features and the sample geographic location information; Calculating a value of a loss function according to the annoyance level labeled value and the annoyance level predicted value output by the initial neural network model; The parameters of the initial neural network model are adjusted according to the value of the loss function until the value of the loss function reaches the target requirement.

8. A device for predicting the annoyance level of an acoustic environment, characterized in that: The device comprises: A first acquisition module is used to acquire a panoramic environment image and geographic location information corresponding to a sampling area; An input module, used for inputting the panoramic environment image and the geographical location information into a pre-trained annoyance prediction model, wherein the annoyance prediction model includes a feature extraction network and a feature mapping network arranged in cascade; The prediction module is used to extract features of the panoramic environment image through the feature extraction network in the annoyance prediction model to obtain image features; and to perform feature mapping processing on the image features and the geographic location information through the feature mapping network to map the image features and the geographic location information to feature values ​​of a single dimension as an annoyance prediction value.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The method comprises instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 7.

Images