Audio loudness abnormality detection method, device and storage medium

Abstract

The present application discloses a method, device and storage medium for detecting audio loudness anomalies, and relates to the field of audio detection technology. The method comprises: obtaining the audio to be detected, dividing the audio to be detected according to a preset time length to obtain multiple audio segments; calculating the audio loudness value corresponding to each of the audio segments, filtering out the target audio loudness value from all the audio loudness values, and using the audio segment corresponding to the target audio loudness value as the target audio segment; judging whether the audio to be detected has loudness anomalies based on the audio loudness difference between the target audio segment and the audio segment to the left of the target audio segment, and the audio loudness difference between the target audio segment and the audio segment to the right of the target audio segment. Automated loudness anomaly detection of audio files is achieved, and audio files with loudness anomalies are promptly discovered from massive audio files.

Description

Technical Field

[0001] The present invention relates to the technical field of audio detection, and in particular to a method, device and storage medium for detecting abnormal audio loudness. Background Art

[0002] The loudness of an audio file may fluctuate, resulting in inconsistent sound quality throughout the entire work and a poor user experience. For example, some podcasts often have fixed audio titles, mid-credits, and end-credits, such as "Welcome to the audiobook platform, I am the host," "Today's program ends here, welcome to the next one, thank you everyone, goodbye!", etc. Because these are fixed, they are generally only recorded once and saved. Each time, the host only needs to record the main text and then insert the fixed titles, end-credits, etc. into the required position in the main text to complete the production of the work; some audiobook readings, such as online articles, have many chapters, even up to thousands of chapters, so it is also common to process the clips and end-credits automatically. In reality, some hosts lack professionalism or thorough consideration in audio editing, or fail to consider the relationship between the loudness of fixed opening, mid-credits, and ending audio clips and the loudness of the main text before and after splicing them during automated processing. This can result in the opening and ending sounds being too loud or too soft, such as the opening sound being significantly louder or softer than the main text, resulting in inconsistent loudness across the entire work. In summary, in the process of implementing the present invention, the inventors have at least discovered that the prior art suffers from inconsistent loudness in audio files. Summary of the Invention

[0003] In view of this, the present invention aims to provide a method, device, and medium for detecting abnormal loudness of audio files, which can automatically detect abnormal loudness of audio files and promptly identify audio files with abnormal loudness from a large number of audio files. The specific scheme is as follows:

[0004] In a first aspect, the present application discloses a method for detecting abnormal audio loudness, comprising:

[0005] Acquire the audio to be detected, and segment the audio to be detected according to preset time lengths to obtain multiple audio segments;

[0006] Calculating the audio loudness value corresponding to each of the audio segments, filtering out a target audio loudness value from all the audio loudness values, and using the audio segment corresponding to the target audio loudness value as the target audio segment;

[0007] Whether the audio to be detected has a loudness abnormality is determined based on an audio loudness difference between the target audio segment and an audio segment to the left of the target audio segment, and an audio loudness difference between the target audio segment and an audio segment to the right of the target audio segment.

[0008] Optionally, the determining whether the audio to be detected has a loudness abnormality based on an audio loudness difference between the target audio segment and an audio segment to the left of the target audio segment, and an audio loudness difference between the target audio segment and an audio segment to the right of the target audio segment, includes:

[0009] Generate a target audio set with the target audio segment as an anchor point according to a target audio set generation rule;

[0010] Selecting a preset number of consecutive audio clips adjacent to the left side of the target audio set as a first audio clip set, and selecting a preset number of consecutive audio clips adjacent to the right side of the target audio set as a second audio clip set;

[0011] Based on the audio loudness difference between the target audio set and the first audio clip set, and the audio loudness difference between the target audio set and the second audio clip set, it is determined whether a loudness anomaly exists in the large set of audio clips containing the target audio clip. Furthermore, based on the determination result of the large set of audio clips, it is determined whether a loudness anomaly exists in the audio to be detected.

[0012] Optionally, before generating the target audio set with the target audio segment as the anchor point according to the target audio set generation rule, the method further includes:

[0013] Filtering out a maximum audio loudness value and a minimum audio loudness value from all the audio loudness values, and calculating a difference between the maximum audio loudness value and the minimum audio loudness value;

[0014] If the difference is less than a first preset threshold, it is determined that there is no loudness abnormality in the audio to be detected;

[0015] If the difference is greater than or equal to the first preset threshold, the operation of generating a corresponding target audio set according to the target audio set generation rule and taking the target audio segment as an anchor point is performed.

[0016] Optionally, generating a target audio set with the target audio segment as an anchor point according to a target audio set generation rule includes:

[0017] Taking the audio segment adjacent to the left of the target audio segment as the first audio segment, calculating the loudness difference between each audio segment and the target audio segment in order from the left, and selecting audio segments with loudness differences less than a second preset threshold as candidate segments, the calculation stops when the first audio segment with a loudness difference greater than or equal to the second preset threshold is found;

[0018] Taking the audio segment adjacent to the right of the target audio segment as the first audio segment, calculating the loudness difference between each audio segment and the target audio segment in order from right to right, and selecting audio segments with loudness differences less than a second preset threshold as candidate segments, and stopping the calculation after the first audio segment with a loudness difference greater than or equal to the second preset threshold is found;

[0019] Based on all the to-be-selected segments and the target audio segment, a target audio set with the target audio segment as an anchor point is generated.

[0020] Optionally, the determining whether there is a loudness abnormality in the large set of audio clips containing the target audio clip based on the audio loudness difference between the target audio set and the first audio clip set, and the audio loudness difference between the target audio set and the second audio clip set, includes:

[0021] Determining a first audio loudness mean value corresponding to the target audio set, a second audio loudness mean value corresponding to the first audio clip set, and a third audio loudness mean value corresponding to the second audio clip set;

[0022] If a difference between the first audio mean loudness value and the second audio mean loudness value is greater than a third preset threshold, or a difference between the first audio mean loudness value and the third audio mean loudness value is greater than the third preset threshold, it is determined that a loudness anomaly exists in the large set of audio segments in which the target audio segment is located.

[0023] Optionally, determining whether the audio to be detected has abnormal loudness according to the judgment result of the large set of audio clips includes:

[0024] If there is abnormal loudness in any of the large set of audio clips, it is determined that the audio to be detected has abnormal loudness;

[0025] If there is no loudness abnormality in all the large sets of audio clips, it is determined that there is no loudness abnormality in the audio to be detected.

[0026] Optionally, filtering out a target audio loudness value from all the audio loudness values ​​includes:

[0027] All the audio loudness values ​​are sorted according to numerical values, and the first N audio loudness values ​​and the last N audio loudness values ​​are taken as the target audio loudness values ​​according to the sorting.

[0028] Optionally, before segmenting the audio to be detected according to a preset time length, the method further includes:

[0029] Performing voice activity detection on the audio to be detected to detect silent segments in the audio to be detected;

[0030] The silent segment is deleted from the audio to be detected.

[0031] In a second aspect, the present application discloses an electronic device, comprising:

[0032] Memory, used to store computer programs;

[0033] The processor is configured to execute the computer program to implement the aforementioned audio loudness abnormality detection method.

[0034] In a third aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein the computer program implements the aforementioned audio loudness anomaly detection method when executed by a processor.

[0035] In this application, the audio to be detected is obtained, and the audio to be detected is segmented according to a preset time length to obtain multiple audio segments; the audio loudness value corresponding to each audio segment is calculated, a target audio loudness value is screened out from all the audio loudness values, and the audio segment corresponding to the target audio loudness value is used as the target audio segment; and based on the audio loudness difference between the target audio segment and the audio segment to the left of the target audio segment, and the audio loudness difference between the target audio segment and the audio segment to the right of the target audio segment, it is determined whether the audio to be detected has a loudness anomaly. It can be seen that by segmenting the audio to be detected into multiple audio segments, determining the target audio segment based on the loudness value of each audio segment, and then determining whether the audio to be detected has a loudness anomaly based on the audio loudness difference between the target audio segment and the audio segments on both sides of the target audio segment, automatic loudness anomaly detection of audio files is achieved, and audio files with loudness anomalies are promptly discovered from a large number of audio files, so that feedback can be provided to the audio uploader for modification or technical repair, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0037] Figure 1 This is a schematic diagram of an abnormal audio loudness;

[0038] Figure 2 This is another diagram of abnormal audio loudness;

[0039] Figure 3 A schematic diagram of the system framework applicable to the audio loudness anomaly detection solution provided in this application;

[0040] Figure 4 A flow chart of an audio loudness anomaly detection method provided by this application;

[0041] Figure 5 A flowchart of a specific audio loudness anomaly detection method provided in this application;

[0042] Figure 6 A schematic diagram of a specific audio clip provided for this application;

[0043] Figure 7 A flowchart of a specific audio loudness anomaly detection method provided in this application;

[0044] Figure 8 This is a structural diagram of an audio loudness anomaly detection device provided by this application;

[0045] Figure 9 This is a structural diagram of an electronic device provided in this application. DETAILED DESCRIPTION

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0047] In the existing technology, the loudness of audio files may fluctuate, resulting in inconsistent loudness of the entire work and a poor user experience. For example, in some podcast programs, there are often some fixed audio titles, mid-titles, and tail-titles, etc. Because these are fixed, they are generally only recorded once and saved. Each time the host only needs to record the main text, and then insert the fixed titles, tail-titles, etc. into the required position of the main text to complete the production of the work; some audiobook readings, such as online articles, have many chapters, even up to thousands of chapters, so it is also common to process the tail-titles through automated processing. In reality, due to the lack of professionalism of some hosts in audio editing, or lack of consideration, or the failure to consider the relationship between the loudness of fixed titles, mid-titles, tail-titles, etc. and the audio loudness of the main text before and after splicing during automated processing, the sound of the titles and tail-titles may be too loud or too small. For example Figure 1 As shown, the sound of the opening is obviously louder than that of the main text, or as Figure 2 As shown in the figure, the sound of the opening is obviously quieter than that of the main text, which results in inconsistent loudness throughout the work. To overcome the above technical problems, this application proposes an audio loudness anomaly detection method that can automatically detect loudness anomalies in audio files and promptly identify audio files with loudness anomalies from a large number of audio files.

[0048] The system framework used in the audio retrieval solution of this application can be found in Figure 3 As shown, it may specifically include: a backend server and a number of user terminals that establish communication connections with the backend server. The user terminals include but are not limited to tablet computers, laptop computers, smart phones, and personal computers (PCs), which are not limited here.

[0049] In this application, the user terminal is used to send the audio to be detected to the backend server, and the backend server executes the steps of the audio loudness anomaly detection method, including obtaining the audio to be detected, dividing the audio to be detected according to a preset time length to obtain multiple audio segments; calculating the audio loudness value corresponding to each of the audio segments, filtering out the target audio loudness value from all the audio loudness values, and using the audio segment corresponding to the target audio loudness value as the target audio segment; based on the audio loudness difference between the target audio segment and the audio segment to the left of the target audio segment, and the audio loudness difference between the target audio segment and the audio segment to the right of the target audio segment, it is determined whether the audio to be detected has a loudness anomaly. The backend server pushes the loudness anomaly detection result of the audio to be detected to the user terminal for display, and can generate a corresponding change prompt when the detection result is a loudness anomaly.

[0050] The present application discloses a method for detecting abnormal audio loudness. Figure 4As shown, the method may include the following steps:

[0051] Step S11: obtaining audio to be detected, and segmenting the audio to be detected according to preset time lengths to obtain multiple audio segments.

[0052] In this embodiment, after obtaining the audio to be detected, the audio to be detected is segmented according to a preset time length to obtain multiple audio segments; that is, the audio segments are segmented, and the time length of each audio segment is t, such as 1 second or 2 seconds per segment.

[0053] Step S12: Calculate the audio loudness value corresponding to each of the audio segments, select a target audio loudness value from all the audio loudness values, and use the audio segment corresponding to the target audio loudness value as the target audio segment.

[0054] In this embodiment, after audio segmentation, the audio loudness value corresponding to each audio segment is calculated. Then, based on all audio loudness values, a target audio loudness value is selected, and the audio segment corresponding to the target audio loudness value is used as the target audio segment. The audio loudness value may be measured in LUFS (Loudness Units relative to Full Scale, dBFS).

[0055] In this embodiment, selecting a target audio loudness value from all the audio loudness values ​​may include sorting all the audio loudness values ​​by numerical value, and selecting the first N and last N audio loudness values ​​based on the sorting as the target audio loudness values. Specifically, the EBU R.128 algorithm is used to sequentially calculate the audio loudness of the segmented audio segments, obtaining the maximum N and minimum N loudness values ​​as the target audio loudness values. These loudness values ​​correspond to time segments that are the target audio segments. EBU R.128 is a recommendation on loudness control issued by the European Broadcasting Union (EBU). Based on the ITU-R BS.1770 standard, this recommendation defines details such as the loudness measurement subject and the integration window length.

[0056] Step S13: Determine whether the audio to be detected has a loudness abnormality based on the audio loudness difference between the target audio segment and the audio segment to the left of the target audio segment, and the audio loudness difference between the target audio segment and the audio segment to the right of the target audio segment.

[0057] In this embodiment, after a target audio segment is determined, the target audio segment is compared with an audio segment to the left of the target audio segment to determine the audio loudness difference. Simultaneously, the target audio segment is compared with an audio segment to the right of the target audio segment to determine the audio loudness difference. Based on the two obtained audio loudness differences, it is then determined whether the audio to be detected has a loudness anomaly. Specifically, audio segments can be selected within a preset distance range to the left of the target audio segment for comparison. Similarly, audio segments can be selected within a preset distance range to the right of the target audio segment for comparison. Specifically, the audio loudness differences can be compared with a preset threshold, and the comparison results can be used to determine whether the audio to be detected has a loudness anomaly. In other words, in this embodiment, the loudness values ​​of the selected target audio segment are compared with the audio segments on both sides of the target audio segment, and based on the loudness differences, it is determined whether the audio to be detected has a loudness anomaly.

[0058] As can be seen from the above, in this embodiment, the audio to be detected is obtained, the audio to be detected is segmented according to a preset time length to obtain multiple audio segments; the audio loudness value corresponding to each audio segment is calculated, a target audio loudness value is screened from all the audio loudness values, and the audio segment corresponding to the target audio loudness value is used as the target audio segment; and the audio loudness difference between the target audio segment and the audio segment to the left of the target audio segment, as well as the audio loudness difference between the target audio segment and the audio segment to the right of the target audio segment, is used to determine whether the audio to be detected has a loudness anomaly. As can be seen, by segmenting the audio to be detected into multiple audio segments, determining a target audio segment based on the loudness value of each audio segment, and then determining whether the audio to be detected has a loudness anomaly based on the audio loudness difference between the target audio segment and the audio segments on both sides of the target audio segment, automatic loudness anomaly detection is achieved for audio files, and audio files with loudness anomalies are promptly identified from a large number of audio files so that feedback can be provided to the audio uploader for modification or technical repair, thereby improving the user experience.

[0059] The present application discloses a specific method for detecting abnormal audio loudness. Figure 5 As shown, the method may include the following steps:

[0060] Step S21: obtaining audio to be detected, and segmenting the audio to be detected according to preset time lengths to obtain multiple audio segments.

[0061] Step S22: calculating the audio loudness value corresponding to each of the audio segments, filtering out a target audio loudness value from all the audio loudness values, and using the audio segment corresponding to the target audio loudness value as the target audio segment.

[0062] Step S23: generating a target audio set with the target audio segment as an anchor point according to a target audio set generation rule.

[0063] In this embodiment, before generating a target audio set with the target audio segment as an anchor point according to the target audio set generation rule, the method may further include: selecting a maximum audio loudness value and a minimum audio loudness value from all the audio loudness values, and calculating the difference between the maximum audio loudness value and the minimum audio loudness value; if the difference is less than a first preset threshold, determining that the detected audio does not have a loudness abnormality; and if the difference is greater than or equal to the first preset threshold, executing the operation of generating a corresponding target audio set with the target audio segment as an anchor point according to the target audio set generation rule. It is understood that if the difference between the maximum audio loudness value and the minimum audio loudness value is less than the first preset threshold, a normal loudness result is directly returned, i.e., the difference between the maximum and minimum audio loudness values ​​is less than a predetermined threshold, indicating that the loudness of the entire audio file is within a certain range and the loudness is normal. The first preset threshold may be 10dB to 12dB. If the difference between the maximum audio loudness value and the minimum audio loudness value is greater than the first preset threshold, it indicates that there is a certain loudness difference in the entire audio file. In this case, it is necessary to further and more carefully determine whether there is a loudness anomaly. Therefore, an operation of generating a corresponding target audio set according to the target audio set generation rule is executed with the target audio segment as the anchor point.

[0064] In this embodiment, after the target audio segment is determined, a target audio set with the target audio segment as the anchor point is generated according to the target audio set generation rule. That is, according to the target audio set generation rule, audio segments around the target audio segment with a loudness similar to that of the target audio segment are formed into a set.

[0065] In this embodiment, the above step S23: generating a target audio set with the target audio segment as an anchor point according to the target audio set generation rule may include the following sub-steps:

[0066] S230: Starting with the audio segment adjacent to the left of the target audio segment as the first audio segment, calculating the loudness difference between each audio segment and the target audio segment in order from the left, and selecting audio segments with loudness differences less than a second preset threshold as candidate segments. Calculation stops when the first audio segment with a loudness difference greater than or equal to the second preset threshold is found.

[0067] In this embodiment, starting from the target audio segment, a search is performed to the left for consecutive audio segments with loudness differences within a second preset threshold range, and the search is stopped until the first audio segment with a loudness difference greater than or equal to the second preset threshold is found. In other words, the candidate segments to the left of the target audio segment are continuous and connected to the target audio segment.

[0068] S231: Starting with the audio segment immediately adjacent to the right side of the target audio segment as the first audio segment, the loudness difference between each audio segment and the target audio segment is calculated in sequence going rightward, and audio segments whose loudness difference is less than a second preset threshold are selected as candidate segments. The calculation is terminated after the first audio segment whose loudness difference is greater than or equal to the second preset threshold is found.

[0069] At the same time, the system searches to the right for consecutive audio segments with loudness differences within a second preset threshold range as candidate segments. Calculation stops when the first audio segment with a loudness difference greater than or equal to the second preset threshold is found. In other words, the candidate segments to the right of the target audio segment are continuous and connected to the target audio segment.

[0070] S232: Based on all the to-be-selected segments and the target audio segment, generate a target audio set with the target audio segment as an anchor point.

[0071] All the candidate segments and the target audio segment are used as a target audio set with the target audio segment as an anchor point. The target audio set is a continuous audio segment.

[0072] Step S24: selecting a preset number of continuous audio segments adjacent to the left side of the target audio set as a first audio segment set, and selecting a preset number of continuous audio segments adjacent to the right side of the target audio set as a second audio segment set.

[0073] In this embodiment, after determining the target audio set, a continuous preset number of audio clips adjacent to the left side of the target audio set are selected as the first audio clip set, and a continuous preset number of audio clips adjacent to the right side of the target audio set are selected as the second audio clip set. If the target audio set is already at the head of the audio to be detected, the first audio clip set on the left cannot be selected. In this case, the second audio clip set on the right can be selected. Similarly, if the target audio set is at the tail of the audio to be detected, only the first audio clip set on the left can be selected. It can be understood that by generating a target audio set, audio clips with loudness values ​​similar to the target audio clip around the target audio clip can be grouped together as a target audio set, and then two more sets are taken on both sides of the target audio set. Finally, loudness anomalies are judged based on the differences between the sets. Compared with the difference detection of a single audio clip, by generating a set and then comparing the sets, false detection can be avoided and the ability to detect loudness anomalies can be improved.

[0074] Step S25: Based on the audio loudness difference between the target audio set and the first audio clip set, and the audio loudness difference between the target audio set and the second audio clip set, determine whether there is a loudness abnormality in the large audio clip set containing the target audio clip. Determine whether there is a loudness abnormality in the audio to be detected based on the determination result of the large audio clip set.

[0075] In this embodiment, based on the audio loudness difference between the target audio set and the first audio clip set, as well as the audio loudness difference between the target audio set and the second audio clip set, it is determined whether there is a loudness abnormality in the large audio clip set where the target audio clip is located. The large audio clip set is a large set consisting of the first audio clip set, the target audio set, and the second audio clip set. Finally, based on the determination result of the large audio clip set, it is determined whether there is a loudness abnormality in the audio to be detected.

[0076] In this embodiment, step S25, which determines whether a loudness anomaly exists within the large set of audio segments containing the target audio segment based on the audio loudness difference between the target audio segment set and the first audio segment set, and the audio loudness difference between the target audio segment set and the second audio segment set, and determines whether a loudness anomaly exists within the audio segment set to be detected based on the determination results of the large set of audio segments, may include the following sub-steps:

[0077] S250: Determine a first audio mean loudness value corresponding to the target audio set, a second audio mean loudness value corresponding to the first audio clip set, and a third audio mean loudness value corresponding to the second audio clip set.

[0078] That is, the average audio loudness values ​​corresponding to the target audio set, the first audio clip set, and the second audio clip set are calculated respectively. That is, the loudness values ​​of all audio clips contained in the set are divided by the number of audio clips in the set to obtain the corresponding average audio loudness values.

[0079] S251: If a difference between the first audio mean loudness value and the second audio mean loudness value is greater than a third preset threshold, or a difference between the first audio mean loudness value and the third audio mean loudness value is greater than the third preset threshold, it is determined that a loudness anomaly exists in the large set of audio segments in which the target audio segment is located.

[0080] That is, if the difference in audio loudness mean between the target audio set and the first audio clip set is greater than a third preset threshold, or if the difference in audio loudness mean between the target audio clip set and the second audio clip set is greater than the third preset threshold, then it is determined that a loudness abnormality exists in the large audio clip set where the target audio clip is located; otherwise, the large audio clip set is normal.

[0081] S252: If there is abnormal loudness in any of the large audio segment sets, it is determined that the audio to be detected has abnormal loudness.

[0082] That is, each target audio segment corresponds to a large audio segment set. If there is abnormal loudness in any of the large audio segment sets in the audio to be detected, it is determined that the audio to be detected has abnormal loudness.

[0083] S253: If there is no abnormal loudness in all of the large audio segment sets, it is determined that the audio to be detected has no abnormal loudness.

[0084] That is, if there is no abnormal loudness in all of the large audio segment sets in the audio to be detected, it is determined that the audio to be detected has no abnormal loudness.

[0085] The following is an example for illustration:

[0086] Suppose after calculating the audio loudness, the maximum N loudness values obtained are represented as MaxVol_1, MaxVol_2, …, MaxVol_N, and their corresponding segment positions are MaxLoc_1, MaxLoc_2, …, MaxLoc_N. The minimum N loudness values are represented as MinVol_1, MinVol_2, …, MinVol_N, and their corresponding segment positions are MinLoc_1, MinLoc_2, …, MinLoc_N.

[0087] Take the target audio segments corresponding to the maximum N loudness values and the minimum N loudness values as anchor points. Let Vol(i) represent the loudness of audio segment i; the target audio set Vol_Set(i, j, k, …) represents a group of consecutive segments with similar loudness, where i, j, k represent audio segments i, j, k; abs() represents taking the absolute value.

[0088] a. If MaxVol_1 - MinVol_1 < Thresh1, that is, the difference between the maximum audio loudness value and the minimum audio loudness value is less than the first preset threshold, directly return the result that the loudness is normal;

[0089] b. If MaxVol_1 - MinVol_1 >= Thresh1, then obtain the target audio set Vol_Set of each anchor segment. The following takes MaxLoc_1 as the target audio segment for illustration:

[0090] For the target audio segment MaxLoc_1, search for consecutive audio segments with loudness within the Thresh2 range to the left, and then add them to the set Vol_Set. That is, if abs(Vol(MaxLoc_1) - Vol(MaxLoc_1 - 1)) < Thresh2, then include the segment (MaxLoc_1 - 1) in the set Vol_Set, and continue to check if abs(Vol(MaxLoc_1) - Vol(MaxLoc_1 - 2)) < Thresh2 to see if the segment (MaxLoc_1 - 2) meets the condition. Keep doing this until the segment (MaxLoc_1 - p) does not meet the condition. Then, the segments (MaxLoc_1 - 1), (MaxLoc_1 - 2),..., (MaxLoc_1 - p + 1) are all included in the set Vol_Set.

[0091] Similarly, for the audio segment MaxLoc_1, search for consecutive audio segments with loudness within the Thresh2 range to the right, and then add them to the set Vol_Set. That is, if abs(Vol(MaxLoc_1) - Vol(MaxLoc_1 + 1)) < Thresh2, then include the segment (MaxLoc_1 + 1) in the set Vol_Set, and continue to check if abs(Vol(MaxLoc_1) - Vol(MaxLoc_1 + 2)) < Thresh2 to see if the segment (MaxLoc_1 + 2) meets the condition. Keep doing this until the segment (MaxLoc_1 + q) does not meet the condition. Then, the segments (MaxLoc_1 + 1), (MaxLoc_1 + 2),..., (MaxLoc_1 + q - 1) are all included in the set Vol_Set. Thus, the consecutive skewness with similar loudness is in the set Vol_Set.

[0092] For example Figure 6 As shown, each block represents each audio segment. Search for segments with similar loudness on the left and right sides of the target audio segment to generate a target audio set containing 6 segments.

[0093] c. On the left and right sides of Vol_Set, take several consecutive segments, such as 4 segments. If each segment is 2 seconds long, then take 8 seconds of audio data as the first audio segment set adj_Set_L and the second audio segment set adj_Set_R.

[0094] d. Calculate the means of Vol_Set, adj_Set_L, and adj_Set_R, denoted as Vol_Set_Mean, adj_Set_L_Mean, and adj_Set_R_Mean respectively.

[0095] e. If abs(Vol_Set_Mean - adj_Set_L_Mean) > Thresh3 or abs(Vol_Set_Mean - adj_Set_R_Mean) > Thresh3, the loudness difference near the anchor point is poor and the result is returned directly, indicating loudness abnormality. Otherwise, repeat steps be for other anchor points.

[0096] f. If no result is returned after the above steps are completed, the loudness is returned as normal.

[0097] The above steps of ae can be executed by a loudness abnormality determiner to determine whether there is a loudness abnormality phenomenon, thereby effectively screening out audio files with loudness abnormality.

[0098] For the specific processes of the above-mentioned steps S21 and S22, reference may be made to the corresponding contents disclosed in the above-mentioned embodiments, which will not be described in detail here.

[0099] As can be seen from the above, in this embodiment, a target audio set is generated with the target audio segment as an anchor point according to the target audio set generation rule; a preset number of consecutive audio segments adjacent to the left side of the target audio set are selected as a first audio segment set, and a preset number of consecutive audio segments adjacent to the right side of the target audio set are selected as a second audio segment set; based on the audio loudness difference between the target audio set and the first audio segment set, and the audio loudness difference between the target audio set and the second audio segment set, it is determined whether there is a loudness anomaly within the large set of audio segments in which the target audio segment is located, and based on the judgment result of the large set of audio segments, it is determined whether there is a loudness anomaly in the audio to be detected. By generating a target audio set, audio segments with similar loudness values ​​around the target audio segment are grouped together as a target audio set, and then two more sets are taken on both sides of the target audio set. Finally, loudness anomalies are determined based on the differences between the sets. Compared with the difference detection of individual audio segments, generating sets and then comparing the sets can avoid false detections and improve the ability of loudness anomaly detection.

[0100] The present application discloses a specific method for detecting abnormal audio loudness. Figure 7 As shown, the method may include the following steps:

[0101] Step S31: obtaining audio to be detected, performing voice activity detection on the audio to be detected, detecting silent segments in the audio to be detected, and deleting the silent segments from the audio to be detected.

[0102] In this embodiment, after obtaining the audio to be detected, voice activity detection (VAD) is first performed on the audio to be detected, thereby detecting silent segments in the audio to be detected and removing these silent segments from the audio to be detected. It is understandable that pauses are common in many audio files, especially in audiobook reading scenarios. Therefore, voice activity detection is used to detect the presence of speech and remove silent pauses to pre-process the audio file, thereby improving the accuracy of loudness calculation.

[0103] Step S32: Segment the audio to be detected according to preset time lengths to obtain multiple audio segments.

[0104] Step S33: calculating the audio loudness value corresponding to each of the audio segments, filtering out a target audio loudness value from all the audio loudness values, and using the audio segment corresponding to the target audio loudness value as the target audio segment.

[0105] Step S34: Determine whether the audio to be detected has a loudness abnormality based on the audio loudness difference between the target audio segment and the audio segment to the left of the target audio segment, and the audio loudness difference between the target audio segment and the audio segment to the right of the target audio segment.

[0106] Among them, regarding the specific process of the above-mentioned steps S32 to S34, reference can be made to the corresponding content disclosed in the above-mentioned embodiment, and no further details will be given here.

[0107] As can be seen from the above, in this embodiment, audio to be detected is obtained, voice activity detection is performed on the audio to be detected, silence segments in the audio to be detected are detected, and the silence segments are deleted from the audio to be detected.

[0108] In this embodiment, after obtaining the audio to be detected, voice activity detection is first performed on the audio to detect silence segments in the audio to be detected, and these silence segments are removed from the audio to be detected. It will be appreciated that voice activity detection is used to detect the presence of speech and remove silence segments to pre-process the audio file, thereby improving the accuracy of loudness calculation.

[0109] Below, taking a certain music APP as an example, the technical solution of this application is explained.

[0110] Suppose a user has installed this music app on their device. After creating an audiobook, they send an audio loudness anomaly detection request to the app's backend server, carrying the audio to be detected. After receiving the audio, the backend server performs voice activity detection on the audio to be detected, detects silent segments in the audio, and removes these segments. The audio to be detected is then segmented into multiple audio segments at 2-second intervals. The audio loudness value corresponding to each audio segment is then calculated and sorted by numerical value. The top N and bottom N audio loudness values, based on the sorting, are selected as target audio loudness values, and the audio segment corresponding to the target audio loudness value is then selected as the target audio segment. Then, according to the target audio set generation rule, a target audio set with the target audio segment as the anchor point is generated; a continuous preset number of audio segments adjacent to the left side of the target audio set are selected as the first audio segment set, and a continuous preset number of audio segments adjacent to the right side of the target audio set are selected as the second audio segment set; based on the audio loudness difference between the target audio set and the first audio segment set, and the audio loudness difference between the target audio set and the second audio segment set, it is determined whether there is a loudness abnormality in the large audio segment set where the target audio segment is located, and based on the judgment result of the large audio segment set, it is determined whether there is a loudness abnormality in the audio to be detected.

[0111] Accordingly, the present application also discloses an audio loudness abnormality detection device, see Figure 8 As shown, the device includes:

[0112] The audio segmentation module 11 is used to obtain the audio to be detected and segment the audio to be detected according to preset time lengths to obtain multiple audio segments;

[0113] a loudness value determining module 12, configured to calculate the audio loudness value corresponding to each of the audio segments, filter out a target audio loudness value from all the audio loudness values, and use the audio segment corresponding to the target audio loudness value as the target audio segment;

[0114] The loudness anomaly detection module 13 is configured to determine whether the audio to be detected has a loudness anomaly based on an audio loudness difference between the target audio segment and an audio segment to the left of the target audio segment, and an audio loudness difference between the target audio segment and an audio segment to the right of the target audio segment.

[0115] As can be seen from the above, in this embodiment, the audio to be detected is obtained, the audio to be detected is segmented according to a preset time length to obtain multiple audio segments; the audio loudness value corresponding to each audio segment is calculated, a target audio loudness value is screened from all the audio loudness values, and the audio segment corresponding to the target audio loudness value is used as the target audio segment; and the audio loudness difference between the target audio segment and the audio segment to the left of the target audio segment, as well as the audio loudness difference between the target audio segment and the audio segment to the right of the target audio segment, is used to determine whether the audio to be detected has a loudness anomaly. As can be seen, by segmenting the audio to be detected into multiple audio segments, determining a target audio segment based on the loudness value of each audio segment, and then determining whether the audio to be detected has a loudness anomaly based on the audio loudness difference between the target audio segment and the audio segments on both sides of the target audio segment, automatic loudness anomaly detection is achieved for audio files, and audio files with loudness anomalies are promptly identified from a large number of audio files so that feedback can be provided to the audio uploader for modification or technical repair, thereby improving the user experience.

[0116] In some specific embodiments, the loudness anomaly detection module 13 may specifically include:

[0117] a target audio set generating unit, configured to generate a target audio set with the target audio segment as an anchor point according to a target audio set generating rule;

[0118] an audio segment set determining unit, configured to select a preset number of consecutive audio segments adjacent to the left side of the target audio segment set as a first audio segment set, and select a preset number of consecutive audio segments adjacent to the right side of the target audio segment set as a second audio segment set;

[0119] The loudness anomaly determination unit is configured to determine whether a loudness anomaly exists within a large set of audio segments containing the target audio segment based on an audio loudness difference between the target audio segment set and the first audio segment set, and an audio loudness difference between the target audio segment set and the second audio segment set, and to determine whether a loudness anomaly exists within the audio segment set to be detected based on the determination result for the large set of audio segments.

[0120] In some specific embodiments, the audio loudness abnormality detection device may specifically include:

[0121] a difference calculation unit, configured to, before generating a target audio set with the target audio segment as an anchor point according to a target audio set generation rule, filter out a maximum audio loudness value and a minimum audio loudness value from all the audio loudness values, and calculate a difference between the maximum audio loudness value and the minimum audio loudness value;

[0122] an abnormality determination unit, configured to determine that there is no loudness abnormality in the audio to be detected if the difference is less than a first preset threshold;

[0123] An operation execution unit is configured to execute, if the difference is greater than or equal to the first preset threshold, an operation of generating a corresponding target audio set according to a target audio set generation rule and taking the target audio segment as an anchor point.

[0124] In some specific embodiments, the target audio set generation unit may specifically include:

[0125] a first candidate-segment determination unit configured to use the audio segment adjacent to the left of the target audio segment as a first audio segment, calculate a loudness difference between each audio segment and the target audio segment in a leftward direction, and select audio segments whose loudness differences are less than a second preset threshold as candidate segments, and stop calculating after the first audio segment whose loudness difference is greater than or equal to the second preset threshold is found;

[0126] a second candidate-segment determination unit, configured to use the audio segment adjacent to the right of the target audio segment as the first audio segment, calculate the loudness difference between each audio segment and the target audio segment in order from right to right, and select audio segments whose loudness difference is less than a second preset threshold as candidate segments, and stop calculating after the first audio segment whose loudness difference is greater than or equal to the second preset threshold is found;

[0127] The target audio set determining unit is configured to generate a target audio set with the target audio segment as an anchor point based on all the candidate segments and the target audio segment.

[0128] In some specific embodiments, the abnormal loudness determination unit may specifically include:

[0129] a mean loudness determination unit, configured to determine a first audio mean loudness corresponding to the target audio set, a second audio mean loudness corresponding to the first audio clip set, and a third audio mean loudness corresponding to the second audio clip set;

[0130] The large set loudness anomaly determination unit is configured to determine that a loudness anomaly exists in the large set of audio segments in which the target audio segment is located if a difference between the first audio mean loudness value and the second audio mean loudness value is greater than a third preset threshold, or a difference between the first audio mean loudness value and the third audio mean loudness value is greater than the third preset threshold.

[0131] In some specific embodiments, the abnormal loudness determination unit may specifically include:

[0132] a first judgment unit, configured to determine that the audio to be detected has abnormal loudness if abnormal loudness exists in any of the large set of audio segments;

[0133] The second judgment unit is configured to determine that the audio to be detected does not have a loudness abnormality if no loudness abnormality exists in all the large sets of audio segments.

[0134] In some specific embodiments, the loudness value determination module 12 may specifically include:

[0135] The target loudness value determining unit is configured to sort all the audio loudness values ​​according to numerical values, and take the first N audio loudness values ​​and the last N audio loudness values ​​as the target audio loudness values ​​according to the sorting.

[0136] In some specific embodiments, the audio loudness abnormality detection device may specifically include:

[0137] a voice activity detection unit, configured to perform voice activity detection on the audio to be detected, and detect silence segments in the audio to be detected;

[0138] A silence segment deleting unit is configured to delete the silence segment from the audio to be detected.

[0139] Furthermore, the present application also discloses an electronic device, see Figure 9 The contents in the drawings should not be considered as any limitation on the scope of use of the present application.

[0140] Figure 9 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the audio loudness anomaly detection method disclosed in any of the aforementioned embodiments.

[0141] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0142] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223 including the audio to be detected, etc. The storage method can be temporary storage or permanent storage.

[0143] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, thereby enabling the processor 21 to operate and process the massive amount of data 223 in the memory 22. The operating system 221 may be Windows Server, NetWare, Unix, Linux, etc. In addition to including computer programs capable of implementing the audio loudness anomaly detection method disclosed in any of the aforementioned embodiments and executed by the electronic device 20, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0144] Furthermore, an embodiment of the present application also discloses a computer storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are loaded and executed by a processor, the steps of the audio loudness anomaly detection method disclosed in any of the aforementioned embodiments are implemented.

[0145] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0146] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0147] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0148] The above describes in detail the audio loudness anomaly detection method, apparatus, device, and medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is intended only to facilitate understanding of the method and core concepts of the present invention. Furthermore, those skilled in the art will appreciate that variations in the specific implementation methods and scope of application are possible based on the concepts of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A method for detecting abnormal audio loudness, characterized in that: include: Acquire the audio to be detected, and segment the audio to be detected according to preset time lengths to obtain multiple audio segments; Calculating the audio loudness value corresponding to each of the audio segments, filtering out a target audio loudness value from all the audio loudness values, and using the audio segment corresponding to the target audio loudness value as the target audio segment; Whether the audio to be detected has a loudness abnormality is determined based on an audio loudness difference between the target audio segment and an audio segment to the left of the target audio segment, and an audio loudness difference between the target audio segment and an audio segment to the right of the target audio segment.

2. The method for detecting abnormal audio loudness according to claim 1, wherein: The determining whether the audio to be detected has a loudness abnormality according to an audio loudness difference between the target audio segment and an audio segment to the left of the target audio segment, and an audio loudness difference between the target audio segment and an audio segment to the right of the target audio segment, includes: Generate a target audio set with the target audio segment as an anchor point according to a target audio set generation rule; Selecting a preset number of consecutive audio clips adjacent to the left side of the target audio set as a first audio clip set, and selecting a preset number of consecutive audio clips adjacent to the right side of the target audio set as a second audio clip set; Based on the audio loudness difference between the target audio set and the first audio clip set, and the audio loudness difference between the target audio set and the second audio clip set, it is determined whether a loudness anomaly exists in the large set of audio clips containing the target audio clip. Furthermore, based on the determination result of the large set of audio clips, it is determined whether a loudness anomaly exists in the audio to be detected.

3. The method for detecting abnormal audio loudness according to claim 2, wherein: Before generating the target audio set with the target audio segment as the anchor point according to the target audio set generation rule, the method further includes: Filtering out a maximum audio loudness value and a minimum audio loudness value from all the audio loudness values, and calculating a difference between the maximum audio loudness value and the minimum audio loudness value; If the difference is less than a first preset threshold, it is determined that there is no loudness abnormality in the audio to be detected; If the difference is greater than or equal to the first preset threshold, the operation of generating a corresponding target audio set according to the target audio set generation rule and taking the target audio segment as an anchor point is performed.

4. The method for detecting abnormal audio loudness according to claim 2, wherein: Generating a target audio set with the target audio segment as an anchor point according to a target audio set generation rule includes: Taking the audio segment adjacent to the left of the target audio segment as the first audio segment, calculating the loudness difference between each audio segment and the target audio segment in order from the left, and selecting audio segments with loudness differences less than a second preset threshold as candidate segments, the calculation stops when the first audio segment with a loudness difference greater than or equal to the second preset threshold is found; Taking the audio segment adjacent to the right of the target audio segment as the first audio segment, calculating the loudness difference between each audio segment and the target audio segment in order from right to right, and selecting audio segments with loudness differences less than a second preset threshold as candidate segments, and stopping the calculation after the first audio segment with a loudness difference greater than or equal to the second preset threshold is found; Based on all the to-be-selected segments and the target audio segment, a target audio set with the target audio segment as an anchor point is generated.

5. The method for detecting abnormal audio loudness according to claim 2, wherein: The determining, based on the audio loudness difference between the target audio set and the first audio clip set, and the audio loudness difference between the target audio set and the second audio clip set, whether there is a loudness abnormality in the large set of audio clips in which the target audio clip is located includes: Determining a first audio loudness mean value corresponding to the target audio set, a second audio loudness mean value corresponding to the first audio clip set, and a third audio loudness mean value corresponding to the second audio clip set; If a difference between the first audio mean loudness value and the second audio mean loudness value is greater than a third preset threshold, or a difference between the first audio mean loudness value and the third audio mean loudness value is greater than the third preset threshold, it is determined that a loudness anomaly exists in the large set of audio segments in which the target audio segment is located.

6. The method for detecting abnormal audio loudness according to claim 5, wherein: The step of determining whether the audio to be detected has abnormal loudness based on the judgment result of the large set of audio segments includes: If there is abnormal loudness in any of the large set of audio clips, it is determined that the audio to be detected has abnormal loudness; If there is no loudness abnormality in all the large sets of audio clips, it is determined that there is no loudness abnormality in the audio to be detected.

7. The method for detecting abnormal audio loudness according to claim 1, wherein: The step of selecting a target audio loudness value from all the audio loudness values ​​includes: All the audio loudness values ​​are sorted according to numerical values, and the first N audio loudness values ​​and the last N audio loudness values ​​are taken as the target audio loudness values ​​according to the sorting.

8. The method for detecting abnormal audio loudness according to any one of claims 1 to 7, wherein: Before segmenting the audio to be detected according to the preset time length, the method further includes: Performing voice activity detection on the audio to be detected to detect silent segments in the audio to be detected; The silent segment is deleted from the audio to be detected.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the audio loudness anomaly detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that Used to store a computer program; wherein when the computer program is executed by a processor, the audio loudness abnormality detection method according to any one of claims 1 to 8 is implemented.

Images