Adaptive audio equipment audio data processing system and method

Abstract

The invention discloses a self-adaptive audio equipment audio data processing system and method, and relates to the technical field of equipment data processing, and the method comprises the steps: obtaining historical video data of monitoring equipment and historical audio data of audio equipment, obtaining a feature region and a target local region, and intercepting a preposed segment in the video data; obtaining video data at the current moment, and obtaining a similarity value corresponding to the current video data; collecting audio data at the current moment so as to obtain an environment spectrogram; and determining a volume adjustment model of the equipment volume, and intelligently adjusting the current volume of the audio equipment in combination with the similarity value. According to the method, the device volume which should be set at the current moment is obtained according to the historical video data and the audio data, the situation that normal conversation is affected due to the fact that the sound of the sound equipment is too loud is avoided, convenience and comfort are brought to a user, the household audio device better meets the actual requirement of the user, and the home entertainment and life quality is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of device data processing, and in particular to an adaptive audio device audio data processing system and method. Background Art

[0002] With the advancement of technology, smart homes can achieve smarter decision-making and automated control, bringing significant development uses. They not only greatly enrich people's entertainment methods and bring endless joy to the family, but also play an important role in creating a warm and harmonious family atmosphere; as an indispensable audio device for home use, speakers play a vital role in people's daily lives. Smart speakers are generally equipped with microphones that can listen to users' instructions at any time. However, in actual use, it is often encountered that if the user calls or needs to talk, the volume of the speaker needs to be manually adjusted so as not to affect the conversation. This is not only troublesome, but also frequently adjusting the volume affects the user experience. Therefore, according to the audio data in the environment and the home surveillance camera, the volume of the audio equipment is intelligently adjusted to avoid the sound of the speaker being too loud to affect normal calls. It can bring convenience and comfort to users, make home audio equipment more in line with the actual needs of users, and further improve the quality of home entertainment and life. Summary of the invention

[0003] The object of the present invention is to provide an adaptive audio device audio data processing system and method to solve the problems raised in the prior art.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] An adaptive audio device audio data processing method comprises the following steps:

[0006] Step S100: acquiring the video data of the monitoring device history and the audio data of the audio device history, and obtaining the characteristic area and the target local area in the monitoring screen according to the amplitude change in the audio data; establishing a plane coordinate system of the monitoring screen, and intercepting the front segment in the video data according to the coordinate position change of the characteristic area and the target local area;

[0007] Step S200: extracting characteristic actions in the preceding segment, obtaining video data at the current moment, comparing and analyzing the current video data with the characteristic actions in the preceding segment, and obtaining a similarity value corresponding to the current video data;

[0008] Step S300: collecting audio data at the current moment, and converting audio input information and audio output information in the audio data into a spectrum graph, thereby obtaining an environment spectrum graph;

[0009] The spectrogram is a tool that visualizes the frequency and amplitude of a sound signal over time. The frequency is the horizontal axis and the amplitude is the vertical axis. Each moment corresponds to a spectrogram. In the spectrogram of the audio output information, there is only the audio information of the sound, while in the spectrogram of the audio input information, it also includes the audio information of the human voice. Therefore, by combining and subtracting the two, we can get the environmental spectrogram.

[0010] Step S400: pre-acquire the historical environmental spectrum diagram of the audio device and the device volume change, determine the volume adjustment model of the device volume, and intelligently adjust the current volume of the audio device in combination with the similarity value.

[0011] Furthermore, step S100 includes:

[0012] Step S110: Acquire the audio data of the audio device for the past M days, the audio data including each moment, as well as the received audio information, output audio information and device volume at each moment; obtain the amplitude of the output audio information at each moment, and take the device volume at a certain moment T0 as V0. If the device volume in the time period S1 before a certain moment T0 is all V0, and the audio volume V1 at the next moment T1 after a certain moment T0 satisfies V0>V1, and the audio volume at subsequent moments decreases successively until a certain moment T0 n , and at a certain time T n The next moment T n+1 , the corresponding device volume is the same, obtain the average amplitude F1 in the time period S1, and the average amplitude F2 from a certain time T0 to a certain time T n The average amplitude F2 within;

[0013] Step S120: if F2<F1, obtain the monitoring screen of the monitoring device at a certain time T0, establish a plane coordinate system of the monitoring screen, and obtain the characteristic area of ​​the characteristic object in the monitoring screen and the local areas corresponding to the local parts of the moving object in the monitoring screen through the neural network algorithm; if the area of ​​intersection between the characteristic area and a certain local area in the monitoring screen is greater than 0, the certain local area is taken as the target local area;

[0014] In this solution, the audio device is a smart speaker, the received audio information is the audio data received by the microphone of the smart speaker, the output audio information is the audio data output by the speaker, and the device volume is the volume of the smart speaker. The reason for judging F2<F1 here is: in some cases, lowering the volume of the speaker may not only be because someone is talking, but also because the sound played by the speaker suddenly increases, and the reason for the increase is generally related to the sudden change of songs, or some sudden background music, so it is necessary to make a judgment to ensure that it is indeed caused by external factors. The audio equipment and monitoring equipment are both deployed in the living room. The characteristic item is the remote control for controlling the volume of the audio equipment. The moving object is a person. The local parts include the hands and mouth. In this step, it is the hands. Specifically: if people pick up the remote control to control the volume of the speaker, and then the volume of the speaker will become smaller, the hand that people pick up the remote control will be used as the target local area; and in general, the volume of the speaker will be adjusted only when talking or making phone calls.

[0015] Step S130: Obtain the center coordinate P of the target local area at a certain time T in the time period S1, obtain the center coordinate of the target local area at each time in the time period S2 starting from time T, and calculate the average coordinate P^, and take the center coordinate of the feature area at time T as P T ; Taking coordinate P as the starting point, P T As the end point, vector V0 is obtained. Starting from time T, until the moment when the intersection area of ​​the target local area and the feature area is not 0, the middle time period is taken as S3. According to the central coordinates of the target local area at each moment in time period S3, P T As the end point, get all vectors;

[0016] Step S140: If 1 / H<L1 / L2<H is satisfied and the angle between each vector and vector V0 is less than the angle threshold, where H is the distance coefficient, L1 is the coordinate P T The distance from coordinate P, L2 is the distance from coordinate P T The distance between the coordinate P^ and the time period S2 is taken as the pre-segment, and then all the pre-segments in the video data are obtained.

[0017] A certain moment T0 is the moment when people are just about to switch the volume of the sound system, the moment T is the moment when people are just about to approach the remote control, the time period S2 is the time period before the moment T when people are answering the phone, and the specific time length is determined according to actual conditions; the time period S3 is the moment when people are just about to approach the remote control, that is, the moment T, and the moment when they are about to touch or have touched the remote control. In this solution, there may be multiple pre-segments, and comprehensive analysis of multiple pre-segments will make the data more reliable.

[0018] Further, step S200 includes:

[0019] Step S210: Obtain all local regions in a certain preceding segment, and obtain the first local region and the second local region therein, obtain the grayscale histogram of the first local region at each moment in the preceding segment, and obtain the grayscale histogram of any two adjacent moments T according to the average grayscale of each grayscale histogram. a-1 and T a The grayscale variance of the grayscale histogram. If the grayscale variance is greater than the variance threshold, then the time T a Marking is performed. If the total duration D1 obtained by adding the marked moments is greater than the first duration threshold, the grayscale change of the first local area is taken as the first characteristic action;

[0020] Step S220: Obtain the area R of the target device in the monitoring image d , if the second local area, area R2 in the monitoring image, is d The total duration D2 of the intersecting area is greater than the area threshold and greater than the second duration threshold, and the change in the intersecting area of ​​the second local area is taken as the second characteristic action; and then, according to the total duration corresponding to all preceding segments, the first total duration average d1 and the second total duration average d2 are obtained, as well as the weight W1=d2 / (d1+d2) of the first characteristic action and the weight W2=d1 / (d1+d2) of the second characteristic action;

[0021] Step S230: Obtain the video data at the current moment. If the total duration D1 corresponding to the first local area of ​​a certain moving object ^ , and the corresponding total duration D2 of the second local area ^ , all satisfy the judgment of step S200, then the first similarity Q1=min(d1,D1 ^ ) / max(d1,D1 ^ ), the second similarity Q2 = min(d2, D2 ^ ) / max(d2,D2 ^ ), and then obtain the similarity value between the current video data and the previous segment: Q = W1*Q1+W1*Q2.

[0022] The first characteristic action is speaking, and the first local area is the mouth. When a person speaks, the grayscale value of the mouth will change; the second characteristic action is the person holding a mobile phone, and the second local area is the hand. The target device is a mobile phone. When a person is making a call, the mobile phone will be in the hand, so based on the overlap between the hand and the mobile phone, it can be judged that someone may be making a call; by comparing the current video data with each characteristic action, a similarity value can be obtained, and based on the similarity value, it can be judged whether the volume should be adjusted and how much the volume should be adjusted.

[0023] Furthermore, step S300 includes: obtaining the received audio information and the output audio information corresponding to each moment in all the preceding segments, and converting the received audio information and the output audio information into a spectrum diagram; if the difference in the amplitude of a certain frequency in the two spectrum diagrams corresponding to a certain moment is greater than the amplitude threshold, then the certain frequency at a certain moment is marked, and if the number of marks of the same certain frequency at each moment is greater than the marking number threshold, then the certain frequency is taken as the changing frequency, and the average value of the difference between the amplitudes is taken as the changing amplitude value of the changing frequency; and then an environmental spectrum diagram is established according to each changing frequency and the corresponding changing amplitude value.

[0024] Furthermore, step S400 includes:

[0025] Step S410: Obtain the device volume V0 of the audio device at a certain time T0 in step S100, and the device volume V1 of the audio device at a certain time T0. n Device volume V n , get the volume change value Y=V0-V corresponding to the audio data n ; Add the amplitude of each frequency in the spectrum diagram at a certain moment in a certain pre-segment, and normalize them to obtain the total amplitude corresponding to each moment, and obtain the amplitude average value D corresponding to a certain pre-segment; mark the moment when the total amplitude is greater than the amplitude threshold, and take the ratio of the sum of each marked moment to the total duration of a certain pre-segment as C, and obtain the environmental audio coefficient X=C*D. According to the least squares method, determine the volume adjustment model of the device volume as Y=k*X+b, where k is the slope and b is the intercept;

[0026] Generally speaking, when people are on the phone, if there is sound at all times, it means that the hands-free function is turned on. When the hands-free function is turned on, the voice is relatively loud, so there is no need to deliberately reduce the volume of the speaker. Therefore, in this case, the volume change of the speaker should be small. If there is not sound at all times, if the volume is adjusted to a small level, some information may be missed. In order to be able to hear the sound normally, the volume change at this time should be large.

[0027] Step S420: If the total duration D1 corresponding to the current time ^ and D2 ^ Meet D1 ^ >d1 and D2 ^ >d2, then get D1 ^ ∪D2 ^ The time period is taken as the monitoring period, and the environmental audio coefficient X corresponding to the monitoring period is obtained according to the corresponding calculation process of a certain preceding segment in step S410. ^ , and then according to the slope k and intercept b, as well as the similarity value Q, the current volume adjustment value of the audio device is obtained Z = G*Q*(k*X ^+b), G is the adjustment coefficient, if Z<Z0, let Z=Z0, Z0 is the adjustment threshold.

[0028] An adaptive audio data processing system for an audio device includes a module for intercepting a pre-segment, a module for obtaining a similarity value, a module for obtaining an environmental spectrum diagram, and a module for adjusting the volume of an audio device;

[0029] The module for capturing the front segment is used to obtain the historical video data of the monitoring device and the historical audio data of the audio device, and obtain the characteristic area and the target local area in the monitoring screen according to the amplitude change in the audio data; establish the plane coordinate system of the monitoring screen, and capture the front segment in the video data according to the coordinate position change of the characteristic area and the target local area;

[0030] A module for obtaining a similarity value is used to extract the characteristic action in the preceding segment, obtain the video data at the current moment, compare and analyze the current video data with the characteristic action in the preceding segment, and obtain the similarity value corresponding to the current video data;

[0031] Obtaining an environmental spectrum graph module: used to collect audio data at the current moment, and convert the audio input information and audio output information in the audio data into a spectrum graph, thereby obtaining an environmental spectrum graph;

[0032] Audio device volume adjustment module: used to obtain the historical environmental spectrum of the audio device and the device volume changes in advance, determine the volume adjustment model of the device volume, and intelligently adjust the current volume of the audio device based on the similarity value.

[0033] Further, the module for intercepting the preceding segment includes an amplitude analysis unit, a unit for obtaining a target local area, a unit for obtaining a vector, and a unit for intercepting the preceding segment;

[0034] Amplitude analysis unit: used to obtain audio data of the audio device, including each moment, as well as the received audio information, output audio information and device volume at each moment; obtain the amplitude of the output audio information at each moment, and obtain the average amplitude of the set time period;

[0035] Obtaining target local area unit: used to obtain the monitoring screen, establish the plane coordinate system of the monitoring screen, obtain the characteristic area of ​​the characteristic object in the monitoring screen, and each local part of the moving object, the corresponding local areas in the monitoring screen, and the target local area;

[0036] Vector obtaining unit: used to obtain the center coordinates of the local target area, and obtain all vectors according to the center coordinates of the feature area and the changes of each coordinate;

[0037] The pre-fragment extraction unit is used to make a judgment and obtain all pre-fragments in the video data.

[0038] Further, the module for adjusting the volume of the audio device includes obtaining a volume adjustment model unit and a volume change value calculation unit;

[0039] Obtaining a volume adjustment model unit: used to obtain a volume change value corresponding to the audio data and an environmental audio coefficient, and determine a volume adjustment model of the device volume according to a least squares method;

[0040] Volume change value calculation unit: used to obtain the monitoring period and the environmental audio coefficient corresponding to the monitoring period, and then obtain the current volume adjustment value of the audio device.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention provides an adaptive audio data processing system and method for audio equipment, including: obtaining the historical video data of the monitoring equipment and the historical audio data of the audio equipment, obtaining the characteristic area and the target local area, and intercepting the front segment in the video data; obtaining the video data at the current moment, and obtaining the similarity value corresponding to the current video data; collecting the audio data at the current moment, and then obtaining the environmental spectrum diagram; determining the volume adjustment model of the device volume, and combining the similarity value to intelligently adjust the current volume of the audio equipment. The present invention obtains the device volume that should be set at the current moment through the historical video data and audio data, avoids the excessive sound of the sound affecting normal calls, brings convenience and comfort to users, makes home audio equipment more in line with the actual needs of users, and improves the quality of home entertainment and life. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic flow chart of an adaptive audio device audio data processing method of the present invention;

[0043] Figure 2 The present invention is a structural diagram of an adaptive audio device audio data processing system. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Example: Figure 1 As shown, the present invention provides an adaptive audio device audio data processing system and method technical solution, comprising the following steps:

[0046] Step S100: Obtain the historical video data of the monitoring device and the historical audio data of the audio device, and obtain the characteristic area and the target local area in the monitoring screen according to the amplitude change in the audio data; establish a plane coordinate system of the monitoring screen, and intercept the front segment in the video data according to the coordinate position change of the characteristic area and the target local area.

[0047] Step S110: Acquire the audio data of the audio device for the past M days, the audio data including each moment, as well as the received audio information, output audio information and device volume at each moment; obtain the amplitude of the output audio information at each moment, and take the device volume at a certain moment T0 as V0. If the device volume in the time period S1 before a certain moment T0 is all V0, and the audio volume V1 at the next moment T1 after a certain moment T0 satisfies V0>V1, and the audio volume at subsequent moments decreases successively until a certain moment T0 n , and at a certain time T n The next moment T n+1 , the corresponding device volume is the same, obtain the average amplitude F1 in the time period S1, and the average amplitude F2 from a certain time T0 to a certain time T n The average amplitude F2 within.

[0048] Step S120: If F2<F1, obtain the monitoring screen of the monitoring device at a certain time T0, establish a plane coordinate system of the monitoring screen, and use a neural network algorithm to obtain the characteristic areas of the characteristic objects in the monitoring screen, as well as the local areas corresponding to the local parts of the moving object in the monitoring screen; if the area of ​​intersection between the characteristic area and a local area in the monitoring screen is greater than 0, then the local area is used as the target local area.

[0049] In this solution, the audio device is a smart speaker, the received audio information is the audio data received by the microphone of the smart speaker, the output audio information is the audio data output by the speaker, and the device volume is the volume of the smart speaker. The reason for judging F2 < F1 here is: in some cases, lowering the volume of the speaker may not only be because someone is talking, but also because the sound played by the speaker suddenly increases. The reason for the increase is generally related to the sudden switching of songs or some sudden background music, so it is necessary to make a judgment to ensure that it is indeed caused by external factors. The audio device and the monitoring device are both deployed in the living room. The characteristic item is the remote control for controlling the volume of the audio device. The moving object is a person. The local parts include the hands and the mouth. In this step, it is the hands. Specifically, if people pick up the remote control to control the volume of the audio, and then the volume of the audio will decrease, the hand of the person picking up the remote control is used as the target local area; and in general, the volume of the audio will be adjusted only when talking or making a phone call.

[0050] Step S130: Obtain the center coordinate P of the target local area at a certain time T in the time period S1, obtain the center coordinate of the target local area at each time in the time period S2 starting from time T, and calculate the average coordinate P^, and take the center coordinate of the feature area at time T as P T ; Taking coordinate P as the starting point, P T As the end point, vector V0 is obtained. Starting from time T, until the moment when the intersection area of ​​the target local area and the feature area is not 0, the middle time period is taken as S3. According to the central coordinates of the target local area at each moment in time period S3, P T As the end point, get all vectors.

[0051] Step S140: If 1 / H<L1 / L2<H is satisfied and the angle between each vector and vector V0 is less than the angle threshold, where H is the distance coefficient, L1 is the coordinate P T The distance from coordinate P, L2 is the distance from coordinate P T and the distance between the coordinate P^, then the time period S2 is taken as the pre-segment, and then all the pre-segments in the video data are obtained.

[0052] A certain moment T0 is the moment when people are just about to switch the volume of the audio system, the moment T is the moment when people are just about to approach the remote control, the time period S2 is the time period before the moment T when people are answering the phone, and the specific time length is determined according to the actual situation; the time period S3 is the moment when people are just about to approach the remote control, that is, the moment T, and the moment when they are about to touch or have touched the remote control. In this solution, there may be multiple pre-segments, and comprehensive analysis of multiple pre-segments will make the data more reliable. The distance coefficient H is adjusted according to the actual situation, and is 1.2 in this embodiment.

[0053] Step S200: extracting characteristic actions in the preceding segment, obtaining video data at the current moment, comparing and analyzing the current video data with the characteristic actions in the preceding segment, and obtaining a similarity value corresponding to the current video data.

[0054] Step S210: Obtain all local regions in a certain preceding segment, and obtain the first local region and the second local region therein, obtain the grayscale histogram of the first local region at each moment in the preceding segment, and obtain the grayscale histogram of any two adjacent moments T according to the average grayscale of each grayscale histogram. a-1 and T a The grayscale variance of the grayscale histogram. If the grayscale variance is greater than the variance threshold, then the time T a Marking is performed. If the total duration D1 obtained by adding the marking moments is greater than the first duration threshold, the grayscale change of the first local area is taken as the first characteristic action.

[0055] Step S220: Obtain the area R of the target device in the monitoring image d , if the second local area of ​​a moving object is in area R2 in the monitoring image, and is d The time duration D2 of the intersection area is greater than the area threshold and greater than the second time duration threshold. The total time duration D2 is taken as the second time period, and the change in the intersection area of ​​the second local area is taken as the second characteristic action. The weight of the first characteristic action is obtained as W1=D2 / (D1+D2), and the weight of the second characteristic action is obtained as W2=D1 / (D1+D2).

[0056] Step S230: Obtain the video data at the current moment. If the grayscale change of the first local area of ​​the target moving object and the change of the intersection area of ​​the second local area meet the judgment of step S200, the first time period TF1 and the second time period TF2 corresponding to the target moving object are obtained, and the first similarity Q1=min(D1,TF1) / max(D1,TF1) and the first similarity Q2=min(D2,TF2) / max(D2,TF2) are obtained, and then the similarity value between the current video data and the previous segment is obtained: Q=W1*Q1+W1*Q2.

[0057] The first characteristic action is speaking, and the first local area is the mouth. When a person speaks, the grayscale value of the mouth will change; the second characteristic action is the person holding a mobile phone, and the second local area is the hand. The target device is a mobile phone. When a person is making a call, the mobile phone will be in the hand, so based on the overlap between the hand and the mobile phone, it can be judged that someone may be making a call; by comparing the current video data with each characteristic action, a similarity value can be obtained, and based on the similarity value, it can be judged whether the volume should be adjusted and how much the volume should be adjusted.

[0058] Step S300: collect audio data at the current moment, and convert audio input information and audio output information in the audio data into a spectrum graph, thereby obtaining an environment spectrum graph.

[0059] The received audio information and the output audio information corresponding to each moment in all the preceding segments are obtained, and both the received audio information and the output audio information are converted into a spectrum diagram; if the difference in the amplitude of a certain frequency in the two spectrum diagrams corresponding to a certain moment is greater than the amplitude threshold, the certain frequency at a certain moment is marked, and if the number of marked frequencies at each moment is greater than the marking number threshold, the certain frequency is taken as the changing frequency, and the average value of the difference between the amplitudes is taken as the changing amplitude value of the changing frequency; and then an environmental spectrum diagram is established according to each changing frequency and the corresponding changing amplitude value.

[0060] A spectrogram is a tool that visualizes how the frequency and amplitude of a sound signal change over time, with frequency as the horizontal axis and amplitude as the vertical axis. Each moment corresponds to a spectrogram. In the spectrogram of audio output information, there is only audio information of the sound system, while in the spectrogram of audio input information, it also includes audio information of the human voice. Therefore, by combining and subtracting the two, an environmental spectrogram can be obtained.

[0061] Step S400: pre-acquire the historical environmental spectrum diagram of the audio device and the device volume change, determine the volume adjustment model of the device volume, and intelligently adjust the current volume of the audio device in combination with the similarity value.

[0062] Step S410: Obtain the device volume V0 of the audio device at a certain time T0 in step S100, and the device volume V1 of the audio device at a certain time T0. n Device volume V n , get the volume change value Y=V0-V corresponding to the audio data n ; Add the amplitude of each frequency in the spectrum diagram at a certain moment in a certain pre-segment, and normalize them to obtain the total amplitude corresponding to each moment, and obtain the amplitude average value D corresponding to a certain pre-segment; mark the moment when the total amplitude is greater than the amplitude threshold, and take the ratio of the sum of each marked moment to the total duration of a certain pre-segment as C, and obtain the environmental audio coefficient X=C*D. According to the least squares method, determine that the volume adjustment model of the device volume is Y=k*X+b, where k is the slope and b is the intercept.

[0063] Generally speaking, when people are on the phone, if there is sound at all times, it means that the hands-free function is turned on. When the hands-free function is turned on, the speaking voice is relatively loud, and there is no need to deliberately reduce the sound of the speaker. Therefore, in this case, the volume change of the speaker should be small. If there is not sound at all times, if the volume is adjusted to a low level, some information may be missed. In order to hear the sound normally, the volume change at this time should be large.

[0064] Step S420: If the total duration D1 corresponding to the current time ^ and D2 ^ Meet D1 ^ >d1 and D2 ^ >d2, then get D1 ^ ∪D2 ^ The time period is taken as the monitoring period, and the environmental audio coefficient X corresponding to the monitoring period is obtained according to the corresponding calculation process of a certain preceding segment in step S410. ^ , and then according to the slope k and intercept b, as well as the similarity value Q, the current volume adjustment value of the audio device is obtained Z = G*Q*(k*X ^ +b), G is the adjustment coefficient, if Z<Z0, let Z=Z0, Z0 is the adjustment threshold.

[0065] Environmental audio coefficient X during monitoring period ^ The calculation method is the same as that in step S410, which is to first calculate the amplitude average, then calculate the ratio, and then multiply them to obtain the ambient audio coefficient X ^ In this embodiment, the adjustment threshold Z0 is the current audio volume multiplied by 0.2.

[0066] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. An adaptive audio device audio data processing method, characterized in that: The following steps are involved: Step S100: acquiring the video data of the monitoring device history and the audio data of the audio device history, and obtaining the characteristic area and the target local area in the monitoring screen according to the amplitude change in the audio data; establishing a plane coordinate system of the monitoring screen, and intercepting the front segment in the video data according to the coordinate position change of the characteristic area and the target local area; Step S200: extracting characteristic actions in the preceding segment, obtaining video data at the current moment, comparing and analyzing the current video data with the characteristic actions in the preceding segment, and obtaining a similarity value corresponding to the current video data; Step S300: collecting audio data at the current moment, and converting audio input information and audio output information in the audio data into a spectrum graph, thereby obtaining an environment spectrum graph; Step S400: pre-acquire the historical environmental spectrum diagram of the audio device and the device volume change, determine the volume adjustment model of the device volume, and intelligently adjust the current volume of the audio device in combination with the similarity value.

2. The method for processing audio data of an adaptive audio device according to claim 1, characterized in that: Step S100 includes: Step S110: Acquire the audio data of the audio device for the past M days, wherein the audio data includes each moment, as well as the received audio information, the output audio information and the device volume at each moment; acquire the amplitude of the output audio information at each moment, and take the device volume at a certain moment T0 as V0. If the device volume in the time period S1 before a certain moment T0 is all V0, and the audio volume V1 at the next moment T1 after a certain moment T0 satisfies V0>V1, and the audio volume at subsequent moments decreases successively until a certain moment T0 n , and at a certain time T n The next moment T n+1 , the corresponding device volume is the same, obtain the average amplitude F1 in the time period S1, and the average amplitude F2 from a certain time T0 to a certain time T n The average amplitude F2 within; Step S120: if F2<F1, obtain the monitoring screen of the monitoring device at a certain time T0, establish a plane coordinate system of the monitoring screen, and obtain the characteristic area of ​​the characteristic object in the monitoring screen and the local areas corresponding to the local parts of the moving object in the monitoring screen through a neural network algorithm; if the area of ​​intersection between the characteristic area and a certain local area in the monitoring screen is greater than 0, take the certain local area as the target local area; Step S130: Obtain the center coordinate P of the target local area at a certain time T in the time period S1, obtain the center coordinate of the target local area at each time in the time period S2 starting from time T, and calculate the average coordinate P^, and take the center coordinate of the feature area at time T as P T ; Taking coordinate P as the starting point, P T As the end point, vector V0 is obtained. Starting from time T, until the moment when the intersection area of ​​the target local area and the feature area is not 0, the middle time period is taken as S3. According to the central coordinates of the target local area at each moment in time period S3, P T As the end point, get all vectors; Step S140: If 1 / H<L1 / L2<H is satisfied and the angle between each vector and vector V0 is less than the angle threshold, where H is the distance coefficient, L1 is the coordinate P T The distance from coordinate P, L2 is the distance from coordinate P T and the distance between the coordinate P^, then the time period S2 is taken as the pre-segment, and then all the pre-segments in the video data are obtained.

3. The method for processing audio data of an adaptive audio device according to claim 2, characterized in that: Step S200 includes: Step S210: Obtain all local regions in a certain preceding segment, and obtain the first local region and the second local region therein, obtain the grayscale histogram of the first local region at each moment in the preceding segment, and obtain the grayscale histogram of any two adjacent moments T according to the average grayscale of each grayscale histogram. a-1 and T a The grayscale variance of the grayscale histogram is calculated. If the grayscale variance is greater than the variance threshold, the time T a Marking is performed. If the total duration D1 obtained by adding the marked moments is greater than the first duration threshold, the grayscale change of the first local area is taken as the first characteristic action; Step S220: Obtain the area R of the target device in the monitoring image d , if the second local area, the area R2 in the monitoring image, is d The total duration D2 of the intersecting area is greater than the area threshold and greater than the second duration threshold, and the change in the intersecting area of ​​the second local area is taken as the second characteristic action; and then, according to the total duration corresponding to all preceding segments, the first total duration average d1 and the second total duration average d2 are obtained, as well as the weight W1=d2 / (d1+d2) of the first characteristic action and the weight W2=d1 / (d1+d2) of the second characteristic action; Step S230: Obtain the video data at the current moment. If the total duration D1 corresponding to the first local area of ​​a certain moving object ^ , and the corresponding total duration D2 of the second local area ^ , all satisfy the judgment of step S200, then the first similarity Q1=min(d1,D1 ^ ) / max(d1,D1 ^ ), the second similarity Q2 = min(d2, D2 ^ ) / max(d2,D2 ^ ), and then obtain the similarity value between the current video data and the previous segment: Q = W1*Q1+W1*Q2.

4. The method for processing audio data of an adaptive audio device according to claim 3, characterized in that: Step S300 includes: obtaining the received audio information and the output audio information corresponding to each moment in all the preceding segments, and converting the received audio information and the output audio information into a spectrum diagram; if the difference in the amplitude of a certain frequency in the two spectrum diagrams corresponding to a certain moment is greater than the amplitude threshold, then a certain frequency at a certain moment is marked, and if the number of marks of the same certain frequency at each moment is greater than the marking number threshold, then the certain frequency is used as the changing frequency, and the average value of the difference between the amplitudes is used as the changing amplitude value of the changing frequency; and then an environmental spectrum diagram is established according to each changing frequency and the corresponding changing amplitude value.

5. The method for processing audio data of an adaptive audio device according to claim 4, characterized in that: Step S400 includes: Step S410: Obtain the device volume V0 of the audio device at a certain time T0 in step S100, and the device volume V1 of the audio device at a certain time T0. n Device volume V n , get the volume change value Y=V0-V corresponding to the audio data n ; Add the amplitude of each frequency in the spectrum diagram at a certain moment in a certain pre-segment, and normalize them to obtain the total amplitude corresponding to each moment, and obtain the amplitude average value D corresponding to a certain pre-segment; mark the moment when the total amplitude is greater than the amplitude threshold, and take the ratio of the sum of each marked moment to the total duration of the certain pre-segment as C, and obtain the environmental audio coefficient X=C*D. According to the least squares method, determine that the volume adjustment model of the device volume is Y=k*X+b, where k is the slope and b is the intercept; Step S420: If the total duration D1 corresponding to the current time ^ and D2 ^ Meet D1 ^ >d1 and D2 ^ >d2, then get D1 ^ ∪D2 ^ The time period is taken as the monitoring period, and the environmental audio coefficient X corresponding to the monitoring period is obtained according to the corresponding calculation process of a certain preceding segment in step S410. ^ , and then according to the slope k and intercept b, as well as the similarity value Q, the current volume adjustment value of the audio device is obtained Z = G*Q*(k*X ^ +b), G is the adjustment coefficient, if Z<Z0, let Z=Z0, Z0 is the adjustment threshold.

6. An audio device audio data processing system, used to execute an adaptive audio device audio data processing method according to any one of claims 1 to 5, characterized in that: The system includes a module for intercepting a pre-segment, a module for obtaining a similarity value, a module for obtaining an environmental spectrum diagram, and a module for adjusting the volume of an audio device; The module for capturing the front segment is used to obtain the historical video data of the monitoring device and the historical audio data of the audio device, and obtain the characteristic area and the target local area in the monitoring screen according to the amplitude change in the audio data; establish the plane coordinate system of the monitoring screen, and capture the front segment in the video data according to the coordinate position change of the characteristic area and the target local area; A module for obtaining a similarity value is used to extract the characteristic action in the preceding segment, obtain the video data at the current moment, compare and analyze the current video data with the characteristic action in the preceding segment, and obtain the similarity value corresponding to the current video data; Obtaining an environmental spectrum graph module: used to collect audio data at the current moment, and convert the audio input information and audio output information in the audio data into a spectrum graph, thereby obtaining an environmental spectrum graph; The audio device volume adjustment module is used to obtain the historical environmental spectrum diagram of the audio device and the device volume change in advance, determine the volume adjustment model of the device volume, and intelligently adjust the current volume of the audio device in combination with the similarity value.

7. The audio data processing system for an audio device according to claim 6, characterized in that: The module for intercepting the preceding segment includes an amplitude analysis unit, a unit for obtaining a target local area, a unit for obtaining a vector, and a unit for intercepting the preceding segment; Amplitude analysis unit: used to obtain audio data of the audio device, including each moment, as well as the received audio information, output audio information and device volume at each moment; obtain the amplitude of the output audio information at each moment, and obtain the average amplitude of the set time period; Obtaining target local area unit: used to obtain the monitoring screen, establish the plane coordinate system of the monitoring screen, obtain the characteristic area of ​​the characteristic object in the monitoring screen, and each local part of the moving object, the corresponding local areas in the monitoring screen, and the target local area; Vector obtaining unit: used to obtain the center coordinates of the local target area, and obtain all vectors according to the center coordinates of the feature area and the changes of each coordinate; The pre-fragment extraction unit is used to make a judgment and obtain all pre-fragments in the video data.

8. The audio data processing system for an audio device according to claim 7, characterized in that: The module for adjusting the volume of the audio device includes a volume adjustment model obtaining unit and a volume change value calculating unit; Obtaining a volume adjustment model unit: used to obtain a volume change value corresponding to the audio data and an environmental audio coefficient, and determine a volume adjustment model of the device volume according to a least squares method; Volume change value calculation unit: used to obtain the monitoring period and the environmental audio coefficient corresponding to the monitoring period, and then obtain the current volume adjustment value of the audio device.

Images