Data processing method, device, equipment and storage medium for monitoring equipment

Abstract

The present application belongs to the technical field of monitoring equipment, and specifically relates to a data processing method, device, equipment and storage medium for monitoring equipment. The method includes: in response to the sound source making a sound, obtaining the distance information between the sound source and the monitoring equipment; determining the sound pickup mode corresponding to the distance information; processing the original audio data collected by the audio acquisition component according to the sound pickup mode; determining the merge mode of the processed audio data; when the merge mode is an offline mode, performing merge processing to obtain the monitoring data to be played; because different sound pickup modes are determined by determining the distance information between the sound source and the monitoring equipment, audio data processing can be carried out in a targeted manner according to different distances, avoiding the problem of inconsistency of processed audio data when the position of the sound source changes, and then the video data and audio data can be merged offline through the merge mode, so that the processed monitoring data can be guaranteed to have a better playback effect.

Description

[Technical field]

[0001] The present application belongs to the technical field of monitoring equipment, and specifically relates to a data processing method, device, equipment and storage medium for monitoring equipment. [Background technology]

[0002] Currently, monitoring equipment needs to process the collected audio data and video data so that the audio data and video data are synchronized and the audio data is clear and free of noise.

[0003] The traditional process of processing audio data includes: performing gain amplification processing on the audio data collected by the audio acquisition component according to a preset gain value; processing the audio data after the gain amplification processing based on a beamforming algorithm to obtain processed audio data; and performing noise suppression on the processed audio data based on a noise suppression algorithm to complete the processing of the audio data.

[0004] However, since the position of the target sound source within the acquisition range of the monitoring device may change, the audio data obtained using the same processing method may not be effective. [Summary of the invention]

[0005] This embodiment provides a data processing method, device, equipment and storage medium for monitoring equipment, which can solve the problem that the audio data obtained using the same processing method may not be effective because the position of the target sound source within the acquisition range of the monitoring equipment may change. This application provides the following technical solutions:

[0006] In a first aspect, a data processing method for a monitoring device is provided, wherein the monitoring device includes an image acquisition component, and the monitoring device is also connected to an audio acquisition component, and the method includes:

[0007] In response to a target sound source making a sound, controlling the image acquisition component to acquire a monitoring image of the target sound source and obtaining distance information between the target sound source and the monitoring device;

[0008] Determine a sound pickup mode corresponding to the distance information; the sound pickup modes include at least two, and different sound pickup modes are adapted to different distance information;

[0009] Processing the original audio data collected by the audio collection component according to the sound pickup mode to obtain processed audio data;

[0010] Determine a merge mode of the processed audio data, wherein the merge mode includes an offline mode and a real-time mode;

[0011] When the merge mode is an offline mode, after obtaining the offline monitoring file composed of each frame of monitoring image, the processed audio data is merged with the offline monitoring file in response to a play request to obtain the monitoring data to be played.

[0012] Optionally, each sound pickup pattern is determined based on a sound pickup algorithm, and the determining of the sound pickup pattern corresponding to the distance information includes:

[0013] Determining algorithm parameters of the sound pickup algorithm based on the distance information;

[0014] The sound pickup pattern is determined based on a result output by a sound pickup algorithm having the algorithm parameters.

[0015] Optionally, it is characterized in that the sound pickup algorithm includes an audio amplification algorithm, a beamforming algorithm and a noise suppression algorithm, and the algorithm parameters of the sound pickup algorithm are determined based on the distance information, including:

[0016] Determine a gain value in an audio amplification algorithm corresponding to the distance information;

[0017] Determine a basic unit parameter value of a phase array in a beamforming algorithm corresponding to the distance information;

[0018] Determine a noise reduction parameter in a noise suppression algorithm corresponding to the distance information.

[0019] Optionally, the monitoring device is equipped with a distance sensor component, and the acquisition posture of the monitoring device is adjustable;

[0020] In response to the target sound source making a sound, controlling the image acquisition component to acquire a monitoring image of the target sound source and obtaining distance information between the target sound source and the monitoring device includes:

[0021] In response to a target sound source making a sound, determining whether the sound made by the target sound source is an abnormal sound using initial audio data collected by the audio collection component;

[0022] In the case where the sound emitted by the target sound source is an abnormal sound, determining a direction of arrival using the initial audio data to obtain a target area where the target sound source is located;

[0023] Control the monitoring device to operate until the monitoring range covers the target area, and control the distance sensor component to collect data from the target area;

[0024] The distance information collected by the distance sensor is obtained.

[0025] Optionally, determining whether the sound emitted by the target sound source is an abnormal sound includes:

[0026] Obtaining a standard volume range of the current monitoring scene of the monitoring device; and determining that the sound emitted by the target sound source is an abnormal sound when the volume of the initial audio data does not fall within the standard volume range;

[0027] and / or,

[0028] Acquire the sound type of the current monitoring scene of the monitoring device; and when the type of the initial audio data is different from the sound type, determine that the sound emitted by the target sound source is an abnormal sound.

[0029] Optionally, determining a track combining mode of the processed audio data includes:

[0030] When the distance information belongs to the first distance range, the track merging mode is determined to be an offline mode; when the distance information belongs to the second distance range, the track merging mode is determined to be a real-time mode; each distance value in the first distance range is greater than each distance value in the second distance range;

[0031] or,

[0032] receiving a setting operation for the track merging mode; if the setting operation indicates an offline mode, determining the track merging mode to be the offline mode; if the setting operation indicates a real-time mode, determining the track merging mode to be the real-time mode;

[0033] or,

[0034] Obtain an output scene type of monitoring data; when the output scene type is a low-latency scene type, determine that the merge mode is an offline mode; when the output scene type is not the delay scene type, determine that the merge mode is a real-time mode.

[0035] Optionally, the method further comprises:

[0036] When the merging mode is a real-time mode, each frame of monitoring image is merged with the processed audio data to obtain monitoring data to be played.

[0037] In a second aspect, a data processing device for a monitoring device is provided, the device comprising:

[0038] A sound source response module, used to control the image acquisition component to acquire a monitoring image of the target sound source in response to the sound of the target sound source, and to obtain distance information between the audio acquisition component and the monitoring device;

[0039] A first determination module is used to determine a sound pickup mode corresponding to the distance information; the sound pickup modes include at least two, and different sound pickup modes are adapted to different distance information;

[0040] A data processing module, used for processing the original audio data collected by the audio collection component according to the sound pickup mode to obtain processed audio data;

[0041] A second determination module is used to determine a merge mode of the processed audio data, wherein the merge mode includes an offline mode and a real-time mode;

[0042] The merging processing module is used to merge the processed audio data with the offline monitoring file in response to a play request after obtaining the offline monitoring file composed of each frame of monitoring image when the merging mode is the offline mode, so as to obtain the monitoring data to be played.

[0043] In a third aspect, an electronic device is provided, the device comprising a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement a data processing method for a monitoring device as described in the first aspect.

[0044] In a fourth aspect, a computer-readable storage medium is provided, wherein a program is stored in the storage medium, and when the program is executed by a processor, it is used to implement a data processing method of a monitoring device as described in the first aspect.

[0045] The beneficial effects of the present application are as follows: by responding to the sound of the target sound source, controlling the image acquisition component to acquire the monitoring image of the target sound source, and acquiring the distance information between the target sound source and the monitoring device; determining the sound pickup mode corresponding to the distance information; the sound pickup mode includes at least two types, and different sound pickup modes are adapted to different distance information; processing the original audio data collected by the audio acquisition component according to the sound pickup mode to obtain processed audio data; determining the merging mode of the processed audio data, the merging mode includes an offline mode and a real-time mode; when the merging mode is the offline mode, after obtaining the offline monitoring file composed of each frame of monitoring image, merging the processed audio data with the offline monitoring file in response to a play request to obtain the audio data to be played. Monitoring data; it can solve the problem that the audio data obtained using the same processing method may not be effective because the position of the target sound source within the acquisition range of the monitoring device may change; on the one hand, different sound pickup modes are determined by determining the distance information between the target sound source and the monitoring device, so that the audio data can be processed in a targeted manner according to different distances, avoiding the problem of inconsistency of the processed audio data heard by the human ear when the position of the target sound source changes; on the other hand, the video data and audio data can be merged through the merge mode in the offline mode, thus avoiding the delay between the merged audio and video data and the display screen, thereby ensuring that the processed monitoring data has a better playback effect.

[0046] In addition, since the audio acquisition component will be affected by the distance when collecting audio data, resulting in delayed collection, that is, the longer the distance, the greater the delay in the audio acquisition component collecting audio data. At this time, if real-time merging is used to process the audio and video data, it will cause a large delay between the audio and video data after merging and the actual picture. Based on the above technical problems, in this embodiment, when the distance information belongs to the first distance range, the merging mode is determined to be an offline mode; when the distance information belongs to the second distance range, the merging mode is determined to be a real-time mode; each distance value in the first distance range is greater than each distance value in the second distance range. In this way, the merging mode can be used in a targeted manner to avoid the problem of a large delay between the audio and video data after merging and the actual picture.

Brief Description of the Drawings

[0047] Figure 1 is a schematic diagram of the structure of a monitoring device provided by an embodiment of the present application;

[0048] Figure 2 is a flowchart of a monitoring device audio data processing method provided by an embodiment of the present application;

[0049] Figure 3 is a flowchart of a monitoring device audio data processing method provided by another embodiment of the present application;

[0050] Figure 4 It is a block diagram of an audio data processing device for monitoring equipment provided by an embodiment of the present application.

[0051] Figure 5 It is a block diagram of an electronic device provided by an embodiment of the present application. [Specific implementation method]

[0052] The technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0054] In the application, unless otherwise specified, directional words such as "up, down, top, bottom" are usually used with reference to the directions shown in the drawings, or with reference to the components themselves in the vertical, perpendicular or gravity directions; similarly, for ease of understanding and description, "inside and outside" refer to the inside and outside relative to the outline of each component itself, but the above directional words are not used to limit the present application.

[0055] The technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0056] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0057] In the application, unless otherwise specified, directional words such as "up, down, top, bottom" are usually used with reference to the directions shown in the drawings, or with reference to the components themselves in the vertical, perpendicular or gravity directions; similarly, for ease of understanding and description, "inside and outside" refer to the inside and outside relative to the outline of each component itself, but the above directional words are not used to limit the present application.

[0058] like Figure 1The figure is a schematic diagram of the structure of a monitoring device provided by an embodiment of the present application. The monitoring device 110 includes at least an image acquisition component 111. The monitoring device 110 is also connected to an audio acquisition component 120 and to a server 130.

[0059] The image acquisition component 111 is used to acquire monitoring images of the target sound source. Optionally, the image acquisition component 111 can be a camera equipped with a color system (Red Green Blue, RGB) detection function, a (Time Of Flight, TOF) camera, or a (charge coupled device camera, CCD) camera, etc. This embodiment does not limit the type of the image acquisition component 111.

[0060] Optionally, the target sound source refers to an object that is emitting sound or a user that is taking action within the monitoring range of the monitoring device. This embodiment does not limit the type of the target sound source.

[0061] The audio collection component 120 is connected to the monitoring device 110 and is used to collect audio data of the target sound source. Optionally, the audio collection component 120 can be a microphone array.

[0062] Optionally, when the audio collection component 120 is a microphone array, the number of microphones in the microphone array may be at least two, and this embodiment does not limit the number of microphones in the microphone array.

[0063] Optionally, the audio data includes but is not limited to: audio coding, audio bit stream, number of audio channels, number of audio quantization bits, audio sampling frequency and bit rate, etc. This embodiment does not limit the type of audio data.

[0064] The monitoring device 110 is connected to the server 130 and is used to send the data collected by the image collection component 111 and the audio collection component 120 to the server 130 .

[0065] In this embodiment, the server 130 is used to: respond to the sound of the target sound source, control the image acquisition component to collect the monitoring image of the target sound source, and obtain the distance information between the audio acquisition component and the monitoring device; determine the sound pickup mode corresponding to the distance information; the sound pickup mode includes at least two types, and different sound pickup modes are adapted to different distance information; process the original audio data collected by the audio acquisition component according to the sound pickup mode to obtain processed audio data; determine the merging mode of the processed audio data, and the merging mode includes an offline mode and a real-time mode; when the merging mode is the offline mode, after obtaining the offline monitoring file composed of each frame of monitoring image, the processed audio data is merged with the offline monitoring file in response to the play request to obtain the monitoring data to be played.

[0066] Optionally, in order to determine the distance information between the target sound source and the monitoring device, the monitoring device is also installed with a distance sensor component 112.

[0067] Optionally, the distance sensor component 112 may be a millimeter wave radar sensor or an infrared radar sensor, etc. This embodiment does not limit the type of the distance sensor component 112 .

[0068] In this embodiment, on the one hand, different sound pickup modes are determined by determining the distance information between the target sound source and the monitoring device, so that the audio data can be processed in a targeted manner according to different distances, avoiding the problem of consistency of processed audio data heard by the human ear when the position of the target sound source changes. On the other hand, the video data and audio data can be merged through the merging mode in the offline mode, thus avoiding the delay between the merged audio and video data and the display screen, thereby ensuring that the processed monitoring data has a better playback effect.

[0069] This embodiment provides a data processing method for a monitoring device, such as Figure 2 In this embodiment, the method is used for a server connected to a monitoring device as an example for explanation. In actual implementation, the method can also be used for other devices, which are connected to the server for communication. The other devices can be remote controllers, mobile phones, tablet computers, wearable devices, and other devices with processing capabilities. This embodiment does not limit the device types of other devices.

[0070] The method includes at least the following steps:

[0071] Step 201, in response to a target sound source making a sound, controlling the image acquisition component to acquire a monitoring image of the target sound source, and acquiring distance information between the audio acquisition component and the monitoring device;

[0072] Optionally, the monitoring device is equipped with a distance sensor component, and the collection posture of the monitoring device is adjustable.

[0073] Accordingly, in response to the target sound source making a sound, obtaining the distance information between the audio collection component and the monitoring device includes at least the following steps S1 to S4:

[0074] Step S1: In response to a target sound source making a sound, using initial audio data collected by an audio collection component to determine whether the sound made by the target sound source is an abnormal sound.

[0075] The initial audio data refers to the unprocessed audio data collected by the audio collection component.

[0076] In one example, determining whether a sound emitted by a target sound source is an abnormal sound includes: obtaining a standard volume range of a current monitoring scene of a monitoring device; and determining that the sound emitted by the target sound source is an abnormal sound when the volume of initial audio data does not fall within the standard volume range.

[0077] Optionally, a method for obtaining the standard volume range of the current monitoring scene of the monitoring device includes but is not limited to one of the following:

[0078] The first method is to obtain the standard volume range pre-stored in the server.

[0079] The second method is to obtain the standard volume range sent by other devices connected to the server for communication. The other devices may be a remote control, a mobile phone, a tablet computer, a wearable device, etc. This embodiment does not limit the device type of the other devices.

[0080] In actual implementation, the standard volume range may be obtained in other ways, and this embodiment does not limit the way to obtain the standard volume range.

[0081] For example: determine that the standard volume range of the current monitoring scene is 30 decibels to 50 decibels. If it is determined that the decibel of the initial audio data collected by the audio acquisition component at this time is 60 decibels, then the sound emitted by the target sound source is determined to be an abnormal sound; if it is determined that the decibel of the initial audio data collected by the audio acquisition component at this time is 40 decibels, then the sound emitted by the target sound source is determined to be a non-abnormal sound.

[0082] In another example, determining whether a sound emitted by a target sound source is an abnormal sound includes: obtaining a sound type of a current monitoring scene of a monitoring device; and determining that the sound emitted by the target sound source is an abnormal sound when the type of initial audio data is different from the sound type.

[0083] Optionally, a method for obtaining the sound type of the current monitoring scene of the monitoring device includes but is not limited to one of the following:

[0084] The first one is to obtain the sound type pre-stored in the server.

[0085] The second method is to obtain the sound type sent by other devices that are connected to the server for communication. The other devices may be a remote control, a mobile phone, a tablet computer, a wearable device, etc. This embodiment does not limit the device type of the other devices.

[0086] In actual implementation, the method for obtaining the sound type may also be other methods, and this embodiment does not limit the method for obtaining the sound type.

[0087] For example, if it is determined that the sound type of the current monitoring scene is the sound type of human speaking, and if it is determined that the sound type collected by the audio collection component at this time is a mechanical sound type, then the sound emitted by the target sound source is determined to be an abnormal sound.

[0088] Step S2: When the sound emitted by the target sound source is an abnormal sound, the initial audio data is used to determine the direction of arrival to obtain the target area where the target sound source is located.

[0089] Optionally, algorithms for determining the direction of arrival using initial audio data include, but are not limited to, correlation method, maximum likelihood method (covariance minimization), multi-signal classification method, signal parameter estimation using rotational invariance technology, eigendecomposition method, or matrix bundle method using array manifold or triangulation, etc. This embodiment does not limit the algorithm for determining the direction of arrival using initial audio data.

[0090] Step S3: Control the monitoring device to run until the monitoring range covers the target area, and control the distance sensor component to collect data on the target area.

[0091] Step S4: Acquire the distance information collected by the distance sensor.

[0092] Step 202: determine the sound pickup pattern corresponding to the distance information.

[0093] Optionally, the sound pickup modes include at least two, and this embodiment does not limit the number of sound pickup modes.

[0094] Among them, different pickup modes are adapted to different distance information.

[0095] Optionally, each sound pickup pattern is determined based on a sound pickup algorithm, and determining the sound pickup pattern corresponding to the distance information includes: determining algorithm parameters of the sound pickup algorithm based on the distance information; and determining the sound pickup pattern based on a result output by the sound pickup algorithm having the algorithm parameters.

[0096] Optionally, the sound pickup algorithm includes an audio amplification algorithm, a beamforming algorithm and a noise suppression algorithm, and the algorithm parameters of the sound pickup algorithm are determined based on the distance information, including: determining the gain value in the audio amplification algorithm corresponding to the distance information; determining the basic unit parameter value of the phase array in the beamforming algorithm corresponding to the distance information; determining the noise reduction parameter in the noise suppression algorithm corresponding to the distance information.

[0097] Among them, the corresponding relationship between distance information and algorithm parameters is pre-stored in the server.

[0098] For example: when the distance information is 0-1 meter, the algorithm parameters corresponding to the distance information are the gain value A1 in the audio amplification algorithm, the basic unit parameter value A2 in the phase array in the beamforming algorithm, and the noise reduction parameter A3 in the noise suppression algorithm; when the distance information is 1 meter-7 meters, the algorithm parameters corresponding to the distance information are the gain value B1 in the audio amplification algorithm, the basic unit parameter value B2 in the phase array in the beamforming algorithm, and the noise reduction parameter B3 in the noise suppression algorithm; when the distance information is 7 meters to 15 meters, the algorithm parameters corresponding to the distance information are the gain value C1 in the audio amplification algorithm, the basic unit parameter value C2 in the phase array in the beamforming algorithm, and the noise reduction parameter C3 in the noise suppression algorithm.

[0099] Step 203: Process the original audio data collected by the audio collection component according to the sound pickup mode to obtain processed audio data.

[0100] Step 204, determining a track-joining mode for the processed audio data.

[0101] Optionally, the track merging mode includes an offline mode and a real-time mode.

[0102] The merge mode refers to a mode in which audio data and video data are played synchronously.

[0103] In one example, determining a merge mode of processed audio data includes: when the distance information belongs to a first distance range, determining the merge mode to be an offline mode; when the distance information belongs to a second distance range, determining the merge mode to be a real-time mode; and each distance value in the first distance range is greater than each distance value in the second distance range.

[0104] The first distance range and the second distance range are pre-stored in the server.

[0105] For example, taking the first distance range as 5 meters to 15 meters and the second distance range as 0 meters to 5 meters, when the distance information indicates that the distance between the target sound source and the monitoring device is 6 meters, the distance information belongs to the first distance range, and the merging mode is determined to be offline merging.

[0106] Since the audio acquisition component will be affected by the distance when collecting audio data, resulting in delayed collection, that is, the longer the distance, the greater the delay in the audio acquisition component collecting audio data. At this time, if real-time merging is used to process the audio and video data, it will cause a large delay between the audio and video data after merging and the actual picture. Based on the above technical problems, in this embodiment, when the distance information belongs to the first distance range, the merging mode is determined to be an offline mode; when the distance information belongs to the second distance range, the merging mode is determined to be a real-time mode; each distance value in the first distance range is greater than each distance value in the second distance range. In this way, the merging mode can be used in a targeted manner to avoid the problem of a large delay between the audio and video data after merging and the actual picture.

[0107] In another example, determining a merge mode of processed audio data includes: receiving a setting operation for the merge mode; if the setting operation indicates an offline mode, determining the merge mode to be an offline mode; if the setting operation indicates a real-time mode, determining the merge mode to be a real-time mode.

[0108] In another example, determining the merging mode of processed audio data includes: obtaining the output scene type of monitoring data; when the output scene type is a low-latency scene type, determining the merging mode to be an offline mode; when the output scene type is not a low-latency scene type, determining the merging mode to be a real-time mode.

[0109] Optionally, the output scenario type for obtaining monitoring data includes but is not limited to one of the following:

[0110] The first one is to obtain the output scene type pre-stored in the server.

[0111] The second method is to obtain the output scene type sent by other devices that are connected to the server for communication. The other devices may be remote controllers, mobile phones, tablet computers, wearable devices, etc. This embodiment does not limit the device type of other devices.

[0112] In actual implementation, the method for obtaining the output scene type may also be other methods, and this embodiment does not limit the method for obtaining the output scene type.

[0113] Step 205, when the merge mode is the offline mode, after obtaining the offline monitoring file composed of each frame of monitoring image, the processed audio data is merged with the offline monitoring file in response to the play request to obtain the monitoring data to be played.

[0114] Optionally, the track merging process is performed based on aligning the timestamp of the offline monitoring file with the timestamp of the processed audio data.

[0115] Optionally, when the merge mode is a real-time mode, each frame of monitoring image is merged with the processed audio data to obtain monitoring data to be played.

[0116] Optionally, the track merging process is performed by aligning the timestamp of the video stream data of each frame of the monitoring image with the timestamp of the audio stream data of the processed audio data.

[0117] In summary, the data processing method of the monitoring device provided in the present embodiment controls the image acquisition component to acquire the monitoring image of the target sound source in response to the sound of the target sound source, and obtains the distance information between the target sound source and the monitoring device; determines the sound pickup mode corresponding to the distance information; the sound pickup mode includes at least two types, and different sound pickup modes are adapted to different distance information; processes the original audio data collected by the audio acquisition component according to the sound pickup mode to obtain processed audio data; determines the merging mode of the processed audio data, and the merging mode includes an offline mode and a real-time mode; when the merging mode is the offline mode, after obtaining the offline monitoring file composed of each frame of monitoring image, the processed audio data is merged with the offline monitoring file in response to a play request. Processing is performed to obtain monitoring data to be played; the problem that the position of the target sound source within the acquisition range of the monitoring device may change and the audio data obtained using the same processing method may not be effective can be solved; on the one hand, different sound pickup modes are determined by determining the distance information between the target sound source and the monitoring device, so that the audio data can be processed in a targeted manner according to different distances, avoiding the problem of consistency of processed audio data heard by the human ear when the position of the target sound source changes; on the other hand, the video data and audio data can be merged through the merge mode in the offline mode, thus avoiding the delay between the merged audio and video data and the display screen, thereby ensuring that the processed monitoring data has a better effect.

[0118] In addition, since the audio acquisition component will be affected by the distance when collecting audio data, resulting in delayed collection, that is, the longer the distance, the greater the delay in the audio acquisition component collecting audio data. At this time, if real-time merging is used to process the audio and video data, it will cause a large delay between the audio and video data after merging and the actual picture. Based on the above technical problems, in this embodiment, when the distance information belongs to the first distance range, the merging mode is determined to be an offline mode; when the distance information belongs to the second distance range, the merging mode is determined to be a real-time mode; each distance value in the first distance range is greater than each distance value in the second distance range. In this way, the merging mode can be used in a targeted manner to avoid the problem of a large delay between the audio and video data after merging and the actual picture.

[0119] In order to more clearly understand the data processing method of the monitoring device, this embodiment uses an example to illustrate the method. This embodiment uses an example in which the monitoring device includes an image acquisition component and the monitoring device is connected to the audio acquisition component. The specific steps are as follows: Figure 3 shown.

[0120] Step 301: Determine whether the audio collection component has collected abnormal sound.

[0121] The method for determining abnormal sound may refer to the above method embodiment.

[0122] Step 302: output the target sound source angle.

[0123] Optionally, the method for determining the target sound source angle is based on the direction of arrival of audio data collected by the audio collection component.

[0124] The method for determining the direction of arrival of audio data may refer to the above method embodiment.

[0125] Step 303, controlling the image acquisition component to adjust the posture toward the target sound source according to the angle of the target sound source.

[0126] In this embodiment, the posture of the monitoring device can be adjusted, and the monitoring device can drive the image acquisition component to adjust its posture.

[0127] Step 304, controlling the synchronous collection of audio data and video data.

[0128] Step 305: output audio and video data.

[0129] Figure 4 is a block diagram of a data processing device for a monitoring device provided by an embodiment of the present application. Figure 1 The device includes at least the following modules: a sound source response module 410 , a first determination module 420 , a data processing module 430 , a second determination module 440 and a track combination processing module 450 .

[0130] The sound source response module 410 is used to control the image acquisition component to acquire a monitoring image of the target sound source in response to the sound emitted by the target sound source, and to obtain the distance information between the audio acquisition component and the monitoring device;

[0131] A first determination module 420 is used to determine a sound pickup mode corresponding to the distance information; the sound pickup modes include at least two, and different sound pickup modes are adapted to different distance information;

[0132] The data processing module 430 is used to process the original audio data collected by the audio collection component according to the sound pickup mode to obtain processed audio data;

[0133] A second determination module 440 is used to determine a merge mode of the processed audio data, where the merge mode includes an offline mode and a real-time mode;

[0134] The merge processing module 450 is used to merge the processed audio data with the offline monitoring file in response to a play request after obtaining the offline monitoring file composed of each frame of monitoring image when the merge mode is the offline mode, so as to obtain the monitoring data to be played.

[0135] For relevant details, refer to the above embodiments.

[0136] It should be noted that: when the data processing device of the monitoring device provided in the above embodiment performs data processing, only the division of the above functional modules is used as an example. In actual applications, the above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the data processing device of the monitoring device is divided into different functional modules to complete all or part of the functions described above. In addition, the data processing device of the monitoring device provided in the above embodiment and the embodiment of the data processing method of the monitoring device belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0137] This embodiment provides an electronic device, such as Figure 5 As shown, the electronic device at least includes a processor 501 and a memory 502 .

[0138] The processor 501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 501 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 501 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 501 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0139] The memory 502 may include one or more computer-readable storage media, which may be non-transitory. The memory 502 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 502 is used to store at least one instruction, which is used to be executed by the processor 501 to implement the data processing method of the monitoring device provided in the method embodiment of the present application.

[0140] In some embodiments, the electronic device may further optionally include: a peripheral device interface and at least one peripheral device. The processor 501, the memory 502 and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Schematically, the peripheral devices include but are not limited to: a radio frequency circuit, a touch display screen, an audio circuit, and a power supply.

[0141] Of course, the electronic device may also include fewer or more components, which is not limited in this embodiment.

[0142] Optionally, the present application also provides a computer-readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the data processing method of the monitoring device of the above method embodiment.

[0143] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A data processing method for a monitoring device, characterized in that: The monitoring device includes an image acquisition component, and the monitoring device is also connected to an audio acquisition component. The method includes: In response to a target sound source making a sound, controlling the image acquisition component to acquire a monitoring image of the target sound source and obtaining distance information between the target sound source and the monitoring device; Determine a sound pickup mode corresponding to the distance information; the sound pickup modes include at least two, and different sound pickup modes are adapted to different distance information; Processing the original audio data collected by the audio collection component according to the sound pickup mode to obtain processed audio data; Determine a merge mode of the processed audio data, wherein the merge mode includes an offline mode and a real-time mode; When the merge mode is the offline mode, after obtaining the offline monitoring file composed of each frame of monitoring image, the processed audio data is merged with the offline monitoring file in response to a play request to obtain monitoring data to be played; The determining the track combining mode of the processed audio data includes: When the distance information belongs to the first distance range, the track merging mode is determined to be an offline mode; when the distance information belongs to the second distance range, the track merging mode is determined to be a real-time mode; each distance value in the first distance range is greater than each distance value in the second distance range; or, Obtain an output scene type of monitoring data; when the output scene type is a low-latency scene type, determine that the merge mode is the offline mode; when the output scene type is not the low-latency scene type, determine that the merge mode is the real-time mode.

2. The method according to claim 1, characterized in that Each sound pickup pattern is determined based on a sound pickup algorithm, and the determining of the sound pickup pattern corresponding to the distance information includes: Determining algorithm parameters of the sound pickup algorithm based on the distance information; The sound pickup pattern is determined based on a result output by a sound pickup algorithm having the algorithm parameters.

3. The method according to claim 2, characterized in that The sound pickup algorithm includes an audio amplification algorithm, a beamforming algorithm and a noise suppression algorithm, and the algorithm parameters of the sound pickup algorithm are determined based on the distance information, including: Determine a gain value in the audio amplification algorithm corresponding to the distance information; Determine a basic unit parameter value of a phase array in the beamforming algorithm corresponding to the distance information; Determine a noise reduction parameter in the noise suppression algorithm corresponding to the distance information.

4. The method according to claim 1, characterized in that: The monitoring device is equipped with a distance sensor component, and the collection posture of the monitoring device is adjustable; In response to the target sound source making a sound, controlling the image acquisition component to acquire a monitoring image of the target sound source and obtaining distance information between the target sound source and the monitoring device includes: In response to a target sound source making a sound, determining whether the sound made by the target sound source is an abnormal sound using initial audio data collected by the audio collection component; In the case where the sound emitted by the target sound source is an abnormal sound, determining a direction of arrival using the initial audio data to obtain a target area where the target sound source is located; Control the monitoring device to operate until the monitoring range covers the target area, and control the distance sensor component to collect data from the target area; The distance information collected by the distance sensor is acquired.

5. The method according to claim 4, characterized in that The determining whether the sound emitted by the target sound source is an abnormal sound includes: Obtaining a standard volume range of the current monitoring scene of the monitoring device; and determining that the sound emitted by the target sound source is an abnormal sound when the volume of the initial audio data does not fall within the standard volume range; and / or, Acquire the sound type of the current monitoring scene of the monitoring device; and when the type of the initial audio data is different from the sound type, determine that the sound emitted by the target sound source is an abnormal sound.

6. The method according to claim 1, characterized in that The step of determining the track combining mode of the processed audio data further includes: A setting operation for the track merging mode is received; when the setting operation indicates the offline mode, the track merging mode is determined to be the offline mode; when the setting operation indicates the real-time mode, the track merging mode is determined to be the real-time mode.

7. The method according to claim 1, characterized in that The method further comprises: When the merging mode is the real-time mode, each frame of monitoring image is merged with the processed audio data to obtain monitoring data to be played.

8. A data processing device using the data processing method of the monitoring device according to any one of claims 1 to 7, characterized in that: The device comprises: A sound source response module, used to control the image acquisition component to acquire a monitoring image of the target sound source in response to the sound emitted by the target sound source, and to obtain the distance information between the audio acquisition component and the monitoring device; A first determination module is used to determine a sound pickup mode corresponding to the distance information; the sound pickup modes include at least two, and different sound pickup modes are adapted to different distance information; A data processing module, used for processing the original audio data collected by the audio collection component according to the sound pickup mode to obtain processed audio data; A second determination module is used to determine a merge mode of the processed audio data, wherein the merge mode includes an offline mode and a real-time mode; The merging processing module is used to merge the processed audio data with the offline monitoring file in response to a play request after obtaining the offline monitoring file composed of each frame of monitoring image when the merging mode is the offline mode, so as to obtain the monitoring data to be played.

9. An electronic device, characterized in that: The device includes a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement a data processing method for a monitoring device as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a program, and when the program is executed by the processor, it is used to implement a data processing method for a monitoring device as described in any one of claims 1 to 7.

Images