Data processing method and device, electronic equipment and storage medium

Abstract

The invention relates to a data processing method and device, electronic equipment and a storage medium, and relates to the technical field of computers. Detecting the audio frequency data, and sending a first starting signal to an acquisition end in the data processing system under the condition that the audio frequency data is detected to meet a video acquisition condition; wherein the acquisition end is used for starting and acquiring video data of the to-be-monitored space as to-be-processed data under the condition that the first starting signal is received; and detecting the to-be-processed data, and generating and outputting alarm information when detecting that an abnormal event exists in the to-be-monitored space. And excessive acquisition of video data is avoided, so that the demand on electric energy is reduced, and the problem of relatively high cost caused by long-time operation of the acquisition end is avoided.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method, device, electronic device and storage medium. Background Art

[0002] With the development of computer technology, many industries use cameras for corresponding data processing and analysis. For example, in the field of home-based elderly care, cameras are used to collect the activities of the elderly in real time, identify abnormal situations of the elderly and issue timely alarms.

[0003] Currently, in indoor scenarios where cameras are used for data collection and detection, the cameras are usually required to be turned on for a long time for uninterrupted data collection, which has high power supply requirements and leads to high costs. Summary of the invention

[0004] The present disclosure provides a data processing method, device, electronic device and storage medium to at least solve the above problems. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a data processing method is provided. The method is applied to a monitoring terminal included in a data processing system. The method includes:

[0006] Collecting audio data of the space to be monitored;

[0007] detecting the audio data, and sending a first start signal to a collection end in the data processing system when detecting that the audio data meets a video collection condition;

[0008] Among them, the acquisition end is used to start and obtain the video data of the monitored space as data to be processed when receiving the first start signal; detect the data to be processed, and generate and output alarm information when an abnormal event is detected in the monitored space.

[0009] According to a second aspect of an embodiment of the present disclosure, a data processing device is provided. The device is applied to a monitoring terminal included in a data processing system. The device includes:

[0010] An audio collection module, used to collect audio data of the space to be monitored;

[0011] An audio detection module, configured to detect the audio data and, if it is detected that the audio data meets the video acquisition condition, send a first start signal to the acquisition end in the data processing system;

[0012] Among them, the acquisition end is used to start and obtain the video data of the monitored space as data to be processed when receiving the first start signal; detect the data to be processed, and generate and output alarm information when an abnormal event is detected in the monitored space.

[0013] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0014] processor;

[0015] a memory for storing instructions executable by the processor;

[0016] The processor is configured to execute the instructions to implement the method as described in any one of the first aspects.

[0017] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device executes a method as described in any one of the first aspects.

[0018] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, wherein the computer program product comprises readable program instructions, and when the readable program instructions are executed by a processor of an electronic device, the electronic device executes a method as described in any one of the first aspects.

[0019] The technical solution provided by the embodiment of the present disclosure brings at least the following beneficial effects: in the embodiment of the present disclosure, by collecting the audio data of the space to be monitored; detecting the audio data, and sending a first start signal to the acquisition end in the data processing system when detecting that the audio data meets the video acquisition condition; wherein the acquisition end is used to start and obtain the video data of the space to be monitored as the data to be processed when receiving the first start signal; detecting the data to be processed, and generating and outputting alarm information when detecting that there is an abnormal event in the space to be monitored. In this way, the embodiment of the present disclosure can detect the audio data of the space to be monitored through the monitoring end, and only start the acquisition end when the video acquisition condition is met, so that the acquisition end detects the abnormal event in the video data, avoids excessive collection of video data, thereby reducing the demand for electric energy, and avoiding the problem of high cost caused by long-term operation of the acquisition end. And because the monitoring end only needs to detect the audio data, the audio data is often much smaller than the video data, thus greatly reducing the overhead of data calculation and storage, and further reducing the data processing cost.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0022] Figure 1 is a flow chart of a data processing method according to an exemplary embodiment;

[0023] Figure 2 is a schematic diagram of a waveform diagram according to an exemplary embodiment;

[0024] Figure 3 is a schematic diagram of a waveform diagram and a fingerprint according to an exemplary embodiment;

[0025] Figure 4 is a schematic diagram of a scenario according to an exemplary embodiment;

[0026] Figure 5 is a structural schematic diagram of a data processing system according to an exemplary embodiment;

[0027] Figure 6 is a structural schematic diagram of yet another data processing system according to an exemplary embodiment;

[0028] Figure 7 is a flow chart of another data processing method according to an exemplary embodiment;

[0029] Figure 8 is a flow chart of another data processing method according to an exemplary embodiment;

[0030] Fig. 9 is a flow chart of yet another data processing method according to an exemplary embodiment;

[0031] Fig.10 is a flow chart of another data processing method according to an exemplary embodiment;

[0032] Fig.11 It is a block diagram of a data processing device according to an exemplary embodiment. DETAILED DESCRIPTION

[0033] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0035] Figure 1 is a flow chart of a data processing method according to an exemplary embodiment. The method can be applied to a monitoring terminal included in a data processing system, such as Figure 1 As shown, the following steps may be included:

[0036] Step 101: Collect audio data of the space to be monitored.

[0037] Step 102: Detect the audio data, and if it is detected that the audio data meets the video acquisition condition, send a first start signal to the acquisition end in the data processing system.

[0038] Among them, the acquisition end is used to start and obtain the video data of the monitored space as data to be processed when receiving the first start signal; detect the data to be processed, and generate and output alarm information when an abnormal event is detected in the monitored space.

[0039] In the embodiments of the present disclosure, the monitoring terminal can be any computer device with sound collection and data processing capabilities. The collection terminal can be a camera with data processing capabilities, a miniature camera, or a common camera. The embodiments of the present disclosure do not limit the type of camera.

[0040] The space to be monitored can be any indoor space with monitoring requirements, or an outdoor space, which is not limited in the embodiments of the present disclosure. For example, the space to be monitored can be a ward in a nursing home, a home of a person living alone, etc.

[0041] Specifically, the above audio data can be collected by a sound collection component (e.g., a microphone), which can be set at the monitoring end or can be set independently, and the embodiments of the present disclosure do not limit this. Specifically, the above audio data can be collected in real time or according to a certain period, and the embodiments of the present disclosure do not limit this.

[0042] The video acquisition condition refers to the condition for triggering video acquisition, which can be preset. The video acquisition condition can be a preset instruction contained in the audio data, or it can be preset abnormal audio data (for example, audio with a higher decibel) contained in the audio data, etc. It can be set according to actual monitoring needs, and the embodiments of the present disclosure do not limit this.

[0043] Specifically, the audio data may be subjected to signal detection, and the audio signal may be matched with a preset abnormal audio signal, and if the match is consistent, it may be confirmed that the video acquisition condition is met. Correspondingly, if the match is inconsistent, it may be confirmed that the video acquisition condition is not met.

[0044] Specifically, the above-mentioned monitoring end and the above-mentioned acquisition end can be connected via a cable or a network cable. Accordingly, the above-mentioned first start signal can be a network wake-up signal or a power control signal, and can be set based on the preset communication protocol or communication instructions of the monitoring end and the acquisition end. The embodiment of the present disclosure does not limit this.

[0045] Among them, the above-mentioned abnormal events can be pre-set, and can be set according to the type of space to be monitored and the actual monitoring needs, and the embodiments of the present disclosure do not limit this. Furthermore, after receiving the first start signal, the acquisition end can start and obtain the video data of the space to be monitored for detection, and can be detected through image recognition, motion capture, infrared sensing and other technologies. Among them, the above-mentioned alarm information can be information indicating the existence of abnormal events in the space to be monitored, which can be text information or voice information. The embodiments of the present disclosure do not limit the type of alarm information.

[0046] Specifically, the above-mentioned collection end can output the alarm information to the monitoring end. Optionally, the data processing system can also include a management end, which can be used to update the data processing programs of the collection end and the monitoring end, and receive data from the monitoring end and the collection end, and process the alarm information, etc. Therefore, the above-mentioned collection end can also directly output the alarm information to the management end.

[0047] Exemplarily, for the monitoring scenario of elderly people living alone, the above abnormal event may be that the elderly person does not take medicine on time. Specifically, the camera can be responsible for detecting whether the elderly person has taken medicine. In particular, when the elderly person has finished taking medicine, the elderly person can actively speak to respond to the camera, and the camera will actively end this task after recognizing the response sound through the collected video data. Therefore, by issuing and receiving voice commands, and cooperating with functions such as the camera's motion recognition, it is possible to detect whether the elderly person's daily behavior of taking medicine on time is normal. If the elderly person does not take medicine on time, the camera can enter a warning state and issue an alarm.

[0048] As another example, the abnormal event may also be a fire hazard. Specifically, the data to be processed can be detected by thermal imaging technology. Since the thermal imaging image will have corresponding color depth changes, if the elderly forget to turn off the fire after cooking (for example, the heat source is turned on for too long, or the elderly do not turn off the heat source but leave the cooking area for a period of time), the camera can determine that there is an abnormal event when it detects that the heat source has not been turned off for a long time, and can actively sound to remind the elderly to turn off the heat source. If the elderly do not respond within a period of time after the sound prompt, the camera can enter the early warning state and issue an alarm.

[0049] Optionally, in one case, the monitoring end may be connected to the acquisition end via an Ethernet cable, and the operation of sending the first start signal to the acquisition end in the data processing system may specifically include:

[0050] S1021. Resume data transmission on the Ethernet cable, and send a first start signal to a collection terminal in the data processing system through the Ethernet network.

[0051] Among them, Ethernet cable refers to the cable that can transmit power and data through Ethernet. Powering the acquisition end through the Ethernet cable can realize Power over Ethernet (PoE) technology. For the acquisition end, PoE technology enables the camera to obtain power and network connection at the same time through a single Ethernet cable without the need for an additional power cable.

[0052] Specifically, PoE technology can transmit power and data simultaneously through a single Ethernet cable, allowing network devices (such as cameras) to obtain power and data connections through a single cable. This technology can use Category 5e (Cat5e) or higher specification Ethernet cables, which contain four pairs of wires, two pairs for data transmission and the other two pairs for power transmission.

[0053] Specifically, data transmission of the Ethernet cable may be restored through a preset communication protocol or instruction.

[0054] Optionally, the monitoring end may include a volatile memory (RAM), a microphone, an Ethernet interface, an electronic switch and a PoE splitter. By controlling the corresponding lines in the cable, the data transmission of the PoE network camera (collection end) can be controlled.

[0055] It should be noted that existing solutions are used to remotely shut down the camera to put it in a low-power mode similar to standby or sleep mode. In this mode, the camera will turn off the image sensor, stop image acquisition and video processing, and minimize other functions to reduce power consumption to achieve energy saving when monitoring or recording is not needed. In the remote shutdown state, the camera usually remains connected to the network and communicates with the management system regularly to receive remote commands or restart when triggered. However, the disadvantage of this solution is that the camera is still connected to the network even in standby or sleep mode, which poses a risk of leaking user privacy data such as sound or images.

[0056] The disclosed embodiment uses PoE technology to enable the acquisition terminal to operate only when it receives power to perform video acquisition, thereby avoiding the problem of privacy leakage and greatly improving the security of data processing.

[0057] In summary, the data processing method provided by the embodiment of the present disclosure is to collect audio data of the space to be monitored; detect the audio data, and send a first start signal to the acquisition end in the data processing system when it is detected that the audio data meets the video acquisition condition; wherein the acquisition end is used to start and obtain the video data of the space to be monitored as the data to be processed when the first start signal is received; detect the data to be processed, and generate and output alarm information when it is detected that there is an abnormal event in the space to be monitored. In this way, the embodiment of the present disclosure can detect the audio data of the space to be monitored through the monitoring end, and start the acquisition end only when the video acquisition condition is met, so that the acquisition end detects the abnormal event in the video data, avoids excessive collection of video data, thereby reducing the demand for electric energy, and avoids the problem of high cost caused by long-term operation of the acquisition end. And because the monitoring end only needs to detect the audio data, the audio data is often much smaller than the video data, thus greatly reducing the overhead of data calculation and storage, and further reducing the data processing cost.

[0058] Optionally, the above-mentioned operation of detecting the audio data may specifically include the following steps in the embodiment of the present disclosure:

[0059] S21. Convert the audio data into a waveform graph, and obtain a fingerprint matrix of the audio data based on the waveform graph as a first matrix.

[0060] S22. Obtain a preset fingerprint matrix of the template to be matched as the second matrix.

[0061] S23: Determine the matching degree between the audio data and the template to be matched based on the first matrix and the second matrix.

[0062] S24: When the matching degree satisfies a preset matching condition, it is determined that the audio data satisfies the video acquisition condition.

[0063] The above-mentioned waveform diagram refers to the image form of the sound waveform signal corresponding to the audio data, and the audio data can be converted into a waveform diagram by calling an audio processing tool or a waveform extraction function, for example, an audio processing library (such as the "librosa" library in Python) can be used to convert the sound signal into a waveform. The above-mentioned fingerprint matrix refers to the matrix composed of the audio fingerprint of the audio data, wherein the audio fingerprint can characterize the unique characteristics of the audio data, and the eigenvalues ​​extracted from the audio data can be encoded into a binary string through a series of encoding and hashing algorithms to obtain the audio fingerprint. Specifically, the above-mentioned operation of obtaining the fingerprint matrix can be implemented by a preset audio fingerprint extraction algorithm, and the embodiments of the present disclosure are not limited to this.

[0064] Among them, the above-mentioned template to be matched refers to a preset audio template, and some audio templates can be collected in advance according to monitoring needs, such as: a roar with a large decibel, or the sound of an object falling on the floor, etc., which is not limited in the embodiments of the present disclosure. Specifically, the number of templates to be matched can be one or more, which is not limited in the embodiments of the present disclosure. Furthermore, the fingerprint matrix of the above-mentioned template to be matched can be calculated in advance and stored in the monitoring terminal, and then the second matrix can be directly obtained from the monitoring terminal without real-time calculation.

[0065] Furthermore, since the fingerprint matrix can characterize the unique features of the audio data, the embodiment of the present disclosure can determine the matching degree between the audio data and the template to be matched based on the first matrix and the second matrix, and determine whether the audio data meets the video acquisition conditions based on the matching degree.

[0066] The preset matching condition may be preset, such as a matching degree not less than a preset matching threshold, or a confidence level within a preset interval. The preset matching degree threshold may be preset, such as 90%, 85% or 95%, and may be set according to actual conditions. Specifically, when the matching degree satisfies the preset matching condition, it indicates that a template to be matched is detected in the audio data. At this time, there may be an abnormal event in the monitored space, and further detection is required by the acquisition end through video data. Therefore, it can be determined that the audio data meets the video acquisition condition.

[0067] Specifically, the above matching degree can be calculated by the difference between the first matrix and the second matrix. Specifically, the second matrix can be subtracted from the first matrix to obtain a third matrix, in which case each vector in the third matrix represents the difference between the audio data and the template to be matched, and then the third matrix is ​​converted into a matching degree through an activation function (Sigmoid).

[0068] In the embodiment of the present disclosure, the audio data is converted into a waveform graph, and a fingerprint matrix of the audio data is obtained based on the waveform graph as a first matrix; a fingerprint matrix of a preset template to be matched is obtained as a second matrix; based on the first matrix and the second matrix, the matching degree of the audio data and the template to be matched is determined; when the matching degree meets the preset matching condition, it is determined that the audio data is detected to meet the video acquisition condition. In this way, whether the video acquisition condition is met can be determined by the matching degree of the template to be matched and the audio data. At the same time, by converting the audio data into a waveform graph for processing, there is no need to use conventional speech recognition algorithms, such as dynamic time warping (DTW), vector quantization (VQ), hidden Markov model (HMM), artificial neural network (ANN) and support vector machine (SVM), etc. These speech recognition algorithms all require special wake-up words, text information conversion and pre-template training, while the embodiment of the present disclosure only needs to convert the audio data into a waveform, which greatly reduces the amount of data processing, thereby reducing the computing requirements on the monitoring end, and can complete audio detection on a lightweight device.

[0069] Optionally, the fingerprint matrix of the audio data obtained based on the waveform diagram as the operation of the first matrix may specifically include the following steps in the embodiment of the present disclosure:

[0070] S31, obtaining a waveform group to be processed from the waveform diagram, and dividing the waveform group to be processed to obtain a plurality of waveform blocks.

[0071] S32. For any of the waveform blocks, obtain the distribution quantity of discrete samples in the waveform block.

[0072] S33, converting the distribution quantity into binary code to obtain the feature vector of the waveform block.

[0073] S34. Arrange the characteristic vectors of each of the waveform blocks in a specified order to obtain a first matrix.

[0074] Among them, since the collected audio data may contain multiple sentences, the embodiment of the present disclosure can obtain the waveform combination corresponding to a sentence from the waveform graph as the waveform group to be processed. Specifically, the embodiment of the present disclosure can use the continuous and uninterrupted waveforms in the waveform graph as the waveform group, and there are intervals between different waveform groups, which can be recorded as the waveform group distance. If the waveform group distance between two adjacent waveform groups is not less than a preset threshold, it can be considered that the two waveform groups belong to two different sentences.

[0075] For example, if the waveform diagram includes waveform groups W1, W2, ..., W n, the preset threshold corresponding to the waveform group distance can be set to d. If the distance between adjacent waveforms is small, and the distance W0 between the first waveform W1 and the previous waveform is large, the last waveform W n With the next waveform W n+1 If the distance between W1 and W is large, n is defined as a set of waveforms to be processed W. Specifically, when W1-W0≥d, W m+1 -W m <d,W n+1 -W n ≥d, 1<m>n, that is, W1, W2, ..., W m , ..., W n The distance between each waveform is small, and the distance between W1 and the previous waveform W0 is large. n And the following waveform W n+1 If the distance is large, W1, W2, ..., W m , ..., W n Defined as a waveform group W to be processed.

[0076] Optionally, the above operation of dividing the waveform group to be processed to obtain multiple waveform blocks may specifically include:

[0077] S311, determining the peak of the waveform group to be processed.

[0078] S312, taking the time corresponding to the peak as the dividing center point, dividing the waveform group to be processed on both sides of the dividing center point respectively to obtain multiple waveform blocks; the areas of the waveform blocks contained on either side of the dividing center point are consistent.

[0079] Among them, the above-mentioned peak refers to the highest point of the waveform. Specifically, random points can be generated by using the Monte Carlo algorithm (for example, 100,000 points are generated in the picture), and the column containing the most random points is found as the peak. Specifically, the moment corresponding to the peak can be used as the center point of the segmentation, and the segmentation is performed to the left and right sides with the peak as the center. Specifically, on either side, the area of ​​each segmentation block is equal, so that the area of ​​each segmentation block on the left or right side is equal, which can ensure that the influence of duration can be ignored during comparison, and only the shape and characteristics of the waveform can be focused on.

[0080] In this way, even if the duration of two utterances is different, as long as their waveform shapes and characteristics are similar, similar results can still be obtained when compared, thus ensuring the consistency of the comparison.

[0081] Optionally, the areas of the waveform blocks on different sides may not be equal.

[0082] Optionally, before the waveform group to be processed is segmented, the embodiment of the present disclosure may also perform normalization processing on the waveform group to be processed, specifically:

[0083] The waveform can be placed in a picture frame of the same size and normalized at the top and bottom and the left and right ends. This operation can ensure that waveforms of different sizes are processed consistently and keep the intercepted parts at the same height. For example, the length from the start point to the end point of the waveform group W is a, a = end point - start point, and the height from the highest point to the lowest point of the waveform group W is b, b = highest point - lowest point, so that the waveform group W can be placed in a rectangular parallelepiped R with a length of a and a height of b.

[0084] Furthermore, outliers in the waveform group to be processed can be removed. Specifically, the parts of the waveform W that fluctuate too low above and below the x-axis can be trimmed. This step can reduce the blank part in the cuboid R and ensure that the vertical range of the waveform is reasonable, so as to improve the utilization rate of the image and reduce the influence of noise. The specific method of trimming is as follows: by setting the threshold h trim To determine whether the fluctuation height of waveform W above and below the x-axis is lower than the threshold, the part below will be clipped:

[0085] h trim =Threshold value of the vertical range of waveform W

[0086] Here, h trim It represents the fluctuation height threshold of the waveform W above and below the x-axis. The x-axis refers to the observation baseline of the main component area of ​​the waveform, which can be the middle line between the peak and the trough.

[0087] Optionally, the left and right ends of the waveform group to be processed can be trimmed to remove the lower amplitude portion of the waveform, thereby reducing the impact of noise and ensuring clarity.

[0088] Optionally, before executing the above step S31, the waveform image may be converted into a grayscale image.

[0089] Furthermore, the above-mentioned discrete samples refer to random points, and the embodiments of the present disclosure may use the Monte Carlo algorithm to estimate the energy of the waveform. Specifically, the Monte Carlo distribution may use random sampling to estimate the probability or quantity, and may obtain approximate results with limited computing resources. Specifically, a number of random points may be generated in a waveform group, and the number of random points in each waveform block may be obtained through the Monte Carlo algorithm, which may characterize the area ratio or energy ratio of the waveform block in the waveform group to be processed. Accordingly, the above-mentioned distribution quantity refers to the number of random points falling in the waveform block.

[0090] Furthermore, after obtaining the distribution quantity, the embodiment of the present disclosure can directly convert it into a binary code to obtain a feature vector of the waveform block. Specifically, for each waveform block, the result of the binary code can be represented as a vector of a fixed length. Each bit of the binary number can be mapped to each position of the vector, thereby forming a vector with a fixed length. For example, the i-th bit in the binary number can be mapped to the i-th position of the vector. The final vector represents the characteristics of this sample and has a fixed length. The value of the vector represents the number of points of the corresponding waveform. Among them, the above-mentioned fixed length can be 16 bits, of course, it can also be other bits, which can be set according to actual needs, and the embodiment of the present disclosure does not limit this.

[0091] Among them, the above-mentioned specified order can be preset, and can be the arrangement order of the waveform blocks in the waveform group to be processed. Of course, other orders can also be set, and the embodiments of the present disclosure are not limited to this.

[0092] Optionally, there may be multiple specified orders, and thus the number of the first matrices may also be multiple, which is not limited in the embodiments of the present disclosure.

[0093] For example, Figure 2 is a schematic diagram of a waveform diagram according to an exemplary embodiment. Figure 2 As shown, the embodiment of the present disclosure slices the image with the peak as the center, and the left side is evenly divided into 10 blocks, and the right side is evenly divided into 10 blocks, which are named from left to right as block 01 to block 20. The number of discrete samples (random points) is counted for each block, and the result is converted into a binary number to improve the efficiency of hardware use.

[0094] It should be noted that since the energy of the waveform is mainly concentrated in the middle, 5 fingerprint matrices (fingerprints) are generated for each waveform by shrinking from both sides to the middle. In the process of shrinking to the middle, the comparison results of larger block combinations are more important to emphasize the necessity of shrinking when the energy of the pictures is similar, and to avoid unnecessary comparison when they are no longer similar. Specifically, fingerprint 1 covers blocks 1 to 20, fingerprint 2 covers blocks 3 to 18, fingerprint 3 covers blocks 5 to 16, fingerprint 4 covers blocks 7 to 14, and fingerprint 5 covers blocks 9 to 12.

[0095] For example, the embodiment of the present disclosure uses the audio data including two audios of "I", "you", "he", and "hello" as an example to conduct experiments, and the waveform diagram and fingerprint can be obtained as follows: Figure 3 As shown, Figure 3 The figure is a schematic diagram of a waveform diagram and a fingerprint according to an exemplary embodiment, showing a waveform diagram of preprocessed audio data and a fingerprint of each audio data.

[0096] In the disclosed embodiment, a waveform group to be processed is obtained from the waveform diagram, and the waveform group to be processed is segmented to obtain a plurality of waveform blocks; for any of the waveform blocks, the distribution number of discrete samples in the waveform block is obtained; the distribution number is converted into a binary code to obtain a feature vector of the waveform block; the feature vectors of each of the waveform blocks are arranged in a specified order to obtain a first matrix. In this way, a first matrix that can characterize the audio characteristics can be obtained to ensure the consistency and accuracy of subsequent matching.

[0097] Optionally, the operation of determining the matching degree between the audio data and the template to be matched based on the first matrix and the second matrix may specifically include:

[0098] S41. Subtract each matrix element in the second matrix from each matrix element in the first matrix to obtain a third matrix.

[0099] S42: Obtain the confidence of the third matrix as the matching degree of the audio data and the template to be matched.

[0100] The above subtraction operation can be implemented by element-by-element subtraction. Specifically, taking the first matrix as A and the second matrix as B as an example, the third matrix can be obtained as:

[0101] C=AB, where each matrix element C contained in C i Characterizes the difference between the i-th element in the second matrix and the i-th element in the first matrix. Wherein, i represents the number of columns in the matrix. It should be noted that the embodiment of the present disclosure can make the number of elements in the first matrix consistent with the number of elements in the second matrix of the model to be matched in advance, ensuring that the rows and columns of the first matrix and the second matrix are consistent and can be subtracted.

[0102] Furthermore, C is an i*j matrix, where j can be 1.

[0103] Furthermore, the embodiment of the present disclosure may obtain the confidence based on the third matrix. Specifically, the embodiment of the present disclosure may use an activation function to obtain the confidence. Specifically, for each C i , the confidence D is calculated by the following formula i :

[0104]

[0105] Among them, a is an adjustment parameter, and its value range is usually between [0, +∞). Generally speaking, a larger a value will cause the confidence function to change more steeply and be more sensitive to differences, while a smaller a value will make the curve relatively smooth and respond relatively gently to differences. The above a can be set according to actual needs, for example, it can be set to 0.0005.

[0106] Furthermore, since there can be multiple first matrices, multiple third matrices and the confidence of each third matrix can be obtained through the above steps. On this basis, the embodiment of the present disclosure can add the confidence of each third matrix to obtain the overall confidence as the matching degree of the audio data and the template to be matched.

[0107] Optionally, a weighted average value may be used as the confidence of the third matrix, specifically:

[0108]

[0109] Here, m represents the number of first matrices. i is the weight of the first matrix i, and the confidence D represents the overall matching degree. In this scenario, the confidence D of each first matrix can be i The weighted average is used to obtain the overall confidence D. The weights can be set according to the actual situation, and can be set according to the number of waveform blocks corresponding to each first matrix. For example, Figure 2 For example, there are 5 first matrices in the , the weights of fingerprints 1 to 5 can be set to: 16, 8, 4, 2, 1 respectively.

[0110] For example, Figure 3 Taking the audio data shown in the figure as an example, the confidence of each audio data can be obtained through the above steps in the embodiment of the present disclosure as shown in the following table:

[0111]

[0112]

[0113] Sigmoid function can map any real number (including binary number) to the interval [0,1], so 0.5 represents the highest confidence, and the confidence decreases towards both sides (0 and 1). As can be seen from the above results, the confidence of the same pronunciation is closer to 0.5, while the confidence of different pronunciations is closer to 1 (due to the use of complement calculation in binary, there is no negative number). Through the summary of the overall confidence D, it is found that the confidence of the same pronunciation and the different pronunciations are clearly separated (in the experiment, the lowest confidence of the same voice is 0.6790, and the highest confidence of different voices is 0.7920), and the threshold can be set more conveniently. This proves the feasibility of the disclosed embodiment to a certain extent.

[0114] Optionally, on this basis, the embodiment of the present disclosure can set the preset matching condition as a matching degree of not less than 0.7 and not greater than 0.8. Of course, other values ​​can also be set, and the embodiment of the present disclosure does not limit this. Optionally, for scenes with more dialect accents or idioms, the confidence interval corresponding to the preset matching condition can also be appropriately expanded.

[0115] Optionally, if the matching is unsuccessful, the peak and segmentation block selection can be repeated to more comprehensively explore the feature space of the unmatched waveform, thereby enhancing the accuracy and robustness of the matching. This comprehensive strategy helps to adapt to the differences of different types of waveforms, making data matching more flexible and applicable.

[0116] In the disclosed embodiment, a third matrix is ​​obtained by subtracting each matrix element in the second matrix from each matrix element in the first matrix; and the confidence of the third matrix is ​​obtained as the matching degree of the audio data and the template to be matched. In this way, the matching degree is calculated by the difference matrix between the first matrix and the second matrix, which can ensure the matching accuracy to a certain extent.

[0117] At the same time, when determining whether the difference vector of the two fingerprints being compared is within the preset fingerprint threshold range, statistics are performed to see whether the summary of all fingerprints is within the overall threshold. The threshold setting can be adjusted according to the specific application scenario and system performance requirements. Finally, the matching result is output. If it matches a certain waveform template, the system performs the corresponding operation. For example, the data line connection may be restored and the camera may be awakened for subsequent processing. The matching result is usually marked by whether the match is successful or not, and further analysis can be performed based on the confidence level of the match.

[0118] The fingerprint threshold can be represented by the mean of the data set. The mean is calculated as follows:

[0119]

[0120] Where N is the length of the waveform vector, x i is the ith element in the waveform vector.

[0121] Fingerprint threshold T Fingerprint It is defined as the fluctuation of the waveform vector within the percentage P range and is calculated as follows:

[0122] T Fingerprint =P×mean

[0123] Where P represents the fluctuation of the waveform vector within a percentage range, and mean is the mean value of the waveform vector in the segmented block.

[0124] The overall threshold refers to the sum of all fingerprints (first feature) that meet the overall threshold. The specific calculation method is as follows:

[0125]

[0126] Where Q is the ratio, indicating how many fingerprints are needed to reach the fingerprint threshold; M is the number of images, that is, the number of the first feature, T Fingerprint,i is the fingerprint threshold of the i-th fingerprint.

[0127] Optionally, the monitoring terminal includes a volatile memory, in which a data processing program is stored, and the data processing program is used to implement the data processing method.

[0128] Specifically, the monitoring end may not be equipped with a network card, so there is no need to assign an IP address, thereby reducing the potential risk of network attacks. The monitoring end mainly includes volatile memory (RAM) and logic circuits for power supply and data transmission control. At each startup, the cloud management platform (management end) can load the data processing program into the running memory of the monitoring end through the PoE network camera, and when the power is off, the RAM will clear all data to ensure that the operating environment of the monitoring end is always in a clean state, while preventing the leakage of personal information when the monitoring end is stolen. It can achieve economic efficiency while maximizing the security of local data.

[0129] Optionally, the above-mentioned data processing system may also include a switch, which is used to connect the monitoring end and the management end. The monitoring end connects the data line every time it is turned on, allowing the PoE network camera to communicate with the cloud management platform. At the same time, the monitoring end, as an external device, allows the PoE network camera to identify and load it. The cloud management platform loads the initialization data (such as data processing programs and templates to be matched, etc.) into the running memory of the monitoring end through the PoE network camera. After the initialization is completed, the monitoring end cuts off the data line and only retains the power line. The PoE network camera enters a sleep state and stops communicating with the network. At the same time, the monitoring end does not have the ability to communicate with the network alone, so the network cannot find the PoE network camera at this time. The monitoring end monitors abnormal conditions by continuously collecting indoor sounds. If an abnormal situation is found, or a pre-set trigger condition is reached, the monitoring end restores the data line connection and wakes up the PoE network camera for further processing.

[0130] Optionally, the embodiment of the present disclosure may further include:

[0131] S61, when the current time reaches a preset time, sending a second start signal to the acquisition end in the data processing system;

[0132] The acquisition end is used to start and output preset prompt information when receiving the second start signal.

[0133] The above current moment refers to the current time point, and the above preset moment may be a time point preset by relevant personnel, or may be set by the user. For example, the user may set a schedule: to perform a certain task at 12 o'clock every day, and the above prompt information may be used to prompt the user to perform the task. The above tasks may be taking medicine, cooking, eating at a community canteen, or getting medicine at a community hospital.

[0134] Specifically, the second start signal may also be a network wake-up signal or a power control signal, and the second start signal may be sent after data transmission on the Ethernet cable is restored.

[0135] For example, an old man likes to cook, and he sets a daily behavior, that is, to start cooking at 5 pm every day. When this pre-set trigger condition is reached (the current time reaches 5 pm), the PoE network camera will also enter the auxiliary state and make a sound to remind the old man to cook. Furthermore, when the old man needs to turn off the heat source after finishing cooking, the thermal imaging function will come into play to determine whether the old man starts and ends cooking.

[0136] Furthermore, if the PoE network camera cannot detect the elderly performing relevant daily behaviors after issuing a task instruction, the camera will be turned on for a secondary judgment. If it is judged that the elderly has an abnormal situation (safety detection), such as not appearing in the designated area within a certain period of time, the PoE network camera can enter the early warning state and issue an alarm.

[0137] As another example, if an elderly person needs to take medicine on time every day, the elderly person can set a daily behavior, that is, take medicine at 3 pm every day. When this pre-set trigger condition is reached (the current time reaches 3 pm), the monitoring end will control the PoE network camera to enter the auxiliary state and make a sound to remind the elderly person to take medicine at a fixed time.

[0138] Optionally, Figure 4 is a schematic diagram of a scenario according to an exemplary embodiment. Figure 4 As shown, the scene may include: a camera 100, a data transmission protection device 200, a cloud management platform 300, and a network 400. The camera corresponds to the acquisition end, the data transmission protection device corresponds to the monitoring end, the cloud management platform corresponds to the management end, and the network is a network cable.

[0139] in, Figure 5 is a structural diagram of a data processing system according to an exemplary embodiment. Figure 5As shown, the data processing system may include a camera, a data transmission protection device, and a cloud management platform. The camera 100 includes a sound acquisition module 11, a video acquisition module 12, a voice recognition module 13, a motion recognition module 14, a target recognition module 15, a thermal imaging module 16, an abnormality judgment and alarm module 17, and an intercom module 18. The data transmission protection device 200 includes a data receiving module 21, a program running module 22, a sound acquisition module 23, and a data control module 24. The cloud management platform 300 includes an application delivery module 31, an information receiving module 32, an alarm processing module 33, and a secondary reasoning module 34.

[0140] Specifically, Figure 6 is a structural diagram of another data processing system according to an exemplary embodiment. Figure 6 As shown, the data transmission protection device can be used to connect a network camera and a switch.

[0141] Among them, the camera 100 is a front-end device, which mainly collects indoor sound, video and other information, and judges the on-site situation, including: sound collection module 11: this module collects indoor sound; video collection module 12: this module collects indoor video; voice recognition module 13: this module recognizes the indoor sound collected by the sound collection module 11; action recognition module 14: this module recognizes the indoor video collected by the video collection module 12; target recognition module 15: this module recognizes whether the monitoring target appears in the specified area; the specified area can be a specific spatial range set by the user, such as a room, a field, etc. The monitoring target refers to the object that the user hopes the system can recognize and monitor, such as the human body. Thermal imaging module 16: This module monitors temperature changes in a specified area; Abnormal judgment and alarm module 17: This module combines the voice recognized by the voice recognition module 13, the action recognized by the action recognition module 14, and the temperature changes monitored by the thermal imaging module 16 to comprehensively judge whether there is any abnormality in the personnel or environment in the room; if an abnormality is determined, an alarm message is sent to the information receiving module 32 of the cloud management platform 300; Intercom module 18: When an alarm is sounded, the personnel in the room can use this module to communicate with the community staff through the alarm processing module 33 of the cloud management platform 300.

[0142] The data transmission protection device 200 serves as a connection between the PoE network camera and the switch, and ensures the security and controllability of data transmission in the privacy space of the network camera. It includes: a data receiving module 21: receiving the program transmitted to the PoE network camera by the application sending module 31 and transmitting it to the program running module 22; a program running module 22: receiving the program transmitted to the PoE network camera by the application sending module 31 via the data receiving module 21 and loading it; a sound collection module 23: this module collects indoor sounds; a data control module 24: controlling the on and off of the data transmission of the PoE cable.

[0143] The data control module 24 controls the data transmission of the PoE cable according to the current state of the system (such as sleep, warning, auxiliary, etc.). In the sleep state, the system cuts off the data transmission of the PoE cable, so that the camera enters a low power consumption mode; after detecting an abnormal situation, the system resumes data transmission, wakes up the camera to enter the warning state, and transmits image and sound data in real time. In addition, under the preset trigger conditions, the camera can enter the auxiliary state to perform specific monitoring tasks.

[0144] Among them, the cloud management platform 300: receives the alarm information sent by the camera 100 and performs subsequent processing, including: application sending module 31: the program running module 22 of the data transmission protection device 200 receives the program (data processing program) transmitted by this module through the PoE network camera via the data receiving module 21 and loads it; information receiving module 32: receives the alarm information sent by the abnormal alarm module 17 of the camera 100; alarm processing module 33: community staff use this module to perform subsequent processing on the alarm information received by the information receiving module 32. When an alarm is issued, community staff can use this module to talk with indoor personnel through the intercom module 18 of the camera 100; secondary reasoning module 34: realizes the coordination of front-end and back-end AI capabilities. When the action recognition module 14 of the camera 100 performs rapid recognition, it can extract images and send them to this module for secondary recognition. Network 400: transmits the alarm information sent by the abnormal alarm module 17 of the camera 100 to the information receiving module 32 of the cloud management platform 300, and transmits the program in the application sending module 31 of the cloud management platform 300 to the PoE network camera (and then handed over to the data receiving module 21 of the data transmission protection device 200 for processing).

[0145] Optionally, the data processing system may include three operating states: a sleep state, an early warning state, and an auxiliary state. The sleep state uses only the sound collection function of the monitoring terminal, and the PoE network camera is not connected to the cloud; the auxiliary state uses the sound, video collection, and thermal imaging functions of the PoE network camera and performs local recognition, and the PoE network camera is not connected to the cloud; the early warning state uses the sound and image collection functions of the PoE network camera and is connected to the cloud.

[0146] Optionally, the above-mentioned monitoring end can also adopt feature analysis technology such as fast Fourier transform (FFT) to effectively distinguish between human voice and television sound and improve recognition accuracy by analyzing the frequency range of the sound signal (for example, human voice is usually in the range of 300Hz to 3.4kHz, while TV sound may be in the range of 100Hz to 20kHz), sound characteristics (such as the rhythm and tone of human voice are usually higher, while TV sound is relatively smooth) and differences in the frequency distribution of sound.

[0147] Optionally, PoE network cameras can also achieve front-end and back-end AI capability collaboration. The front-end performs rapid detection based on the original image, extracts the captured high-definition image and sends it to the cloud for secondary reasoning and recognition. The cloud (cloud management platform) has algorithms such as early fire detection, elderly safety detection, and elderly fall detection.

[0148] Figure 7 is a flowchart of another data processing method according to an exemplary embodiment. Figure 7 As shown, including:

[0149] S201: Power on and initialization: The cloud management platform loads the program into the program running module of the monitoring end, namely the volatile memory (RAM), through the PoE network camera. The monitoring end connects the data line so that the PoE network camera can communicate with the cloud management platform.

[0150] S202: Cut off the network connection: After the initialization is completed, the monitoring end cuts off the data line and keeps the power line. At this time, the PoE network camera enters a sleep state and stops communicating with the network.

[0151] S203: Monitoring and judging abnormal situations: When the PoE network camera is in sleep mode, the monitoring end collects indoor sound through the sound collection module and performs pre-processing steps such as filtering and enhancement to improve the clarity and accuracy of the signal. Subsequently, it is handed over to the program operation module for real-time monitoring and analysis to determine whether there is an abnormal situation.

[0152] S204: Response to abnormal situations: If an abnormal sound is detected, the monitoring end will restore the data line connection and wake up the PoE network camera, which will perform further processing.

[0153] Through the above operations, PoE technology can be used to achieve single-cable transmission of power and data, thereby reducing costs. At each startup, the cloud management platform loads the program into the running memory of the monitoring end through the PoE network camera, and clears all data when the power is off to ensure that the operating environment of the device remains clean. This design not only effectively reduces costs, but also maximizes the security of local data and reduces the risk of local data tampering.

[0154] Figure 8 is a flowchart of another data processing method according to an exemplary embodiment. Fig. 9 is a flowchart of another data processing method according to an exemplary embodiment. Fig.10 is a flowchart of another data processing method according to an exemplary embodiment. Figure 8 , 9As shown in Figure 10, it is possible to monitor whether the daily life of the elderly is normal. By setting parameters such as indoor personnel, indoor environment and daily behavior, the PoE network camera is awakened when the preset trigger conditions are met, switches to the auxiliary state, provides prompts for the daily life of the elderly, and judges whether it is completed normally. In the case of normal completion, the PoE network camera will return to the sleep state. However, if an abnormality is detected in the elderly or the environment, the PoE network camera will switch from the auxiliary state to the early warning state and issue a corresponding alarm. By setting preset conditions and the corresponding state switching mechanism, the monitoring, assistance and emergency response of the elderly's life can be achieved to ensure their safety and well-being.

[0155] At present, the widely used speech recognition algorithms include dynamic time warping (DTW), vector quantization (VQ), hidden Markov model (HMM), artificial neural network (ANN) and support vector machine (SVM). When implementing speech recognition tasks, these algorithms must undergo highly computationally intensive operations such as feature extraction, template training and matching. At the same time, in order to store the speech recognition library, the device needs to have a large storage space. In addition, achieving long-term efficient processing requires a lot of resource investment.

[0156] At present, the performance of the central processing unit (CPU) on terminal devices (such as mobile phones and AI cameras) has not yet reached a sufficiently powerful level to support various voice recognition algorithms, especially the operation of deep learning-based algorithms on the terminal. Therefore, the current mainstream voice recognition solution is to perform sound collection and pre-processing operations on the terminal, and place the voice recognition algorithm on the server side (i.e., the cloud) to run. However, this solution has some defects, such as user privacy leakage (voice data on the terminal is transmitted to the server) and inability to use in an environment without a network. In addition, keeping the device online for a long time may also face the risk of network intrusion, resulting in leakage of user image data. If considering deploying edge devices, such as super brain, to assist voice recognition, it will bring significant additional costs in terms of equipment purchase, network deployment, electricity collection and use, and property management. It is even less feasible to deploy edge devices in users' homes.

[0157] The data processing system and data processing method provided by the embodiment of the present disclosure do not need to store a huge speech recognition library for analysis, but only need to store simple abnormal sound templates. Accordingly, there is no need to deploy edge devices with powerful computing and storage capabilities, such as super brains, for processing, nor is there a need to transmit data to the back-end server for complex training and reasoning. The monitoring end is configured with corresponding algorithms to realize the judgment of simple sounds, which meets the computing and storage requirements that can be easily met by lightweight front-end devices. In addition, the judgment of simple voices and secondary confirmation mechanisms does not require the use of complex speech recognition algorithms such as dynamic time warping (DTW), vector quantization (VQ), hidden Markov model (HMM), artificial neural network (ANN), support vector machine (SVM), etc., to ensure high accuracy. The collected sound is converted into a waveform and the waveform vector is calculated, and then compared with the waveform vector of the abnormal sound stored in the front-end device. Due to the extremely low amount of calculation, the confidence interval of the vector can be adjusted according to the actual situation (such as idioms, dialect accents). When the confidence intervals of the vectors overlap, the waveform can be further segmented for matching, and the recognition results can be quickly output to improve the real-time performance of the system detection and reduce the computing, storage and transmission overhead. At the same time, by calculating the waveform vector, the waveform displacement problem is avoided, and there is no need for dynamic time warping (DTW). In the sleep state and auxiliary state, the device can reduce power consumption and protect user privacy. By judging the waveform vector instead of the wake-up word, the device can further reduce the amount of calculation and power consumption.

[0158] In addition, once an abnormal event is detected, the monitoring end will wake up the PoE network camera from sleep mode to warning mode to conduct a secondary confirmation of the indoor abnormal event, thereby improving the accuracy of abnormal event confirmation and buying more time for timely handling of abnormal events.

[0159] Fig.11 is a block diagram of a data processing device according to an exemplary embodiment. Fig.11 As shown, the device 80 is applied to a monitoring terminal included in a data processing system, and the device includes:

[0160] The audio collection module 801 is used to collect audio data of the space to be monitored;

[0161] An audio detection module 802, configured to detect the audio data and, if it is detected that the audio data meets the video acquisition condition, send a first start signal to the acquisition end in the data processing system;

[0162] Among them, the acquisition end is used to start and obtain the video data of the monitored space as data to be processed when receiving the first start signal; detect the data to be processed, and generate and output alarm information when an abnormal event is detected in the monitored space.

[0163] Optionally, the audio detection module includes:

[0164] A conversion submodule, used for converting the audio data into a waveform image, and obtaining a fingerprint matrix of the audio data based on the waveform image as a first matrix;

[0165] The matrix acquisition submodule is used to acquire a preset fingerprint matrix of the template to be matched as the second matrix;

[0166] A matching submodule, used for determining the matching degree between the audio data and the template to be matched based on the first matrix and the second matrix;

[0167] The determination submodule is used to determine that the audio data meets the video acquisition condition when the matching degree meets the preset matching condition.

[0168] Optionally, the conversion submodule is specifically used for:

[0169] Acquire a waveform group to be processed from the waveform graph, and divide the waveform group to be processed into multiple waveform blocks;

[0170] For any of the waveform blocks, obtaining the distribution quantity of discrete samples in the waveform block;

[0171] Convert the distribution quantity into binary code to obtain a feature vector of the waveform block;

[0172] The characteristic vectors of each of the waveform blocks are arranged in a specified order to obtain a first matrix.

[0173] Optionally, the matching submodule is specifically used for:

[0174] Subtract each matrix element in the second matrix from each matrix element in the first matrix to obtain a third matrix;

[0175] The confidence of the third matrix is ​​obtained as the matching degree of the audio data and the template to be matched.

[0176] Optionally, the conversion submodule is further used for:

[0177] Determining the peak of the waveform group to be processed;

[0178] The moment corresponding to the peak is taken as the dividing center point, and the waveform group to be processed is divided on both sides of the dividing center point to obtain multiple waveform blocks; the areas of the waveform blocks contained on either side of the dividing center point are consistent.

[0179] Optionally, the device further comprises:

[0180] A sending module, configured to send a second start signal to a collection end in the data processing system when the current time reaches a preset time;

[0181] The acquisition end is used to start and output preset prompt information when receiving the second start signal.

[0182] Optionally, the monitoring end includes a volatile memory, in which a data processing program is stored, and the data processing program is used to implement the data processing method.

[0183] Optionally, the audio detection module further includes:

[0184] The recovery submodule is used to recover the data transmission of the Ethernet cable and send a first start signal to the acquisition end in the data processing system through the Ethernet network.

[0185] In summary, the data processing device provided by the embodiment of the present disclosure collects audio data of the space to be monitored; detects the audio data, and sends a first start signal to the acquisition end in the data processing system when it is detected that the audio data meets the video acquisition condition; wherein the acquisition end is used to start and obtain the video data of the space to be monitored as the data to be processed when the first start signal is received; detects the data to be processed, and generates and outputs alarm information when it is detected that there is an abnormal event in the space to be monitored. In this way, the embodiment of the present disclosure can detect the audio data of the space to be monitored through the monitoring end, and only start the acquisition end when it meets the video acquisition condition, so that the acquisition end detects abnormal events in the video data, avoids excessive collection of video data, thereby reducing the demand for electric energy, and avoids the problem of high cost caused by long-term operation of the acquisition end. And because the monitoring end only needs to detect the audio data, the audio data is often much smaller than the video data, thus greatly reducing the overhead of data calculation and storage, and further reducing the data processing cost.

[0186] According to one embodiment of the present disclosure, an electronic device is provided, comprising: a processor and a memory for storing processor executable instructions, wherein the processor is configured to implement the steps in the data processing method in any of the above embodiments when executing.

[0187] According to an embodiment of the present disclosure, a storage medium is further provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the steps in the data processing method in any of the above embodiments.

[0188] According to one embodiment of the present disclosure, a computer program product is also provided. The computer program product includes readable program instructions. When the readable program instructions are executed by a processor of an electronic device, the electronic device can execute the steps in the data processing method in any of the above embodiments.

[0189] The user information (including but not limited to the user's device information, user personal information, etc.) and related data involved in this disclosure are all information authorized by the user or by all parties.

[0190] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0191] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data processing method, characterized in that: The method is applied to a monitoring terminal included in a data processing system, and the method includes: Collect audio data of the space to be monitored; detecting the audio data, and sending a first start signal to a collection end in the data processing system when detecting that the audio data meets a video collection condition; Among them, the acquisition end is used to start and obtain the video data of the monitored space as data to be processed when receiving the first start signal; detect the data to be processed, and generate and output alarm information when an abnormal event is detected in the monitored space.

2. The method according to claim 1, characterized in that The detecting of the audio data comprises: Convert the audio data into a waveform graph, and obtain a fingerprint matrix of the audio data based on the waveform graph as a first matrix; Obtaining a preset fingerprint matrix of the template to be matched as the second matrix; Determining the matching degree between the audio data and the template to be matched based on the first matrix and the second matrix; In the case where the matching degree satisfies a preset matching condition, it is determined that the audio data is detected to satisfy the video acquisition condition.

3. The method according to claim 2, characterized in that The fingerprint matrix of the audio data obtained based on the waveform diagram, as a first matrix, includes: Acquire a waveform group to be processed from the waveform graph, and divide the waveform group to be processed into multiple waveform blocks; For any of the waveform blocks, obtaining the distribution quantity of discrete samples in the waveform block; Converting the distribution quantity into binary code to obtain a feature vector of the waveform block; The characteristic vectors of each of the waveform blocks are arranged in a specified order to obtain a first matrix.

4. The method according to claim 2, characterized in that: The determining the matching degree between the audio data and the template to be matched based on the first matrix and the second matrix includes: Subtract each matrix element in the second matrix from each matrix element in the first matrix to obtain a third matrix; The confidence of the third matrix is ​​obtained as the matching degree of the audio data and the template to be matched.

5. The method according to claim 3, characterized in that: The step of dividing the waveform group to be processed into a plurality of waveform blocks includes: Determining the peak of the waveform group to be processed; The moment corresponding to the peak is taken as the dividing center point, and the waveform group to be processed is divided on both sides of the dividing center point to obtain multiple waveform blocks; the areas of the waveform blocks contained on either side of the dividing center point are consistent.

6. The method according to claim 1, characterized in that The method further comprises: When the current time reaches the preset time, sending a second start signal to the acquisition end in the data processing system; The acquisition end is used to start and output preset prompt information when receiving the second start signal.

7. The method according to claim 1, characterized in that The monitoring end includes a volatile memory, in which a data processing program is stored, and the data processing program is used to implement the data processing method.

8. The method according to any one of claims 1 to 7, characterized in that: The monitoring end is connected to the acquisition end through an Ethernet cable; the sending of a first start signal to the acquisition end in the data processing system includes: The data transmission of the Ethernet cable is restored, and a first start signal is sent to the acquisition terminal in the data processing system through the Ethernet network.

9. A data processing device, characterized in that: The device is applied to a monitoring terminal included in a data processing system, and the device includes: An audio collection module, used to collect audio data of the space to be monitored; An audio detection module, configured to detect the audio data and, if it is detected that the audio data meets the video acquisition condition, send a first start signal to the acquisition end in the data processing system; Among them, the acquisition end is used to start and obtain the video data of the monitored space as data to be processed when receiving the first start signal; detect the data to be processed, and generate and output alarm information when an abnormal event is detected in the monitored space.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 8.

11. A storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device executes the method as claimed in any one of claims 1 to 8.

Images