Infant crying detection method and device based on audio and video fusion

A detection method, audio and video technology, applied in neural learning methods, speech analysis, instruments, etc., can solve the problems of low recognition accuracy, difficult audio recognition of baby crying, limited data, etc., to achieve improved accuracy and good practical value , the effect of improving the accuracy

Pending Publication Date: 2022-06-03
SOUTH CHINA NORMAL UNIVERSITY
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, previous cry detection algorithms are limited by data, and it is difficult to recognize babies crying in complex environment

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Infant crying detection method and device based on audio and video fusion
  • Infant crying detection method and device based on audio and video fusion
  • Infant crying detection method and device based on audio and video fusion

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0050] In order to make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0051] It should be clear that the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by persons of ordinary skill in the art without creative work fall within the protection scope of the embodiments of the present application.

[0052] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. As used in the embodiments of this application and the appended claims, the singular forms "a," "the," and "the" are intended to include the plur...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a baby crying detection method and device based on audio and video fusion. The infant crying detection method based on audio and video fusion comprises the following steps: acquiring an audio and video file to be detected, and splitting the audio and video file to obtain audio data and video data; extracting a Mel spectrogram of the audio data, and extracting abstract features of the Mel spectrogram by using a ResNet neural network to obtain voice features; extracting grayscale images of continuous frames in the video data, and extracting abstract features of the grayscale images by using a 3DCNN network to obtain visual features; fusing the voice features and the visual features to obtain mixed features; and inputting the mixed features into a trained ResNet residual network to obtain a crying detection result. According to the infant crying detection method and device based on audio and video fusion, voice and visual information are combined for automatic recognition of infant crying, and the accuracy of infant crying detection is improved.

Description

technical field [0001] The invention relates to the field of computer image speech processing, in particular to a method and device for detecting infant crying based on audio and video fusion. Background technique [0002] Automatic detection of crying plays an important role in baby monitoring. For example, baby cry detection can reduce parental care time and family burden. In hospitals, high-precision cry detection reduces the workload of hospital staff and helps improve cry pathology diagnosis. [0003] In previous studies, researchers found that early infant voices have the same vocal characteristics, but these characteristics disappear over time due to the influence of living environment. Based on this finding, many efforts have been devoted to the design of audio feature analysis and detection algorithms for infant crying. Most of the baby cry detection works use frequency domain features as input, such as MFCC and LFCC. These features are very effective in voice r...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G10L17/26G10L17/02G10L17/18G10L17/04G10L25/24G10L25/57G06N3/04G06N3/08
CPCG10L17/26G10L17/02G10L17/18G10L17/04G10L25/24G10L25/57G06N3/08G06N3/045
Inventor 余松森刘朋
Owner SOUTH CHINA NORMAL UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap