Picture capture detection method and device based on family abnormal sound recognition

Abstract

The invention belongs to the technical field of data processing, and particularly relates to a picture capture detection method and device based on family abnormal sound recognition. The method comprises the following steps: S1, acquiring sound features through an audio acquisition unit of a camera, and forming original feature vectors; s2, outputting a sound category through a pre-trained family abnormal sound recognition model; and S3, determining whether the sound category belongs to the family abnormal sound or not, and when the sound category belongs to the family abnormal sound, rotating the camera to the sound source generation direction of the family abnormal sound, and capturing a picture of the sound source direction. According to the invention, the accuracy, timeliness and comprehensiveness of monitoring are improved.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular relates to a screen capture detection method and device based on abnormal household sound recognition. Background Art

[0002] With the development of society and the improvement of people's living standards, family safety has become an issue of increasing concern. Traditional home security monitoring systems mainly rely on video monitoring and manual monitoring, but there are problems such as high false alarm rate, untimely response, and limited coverage. Especially when family members are not at home, how to promptly detect and deal with sudden abnormal situations has become a major problem.

[0003] In a home environment, abnormal sounds such as glass breaking, heavy objects falling, screaming, etc. are often early warning signals of accidents or dangerous situations. Being able to detect and identify these sounds in a timely manner and capture the scene simultaneously can not only improve the effectiveness of home security monitoring, but also provide important evidence for post-event analysis.

[0004] In the prior art, sound sensors can be used to monitor the sound in the environment. When a specific abnormal sound is detected, an alarm signal is issued. However, this method has a single function, is easily disturbed by background noise, has a high false alarm rate, cannot provide on-site images, and lacks comprehensive recording and analysis of events. In addition, the home environment can be monitored in real time by a camera, video images can be captured and recorded, and abnormal situations can be detected by combining mobile detection technology. The disadvantage is that it relies on visual detection and cannot detect abnormal sounds. It is easily affected by factors such as light and viewing angle. The video monitoring system usually needs to record continuously or start the recording function frequently, which not only takes up a lot of storage space, but also may miss key abnormal sound events. On this basis, the smart home system has gradually developed, which integrates a variety of sensors for comprehensive monitoring, manages and controls various devices through the smart home platform, and realizes remote alarm and control. However, the installation and maintenance costs of the smart home system are high, and it is easily affected by environmental noise and sensor sensitivity. The accuracy and response speed of sound recognition need to be improved. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a screen capture detection method and device based on abnormal sound recognition in the home, aiming to improve the level of home security monitoring through intelligent means, and to solve the shortcomings of the existing home security monitoring system by combining sound recognition technology and video monitoring technology.

[0006] The first aspect of the present application provides a screen capture detection method based on abnormal sound recognition in the home, which mainly includes:

[0007] Step S1, obtaining sound features through the audio acquisition unit of the camera and forming an original feature vector;

[0008] Step S2: Output the sound category through the pre-trained household abnormal sound recognition model;

[0009] Step S3, determining whether the sound category belongs to abnormal household sounds. If it belongs to abnormal household sounds, rotating the camera to the direction of the sound source of the abnormal household sounds to capture the picture in the direction of the sound source.

[0010] Preferably, step S2 further comprises:

[0011] Step S21, determining the type of sound based on a pre-trained household abnormal sound recognition model;

[0012] Step S22, mapping the recognized sound type into a continuous vector having the same dimension as the original feature vector through an embedding matrix, and superimposing the new feature vector with the original feature vector to form a new feature vector;

[0013] Step S23: For the new feature vector, output the detailed sound category of the sound based on the pre-trained household abnormal sound recognition model.

[0014] Preferably, in step S3, determining whether the sound category belongs to abnormal household sound includes:

[0015] Step S31, determining each sound category and its discrimination probability output by the household abnormal sound recognition model;

[0016] Step S32: for each sound category, determine whether it is in the preset sound categories to be triggered for awakening, and compare the discrimination probability with the preset awakening sensitivity. When the sound category is in the preset sound categories to be triggered for awakening and its discrimination probability is not lower than the awakening sensitivity, determine that the sound category belongs to abnormal household sound.

[0017] Preferably, before step S31, the method further includes obtaining the type of sound and the wake-up sensitivity configured by the user on the terminal for triggering wake-up.

[0018] Preferably, step S3 further includes:

[0019] Step S4: Push the image of the location where the abnormal household sound is generated to the terminal via the cloud.

[0020] The second aspect of the present application provides a screen capture detection device based on abnormal sound recognition in the home, which mainly includes:

[0021] A sound feature acquisition module, used to acquire sound features through the audio acquisition unit of the camera and form an original feature vector;

[0022] A sound category recognition module, used for outputting the sound category through a pre-trained abnormal home sound recognition model;

[0023] The module for capturing images of abnormal household sounds is used to determine whether the sound category belongs to abnormal household sounds. If it belongs to abnormal household sounds, the camera is rotated to the direction of the sound source of the abnormal household sounds to capture the image in the direction of the sound source.

[0024] Preferably, the sound category recognition module includes:

[0025] A sound type recognition unit, used to determine the sound type based on a pre-trained abnormal home sound recognition model;

[0026] A feature vector combining unit, used for mapping the recognized sound type into a continuous vector having the same dimension as the original feature vector through an embedding matrix, and superimposing the new feature vector with the original feature vector;

[0027] The sound discrimination classification recognition unit is used to output the detailed sound category of the sound based on the pre-trained household abnormal sound recognition model for the new feature vector.

[0028] Preferably, the household abnormal sound and picture capturing module comprises:

[0029] A sound category and its discrimination probability acquisition unit, used to determine each sound category and its discrimination probability output by the family abnormal sound recognition model;

[0030] The household abnormal sound discrimination unit is used to judge whether each sound category is among the preset sound categories to be triggered for awakening, and compare the discrimination probability with the preset awakening sensitivity. When the sound category is among the preset sound categories to be triggered for awakening and its discrimination probability is not lower than the awakening sensitivity, it is determined that the sound category belongs to the household abnormal sound.

[0031] Preferably, the screen capture detection device further comprises a terminal configuration module, which is used to obtain the type of sound triggering wake-up and the wake-up sensitivity configured by the user in the terminal.

[0032] Preferably, the picture capture detection device further comprises:

[0033] The image processing module is used to push the image of the source of abnormal household sounds to the terminal through the cloud.

[0034] This application improves the accuracy, timeliness and comprehensiveness of monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flowchart of a preferred embodiment of the screen capture detection method based on abnormal sound recognition in the home of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical scheme and advantages of the implementation of this application clearer, the technical scheme in the implementation of this application will be described in more detail in combination with the drawings in the implementation of this application. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described implementation is a part of the implementation of this application, not all of the implementations. The implementation described below with reference to the drawings is exemplary and is intended to be used to explain this application, and cannot be understood as a limitation on this application. Based on the implementation in this application, all other implementations obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The implementation of this application is described in detail below in combination with the drawings.

[0037] The first aspect of the present application provides a screen capture detection method based on abnormal sound recognition in a home, such as Figure 1 As shown, it mainly includes:

[0038] Step S1: Acquire sound features through the audio acquisition unit of the camera and form an original feature vector.

[0039] Step S2: Output the sound category through the pre-trained household abnormal sound recognition model.

[0040] It can be understood that the household abnormal sound recognition model is formed through pre-training. During training, based on the marked sound feature vectors, the household abnormal sound recognition model is controlled to output the sound category, which is then compared with the marked sound category to continuously correct the various parameters in the household abnormal sound recognition model. After multiple rounds of training, a model is obtained that can more accurately identify abnormal sounds in the home environment, reduce the false alarm rate, and the model can identify multiple types of abnormal sounds, rather than being limited to specific sound types, thereby improving the comprehensiveness of detection.

[0041] In some optional implementations, step S2 further includes:

[0042] Step S21, determining the type of sound based on a pre-trained household abnormal sound recognition model;

[0043] Step S22, mapping the recognized sound type into a continuous vector having the same dimension as the original feature vector through an embedding matrix, and superimposing the new feature vector with the original feature vector to form a new feature vector;

[0044] Step S23: For the new feature vector, output the detailed sound category of the sound based on the pre-trained household abnormal sound recognition model.

[0045] This embodiment extracts sound features through the backbone network to form an original feature vector, and identifies the sound type through the detection branch. It is then mapped into a continuous vector of a specified dimension through an embedding matrix, and superimposed with the original feature vector to form a new feature vector for re-identification to obtain the specific category of the sound. The two branches of the home abnormal sound recognition model can further improve the accuracy of sound category detection.

[0046] In this embodiment, the sound type determined in step S21 may be, for example, human sound, animal sound, or object sound. The sound category given in step S23 refers to a more detailed classification, mainly including the following:

[0047] (1) A fall or bump sound, which may be caused by someone falling, bumping into furniture, or falling to the ground. Potential problems may be injuries, fainting, or other health problems.

[0048] (2) Glass breaking, which may be caused by breaking a window, mirror, or glassware. Potential problems may be accidents, break-ins, domestic violence, etc.

[0049] (3) The sound of heavy objects falling may be caused by heavy objects falling, furniture tipping over, or other heavy objects moving. Potential problems may be unstable furniture falling, danger caused by children climbing, or family members' operating errors.

[0050] (4) Screaming or shouting, which may be caused by shock, pain, anger or quarrel. Potential problems may be family conflicts, accidental injuries, violent incidents, etc.

[0051] (5) The sound of hurried footsteps may be caused by rushing to get somewhere, escaping from danger, or chasing someone or a pet. Potential problems may be emergencies, fires, nervousness, etc.

[0052] (6) Crying. Possible causes include emotional agitation, physical pain, and psychological stress. Potential problems may include emotional distress, family disputes, and health problems.

[0053] (7) Groaning or groaning, which may be caused by physical discomfort, injury, or fatigue. The underlying problem may be illness, psychological stress, or physical fatigue.

[0054] (8) Banging noises, which may be caused by knocking on the door, hitting the wall or floor, or hitting an object. Potential problems may be emergency help, venting anger, children playing, etc.

[0055] (9) Cries for help, which may be caused by seeking help, feeling fear or distress. Potential problems may be sudden illness, accidents, intruders, etc.

[0056] (10) Abnormal household appliance sounds may be caused by household appliance failure or abnormal operation, such as strange noises or sparks. Potential problems may be appliance failure, circuit problems, or fire hazards.

[0057] (11) Unusual pet noises, which may be caused by the pet being in pain, fear or anxiety. Potential problems may be that the pet is injured, sick or has noticed something unusual.

[0058] Step S3, determining whether the sound category belongs to abnormal household sounds. If it belongs to abnormal household sounds, rotating the camera to the direction of the sound source of the abnormal household sounds to capture the picture in the direction of the sound source.

[0059] In some optional implementations, in step S3, determining whether the sound category belongs to abnormal household sound includes:

[0060] Step S31, determining each sound category and its discrimination probability output by the household abnormal sound recognition model;

[0061] Step S32: for each sound category, determine whether it is in the preset sound categories to be triggered for awakening, and compare the discrimination probability with the preset awakening sensitivity. When the sound category is in the preset sound categories to be triggered for awakening and its discrimination probability is not lower than the awakening sensitivity, determine that the sound category belongs to abnormal household sound.

[0062] In some optional implementations, the step S31 further includes obtaining the type of sound and the wake-up sensitivity configured by the user on the terminal for triggering wake-up.

[0063] In step S3, first, the camera wake-up logic can be set on the terminal, that is, the keywords and wake-up sensitivity for waking up the camera for shooting are set, the keywords correspond to the sound categories output by the household abnormal sound recognition model, and the wake-up sensitivity corresponds to the probability of each sound category output by the household abnormal sound recognition model. For example, the keyword is set to abnormal household appliance sound, and its wake-up sensitivity is set to 80. When the output result of the household abnormal sound recognition model is that the discrimination probability of 90% indicates that the monitored sound is an abnormal household appliance sound, the camera is woken up and controlled to take pictures. When taking pictures, it is preferred to use the sound source positioning module of the camera to rotate the camera to the direction of the sound source, capture the picture in the direction of the sound source, and provide the user with intuitive visual evidence.

[0064] In some optional implementations, step S3 further includes:

[0065] Step S4: Push the image of the location where the abnormal household sound is generated to the terminal via the cloud.

[0066] In this embodiment, the sound source location image of abnormal household sounds is pushed to the user's mobile terminal in real time through the cloud server, and the user can immediately view and handle abnormal situations, thereby improving the efficiency of home security management.

[0067] This application sets the camera wake-up word and sensitivity through the terminal device, combines sound recognition, sound source location, image capture, cloud transmission and push, to achieve integrated home security monitoring, ensuring efficient, accurate and intelligent abnormal sound recognition and image capture in the home.

[0068] The second aspect of the present application provides a screen capture detection device based on abnormal sound recognition in the home, which mainly includes:

[0069] A sound feature acquisition module, used to acquire sound features through the audio acquisition unit of the camera and form an original feature vector;

[0070] A sound category recognition module, used for outputting the sound category through a pre-trained abnormal home sound recognition model;

[0071] The module for capturing images of abnormal household sounds is used to determine whether the sound category belongs to abnormal household sounds. If it belongs to abnormal household sounds, the camera is rotated to the direction of the sound source of the abnormal household sounds to capture the image in the direction of the sound source.

[0072] In some optional implementations, the sound category recognition module includes:

[0073] A sound type recognition unit, used to determine the sound type based on a pre-trained abnormal home sound recognition model;

[0074] A feature vector combining unit, used for mapping the recognized sound type into a continuous vector having the same dimension as the original feature vector through an embedding matrix, and superimposing the new feature vector with the original feature vector;

[0075] The sound discrimination classification recognition unit is used to output the detailed sound category of the sound based on the pre-trained household abnormal sound recognition model for the new feature vector.

[0076] In some optional implementations, the home abnormal sound image capturing module includes:

[0077] A sound category and its discrimination probability acquisition unit, used to determine each sound category and its discrimination probability output by the family abnormal sound recognition model;

[0078] The household abnormal sound discrimination unit is used to judge whether each sound category is among the preset sound categories to be triggered for awakening, and compare the discrimination probability with the preset awakening sensitivity. When the sound category is among the preset sound categories to be triggered for awakening and its discrimination probability is not lower than the awakening sensitivity, it is determined that the sound category belongs to the household abnormal sound.

[0079] In some optional implementations, the screen capture detection device further includes a terminal configuration module, which is used to obtain the type of sound triggering wake-up and the wake-up sensitivity configured by the user in the terminal.

[0080] In some optional implementations, the image capture detection device further includes:

[0081] The image processing module is used to push the image of the source of abnormal household sounds to the terminal through the cloud.

[0082] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A screen capture detection method based on abnormal sound recognition in the home, characterized in that: include: Step S1, obtaining sound features through the audio acquisition unit of the camera and forming an original feature vector; Step S2: Output the sound category through the pre-trained household abnormal sound recognition model; Step S3, determining whether the sound category belongs to abnormal household sounds. If it belongs to abnormal household sounds, rotating the camera to the direction of the sound source of the abnormal household sounds to capture the picture in the direction of the sound source.

2. The image capture detection method based on abnormal sound recognition in the home as claimed in claim 1, characterized in that: Step S2 further comprises: Step S21, determining the type of sound based on a pre-trained household abnormal sound recognition model; Step S22, mapping the recognized sound type into a continuous vector having the same dimension as the original feature vector through an embedding matrix, and superimposing the new feature vector with the original feature vector to form a new feature vector; Step S23: For the new feature vector, output the detailed sound category of the sound based on the pre-trained household abnormal sound recognition model.

3. The image capture detection method based on abnormal sound recognition in the home as claimed in claim 1, characterized in that: In step S3, determining whether the sound category belongs to abnormal household sound includes: Step S31, determining each sound category and its discrimination probability output by the household abnormal sound recognition model; Step S32: for each sound category, determine whether it is in the preset sound categories to be triggered for awakening, and compare the discrimination probability with the preset awakening sensitivity. When the sound category is in the preset sound categories to be triggered for awakening and its discrimination probability is not lower than the awakening sensitivity, determine that the sound category belongs to abnormal household sound.

4. The image capture detection method based on household abnormal sound recognition as claimed in claim 3, characterized in that: Before step S31, the method further includes obtaining the type of sound and the wake-up sensitivity configured by the user on the terminal for triggering wake-up.

5. The image capture detection method based on abnormal sound recognition in the home as claimed in claim 1, characterized in that: Step S3 further includes: Step S4: Push the image of the location where the abnormal household sound is generated to the terminal via the cloud.

6. A screen capture detection device based on abnormal sound recognition in the home, characterized in that: include: A sound feature acquisition module, used to acquire sound features through the audio acquisition unit of the camera and form an original feature vector; A sound category recognition module, used for outputting the sound category through a pre-trained abnormal home sound recognition model; The module for capturing images of abnormal household sounds is used to determine whether the sound category belongs to abnormal household sounds. If it belongs to abnormal household sounds, the camera is rotated to the direction of the sound source of the abnormal household sounds to capture the image in the direction of the sound source.

7. The image capture detection device based on household abnormal sound recognition as claimed in claim 6, characterized in that: The sound category recognition module comprises: A sound type recognition unit, used to determine the sound type based on a pre-trained abnormal home sound recognition model; A feature vector combining unit, used for mapping the recognized sound type into a continuous vector having the same dimension as the original feature vector through an embedding matrix, and superimposing the new feature vector with the original feature vector; The sound discrimination classification recognition unit is used to output the detailed sound category of the sound based on the pre-trained household abnormal sound recognition model for the new feature vector.

8. The image capture detection device based on household abnormal sound recognition as claimed in claim 6, characterized in that: The family abnormal sound picture capturing module comprises: A sound category and its discrimination probability acquisition unit, used to determine each sound category and its discrimination probability output by the family abnormal sound recognition model; The household abnormal sound discrimination unit is used to judge whether each sound category is among the preset sound categories to be triggered for awakening, and compare the discrimination probability with the preset awakening sensitivity. When the sound category is among the preset sound categories to be triggered for awakening and its discrimination probability is not lower than the awakening sensitivity, it is determined that the sound category belongs to the household abnormal sound.

9. The image capture detection device based on household abnormal sound recognition as claimed in claim 8, characterized in that: The screen capture detection device further includes a terminal configuration module, which is used to obtain the type of sound triggering wake-up and the wake-up sensitivity configured by the user in the terminal.

10. The image capture detection device based on household abnormal sound recognition as claimed in claim 6, characterized in that: The picture capture detection device also includes: The image processing module is used to push the image of the source of abnormal household sounds to the terminal through the cloud.

Images