Method and system for providing a selective audio in a video environment

IN607434BActive Publication Date: 2026-10-28SAMSUNG ELECTRONICS CO LTD
View PDF -1 Cites 0 Cited by

Patent Information

Application Number
IN202211067436
Authority / Receiving Office
IN · IN
Patent Type
Patents
Current Assignee / Owner
Filing Date
2022-11-23
Publication Date
2026-10-28
Estimated Expiration
2042-11-23

AI Technical Summary

Technical Problem

Current mobile devices struggle to separate and record high-quality, personalized audio in video recordings, especially in noisy environments and with multiple subjects, due to limitations in microphone arrays and chipset resources, leading to poor audio quality and inability to focus on specific subjects.

Method used

A method and system utilizing an ultra-wide band (UWB) connection between a source device and a destination device to enable secure audio transmission, processing audio signals with a single microphone, and generating an audio sphere with directional information, allowing for selective audio integration with video based on entity parameters like environmental specifications and activity.

Benefits of technology

Enables high-quality, selective audio recording by focusing on specific subjects and filtering out background noise, improving video audio quality and allowing for precise audio selection and integration with video, even in crowded environments.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

5 The present invention relates to a method for providing a selective audio in a video environment. The method (100) includes enabling a secure connection between a source device and a destination device while recording a video of an entity carrying the source device from the destination device. The method (100) further includes performing a plurality of functions on audio received by the 10 source device and providing directional information of each of one or more audio separated from the received audio to the destination device. The method (100) further includes generating an audio sphere around the entity recorded in the video, selecting a portion of the generated audio sphere at the destination device, and thereafter integrating the selected portion of audio with the video of the entity 15 recorded by the destination device. FIG. 1
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present invention relates in general to video recording and, moreparticularly, to a method and system for providing a selective audio in a videoenvironment.BACKGROUND

[0002] With the development of technologies, the mobile device has become anessential part of everybody's lives, and the most user-attractive feature of themobile device is its camera. The camera is a photographic tool that creates apicture using electronic circuitry. At present, mobile devices include a plurality ofcameras and a plurality of microphones, to perform various functions, includingcapturing images and video. The video not only focuses on images but also onaudio. These cameras use extensive software resources to intelligently enhanceimages, but no function is performed to enhance audio quality. As a result, care isalso required to enhance audio quality by removing noise, as this can degradevideo quality and also distract video viewers from information given by theperson getting recorded in the video. Therefore, good-quality of the audio isessential for creating a good video. The audio noise is generally defined as anybackground audio such as fans, whispering, talking, sounds of vehicles, buzz fromfaulty audio wires, or other ambient noises.

[0003] However, with the present mobile devices, it is difficult to separate out theaudio of unwanted subjects while recording a video. Additionally, in the case ofmultiple subjects, it is not possible to record the video while selecting andtracking only a specific subject. Additionally, due to chipset limitations, when oneapplication uses the microphone resource, other applications cannot utilize themicrophone resource. As a result, even on today's mobile devices, achievingclean, personalized, and only user-required audio in the video is impossible,especially in noisy environments with large crowds.

[0004] Therefore, there is a need for a system and method that can overcome theaforementioned drawback and provide a selective audio while recording a videoor in a video environment.

[0005] At present, the existing methodology for automatic localization of theaudio in various systems is performed by computing difference in arrival times ofthe audio at each of the plurality of microphones. Hence, the methodology isfundamentally limited by the number of microphones used and also by the spacebetween the microphones. In cases where the number of microphones is less orthe space between the microphones is smaller, the calculated result can be buggy.

[0006] Thus, in a world where researchers are working on reducing the size andweight of the devices, the application of such mic arrays is nearly impossible.

[0007] Further, there exist numerous prior arts that also provide a system andmethod to separate audio from multiple sources.

[0008] The existing prior art discloses an electronic device and control method.The electronic device comprises a communication interface for communicatingwith an external apparatus in a UWB manner, a microphone, a camera, a sensor, adisplay, and a processor. In one embodiment, the processor obtains distanceinformation and angle information with respect to the external apparatus, on thebasis of data received from the external apparatus, obtains photographingdirection information, on the basis of a detected direction of the camera, identifiesthe external apparatus, on the basis of the obtained photographing directioninformation, distance information, and angle information, controls the microphoneor communication interface to obtain an audio signal, on the basis of a distancefrom the external apparatus, and controls the display to display a UI indicating theamplitude of the obtained audio signal, together with a displayed image.However, the prior art does not disclose providing directional information foreach of one or more audio separated from the received audio, generating an audiosphere including directions for each of the one or more audio, and selecting aportion of the generated audio sphere at the destination device automaticallybased on one or more parameters such as environmental specifications, a profileof the entity, and activity of the entity. Further, the prior art doesn't focus on thecontent of the audio. In addition, the prior art is silent about maintaining themotion of the entity through UWB connection and providing the directional audiobeing selected automatically as per the one or more parameters to the destinationdevice so that only selective audio becomes part of the video recorded and noextra voice gets recorded.

[0009] Further, the existing prior art discloses a method for processing soundsource in terminal and capable of receiving an outside sound source through atleast two microphones and displaying a position of the outside sound source on adisplay unit, and a terminal using the same. In one embodiment, the terminalincludes at least two microphones configured to receive an outside sound source,a display unit configured to display a predetermined data, and a controllerconfigured to obtain position information of the sound source using at least one ofan amplitude, a phase and a period of the outside sound source received from thetwo microphones, and controllably display the position information of the soundsource on a display unit. However, the prior art does not disclose providingdirectional information for each of one or more audio separated from the receivedaudio, generating an audio sphere including directions for each of the one or moreaudio, and selecting a portion of the generated audio sphere at the destinationdevice automatically based on one or more parameters such as environmentalspecifications, a profile of the entity, and activity of the entity. Further, the priorart doesn't focus on the content of the audio. In addition, the prior art is silentabout maintaining the motion of the entity through UWB connection andproviding the directional audio being selected automatically as per the one ormore parameters to the destination device so that only selective audio becomespart of the video recorded and no extra voice gets recorded.

[0010] Therefore, in light of the foregoing discussion, there exists a need toovercome the above mentioned drawbacks associated with the existing methodand system for providing selective audio in the video environment.SUMMARY

[0011] The present invention provides a method for selective audio in a videoenvironment. The method comprises enabling a secure connection between asource device and a destination device while recording a video of an entitycarrying the source device from the destination device. In one embodiment, thesecure connection is an ultra-wide band (UWB) connection between UWBtransmitter / receiver of the source device and the destination device. The methodfurther comprises performing a plurality of functions on audio received by thesource device and providing directional information of each of one or more audioseparated from the received audio to the destination device. The source devicereceives the audio via a single microphone. In one embodiment, the directionalinformation of each of one or more audio is determined by performing stepsincluding receiving the audio via a single microphone of the source device,performing digital audio pre-processing of the received audio to provideprocessed audio, separating one or more audio from the processed audio,extracting a plurality of features including frequency, phase angle, time delay, andamplitude from each of the separated one or more audio, and determiningdirectional information of each of the one or more audio based on extractedplurality of features and providing to the destination device.

[0012] The method further comprises generating an audio sphere around theentity recorded in the video. In one embodiment, the audio sphere is generatedaround the entity recorded in the video by performing steps including receivingdirectional information of each of the one or more audio and generating the audiosphere using the received directional information, extracting the entity recorded inthe video and providing one or more frames including the entity, adjusting thegenerated audio sphere based on the entity in the one or more frames, andaugmenting the adjusted audio sphere on the video and displaying on userinterface of the destination device. The method further comprises selecting aportion of the generated audio sphere at the destination device. In oneembodiment, the portion of the generated audio sphere is selected by a user at thedestination device. In another embodiment, the portion of the generated audiosphere is selected automatically based on one or more parameters of the entityincluding environmental parameters such as state and surrounding of the entity,profile of the entity including at least but not limited to age and profession in casethe entity is a person, activity such as dancing, singing, speaking, householdchores or any other activity performed by the person, location, date, and time.Thereafter, the method comprises integrating the selected portion of audio withthe video of the entity recorded by the destination device. In one embodiment, theselected portion of audio is integrated with the video of the entity recorded by thedestination device by performing steps including localizing exact position of thedestination device using time of flight and phase angle of received ultra-wideband (UWB) radio, tracking the source device and the destination device in realtime in case a change in relative position of the source device and the destinationdevice is determined, changing direction of the UWB radio based on the changein relative position of the source device and the destination device, loading audioin the UWB radio and transmitting the UWB radio to the destination device,receiving the selected portion of audio loaded in the UWB radio by the UWBpaired destination device in a fast and secured manner, and integrating thereceived selected portion of audio with the video recorded by the destinationdevice. In one embodiment, the received selected portion of audio is integratedwith the video by performing steps including receiving video recorded by thedestination device and performing amplification of the received video, receivingthe selected portion of audio from the source device and performing amplificationof the received audio, performing scanning and synchronization of the amplifiedvideo and amplified audio, and combining the scanned and synchronized videoand audio to provide the audio integrated video.

[0013] In accordance with another embodiment, the present invention provides asystem for providing selective audio in a video environment. The systemcomprises an automated secure connection providing module positioned at sourcedevice and destination device is configured for enabling a secure connectionbetween the source device and the destination device. In one embodiment, thesource device is carried by an entity recorded in a video and the destination deviceis a device used for recording video of the entity carrying the source device.

[0014] The system further comprises a directional audio information sharingmodule positioned at the source device is configured for performing a plurality offunctions on audio received by the source device and providing directionalinformation of each of one or more audio separated from the received audio to thedestination device. In one embodiment, the directional audio information sharingmodule comprises a digital audio pre-processing sub-module for receiving theaudio via a single microphone of the source device and performing digital audiopre-processing of the received audio to provide processed audio, an audio waveseparating sub-module for separating one or more audio from the processedaudio, an audio wave feature extracting sub-module for extracting a plurality offeatures including frequency, phase angle, time delay, and amplitude from each ofthe separated one or more audio, and a directional trained audio machine learningsub-module determining directional information of each of the one or more audiobased on extracted plurality of features and providing to the destination device.

[0015] The system further comprises an audio sphere augmenting module,positioned at the destination device is configured for receiving the directionalinformation of each of one or more audio and generating an audio sphereincluding directions of each of the one or more audio received by the entityrecorded in the video. In one embodiment, the audio sphere augmenting modulecomprises a directional audio sphere generating sub-module configured forreceiving directional information of each of the one or more audio and generatingthe audio sphere using the received directional information, an entity extractingsub-module configured for extracting the entity recorded in the video andproviding one or more frames including the person, a sphere direction adjustingsub-module configured for adjusting the generated audio sphere based on theentity in the one or more frames, an audio augmenting sub-module configured foraugmenting the adjusted audio sphere on the video and displaying on userinterface of the destination device.

[0016] The system further comprises an intelligent audio selection modulepositioned at the destination device is configured for selecting a portion of thegenerated audio sphere based on one or more parameters. In one embodiment, theintelligent audio selection module comprises an audio labeler for labeling each ofthe one or more audio in the audio sphere and enabling the user to select a portionat the destination device and a trained machine learning sub-module forautomatically selecting a portion of the generated audio sphere in case no portionof the audio sphere is selected by the user.

[0017] The system further comprises a directional audio transfer module,including some sub-modules positioned at the source device and some at thedestination device is configured for integrating the selected portion of audio withthe video of the entity recorded by the destination device. In one embodiment, thedirectional audio transfer module comprises a UWB locating sub-modulepositioned at the source device is configured for localizing exact position of thedestination device using time of flight and phase angle of received ultra-wideband (UWB) radio, a real time user tracking sub-module positioned at the sourcedevice is configured for tracking the source device and the destination device inreal time in case a change in relative position of the source device and thedestination device is determined, an automatic audio adjustment sub-modulepositioned at the source device is configured for changing direction of the UWBradio carrying the selected portion of audio based on the change in relativeposition of the source device and the destination device, a directional audiotransmitting sub-module positioned at the source device is configured for loadingthe selected portion of audio and transmitting the loaded audio using the UWBradio to the destination device, a UWB receiving sub-module positioned at thedestination device is configured for receiving the selected portion of audio in afast and secured manner, and a directed audio codec integrator positioned at thedestination device is configured for integrating the received selected portion ofaudio with the video recorded by the destination device.

[0018] In one embodiment, directed audio codec integrator comprises a videoamplifying sub-module configured for receiving video recorded by the destinationdevice and performing amplification of the received video, an audio amplifyingsub-module configured for receiving the selected portion of audio from the sourcedevice and performing amplification of the received audio, a scan andsynchronization performing sub-module configured for performing scanning andsynchronization of the amplified video and audio, and a combining sub-moduleconfigured for combining the scanned and synchronized video and audio toprovide the audio integrated video.

[0019] The foregoing summary is illustrative only and is not intended to be inany way limiting. In addition to the illustrative aspects, embodiments, andfeatures described earlier, further aspects, embodiments, and features will becomeapparent by reference to the drawings and the following detailed description.BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated herein and constitutea part of this disclosure, illustrate exemplary embodiments, and together with thedescription, serve to explain the disclosed principles. The same numbers are usedthroughout the figures to reference like features and components, wherein:

[0021] FIG. 1 illustrates a flow diagram showing a method for providing aselective audio in a video environment, in accordance with one or moreexemplary embodiments of the present disclosure;

[0022] FIG. 2 illustrates a block diagram of a system for providing the selectiveaudio in the video environment, in accordance with one or more exemplaryembodiments of the present disclosure;

[0023] FIG. 3 illustrates a pictorial representation of operation of an automatedsecure connection providing module, in accordance with one or more exemplaryembodiments of the present disclosure;

[0024] FIG. 4 illustrates a block diagram of a directional audio informationsharing module, in accordance with one or more exemplary embodiments of thepresent disclosure;

[0025] FIG. 5 illustrates a flow diagram showing working of the directional audioinformation sharing module, in accordance with one or more exemplaryembodiments of the present disclosure;

[0026] FIG. 6 illustrates a block diagram of an audio sphere augmenting module,in accordance with one or more exemplary embodiments of the present disclosure;

[0027] FIG. 7 illustrates a flow diagram showing working of the audio sphereaugmenting module, in accordance with one or more exemplary embodiments ofthe present disclosure;

[0028] FIG. 8 illustrates a block diagram of an intelligent audio selection module,in accordance with one or more exemplary embodiments of the present disclosure;

[0029] FIG. 9 illustrates a block diagram of a directional audio transfer module,in accordance with one or more exemplary embodiments of the present disclosure;

[0030] FIG. 10 illustrates a flow diagram showing working of the directionalaudio transfer module, in accordance with one or more exemplary embodimentsof the present disclosure;

[0031] FIG. 11 illustrates a block diagram of a directed audio codec integrator, inaccordance with one or more exemplary embodiments of the present disclosure;

[0032] FIG. 12 illustrates a flow diagram showing working of the directed audiocodec integrator, in accordance with one or more exemplary embodiments of thepresent disclosure;

[0033] FIG. 13 illustrates a pictorial representation of a first use case disclosesselection of a required audio in a crowded environment while capturing the video,in accordance with one or more exemplary embodiments of the present disclosure;

[0034] FIG. 14 illustrates a pictorial representation of a second use case disclosesselection of required audio while user capturing video of another user using backcamera and sharing the video to another user via a video call, in accordance withone or more exemplary embodiments of the present disclosure;

[0035] FIG. 15 illustrates a pictorial representation of a third use case disclosesselection of required audio while user is capturing his own video and sharing thevideo to another user via the video call, in accordance with one or moreexemplary embodiments of the present disclosure.DETAILED DESCRIPTION OF EMBODIMENTS

[0036] In the following description, numerous specific details are set forth inorder to provide a thorough understanding of the present disclosure. It will beapparent, however, to one skilled in the art that these specific details are onlyexemplary and not intended to be limiting. Additionally, it may be noted that thesystems and / or methods are shown in block diagram form only in order to avoidobscuring the present disclosure. It is to be understood that various omissions andsubstitutions of equivalents may be made as circumstances may suggest or renderexpedient to cover various applications or implementations without departingfrom the spirit or the scope of the present disclosure. Further, it is to beunderstood that the phraseology and terminology employed herein are for thepurpose of clarity of the description and should not be regarded as limiting.

[0037] Furthermore, in the present description, references to "one embodiment"or "an embodiment" means that a particular feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment of the present disclosure. The appearance of the phrase "in oneembodiment" in various places in the specification is not necessarily referring tothe same embodiment, nor are separate or alternative embodiments mutuallyexclusive of other embodiments. Further, the terms "a" and "an" used herein donot denote a limitation of quantity, but rather denote the presence of at least oneof the referenced items. Moreover, various features are described which may beexhibited by some embodiments and not by others. Similarly, variousrequirements are described, which may be requirements for some embodimentsbut not for other embodiments.

[0038] Referring to FIG. 1, a flow diagram showing a method (100) for providingselective audio in a video environment is disclosed. The method may be explainedin conjunction with a system disclosed in FIG.2. In the flow diagram, each blockmay represent a module, segment, or portion of code, which comprises one ormore executable instructions for implementing the specified logical function(s). Itshould also be noted that in some alternative implementations, the functions notedin the blocks may occur out of the order noted in the drawings. For example, twoblocks shown in succession in FIG. 1 may be executed substantially concurrentlyor the blocks may sometimes be executed in the reverse order, depending uponthe functionality involved. Any process descriptions or blocks in flowchartsshould be understood as representing modules, segments, or portions of code thatinclude one or more executable instructions for implementing specific logicalfunctions or steps in the process, and alternate implementations are includedwithin the scope of the example embodiments in which functions may beexecuted out of order from that shown or discussed, including substantiallyconcurrently or in reverse order, depending on the functionality involved. Inaddition, the process descriptions or blocks in flow charts should be understood asrepresenting decisions made by a hardware structure such as a state machine. Theflow diagram starts at step (102) and proceeds to step (110).

[0039] At first, a secure connection between a source device and a destinationdevice is enabled, at step 102. In one exemplary embodiment, the source devicemay be at least one of electronic devices including mobile phone, gear, anearphone, a smart watch, a headphone, a mobile device, or any other electronicdevice equipped with mic (microphone) and the destination device may be amobile device, or any other electronic device equipped with a camera module.The secure connection is enabled between the source device and the destinationdevice when a video of an entity carrying the source device is being recordedfrom the destination device. In one embodiment, the secure connection is an ultrawide band (UWB) connection between UWB transmitter / receiver of the sourcedevice and the destination device. The UWB is a short-range wirelesscommunication protocol that operates at very high frequencies of radio waves tocapture highly accurate spatial and directional data. It should be noted that theenablement of the secure connection includes searching of nearby UWBtransmitter / receiver and establishing connection with desired UWB transmitter / receiver selected by the user. In another embodiment, the secure connection mayinclude any other connection that can successfully implement the invention.

[0040] Successively, a plurality of functions is performed on audio received bythe source device and directional information of each of one or more audioseparated from the received audio is provided to the destination device, at step104. In one embodiment, the audio is received by the source device via a singlemicrophone.

[0041] Successively, an audio sphere is generated, at step 106, around the entityrecorded in the video. In one embodiment, the audio sphere includes directions ofeach of the one or more audio received by the entity.

[0042] Successively, a portion of the generated audio sphere is selected, at step108, at the destination device. In one embodiment, the portion of the generatedaudio sphere is selected by a user at the destination device. In anotherembodiment, the portion of the generated audio sphere is selected automaticallybased on one or more parameters of the entity including environmental parameterssuch as state and surrounding of the entity, profile of the entity including at leastbut not limited to age and profession in case the entity is a person, activity such asdancing, singing, speaking, household chores or any other activity performed bythe person, location, date, and time.

[0043] Thereafter, the selected portion of audio is integrated with the video of theentity recorded by the destination device, at step 110. The selected portion ofaudio integrated with the video is then displayed on user interface of thedestination device.

[0044] Referring to FIG. 2 a block diagram of a system (200) for providing theselective audio in the video environment is illustrated, in accordance with one ormore exemplary embodiments of the present disclosure. The system (200) maycomprise a plurality of modules for adjusting the privacy level automatically. Atleast one of the plurality of modules may be implemented through an AI model. Afunction associated with AI may be performed through the non-volatile memory,the volatile memory, and the processor.

[0045] The processor may include one or a plurality of processors. At this time,one or a plurality of processors may be a general purpose processor, such as acentral processing unit (CPU), an application processor (AP), or the like, agraphics-only processing unit such as a graphics processing unit (GPU), a visualprocessing unit (VPU), and / or an AI-dedicated processor such as a neuralprocessing unit (NPU).

[0046] The one or a plurality of processors control the processing of the inputdata in accordance with a predefined operating rule or artificial intelligence (AI)model stored in the non-volatile memory and the volatile memory. The predefinedoperating rule or artificial intelligence model is provided through training orlearning.

[0047] Here, being provided through learning means that, by applying a learningalgorithm to a plurality of learning data, a predefined operating rule or AI modelof a desired characteristic is made. The learning may be performed in a deviceitself in which AI according to an embodiment is performed, and / or may beimplemented through a separate server / system.

[0048] The AI model may consist of a plurality of neural network layers. Eachlayer has a plurality of weight values, and performs a layer operation throughcalculation of a previous layer and an operation of a plurality of weights.Examples of neural networks include, but are not limited to, convolutional neuralnetwork (CNN), deep neural network (DNN), recurrent neural network (RNN),restricted Boltzmann Machine (RBM), deep belief network (DBN), bidirectionalrecurrent deep neural network (BRDNN), generative adversarial networks (GAN),and deep Q-networks.

[0049] The learning algorithm is a method for training a predetermined targetdevice (for example, a robot) using a plurality of learning data to cause, allow, orcontrol the target device to make a determination or prediction. Examples oflearning algorithms include, but are not limited to, supervised learning,unsupervised learning, semi-supervised learning, or reinforcement learning.

[0050] In one embodiment, the system (200) comprises an automated secureconnection providing module (202) configured for enabling a secure connectionbetween the source device and the destination device. It should be noted that theautomated secure connection providing module (202) is positioned at a sourcedevice and at a destination device also. In one embodiment, the source device iscarried by an entity recorded in a video and the destination device is a device usedfor recording video of the entity carrying the source device. The entity can be aperson or a group of people or part of people who share common attributes. Theentity may also be part of people who share or do not share certain single ormultiple attributes. Further, the entity can also be an object, including but notlimited to a product, location, or item. The automated secure connection providingmodule (202) can be explained in detail in conjunction with FIG. 3.

[0051] Referring to FIG. 3, a pictorial representation of operation of theautomated secure connection providing module (202) is illustrated, in accordancewith one or more exemplary embodiments of the present disclosure. Asillustrated, a user is able to see all nearby entities with UWB transmitter / receiverand select any one as per requirement. After the selection of the required entity,the user may open camera of the destination device and point toward the entity infocus. Successively, the destination device may start ranging. In one embodiment,the ranging refers to computing time of flight (ToF) between the source devicecarrying by the entity and the destination device in order to detect UWBtransmitter / receiver of the entity. The computation of the ToF is performed usingequation given below:ToF= TLoop-TReplay / 2

[0052] It should be noted that in order to enable the secure connections the sourcedevice and the destination device includes automated secure connection providingmodule (202), graphic user interface (GUI), and UWB communication interface.In one embodiment the graphic user interface (also referred to as user interface) isan interface through which a user interacts with electronic devices such as mobilephones, computers, and smartphones through icons, menus and other visualindicators or graphics. The GUI graphically display information and related usercontrols.

[0053] The system (200) further comprises a directional audio informationsharing module (204) configured for performing a plurality of functions on audioreceived by the source device and providing directional information of each ofone or more audio separated from the received audio to the destination device. Inone embodiment, the directional audio information sharing module (204) ispositioned at the source device, the directional audio information sharing module(204) can be explained in conjunction with FIG. 4 and FIG 5.

[0054] Referring to FIG. 4 a block diagram of the directional audio informationsharing module (204) is illustrated, in accordance with one or more exemplaryembodiments of the present disclosure. As illustrated, directional audioinformation sharing module (204) comprises a digital audio pre-processing submodule (402). In one embodiment, the digital audio pre-processing sub-module(402) is configured for receiving the audio via a single microphone of the sourcedevice and performing digital audio pre-processing of the received audio toprovide processed audio. The digital audio pre-processing is performed to preparethe received audio for the primary processing or for further analysis.

[0055] The directional audio information sharing module (204) further comprisesan audio wave separating sub-module (404). In one embodiment, the audio waveseparating sub-module (404) is configured for separating one or more audio fromthe processed audio. The directional audio information sharing module (204)further comprises an audio wave feature extracting sub-module (406). The audiowave feature extracting sub-module (406) is configured for extracting a pluralityof features from each of the separated one or more audio. The directional audioinformation sharing module (204) further comprises a directional trained audiomachine learning sub-module (408). In one embodiment, the directional trainedaudio machine learning sub-module (408) is configured for determiningdirectional information of each of the one or more audio based on extractedplurality of features including frequency, phase angle, time delay, and amplitudeand providing the determined directional information to the destination device.The directional trained audio machine learning sub-module (408) providesdirectional information of each of the one or more audio to automated secureconnection providing module (202) and one or more audio along with directionalinformation of each of the one or more audio to an intelligent audio selectionmodule.

[0056] Referring to FIG. 5 a flow diagram showing working of the directionalaudio information sharing module is illustrated, in accordance with one or moreexemplary embodiments of the present disclosure. As illustrated, the audio isreceived, at step 502, via a single microphone of the source device. Successively,digital audio pre-processing of the received audio is performed, at step 504 toprovide processed audio. Successively, one or more audio is separated, at step506, from the processed audio. Successively, a plurality of features are extractedfrom each of the separated one or more audio, at step 506. Thereafter, directionalinformation of each of the one or more audio is determined, at step 508, based onextracted plurality of features and provided to the destination device.

[0057] The system (200) further comprises an audio sphere augmenting module(206) configured for receiving the directional information of each of one or moreaudio and generating an audio sphere around the entity recorded in the video. Theaudio sphere generally includes directions of each of the one or more audioreceived by the entity. In one embodiment, the audio sphere augmenting module(206) is positioned at the destination device, the audio sphere augmenting module(206) can be explained in conjunction with FIG. 6 and FIG 7.

[0058] Referring to FIG. 6 a block diagram of the audio sphere augmentingmodule (206) is illustrated, in accordance with one or more exemplaryembodiments of the present disclosure. As illustrated, the audio sphereaugmenting module (206) comprises a directional audio sphere generating submodule (602). In one embodiment, the directional audio sphere generating submodule (602) is configured for receiving directional information of each of theone or more audio from the directional audio information sharing module (204)and generating the audio sphere using the received directional information.

[0059] The audio sphere augmenting module (206) further comprises an entityextracting sub-module (604). In one embodiment, the entity extracting submodule (604) is configured for extracting the entity recorded in the video andproviding one or more frames which includes the entity. The audio sphereaugmenting module (206) further comprises a sphere direction adjusting submodule (606). The sphere direction adjusting sub-module (606) is configured forreceiving the generated audio sphere from the directional audio sphere generatingsub-module (602) and the one or more frames including the entity from the entityextracting sub-module (604) and adjusting the generated audio sphere based onthe entity in the one or more frames. The audio sphere augmenting module (206)further comprises an audio augmenting sub-module (608). In one embodiment,the audio augmenting sub-module (608) is configured for augmenting the adjustedaudio sphere on the video and displaying on user interface of the destinationdevice.

[0060] Referring to FIG. 7 a flow diagram showing working of the audio sphereaugmenting module is illustrated, in accordance with one or more exemplaryembodiments of the present disclosure. As illustrated, directional information ofeach of the one or more audio is received and the audio sphere is generated usingthe received directional information, at step 702. Successively, the entity recordedin the video is extracted and one or more frames including the entity are provided,at step 704. Successively, the generated audio sphere is adjusted, at step 706,based on the entity in the one or more frames. Thereafter, the adjusted audiosphere is augmented on the video and displayed on user interface of thedestination device, at step 708.

[0061] The system (200) further comprises an intelligent audio selection module(208) configured for selecting a portion of the generated audio sphere based onone or more parameters. The intelligent audio selection module (208) can beexplained in conjunction with FIG. 8.

[0062] Referring to FIG.8, a block diagram of the intelligent audio selectionmodule is illustrated, in accordance with one or more exemplary embodiments ofthe present disclosure. In one embodiment, the intelligent audio selection module(208) is positioned at the destination device. The intelligent audio selectionmodule (208) comprises an audio labeler (802) for labeling each of the one ormore audio in the audio sphere and enabling the user to select a portion at thedestination device and a trained machine learning sub-module (804) forautomatically selecting a portion of the generated audio sphere in case no portionof the audio sphere is selected by the user. In one embodiment, the portion of thegenerated audio sphere is selected by a user at the destination device. In anotherembodiment, the portion of the generated audio sphere is selected automaticallybased on one or more parameters of the entity including environmental parameterssuch as state and surrounding of the entity, profile of the entity including at leastbut not limited to age and profession in case the entity is a person, activity such asdancing, singing, speaking, household chores or any other activity performed bythe person, location, date, and time.

[0063] The system (200) further comprises a directional audio transfer module(210). The directional audio transfer module (210) is configured for integratingthe selected portion of audio with the video of the entity recorded by thedestination device. In one embodiment, the directional audio transfer module(210) comprises some sub-modules positioned at the source device and some atthe destination device, which can be explained in conjunction with FIG. 9 andFIG. 10.

[0064] Referring to FIG. 9, a block diagram of the directional audio transfermodule (210) is illustrated, in accordance with one or more exemplaryembodiments of the present disclosure. As illustrated, the directional audiotransfer module (210) comprises a UWB locating sub-module (902) positioned atthe source device and is configured for localizing exact position of the destinationdevice using time of flight and phase angle of received ultra-wide band (UWB)radio. It should be noted that the UWB provides accurate positioning and radiofrequency security, which other wireless connectivity technologies such asBluetooth and Wi-Fi generally do not provides and the most significant advantageof the UWB is the added portion of physical layer that is used to send and receivedata packets and provide a critical security extension not available in otherwireless connectivity technologies. The directional audio transfer module (210)further comprises a real time user tracking sub-module (904) positioned at thesource device and is configured for tracking the source device and the destinationdevice in real time in case a change in relative position of the source device andthe destination device is determined. It should be noted that the real time usertracking sub-module (904) triggers the UWB wave interaction between UWBantennas of the source device and the destination device as soon as there is amotion or a change in relative position between these devices is determined.

[0065] The directional audio transfer module (210) further comprises anautomatic audio adjustment sub-module (906) positioned at the source device andis configured for changing direction of the UWB radio carrying the selectedportion of audio based on the change in relative position of the source device andthe destination device. The directional audio transfer module (210) furthercomprises a directional audio transmitting sub-module (908) positioned at thesource device and is configured for loading the selected portion of audio andtransmitting the loaded audio using the UWB radio to the destination device. Thedirectional audio transfer module (210) further comprises a UWB receiving submodule (910) positioned at the destination device and is configured for receivingthe selected portion of audio in a fast and secured manner. The directional audiotransfer module (210) further comprises a directed audio codec integrator (912)positioned at the destination device and is configured for integrating the receivedselected portion of audio with the video recorded by the destination device. Thedirected audio codec integrator (912) can be explained in detail using FIG. 11.

[0066] Referring to FIG. 10, a flow diagram showing working of the directionalaudio transfer module is illustrated, in accordance with one or more exemplaryembodiments of the present disclosure As illustrated, exact position of thedestination device is localized, at step 1002, using time of flight and phase angleof received ultra-wide band (UWB) radio. Successively, the source device and thedestination device are tracked, at step 1004, in real time in case a change inrelative position of the source device and the destination device is determined.Successively, the direction of the UWB radio is changed based on the change inrelative position of the source device and the destination device, at step 1006. Itshould be noted that in case no change is determined in relative position of thesource device and the destination device, no change in the direction of the UWBradio is performed. Successively, the selected portion of audio is loaded in theUWB radio and transmitted the UWB radio to the destination device, at step 1008.Successively, the selected portion of audio loaded in the UWB radio is received,at step 1010, by the UWB paired destination device in a fast and secured manner.Thereafter, the received selected portion of audio is integrated with the videorecorded by the destination device, at step 1012. The integration of the receivedselected portion of audio is with the video is explained in conjunction with FIG.12.

[0067] Referring to FIG. 11, a block diagram of a directed audio codec integratoris illustrated, in accordance with one or more exemplary embodiments of thepresent disclosure. As illustrated, the directed audio codec integrator (912)comprises a video amplifying sub-module (1102). The video amplifying submodule (1102) is configured for receiving video recorded by the destinationdevice and performing amplification of the received video. In one exemplaryembodiment, the video amplifying sub-module (1102) comprises a resistor, acapacitor, and a video operational amplifying chip, an input of the videooperational amplifying chip receives the video recorded by the destination deviceand provides the amplified video at the output of the video operational amplifyingchip. In another embodiment, the video amplifying sub-module (1102) comprisesany other arrangement of electronic components to perform amplification of thevideo.

[0068] The directed audio codec integrator (912) further comprises an audioamplifying sub-module (1104). The audio amplifying sub-module (1104) isconfigured for receiving the selected portion of audio from the source device andperforming amplification of the received audio. In one exemplary embodiment,the audio amplifying sub-module (1104) comprises an audio amplifying chip, aninput of the audio amplifying chip receives the selected portion of audio from thesource device and provides the amplified audio at the output of the videooperational amplifying chip. In another embodiment, the audio amplifying submodule (1104) comprises any other arrangement of electronic components toperform amplification of the audio.

[0069] The directed audio codec integrator (912) further comprises a scan andsynchronization performing sub-module (1106). The scan and synchronizationperforming sub-module (1106) is configured for performing scanning andsynchronization of the amplified video and audio. The directed audio codecintegrator (912) further comprises a combining sub-module (1108). Thecombining sub-module (1108) is configured for combining the scanned andsynchronized video and audio to provide the audio integrated video.

[0070] Referring to FIG. 12, a flow diagram showing working of the directedaudio codec integrator is illustrated, in accordance with one or more exemplaryembodiments of the present disclosure As illustrated, video recorded by thedestination device is received and amplification of the received video isperformed, at step 1202. The video is recorded by the destination device using acamera positioned at surface of the destination device. Successively, the selectedportion of audio is received from the source device and amplification of thereceived audio is performed, at step 1204. Successively, scanning andsynchronization of the amplified video and amplified audio is performed, at step1206. Thereafter, the scanned and synchronized video and audio is combined toprovide the audio integrated video, at step 1208.

[0071] Referring to FIG. 13, a pictorial representation of a first use case thatdiscloses the selection of a required audio in a crowded environment whilecapturing the video, in accordance with one or more exemplary embodiments ofthe present disclosure. As illustrated in image "A", a user is recording a video of aperson dancing in a crowded place using the camera of his mobile phone. Thevideo recorded by the user is highly focused visually, but the audio recording isnot focused, as the mobile phone of the user receives audio from 360-degreesurroundings of the person, which includes unwanted noise or instruction by thecameraman. In order to overcome the above-mentioned drawbacks associatedwith the existing method and system, the invention is used and image "B"illustrates the scenario where the invention is being utilized for recording theselective audio in the video environment.

[0072] As illustrated, the user is recording a video of the person dancing in thecrowded place using the camera of his mobile phone. The present invention firstpairs the mobile phone of the user with the mobile phone of the person dancing ina crowded place using a UWB wireless connection. The present invention thengenerates an audio sphere including directions for each audio received by theperson and enables the user to select a portion of audio the user is interested inreceiving with video on his mobile phone. The present invention then provideshighly focused video and highly focused audio recordings of the person to theuser.

[0073] Referring to FIG. 14, a pictorial representation of a second use case thatdiscloses the selection of required audio while capturing video of another userusing back camera and sharing the video via a video call, in accordance with oneor more exemplary embodiments of the present disclosure. As illustrated in image"A", a user 1 is recording a video of a person using back camera of his mobilephone and sharing the video to user 2. The video recorded by the user1 is highlyfocused visually, but the audio recording is not focused, as the mobile phone ofthe user 1 receives audio from surrounding of the person that includes voice of theperson and unwanted noises or noise from his own surrounding. In order toovercome the above-mentioned drawbacks associated with the existing methodand system, the invention is used and image "B" illustrates the scenario where theinvention is being utilized for recording the selective audio in the videoenvironment.

[0074] As illustrated, the user1 is recording a video of the person using backcamera of his mobile phone. The present invention first pairs the mobile phone ofthe user1 with the earphone of the person using a UWB wireless connection. Thepresent invention then generates an audio sphere around the person includingdirections for each audio received by the person and enables the user1 to select aportion of audio the user1 is interested in receiving with video on his mobilephone. The present invention then provides highly focused video and highlyfocused audio recordings of the person to the user1 which the user1 then transmitto the user 2 through video call.

[0075] Referring to FIG. 15, a pictorial representation of a third use case thatdiscloses the selection of required audio while the user is capturing his own videoand sharing it via the video call, in accordance with one or more exemplaryembodiments of the present disclosure. As illustrated in image "A", user 1 isrecording his own video using the front camera of his mobile phone and sharingthe video with user 2. The video recorded by the user1 is highly focused visually,but the audio recording is not focused, as the mobile phone of the user 1 receivesaudio from his surrounding which includes the voice of two persons talking toeach other. The recorded video is then sent by user1 to user 2 via video call,which user 2 is unable to comprehend, and sends a response to user 1 about thenoise in the background. In order to overcome the above-mentioned drawbacksassociated with the existing method and system, the invention is used, and image"B" illustrates the scenario where the invention is being utilized for recordingselective audio in the video environment.

[0076] As illustrated, the user 1 is recording his own video using the frontcamera of his mobile phone and sharing the video with user 2. The presentinvention first pairs the mobile phone of the user1 with watches carrying by thetwo persons standing nearby to the user1 using a UWB wireless connection. Thepresent invention then generates an audio sphere around the user1 includingdirections for each audio received by the user1 and enables the user1 to select aportion of audio the user1 is interested in recording with video on his mobilephone. The present invention then provides highly focused video and highlyfocused audio recordings of the user1 which the user1 then transmits to the user 2through video call.

[0077] It has thus been seen that the system and method for the selective audio inthe video environment according to the present invention achieve the purposeshighlighted earlier. Such a system and method can in any case undergo numerousmodifications and variants, all of which are covered by the same innovativeconcept, moreover, all of the details can be replaced by technically equivalentelements. The scope of protection of the invention is therefore defined by theattached claims.

Claims

1. A method (100) for providing a selective audio in a video environment, the method (100) comprises: enabling a secure connection between a source device and a destination device while recording a video of an entity carrying the source device from the destination device; performing a plurality of functions on audio received by the source device and providing directional information of each of one or more audio separated from the received audio to the destination device; generating an audio sphere around the entity recorded in the video, wherein the audio sphere includes directions of each of the one or more audio received by the entity; selecting a portion of the generated audio sphere at the destination device; and integrating the selected portion of audio with the video of the entity recorded by the destination device.

2. The method (100) as claimed in claim 1, wherein the secure connection is an ultra-wide band (UWB) connection between UWB transmitter / receiver of the source device and the destination device.

3. The method (100) as claimed in claim 1, wherein the source device receives the audio via a single microphone.

4. The method (100) as claimed in claim 1, wherein the directional information of each of one or more audio is determined by performing steps: receiving the audio via a single microphone of the source device; performing digital audio pre-processing of the received audio to provide processed audio; separating one or more audio from the processed audio; extracting a plurality of features from each of the separated one or more audio; and determining directional information of each of the one or more audio based on extracted plurality of features and providing to the destination device.

5. The method (100) as claimed in claim 4, wherein the extracted plurality of features include frequency, phase angle, time delay, and amplitude.

6. The method (100) as claimed in claim 1, wherein the audio sphere is generated around the entity recorded in the video by performing steps including: receiving directional information of each of the one or more audio and generating the audio sphere using the received directional information; extracting the entity recorded in the video and providing one or more frames including the entity; adjusting the generated audio sphere based on the entity in the one or more frames; and augmenting the adjusted audio sphere on the video and displaying on user interface of the destination device.

7. The method (100) as claimed in claim 1, wherein the portion of the generated audio sphere is selected by a user at the destination device.

8. The method (100) as claimed in claim 1, wherein the portion of the generated audio sphere is selected automatically based on one or more parameters of the entity including environmental parameters such as state and surrounding of the entity, profile of the entity including at least but not limited to age and profession in case the entity is a person, activity such as dancing, singing, speaking, household chores or any other activity performed by the person, location, date, and time.

9. The method (100) as claimed in claim 1, wherein the selected portion of audio is integrated with the video of the entity recorded by the destination device by performing steps: localizing exact position of the destination device using time of flight and phase angle of received ultra-wide band (UWB) radio; tracking the source device and the destination device in real time in case a change in relative position of the source device and the destination device is determined; changing direction of the UWB radio based on the change in relative position of the source device and the destination device; loading the selected portion of audio in the UWB radio and transmitting the UWB radio to the destination device; receiving the selected portion of audio loaded in the UWB radio by the UWB paired destination device in a fast and secured manner; and integrating the received selected portion of audio with the video recorded by the destination device.

10. The method (100) as claimed in claim 9, wherein the received selected portion of audio is integrated with the video by performing steps including: receiving video recorded by the destination device and performing amplification of the received video; receiving the selected portion of audio from the source device and performing amplification of the received audio; performing scanning and synchronization of the amplified video and amplified audio; and combining the scanned and synchronized video and audio to provide the audio integrated video.

11. A system (200) for providing a selective audio in a video environment, the system (200) comprising: an automated secure connection providing module (202) positioned at source device and destination device is configured for enabling a secure connection between the source device and the destination device, wherein the source device is carried by an entity recorded in a video and the destination device is a device used for recording video of the entity carrying the source device; a directional audio information sharing module (204) positioned at the source device is configured for performing a plurality of functions on audio received by the source device and providing directional information of each of one or more audio separated from the received audio to the destination device; an audio sphere augmenting module (206), positioned at the destination device is configured for receiving the directional information of each of one or more audio and generating an audio sphere around the entity recorded in the video, wherein the audio sphere includes directions of each of the one or more audio received by the entity; an intelligent audio selection module (208), positioned at the destination device is configured for selecting a portion of the generated audio sphere based on one or more parameters; and a directional audio transfer module (210), including some submodules positioned at the source device and some at the destination device is configured for integrating the selected portion of audio with the video of the entity recorded by the destination device.

12. The system (200) as claimed in claim 11, wherein the secure connection is an ultra-wide band (UWB) connection between UWB transmitter / receiver of the source device and the destination device.

13. The system (200) as claimed in claim 11, wherein the source device is configured for receiving the audio via a single microphone.

14. The system (200) as claimed in claim 11, wherein the directional audio information sharing module (204) comprises: a digital audio pre-processing sub-module (402) for receiving the audio via a single microphone of the source device and performing digital audio pre-processing of the received audio to provide processed audio; an audio wave separating sub-module (404) for separating one or more audio from the processed audio; an audio wave feature extracting sub-module (406) for extracting a plurality of features from each of the separated one or more audio; and a directional trained audio machine learning sub-module (408) determining directional information of each of the one or more audio based on extracted plurality of features and providing to the destination device.

15. The system (200) as claimed in claim 14, wherein the extracted plurality of features include frequency, phase angle, time delay, and amplitude.

16. The system (200) as claimed in claim 11, wherein the audio sphere augmenting module (206) comprises: a directional audio sphere generating sub-module (602) configured for receiving directional information of each of the one or more audio and generating the audio sphere using the received directional information; an entity extracting sub-module (604) configured for extracting the entity recorded in the video and providing one or more frames including the person; a sphere direction adjusting sub-module (606) configured for adjusting the generated audio sphere based on the entity in the one or more frames; and an audio augmenting sub-module (608) configured for augmenting the adjusted audio sphere on the video and displaying on user interface of the destination device.

17. The system (200) as claimed in claim 11, wherein the intelligent audio selection module (208) comprises an audio labeler (802) for labeling each of the one or more audio in the audio sphere and enabling the user to select a portion at the destination device and a trained machine learning submodule (804) for automatically selecting a portion of the generated audio sphere in case no portion of the audio sphere is selected by the user.

18. The system (200) as claimed in claim 17, wherein the portion is selected automatically based on one or more parameters of the entity including environmental parameters such as state and surrounding of the entity, profile of the entity including at least but not limited to age and profession in case the entity is a person, activity such as dancing, singing, speaking, household chores or any other activity performed by the person, location, date, and time.

19. The system (200) as claimed in claim 11, wherein the directional audio transfer module (210), comprises: a UWB locating sub-module (902), positioned at the source device is configured for localizing exact position of the destination device using time of flight and phase angle of received ultra-wide band (UWB) radio; a real time user tracking sub-module (904) positioned at the source device is configured for tracking the source device and the destination device in real time in case a change in relative position of the source device and the destination device is determined; an automatic audio adjustment sub-module (906) positioned at the source device is configured for changing direction of the UWB radio carrying the selected portion of audio based on the change in relative position of the source device and the destination device; a directional audio transmitting sub-module (908) positioned at the source device is configured for loading the selected portion of audio and transmitting the loaded audio using the UWB radio to the destination device; a UWB receiving sub-module (910) positioned at the destination device is configured for receiving the selected portion of audio in a fast and secured manner; and a directed audio codec integrator (912) positioned at the destination device is configured for integrating the received selected portion of audio with the video recorded by the destination device.

20. The system (200) as claimed in claim 19, wherein the directed audio codec integrator (912) comprises: a video amplifying sub-module (1102) configured for receiving video recorded by the destination device and performing amplification of the received video; an audio amplifying sub-module (1104) configured for receiving the selected portion of audio from the source device and performing amplification of the received audio; a scan and synchronization performing sub-module (1106) configured for performing scanning and synchronization of the amplified video and audio; and a combining sub-module (1108) configured for combining the scanned and synchronized video and audio to provide the audio integrated video.