Android-based voice remark method and device

Abstract

The invention provides an Android-based voice remark method and device, and the method comprises the steps: copying first audio data as second audio data from a current time point if a remark request is received when a multimedia file is recorded on Android equipment, and carrying out the coding processing of the second audio data, so as to generate remark audio data; after the remark request is finished, copying is stopped, and remark audio data generated in the period serve as voice remarks to be stored; according to the invention, voice remarks can be generated along with audio and video recording.

Description

Technical Field

[0001] The present application relates to the field of voice processing technology, and in particular to an Android-based voice annotation method and device. Background Art

[0002] In the existing Android voice note solutions, there are few ways to implement voice notes, and the applications are severely fragmented. Although there are some third-party applications that can implement voice notes, these applications often have different focuses, some focus on recording, and some focus on text transcription, lacking a comprehensive solution. Users need to switch between multiple applications, which increases the complexity of use. Many Android applications do not provide a convenient way to integrate voice recording with other functions (such as calendar, task management), resulting in users being unable to complete multiple tasks in one application.

[0003] In addition, there are many restrictions on implementing voice notes. Although some applications provide basic voice recognition functions, they lack advanced functions such as intelligent classification and automatic summarization, which requires users to spend more time and energy when processing voice notes. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a voice annotation method and device based on Android, which can generate voice annotations along with audio and video recordings.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides an Android-based voice note method, which is applied to an Android device, wherein the Android device includes a recording function, and the method includes:

[0007] In response to the recording request, recording a multimedia file through the Android device, wherein the multimedia file includes video data and first audio data;

[0008] In the process of recording the multimedia file, in response to the start of the annotation request, the first audio data is copied from the current time to obtain the second audio data, and the second audio data is encoded to obtain the annotation audio data;

[0009] In response to the end of the comment request, the copying of the first audio data is stopped, and the comment audio data obtained from the start to the end of the comment request is used as the voice comment.

[0010] In a second aspect, an embodiment of the present application further provides an Android-based voice note device, which is applied to an Android device, wherein the Android device includes a recording function, and the device includes:

[0011] A recording module, configured to record a multimedia file through the Android device in response to a recording request, wherein the multimedia file includes video data and first audio data;

[0012] a first annotation module, configured to copy the first audio data from a current time to obtain second audio data in response to a start of an annotation request during recording of the multimedia file, and to encode the second audio data to obtain annotation audio data;

[0013] The second annotation module is used to stop copying the first audio data in response to the end of the annotation request, and use the annotated audio data obtained from the beginning to the end of the annotation request as the voice annotation.

[0014] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to execute the Android-based voice annotation method described in any one of the first aspects.

[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the Android-based voice annotation method described in any one of the first aspects is executed.

[0016] The embodiments of the present application have the following beneficial effects:

[0017] (1) Immediacy and convenience: This method allows users to add voice notes while recording multimedia files (such as videos). This instant feedback mechanism greatly improves the convenience of user operation, allowing users to complete notes without interrupting the recording process or switching to other applications, effectively saving time and operation steps.

[0018] (2) Flexibility: By copying and encoding the first audio data from the current time, users can freely choose to start and end the annotation request at any time during the recording process, thereby flexibly adding multiple voice annotations to the multimedia file. This flexibility enables users to annotate key information in detail or provide immediate feedback as needed.

[0019] (3) Maintaining the integrity of the original data: When generating voice notes during the recording process, the method only copies the first audio data without directly modifying the original multimedia file. This ensures the integrity and availability of the original data, and the user can access and edit the original file and its notes at any time in subsequent processing.

[0020] (4) Efficient data management: By managing the annotated audio data as independent voice notes, this method provides users with an efficient data management method. Users can easily view, edit, or delete voice notes without modifying the entire multimedia file, which improves the efficiency and accuracy of data management.

[0021] (5) Improve user experience: Taking all the above advantages into consideration, this method significantly improves the user experience when recording and managing multimedia files. Users can focus more on content creation and recording, while easily completing annotation work, reducing operational complexity and time cost.

[0022] In summary, this Android-based voice annotation method brings significant benefits to users by providing an instant, flexible, efficient annotation function that maintains the integrity of the original data, and improves the overall experience of multimedia recording and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 is a flowchart of steps S101-S103 provided in an embodiment of the present application;

[0025] Figure 2 It is a flowchart of steps S201-S202 provided in an embodiment of the present application;

[0026] Figure 3 It is a flowchart of steps S301-S303 provided in an embodiment of the present application;

[0027] Figure 4 It is a flowchart of steps S401-S403 provided in an embodiment of the present application;

[0028] Figure 5 It is a flowchart of steps S501-S503 provided in an embodiment of the present application;

[0029] Figure 6It is a principle block diagram of the voice note method based on Android provided in an embodiment of the present application;

[0030] Figure 7 It is a structural diagram of an Android-based voice note device provided in an embodiment of the present application;

[0031] Figure 8 It is a schematic diagram of the composition structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0032] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0033] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0034] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0035] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0036] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0038] See also Figure 1 , Figure 1 This is a flow chart of steps S101-S103 of the voice note method based on Android provided in the embodiment of the present application, which will be combined with Figure 1 Steps S101-S103 are shown for explanation.

[0039] In step S101, in response to a recording request, a multimedia file is recorded by the Android device, wherein the multimedia file includes video data and first audio data.

[0040] For examples, see Figure 6 , Figure 6 is a principle block diagram of a voice note method based on Android provided in an embodiment of the present application, such as Figure 6 As shown, first start the recording mode and collect the audio and video to be recorded. This is the starting point of the recording process. The user triggers the recording request through interface operation or program logic.

[0041] Use Android's Camera2 API and OpenGL ES to capture the camera preview image and use it as a video input source. The Camera2 API is used to control the camera hardware, including preview, photo taking, and video recording. OpenGL ES is used for image processing, including texture creation, texture binding, and texture drawing. The main purpose here is to convert the image data of the camera preview into a texture for subsequent video encoding. Since the camera preview usually outputs data in YUV format, and it is more common for OpenGL ES to process RGB format, format conversion is required. The texture (i.e., the camera preview image) is then mapped to the drawing surface of OpenGL ES to achieve the preview effect.

[0042] Use the AudioRecord class to record audio data from the microphone. Set the following parameters:

[0043] Sampling rate: The number of samples per second, which affects the audio clarity and file size.

[0044] Number of channels: The number of audio channels, such as mono or stereo.

[0045] Audio format: such as PCM, which represents uncompressed raw audio data.

[0046] For video encoding, use the MediaCodec class to encode the camera texture (actually the processed video frames) into a video stream.

[0047] Coding format: such as H.264 or H.265, which is the standard for video compression.

[0048] Video parameters: including video width, height, frame rate, bit rate, etc., which affect the quality and file size of the video.

[0049] Input data: Texture data from OpenGL ES, which is processed appropriately and used as input to the encoder.

[0050] For audio encoding, the MediaCodec class is also used to encode PCM audio data into an AAC audio stream.

[0051] Audio format: such as AAC, which is a standard for audio compression.

[0052] Sampling rate, number of channels, bit rate: the same as those used for audio recording. Ensure that the encoding configuration matches the recording configuration.

[0053] When the actual recording is turned on, use the MediaMuxer API to create an MP4 file and add the recorded video and audio streams to the MP4 file. MediaMuxer encapsulates the encoded video and audio data to generate the final MP4 file.

[0054] In step S102, during recording of the multimedia file, in response to the start of the annotation request, the first audio data is copied from the current time to obtain second audio data, and the second audio data is encoded to obtain annotated audio data.

[0055] For examples, please see Figure 6, the system first detects a user-initiated annotation request. This can be achieved through button clicks on the interface, gestures, or voice commands. When the annotation request is triggered, the system records the current timestamp (or frame number), which will be the starting point of the annotated audio data. Starting from this timestamp, the system copies the audio stream being recorded (the first audio data) in real time. This means that while the audio data is being recorded, it is also copied to a separate buffer or stream, which we call the "second audio data". During the copying process, it is necessary to ensure that the second audio data is completely synchronized with the first audio data. This means that the two should correspond in time so that the annotated audio data can be accurately inserted into the original audio stream later. Next, the system encodes the second audio data. The encoding configuration should be selected according to the requirements, such as audio format (such as AAC), bit rate, sampling rate, etc. These parameters can vary depending on the purpose of the annotated audio and the final storage format. Since the annotation request is initiated during the recording process, the encoding process needs to be real-time or near real-time. The audio data is encoded while or immediately after being copied to ensure that no excessive delay is introduced. After the encoding process, the obtained audio data is the remark audio data. These data will be kept in the memory or stored in the storage medium in the form of files for subsequent use.

[0056] When the user ends the annotation request (triggered by the corresponding operation), the system will record the current timestamp (or frame number) again, which will be the end point of the annotated audio data. From this timestamp, the system will stop copying and encoding the first audio data. At this point, the annotated audio data has been completely generated and is ready for subsequent operations.

[0057] In step S103, in response to the end of the annotation request, the copying of the first audio data is stopped, and the annotated audio data obtained from the start to the end of the annotation request is used as the voice annotation.

[0058] Here, when the user ends the annotation request through an interface operation (such as clicking a stop button), a gesture, or a voice command, the system detects this event. The system records the current timestamp (or frame number), which marks the end point of the annotated audio data. Once the end of the annotation request is detected, the system immediately stops the copying operation of the first audio data. The audio data copied from the beginning to the end of the annotation request (i.e., the second audio data) is now a complete and independent audio segment.

[0059] It should be noted that if the audio data is encoded while being copied, then it is necessary to ensure that the encoding process is also completed. If the encoding is performed asynchronously, the system needs to wait for the encoding task to be completed.

[0060] In some embodiments, see Figure 2 , Figure 2 It is a flowchart of steps S201-S202 provided in an embodiment of the present application. The method also includes steps S201-S202, which will be described in combination with each step.

[0061] In step S201, in response to a first play request for the multimedia file, the multimedia file is played through a graphical user interface.

[0062] In step S202, during the playing of the multimedia file, the position of the voice note is displayed in the playing progress control.

[0063] Here, when the user initiates a request to play a multimedia file through a graphical user interface (GUI), the system recognizes and responds to this request. Once a playback request is received, the system uses the appropriate media playback API (such as Android's MediaPlayer, ExoPlayer, or VideoView, etc.) to load and play the multimedia file. During playback, the GUI displays the video screen and synchronized audio (including the original audio and possible voice notes). When playing a multimedia file, the system tracks the current playback progress and compares it with the timestamp of the voice notes, and quickly queries it during playback through a list or database record containing the time information of the voice notes. The start and end positions of the voice notes are marked on the playback progress control (such as a progress bar, timeline, or playback slider). When the playback progress approaches or reaches the position of the voice note, the system can also remind the user through GUI elements (such as pop-ups, prompt boxes, or highlights).

[0064] In some embodiments, see Figure 3 , Figure 3 It is a flowchart of steps S301-S303 provided in an embodiment of the present application. The method also includes steps S301-S303, which will be described in combination with each step.

[0065] In step S301, in response to a second play request for the voice note, a note position of the voice note is determined, wherein the note position includes a start time and an end time.

[0066] In step S302, a playback interval is determined based on the start time and the end time.

[0067] In step S303, the multimedia file is played based on the playing interval.

[0068] Here, the embodiment of the present application can also perform specific playback for voice notes. Specifically, when a user initiates a playback request for voice notes through a graphical user interface (GUI), the system recognizes and responds to this request. It is usually achieved by clicking, touching or selecting a specific GUI element (such as an item in a voice note list, a mark on a progress bar, etc.). The system queries the stored voice note information to find a voice note that matches the request. The voice note information contains the start time and end time of the note, and these timestamps represent the position of the voice note in the multimedia file. Once the start time and end time of the voice note are obtained, the system can calculate the interval of the multimedia file that needs to be played. This interval starts from the start time of the voice note and ends at the end time. Use appropriate media playback APIs (such as Android's MediaPlayer, ExoPlayer, etc.) to load and play multimedia files. During playback, set the playback start position to the start time of the voice note, and (optionally) stop playing or automatically jump to the next voice note / continue playing the remaining content when the playback reaches the end time. The GUI can be updated to reflect the current playback status, including marks on the playback progress bar, current playback time display, etc.

[0069] In some embodiments, see Figure 4 , Figure 4 It is a flowchart of steps S401-S403 provided in an embodiment of the present application. The method also includes steps S401-S403, which will be described in combination with each step.

[0070] In step S401, in response to the end of recording, the multimedia file and at least one voice note are determined.

[0071] In step S402, the multimedia file and at least one voice note are stored.

[0072] In step S403, in response to the viewing operation on the multimedia file, the file information and the note information of the multimedia file are displayed, wherein the note information includes at least one sub-information, and the at least one sub-information corresponds one-to-one with at least one of the voice notes.

[0073] Here, the end of recording can be triggered by the user (such as clicking the stop button) or automatically detected by the system (such as reaching the preset recording time, storage space limit, etc.). The system needs to organize and determine the multimedia files and the voice notes associated with them during the recording process, and store the multimedia files and voice notes in a persistent storage medium, such as the internal storage of the device, an external SD card, or a cloud storage service. For the voice notes, it is necessary to associate them with the multimedia files so that they can be easily retrieved and played later. When the user initiates a viewing operation on the multimedia file through the GUI (such as clicking an item in the file list, selecting a file preview, etc.), the system recognizes and responds to this request, and displays the file information of the multimedia file, such as the file name, size, creation time, modification time, etc. At the same time, the note information associated with the multimedia file is displayed. The sub-information may include a brief description of the voice note (such as "Here is an important explanation"), a timestamp (start time and end time), a play button (allowing the user to directly play the voice note), etc.

[0074] In some embodiments, see Figure 5 , Figure 5 It is a flowchart of steps S501-S503 provided in an embodiment of the present application. The method also includes steps S501-S503, which will be described in combination with each step.

[0075] In step S501, speech recognition is performed on the voice note to obtain a target text.

[0076] In step S502, feature extraction is performed on the target text to obtain target features.

[0077] In step S503, a label or category of the multimedia file is determined based on the target feature.

[0078] Here, speech recognition technology (such as automatic speech recognition ASR system) is used to convert the audio data of the speech annotation into text data. The target text obtained by speech recognition is used as input for text feature extraction. Feature extraction may include Bag of Words, TF-IDF (Term Frequency-Inverse Document Frequency), Word Embeddings (such as Word2Vec, GloVe, etc.) or other text representation methods. These features can capture key information in the text, such as themes, emotions, keywords, etc., to facilitate subsequent classification or label determination. The extracted text features are processed using machine learning or deep learning models (such as classifiers, clustering algorithms, etc.) to determine the label or classification of the multimedia file. The label can be a predefined keyword or phrase that describes the content or theme of the multimedia file. Classification may classify the multimedia file into a broader category, such as education, entertainment, news, etc. In this process, the model needs to be trained to learn how to associate text features with specific labels or classifications. The training data may include multimedia files with known labels or classifications and the corresponding speech annotation text.

[0079] In summary, the embodiments of the present application have the following beneficial effects:

[0080] (1) Improving the consistency of user experience: The embodiments of the present application reduce the need for users to switch between multiple applications by enabling instant generation of audio and video recording and voice notes in the same application, greatly improving the consistency and convenience of user operations. Users can easily add voice notes while recording multimedia files without interrupting the current task or switching applications, thereby improving the overall user experience.

[0081] (2) Enhanced comprehensiveness and practicality of functions: Compared with third-party applications with single functions on the market, the embodiments of the present application provide a more comprehensive voice note solution. It supports the instant generation of voice notes, allowing users to complete multiple tasks in one application and improve work efficiency.

[0082] (3) Intelligent processing to reduce the burden on users: The embodiments of the present application not only implement basic voice recognition functions, but also introduce functions such as intelligent classification and automatic summarization. These functions can automatically analyze the content of voice notes, add appropriate tags or generate summaries to them, help users understand and review the notes information more quickly, and reduce the burden on users when processing a large number of voice notes.

[0083] (4) Promote effective management and utilization of information: By integrating multimedia files and voice notes, the embodiments of the present application make information organization and retrieval more efficient. Users can quickly locate key content in multimedia files based on voice notes, and can also classify and archive multimedia files based on the note information for subsequent reference and sharing.

[0084] (5) Improving user work efficiency and creativity: The implementation of the embodiments of the present application will provide powerful support for users in multiple scenarios such as creation, learning, and work. Users can record videos or audios anytime and anywhere, and instantly add voice notes to record inspiration, explain content, or mark key points, thereby improving work efficiency.

[0085] To sum up, the embodiments of the present application solve many problems in the existing Android voice notes solution, realize the function of generating voice notes along with audio and video recordings, bring many beneficial effects to users, not only improve user experience and work efficiency, but also promote the healthy development of the Android application ecosystem.

[0086] Based on the same inventive concept, an Android-based voice annotation device corresponding to the Android-based voice annotation method in the first embodiment is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned Android-based voice annotation method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0087] like Figure 7 As shown, Figure 7 : is a structural diagram of an Android-based voice note device 700 provided in an embodiment of the present application. The Android-based voice note device 700 includes:

[0088] The recording module 701 is used to record a multimedia file through the Android device in response to a recording request, wherein the multimedia file includes video data and first audio data;

[0089] The first remark module 702 is used for, in the process of recording the multimedia file, in response to the start of the remark request, copying the first audio data from the current time to obtain second audio data, and encoding the second audio data to obtain remark audio data;

[0090] The second annotation module 703 is used to stop copying the first audio data in response to the end of the annotation request, and use the annotated audio data obtained from the beginning to the end of the annotation request as the voice annotation.

[0091] Those skilled in the art should understand that Figure 7 The implementation functions of each unit in the Android-based voice note device 700 shown can be understood by referring to the relevant description of the aforementioned Android-based voice note method. Figure 7The functions of each unit in the Android-based voice annotation device 700 shown can be implemented by a program running on a processor, or can be implemented by a specific logic circuit.

[0092] In a possible implementation, the device further includes a playing module 704, and the playing module 704 further includes:

[0093] In response to a first play request for the multimedia file, playing the multimedia file through a graphical user interface;

[0094] And, during the playing of the multimedia file, the position of the voice note is displayed in the playing progress control.

[0095] In a possible implementation, the playing module 704 further includes:

[0096] In response to a second play request for the voice note, determining a note position of the voice note, wherein the note position includes a start time and an end time;

[0097] Determine a playback interval based on the start time and the end time;

[0098] The multimedia file is played based on the playback interval.

[0099] In a possible implementation, the device further includes a storage module 705, and the storage module 705 further includes:

[0100] In response to the recording being finished, determining the multimedia file and at least one of the voice notes;

[0101] Storing the multimedia file and at least one of the voice notes;

[0102] In response to a viewing operation on the multimedia file, file information and remark information of the multimedia file are displayed, wherein the remark information includes at least one sub-information, and the at least one sub-information corresponds one-to-one with at least one of the voice remark.

[0103] In a possible implementation, the storage module 705 further includes:

[0104] Performing speech recognition on the speech notes to obtain a target text;

[0105] Extracting features from the target text to obtain target features;

[0106] A label or classification of the multimedia file is determined based on the target feature.

[0107] In a possible implementation, the video data is recorded based on the Camera2API of the Android system, the first audio data is recorded based on the AudioRecord class of the Android system, the first audio data is in PCM format, and the video data and the first audio data are encapsulated based on the MediaMuxer API of the Android system.

[0108] In a possible implementation manner, the first remark module 702 encodes the second audio data to obtain remark audio data, including:

[0109] Create an audio encoder based on the Android system's MediaCodec class;

[0110] Configure encoding parameters of the audio encoder, wherein the encoding parameters include an audio format, and the audio format is an AAC format;

[0111] The second audio data is input into the audio encoder, and the remark audio data in the AAC format is output through the audio encoder.

[0112] The above Android-based voice note device has the following beneficial effects:

[0113] (1) Improving the consistency of user experience: The embodiments of the present application reduce the need for users to switch between multiple applications by enabling instant generation of audio and video recording and voice notes in the same application, greatly improving the consistency and convenience of user operations. Users can easily add voice notes while recording multimedia files without interrupting the current task or switching applications, thereby improving the overall user experience.

[0114] (2) Enhanced comprehensiveness and practicality of functions: Compared with third-party applications with single functions on the market, the embodiments of the present application provide a more comprehensive voice note solution. It supports the instant generation of voice notes, allowing users to complete multiple tasks in one application and improve work efficiency.

[0115] (3) Intelligent processing to reduce the burden on users: The embodiments of the present application not only implement basic voice recognition functions, but also introduce functions such as intelligent classification and automatic summarization. These functions can automatically analyze the content of voice notes, add appropriate tags or generate summaries to them, help users understand and review the notes information more quickly, and reduce the burden on users when processing a large number of voice notes.

[0116] (4) Promote effective management and utilization of information: By integrating multimedia files and voice notes, the embodiments of the present application make information organization and retrieval more efficient. Users can quickly locate key content in multimedia files based on voice notes, and can also classify and archive multimedia files based on the note information for subsequent reference and sharing.

[0117] (5) Improving user work efficiency and creativity: The implementation of the embodiments of the present application will provide powerful support for users in multiple scenarios such as creation, learning, and work. Users can record videos or audios anytime and anywhere, and instantly add voice notes to record inspiration, explain content, or mark key points, thereby improving work efficiency.

[0118] To sum up, the embodiments of the present application solve many problems in the existing Android voice notes solution, realize the function of generating voice notes along with audio and video recordings, bring many beneficial effects to users, not only improve user experience and work efficiency, but also promote the healthy development of the Android application ecosystem.

[0119] like Figure 8 As shown, Figure 8 The present invention provides a schematic diagram of the structure of an electronic device 800 according to an embodiment of the present invention. The electronic device 800 includes:

[0120] A processor 801, a storage medium 802 and a bus 803, wherein the storage medium 802 stores machine-readable instructions executable by the processor 801. When the electronic device 800 is running, the processor 801 communicates with the storage medium 802 via the bus 803, and the processor 801 executes the machine-readable instructions to perform the steps of the Android-based voice annotation method described in the embodiment of the present application.

[0121] In actual application, the components in the electronic device 800 are coupled together via bus 803. It is understood that bus 803 is used to realize the connection and communication between these components. Bus 803 includes not only a data bus, but also a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 8 Various buses are labeled as bus 803.

[0122] The above electronic device has the following beneficial effects:

[0123] (1) Improving the consistency of user experience: The embodiments of the present application reduce the need for users to switch between multiple applications by enabling instant generation of audio and video recording and voice notes in the same application, greatly improving the consistency and convenience of user operations. Users can easily add voice notes while recording multimedia files without interrupting the current task or switching applications, thereby improving the overall user experience.

[0124] (2) Enhanced comprehensiveness and practicality of functions: Compared with third-party applications with single functions on the market, the embodiments of the present application provide a more comprehensive voice note solution. It supports the instant generation of voice notes, allowing users to complete multiple tasks in one application and improve work efficiency.

[0125] (3) Intelligent processing to reduce the burden on users: The embodiments of the present application not only implement basic voice recognition functions, but also introduce functions such as intelligent classification and automatic summarization. These functions can automatically analyze the content of voice notes, add appropriate tags or generate summaries to them, help users understand and review the notes information more quickly, and reduce the burden on users when processing a large number of voice notes.

[0126] (4) Promote effective management and utilization of information: By integrating multimedia files and voice notes, the embodiments of the present application make information organization and retrieval more efficient. Users can quickly locate key content in multimedia files based on voice notes, and can also classify and archive multimedia files based on the note information for subsequent reference and sharing.

[0127] (5) Improving user work efficiency and creativity: The implementation of the embodiments of the present application will provide powerful support for users in multiple scenarios such as creation, learning, and work. Users can record videos or audios anytime and anywhere, and instantly add voice notes to record inspiration, explain content, or mark key points, thereby improving work efficiency.

[0128] To sum up, the embodiments of the present application solve many problems in the existing Android voice notes solution, realize the function of generating voice notes along with audio and video recordings, bring many beneficial effects to users, not only improve user experience and work efficiency, but also promote the healthy development of the Android application ecosystem.

[0129] The embodiment of the present application also provides a computer-readable storage medium, which stores executable instructions. When the executable instructions are executed by at least one processor 801, the Android-based voice annotation method described in the embodiment of the present application is implemented.

[0130] In some embodiments, the storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface storage, an optical disk, or a compact disc read-only memory (CD ROM); it can also be various devices including one or any combination of the above memories.

[0131] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0132] As an example, executable instructions may, but do not necessarily, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (for example, files storing one or more modules, subroutines, or code portions).

[0133] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0134] The computer-readable storage medium has the following advantages:

[0135] (1) Improving the consistency of user experience: The embodiments of the present application reduce the need for users to switch between multiple applications by enabling instant generation of audio and video recording and voice notes in the same application, greatly improving the consistency and convenience of user operations. Users can easily add voice notes while recording multimedia files without interrupting the current task or switching applications, thereby improving the overall user experience.

[0136] (2) Enhanced comprehensiveness and practicality of functions: Compared with third-party applications with single functions on the market, the embodiments of the present application provide a more comprehensive voice note solution. It supports the instant generation of voice notes, allowing users to complete multiple tasks in one application and improve work efficiency.

[0137] (3) Intelligent processing to reduce the burden on users: The embodiments of the present application not only implement basic voice recognition functions, but also introduce functions such as intelligent classification and automatic summarization. These functions can automatically analyze the content of voice notes, add appropriate tags or generate summaries to them, help users understand and review the notes information more quickly, and reduce the burden on users when processing a large number of voice notes.

[0138] (4) Promote effective management and utilization of information: By integrating multimedia files and voice notes, the embodiments of the present application make information organization and retrieval more efficient. Users can quickly locate key content in multimedia files based on voice notes, and can also classify and archive multimedia files based on the note information for subsequent reference and sharing.

[0139] (5) Improving user work efficiency and creativity: The implementation of the embodiments of the present application will provide powerful support for users in multiple scenarios such as creation, learning, and work. Users can record videos or audios anytime and anywhere, and instantly add voice notes to record inspiration, explain content, or mark key points, thereby improving work efficiency.

[0140] To sum up, the embodiments of the present application solve many problems in the existing Android voice notes solution, realize the function of generating voice notes along with audio and video recordings, bring many beneficial effects to users, not only improve user experience and work efficiency, but also promote the healthy development of the Android application ecosystem.

[0141] In the several embodiments provided in the present application, it should be understood that the disclosed methods and electronic devices can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0142] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0144] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a platform server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0145] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A voice note method based on Android, characterized in that: Applied to an Android device, the Android device includes a recording function, and the method includes: In response to the recording request, recording a multimedia file through the Android device, wherein the multimedia file includes video data and first audio data; In the process of recording the multimedia file, in response to the start of the annotation request, the first audio data is copied from the current time to obtain the second audio data, and the second audio data is encoded to obtain the annotation audio data; In response to the end of the comment request, the copying of the first audio data is stopped, and the comment audio data obtained from the start to the end of the comment request is used as the voice comment.

2. The method according to claim 1, characterized in that The method further comprises: In response to a first play request for the multimedia file, playing the multimedia file through a graphical user interface; And, during the playing of the multimedia file, the position of the voice note is displayed in the playing progress control.

3. The method according to claim 1, characterized in that The method further comprises: In response to a second play request for the voice note, determining a note position of the voice note, wherein the note position includes a start time and an end time; Determine a playback interval based on the start time and the end time; The multimedia file is played based on the playback interval.

4. The method according to claim 1, characterized in that: The method further comprises: In response to the recording being finished, determining the multimedia file and at least one of the voice notes; Storing the multimedia file and at least one of the voice notes; In response to a viewing operation on the multimedia file, file information and remark information of the multimedia file are displayed, wherein the remark information includes at least one sub-information, and the at least one sub-information corresponds one-to-one with at least one of the voice remark.

5. The method according to claim 1, characterized in that The method further comprises: Performing speech recognition on the speech notes to obtain a target text; Extracting features from the target text to obtain target features; A label or classification of the multimedia file is determined based on the target feature.

6. The method according to claim 1, characterized in that The video data is recorded based on the Camera2API of the Android system, the first audio data is recorded based on the AudioRecord class of the Android system, the first audio data is in PCM format, and the video data and the first audio data are encapsulated based on the MediaMuxer API of the Android system.

7. The method according to claim 1, characterized in that The encoding process of the second audio data to obtain the remark audio data includes: Create an audio encoder based on the Android system's MediaCodec class; Configure encoding parameters of the audio encoder, wherein the encoding parameters include an audio format, and the audio format is an AAC format; The second audio data is input into the audio encoder, and the remark audio data in the AAC format is output through the audio encoder.

8. A voice note device based on Android, characterized in that: Applied to an Android device, the Android device includes a recording function, and the apparatus includes: A recording module, configured to record a multimedia file through the Android device in response to a recording request, wherein the multimedia file includes video data and first audio data; a first annotation module, configured to copy the first audio data from a current time to obtain second audio data in response to a start of an annotation request during recording of the multimedia file, and to encode the second audio data to obtain annotation audio data; The second annotation module is used to stop copying the first audio data in response to the end of the annotation request, and use the annotated audio data obtained from the beginning to the end of the annotation request as the voice annotation.

9. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to execute the Android-based voice annotation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the Android-based voice annotation method according to any one of claims 1 to 7 is executed.

Images