Audio content conversion method, computer device and storage medium
Through the audio content conversion method, the audio of audio programs is converted into text or sign language in real time and sent to the terminal device, solving the problem that hearing-impaired people need to search for subtitles or sign language in advance, realizing instant access to audio content, improving user experience and reducing network load.
Patent Information
- Application Number
- CN202210255871.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-15
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-03-15
AI Technical Summary
People with hearing impaired people need to pre-search content with subtitles or sign language when watching audio programs, resulting in poor user experience.
Provides an audio content conversion method, by receiving an audio conversion request from a terminal device, determining the currently played audio content, converting it into text or sign language content, and sending it to the terminal device in real time for display, including calling the voice recognition engine and dynamic updates. Content to adapt to interface changes.
Without pre-search, hearing-impaired people can instantly access audio content, improving user experience and reducing network bandwidth and latency through feature recognition and sharing processing distribution.
Smart Images

Figure CN114827720B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of audio processing, and in particular to an audio content conversion method, a computer device and a storage medium. Background Art
[0002] According to statistics, there are more than 27 million people with hearing impairments in my country. Because they have lost most of their hearing, they cannot hear the sound of audio programs (such as audio-only programs or video programs with audio), which seriously hinders their normal study, life and entertainment.
[0003] For some of the current audio programs, subtitles or sign language are prepared in advance and then played simultaneously with the program. However, the number of such programs with subtitles or sign language is relatively small. When watching audio programs, people with hearing impairments often need to specifically search for programs with subtitles or sign language and then watch them.
[0004] It should be noted that, in the prior art, hearing-impaired people can only search for programs with subtitles or sign language in advance and then watch them, resulting in a poor user experience. Summary of the invention
[0005] The present application provides an audio content conversion method, a computer device, and a storage medium to solve the technical problem in the prior art that hearing-impaired people can only pre-search for programs with subtitles or sign language and then watch them, resulting in a poor user experience.
[0006] According to a first aspect of the present invention, a method for converting audio content is provided, the method comprising: step S11, receiving an audio conversion request sent by a terminal device; step S13, determining the audio content currently played by the terminal device based on the audio conversion request; step S15, converting the audio content to obtain target content; step S17, sending the target content to the terminal device for display.
[0007] Furthermore, the target content includes at least one or more of the following: text content and sign language content, wherein the text content includes at least one language.
[0008] Furthermore, converting the audio content to obtain the target content includes: calling a speech recognition engine to convert the audio content into the target content.
[0009] Furthermore, determining the audio content currently being played by the terminal device based on the audio conversion request includes: obtaining the station logo of the current display interface of the terminal device from the audio conversion request; and matching in a live TV resource pool based on the station logo to obtain the audio content currently being played by the terminal device.
[0010] Furthermore, determining the audio content currently being played by the terminal device based on the audio conversion request includes: intercepting audio and / or video of a preset length from the current display interface of the terminal device according to the audio conversion request; and searching for the audio and / or video of the preset length in an audio and video content resource pool to obtain the audio content currently being played by the terminal device.
[0011] Furthermore, after sending the target content to the terminal device for display, the method further includes: detecting a state of a current display interface of the terminal device, and dynamically updating the target content based on the state of the current display interface.
[0012] Furthermore, dynamically updating the target content based on the state of the current display interface includes: upon detecting that the station logo of the current display interface of the terminal device has changed, reacquiring the audio content currently played by the terminal device based on the changed station logo; and executing steps S15 to S17.
[0013] Furthermore, dynamically updating the target content based on the state of the current display interface includes: when it is detected that the current display interface of the terminal device receives a content adjustment instruction from the user, re-capturing audio and / or video of a preset length from the current display interface of the terminal device; and executing steps S15 to S17.
[0014] According to a second aspect of the present invention, a computer device is provided, comprising a memory and a processor, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the method described in any one of the first aspects is executed.
[0015] According to a third aspect of the present invention, there is provided a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in any one of the first aspects is executed.
[0016] The present application provides an audio content conversion method, a computer device, and a storage medium to solve the technical problem in the prior art that hearing-impaired people can only pre-search for programs with subtitles or sign language and then watch them, resulting in a poor user experience.
[0017] The present application provides a method for converting audio content, a computer device, and a storage medium, the method comprising: receiving an audio conversion request sent by a terminal device; determining the audio content currently played by the terminal device based on the audio conversion request; converting the audio content to obtain target content; and sending the target content to the terminal device for display. The present application solves the technical problem in the prior art that hearing-impaired people can only pre-search for programs with subtitles or sign language and then watch them, resulting in poor user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0019] Figure 1 A flowchart of a method for converting audio content according to an embodiment of the present invention;
[0020] Figure 2 A flowchart of an optional audio content conversion method according to an embodiment of the present invention; and
[0021] Figure 3 The present invention is a flowchart of an optional audio content conversion method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0022] In order to make the above and other features and advantages of the present invention more clear, the present invention is further described below in conjunction with the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explaining to those skilled in the art and are only exemplary and not restrictive.
[0023] In the following description, many specific details are set forth to provide a thorough understanding of the present invention. However, it is apparent to one of ordinary skill in the art that specific details need not be adopted to practice the present invention. In other cases, well-known steps or operations are not described in detail to avoid obscuring the present invention.
[0024] Embodiment 1
[0025] The present invention provides a method for converting audio content, such as Figure 1 As shown, the method may include:
[0026] Step S11: receiving an audio conversion request sent by a terminal device.
[0027] Specifically, in the present solution, a device with data processing function, such as a server, may be used as the executor of the present solution. The terminal device may be a playback device for playing audio programs, such as a smart TV. When a user watches an audio program through the terminal device, the user may send an audio conversion request to the server through the terminal device. The audio conversion request is used to request conversion of the audio content in the audio program being watched. It should be noted that the audio program may be pure audio content or video content including audio.
[0028] For example, when a hearing-impaired person is watching an audio program played on a smart TV, since the hearing-impaired person cannot hear the sound, the hearing-impaired person can send an audio conversion request to the smart TV through an operating device such as a remote control. After receiving the audio conversion request, the smart TV sends the audio conversion request to a server that has established a communication relationship with the smart TV.
[0029] It should also be noted that the audio program played by the above-mentioned playback device can be a real-time live program or an on-demand program.
[0030] Step S13: determining the audio content currently being played by the terminal device based on the audio conversion request.
[0031] Specifically, in this solution, the server can determine the audio content in the playback content of the audio program based on the audio conversion request, for example, obtaining the audio content in the video content from the video content.
[0032] Step S15, converting the audio content to obtain target content.
[0033] Specifically, in this solution, the server can process the audio content to obtain the converted target requirements.
[0034] Step S17: sending the target content to the terminal device for display.
[0035] Specifically, in this solution, the server can send the above target content to the terminal device for display, and the hearing-impaired person can understand the audio through the displayed content.
[0036] Through the above steps S11 to S17, when a hearing-impaired person receives an audio program, he or she does not need to search in advance, but only needs to send an audio conversion request to the terminal device. This solution automatically converts the audio into the target content and then sends it to the terminal device for display. Therefore, this solution solves the technical problem in the prior art that the hearing-impaired person can only search in advance for programs with subtitles or sign language and then watch them, resulting in a poor user experience.
[0037] Optionally, the target content includes at least one or more of the following: text content and sign language content, wherein the text content includes at least one language.
[0038] Specifically, this solution can convert audio content into text for display, or into sign language for display. After converting the audio into text, this solution can also translate the text into multiple other target languages, making it easier for hearing-impaired people of different nationalities to obtain information, greatly improving the user experience.
[0039] Optionally, the step S15 of converting the audio content to obtain the target content may include:
[0040] Step S151, calling a speech recognition engine to convert the audio content into the target content.
[0041] Specifically, in this solution, the server can call a real-time language recognition engine to convert audio into text.
[0042] Optional, such as Figure 2 As shown, step S13 determines the audio content currently played by the terminal device based on the audio conversion request, including:
[0043] Step S131, obtaining the logo of the current display interface of the terminal device from the audio conversion request.
[0044] Step S132: matching is performed in a TV live resource pool based on the station logo to obtain the audio content currently being played by the terminal device.
[0045] Specifically, in the present scheme, when the above-mentioned audio program is a live program (such as a program broadcast live by a TV station), when a hearing-impaired person initiates an audio conversion request, the terminal device automatically performs feature recognition to identify the station logo of the currently playing live program, and then sends it to the server. That is, the above-mentioned audio conversion request includes the station logo information, and the server can obtain the audio program being played by the terminal device based on the station logo. That is, after receiving the station logo information, the server can match it in the TV live resource pool based on the station logo to obtain the audio content currently playing by the terminal device, and then send the text or sign language converted in real time from the audio content to the terminal device for display.
[0046] It should also be noted that if multiple users (multiple hearing-impaired people) are watching the same live program, the server only performs an audio conversion operation once to achieve shared information distribution. That is, users of the same TV station share one voice engine call, and each user does not need to call it repeatedly.
[0047] Optionally, before the above step S131, the solution of the present application may further include:
[0048] The server obtains mainstream live TV stations, radio stations, and streaming media in real time and marks them to establish a seed video or audio library. The purpose of the above marking is to establish basic information, such as which TV station and which type of program. When the program is broadcast live and matches the user's needs, it can be identified as a seed and distributed to the corresponding user according to the matching rules.
[0049] Optional, such as Figure 3 As shown, step S13 determines the audio content currently played by the terminal device based on the audio conversion request, which may include:
[0050] Step S134: intercepting audio and / or video of a preset duration from the current display interface of the terminal device according to the audio conversion request.
[0051] Step S135 , searching the audio and / or video of the preset duration in the audio and video content resource pool to obtain the audio content currently being played by the terminal device.
[0052] Specifically, in this solution, when the content currently played by the terminal device is a on-demand program (such as a program played on demand on a video platform), the server can capture audio and / or video of a preset length from the current display interface of the terminal device, and then search in the audio and video content resource pool to obtain the audio content currently played by the terminal device.
[0053] It should be noted that the present solution can pre-establish a library of on-demand content, and then establish a matching audio clip search algorithm, a video clip search algorithm, and an image search algorithm.
[0054] Optionally, in step S17, after sending the target content to the terminal device for display, the method further includes:
[0055] Step S19: detecting the state of the current display interface of the terminal device, and dynamically updating the target content based on the state of the current display interface.
[0056] Specifically, this solution can always detect the status of the current display interface of the terminal device. If the status of the user's current display interface changes, it means that the content the user is watching has changed. This solution can update the target content displayed in the terminal device in real time to ensure that the target content is displayed in real time and accurately.
[0057] Optionally, step S19 dynamically updating the target content based on the state of the current display interface includes:
[0058] Step S191, when it is detected that the station logo of the current display interface of the terminal device has changed, the audio content currently being played by the terminal device is reacquired based on the changed station logo.
[0059] Step S192, execute steps S15 to S17.
[0060] Specifically, in this scheme, if the user (a person with hearing impairment) changes the channel, the current interface (station logo position) played by the terminal device will change. This scheme can monitor the current station logo played by the terminal device at intervals of seconds (for example, once every 30 seconds). If it is determined that the station logo has changed, this scheme re-acquires the audio content currently played by the terminal device, and then executes the method of steps S15 to S17.
[0061] Optionally, step S19 dynamically updating the target content based on the state of the current display interface includes:
[0062] Step S193, when it is detected that the current display interface of the terminal device receives a content adjustment instruction from the user, re-capturing the audio and / or video of a preset duration from the current display interface of the terminal device;
[0063] Step S194, execute steps S15 to S17.
[0064] Specifically, in this solution, in this solution, if the user (a hearing-impaired person) fast-forwards or rewinds the on-demand program (i.e., sends the above-mentioned content adjustment instructions to the terminal device), the current display interface of the terminal device will also change. This solution can capture audio and / or image clips of a preset duration (e.g., 5 seconds) of the current display interface of the terminal device at intervals of seconds (e.g., once every 30 seconds), and then re-acquire the audio content currently being played by the terminal device, and then execute the method of steps S15 to S17.
[0065] It should be noted that if the user's on-demand content is not found in the established content library, the on-demand content audio and conversion information can be uploaded and updated.
[0066] In summary, this solution can convert audio content into text or sign language in real time so that people with hearing impairments can obtain information in a timely manner. Through feature recognition technology, this solution can aggregate needs, realize shared processing and distribution, reduce network bandwidth and latency, and improve user experience.
[0067] It should be understood that the specific features, operations and details described hereinabove about the method of the present invention may also be similarly applied to the device and system of the present invention, or, vice versa. In addition, each step of the method of the present invention described above may be performed by the corresponding parts or units of the device or system of the present invention.
[0068] It should be understood that each module / unit of the device of the present invention can be implemented in whole or in part by software, hardware, firmware or a combination thereof. Each of the modules / units can be embedded in the processor of the computer device in the form of hardware or firmware or independent of the processor, or can be stored in the memory of the computer device in the form of software for the processor to call to perform the operation of each module / unit. Each of the modules / units can be implemented as an independent component or module, or two or more modules / units can be implemented as a single component or module.
[0069] In one embodiment, a computer device is provided, which may be a terminal device, including a memory and a processor, wherein the memory stores computer instructions executable by the processor, and the computer instructions instruct the processor to execute the steps of the method in the first embodiment of the present invention when executed by the processor. The computer device may be a server, a terminal, or any other electronic device with necessary computing and / or processing capabilities in a broad sense. In one embodiment, the computer device may include a processor, a memory, a network interface, a communication interface, etc. connected via a system bus. The processor of the computer device may be used to provide necessary computing, processing, and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and an internal memory. An operating system, a computer program, etc. may be stored in or on the non-volatile storage medium. The internal memory may provide an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface and the communication interface of the computer device may be used to connect and communicate with external devices via a network. The steps of the method of the present invention are executed by the processor.
[0070] The present invention may be implemented as a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the steps of the method of Embodiment 1 of the present invention to be executed. In one embodiment, the computer program is distributed on a plurality of computer devices or processors coupled to a network so that the computer program is stored, accessed, and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, may be performed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations may be performed by one or more computer devices or processors, and one or more other method steps / operations may be performed by one or more other computer devices or processors. One or more computer devices or processors may perform a single method step / operation, or perform two or more method steps / operations.
[0071] It will be appreciated by those skilled in the art that the method steps of the present invention may be performed by instructing related hardware such as a computer device or a processor through a computer program, and the computer program may be stored in a non-temporary computer-readable storage medium, which causes the steps of the present invention to be performed when the computer program is executed. Depending on the circumstances, any reference to memory, storage, database, or other media herein may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.
[0072] The various technical features described above can be combined arbitrarily. Although all possible combinations of these technical features are not described, any combination of these technical features should be considered to be covered by this specification as long as there is no contradiction in such combination.
[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for converting audio content, characterized in that: The method comprises: Step S11, receiving an audio conversion request sent by a terminal device; Step S13, determining the audio content currently played by the terminal device based on the audio conversion request; Step S15, converting the audio content to obtain target content; Step S17, sending the target content to the terminal device for display; Wherein, in the case where the currently played audio content is a live program, determining the audio content currently played by the terminal device based on the audio conversion request includes: obtaining the station logo of the current display interface of the terminal device from the audio conversion request; matching in a TV live resource pool based on the station logo to obtain the audio content currently played by the terminal device; In the case where the currently played audio content is a video-on-demand program, determining the audio content currently played by the terminal device based on the audio conversion request includes: intercepting audio and / or video of a preset duration from the current display interface of the terminal device according to the audio conversion request; searching for the audio and / or video of the preset duration in an audio and video content resource pool to obtain the audio content currently played by the terminal device; After sending the target content to the terminal device for display, the method further includes: detecting a state of a current display interface of the terminal device; In the case where the currently played audio content is a live program, when it is detected that the station logo of the current display interface of the terminal device has changed, the audio content currently played by the terminal device is re-acquired based on the changed station logo, and steps S15 to S17 are executed; In the case where the currently playing audio content is a on-demand program, when it is detected that the current display interface of the terminal device has received a content adjustment instruction from the user, the audio and / or video of a preset length is re-captured from the current display interface of the terminal device, and steps S15 to S17 are executed; the content adjustment instruction includes a content adjustment instruction corresponding to a fast-forward operation and a content adjustment instruction corresponding to a fast-rewind operation.
2. The method according to claim 1, characterized in that: The target content includes at least one or more of the following: Text content and sign language content, wherein the text content includes at least one language.
3. The method according to claim 1, characterized in that The audio content is converted and processed to obtain target content including: A speech recognition engine is called to convert the audio content into the target content.
4. A computer device, comprising a memory and a processor, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the method according to any one of claims 1 to 3 is executed.
5. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method of any one of claims 1 to 3 is performed.
