Subtitle generation method, controller, wearable device and storage medium

Abstract

The invention provides a subtitle generation method, a controller, wearable equipment and a storage medium, and is suitable for the technical field of wearable equipment. The method comprises the following steps: receiving audio data sent by communication equipment; wherein the audio data is used for being played to a target object wearing the wearable device; performing character conversion based on the audio data by adopting a preset subtitle generation pipeline to obtain character information; and controlling a display screen to display subtitles corresponding to the character information based on the character information. According to the embodiment of the invention, the environment sound does not need to be collected through a microphone of the wearable device when the audio data is played, so that the accuracy of audio character conversion is improved, meanwhile, the risk of privacy leakage caused by cloud data transmission is avoided, and the character conversion efficiency is also improved. Moreover, according to the embodiment of the invention, language selection can be carried out based on the attitude information, and the azimuth information can be displayed on the display screen, so that the user can carry out intelligent selection according to the audio data in which the user is interested.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wearable devices, and in particular to a subtitle generation method, a controller, a wearable device, and a storage medium. BACKGROUND

[0002] Currently, wearable devices (for example, AR glasses) mainly collect environmental sound through microphones and then convert the environmental sound into text. The wearable device needs to send the audio data obtained by collecting the environmental sound to the cloud for audio-to-text conversion and then return the converted data to the wearable device.

[0003] However, the microphone is susceptible to environmental noise when collecting environmental sound, and the recognition error rate increases by 42% when the actual measured noise is greater than 85 decibels, thereby reducing the accuracy of audio-to-text conversion. At the same time, sending audio data to the cloud for processing and receiving the converted data returned by the cloud not only has the risk of privacy leakage but also reduces the efficiency of text conversion. SUMMARY

[0004] Therefore, the embodiments of the present application provide a subtitle generation method, a controller, a wearable device, and a storage medium to solve the problem of privacy leakage or low accuracy and efficiency of audio-to-text conversion in the prior art.

[0005] A first aspect of the embodiments of the present application provides a subtitle generation method applied to a wearable device including a display screen, and the method includes:

[0006] receiving audio data sent by a communication device; wherein the audio data is used to play to a target object wearing the wearable device;

[0007] using a preset subtitle generation pipeline to perform text conversion based on the audio data and obtain text information;

[0008] controlling the display screen to display subtitles corresponding to the text information based on the text information.

[0009] In one possible implementation, the audio data is encrypted data.

[0010] Using a preset subtitle generation pipeline to perform text conversion based on the audio data and obtain text information includes:

[0011] obtaining decryption information; wherein the decryption information is obtained based on physiological characteristics or behavioral characteristics;

[0012] decrypting the audio data based on the decryption information to obtain decrypted audio data;

[0013] using the preset subtitle generation pipeline to perform text conversion on the decrypted audio data to obtain the text information.

[0014] In a possible implementation, the subtitle generation pipeline includes a local machine learning engine;

[0015] The decrypted audio data is converted into text by using the preset subtitle generation pipeline, and text information is obtained, including:

[0016] The decrypted audio data is extracted for voice, and voice feature information is obtained;

[0017] The voice feature information is converted into text information by using the local machine learning engine.

[0018] In a possible implementation, after receiving the audio data sent by the communication device, and before converting the audio data into text by using the preset subtitle generation pipeline, the method further includes:

[0019] The wearable device is controlled to display language selection information; wherein the language selection information is used to provide at least two selectable languages;

[0020] Obtaining posture information of the target object; wherein the posture information includes at least one of a deflection angle and a deflection duration;

[0021] Determining a target language selected by the target object based on the posture information;

[0022] The audio data is converted into text by using the preset subtitle generation pipeline, and text information is obtained, including:

[0023] The audio data and the target language are converted into text by using the preset subtitle generation pipeline, and text information is obtained.

[0024] In a possible implementation, before receiving the audio data sent by the communication device, the method further includes:

[0025] Obtaining orientation information of at least one communication device; wherein the orientation information includes at least one of angle information and distance information;

[0026] The wearable device is controlled to display each orientation information.

[0027] In a possible implementation, the audio data sent by the communication device is received, including:

[0028] The audio data sent by the communication device is received when it is detected that the distance between the target object and the communication device is within a preset distance range.

[0029] A second aspect of the embodiment of the application provides a controller, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method of the first aspect when executing the computer program.

[0030] A third aspect of the embodiments of the present application provides a wearable device, comprising a display screen and the controller of the second aspect.

[0031] The controller is electrically connected with the display screen.

[0032] In a possible implementation, the wearable device comprises a head-mounted device.

[0033] The head-mounted device comprises any one of the following: a virtual reality (VR) glasses, an augmented reality (AR) glasses, a mixed reality (MR) glasses, and an artificial intelligence (AI) glasses.

[0034] A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method of the first aspect.

[0035] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0036] The subtitle generation method of the embodiments of the present application can receive audio data sent by a communication device, and does not need to collect environmental sound through a microphone of the wearable device, thereby avoiding environmental noise interference and improving the accuracy of audio text conversion. Then, a preset subtitle generation pipeline is adopted to perform text conversion based on the audio data to obtain text information, that is, the text conversion can be performed through the pre-stored subtitle generation pipeline, and does not need to be sent to the cloud for text conversion. Further, based on the text information, the display screen is controlled to display subtitles corresponding to the text information, for a user to view subtitles corresponding to the audio data.

[0037] Therefore, the embodiments of the present application perform text conversion on audio data, and do not need to collect environmental sound through a microphone of the wearable device when the audio data is played, thereby improving the accuracy of audio text conversion. Meanwhile, the embodiments of the present application do not need to send the audio data to the cloud for processing and then receive converted data returned by the cloud, thereby avoiding the risk of privacy leakage caused by cloud data transmission and improving the efficiency of text conversion.

[0038] It can be understood that the beneficial effects of the second aspect to the fourth aspect can be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0040] Figure 1 is a frame schematic diagram of a wearable device provided by an embodiment of the present application;

[0041] Figure 2 is a flowchart of a subtitle generation method provided by an embodiment of the present application;

[0042] Figure 3 is a flowchart of a method for obtaining text information provided by an embodiment of the present application;

[0043] Figure 4 is a flowchart of another subtitle generation method provided by an embodiment of the present application;

[0044] Figure 5 is a structural schematic diagram of a subtitle generation apparatus provided by an embodiment of the present application;

[0045] Figure 6 is a structural schematic diagram of a controller provided by an embodiment of the present application. DETAILED DESCRIPTION

[0046] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular sequences of steps, techniques, etc., in order to provide a thorough understanding of the present embodiments. However, it will be apparent to those skilled in the art that the present embodiments can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, and circuits are omitted so as not to obscure the description of the present embodiments.

[0047] It is to be understood that the terminology "includes", "has", "holds", "contains" and / or "comprising", when used in this specification and in the following claims, indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0048] It is also to be understood that the terminology "and / or" when used in this specification and in the following claims, refers to at least one of the items, or any combination of one or more of the items, and includes all possible combinations of one or more of the items.

[0049] As used in this specification and in the claims, the term "if" can be interpreted as meaning "when", or "once", or "in response to a determination", or "in response to detecting", as appropriate, depending on the context. Similarly, the phrase "if determined", or "if detected [the described condition or event]" can be interpreted as meaning "once determined", or "in response to a determination", or "once detected [the described condition or event]", or "in response to detecting [the described condition or event]", as appropriate, depending on the context.

[0050] In addition, in the description of the present application and the appended claims, the terms "first", "second", "third", etc. are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0051] In the present application, the reference to "one embodiment" or "some embodiments" means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in the present description are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "including but not limited to", unless otherwise specifically emphasized.

[0052] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. It should be pointed out that the following embodiments can be mutually referenced, borrowed or combined. For the same terms, similar features and similar implementation steps in different embodiments, they will not be described repeatedly.

[0053] Referring to Figure 1 As shown in the figure, the present embodiment provides a frame schematic diagram of a wearable device 10. As shown in the figure, Figure 1 As shown in the figure, the wearable device 10 includes a display screen 101 and a controller 102; the controller 102 is electrically connected with the display screen 101.

[0054] The controller 102 is configured to receive audio data sent by a communication device; the audio data is configured to play to a target object wearing the wearable device 10; a preset subtitle generation pipeline is adopted to perform text conversion based on the audio data to obtain text information; and the display screen 101 is controlled to display subtitles corresponding to the text information based on the text information.

[0055] Specifically, the controller 102 of the present embodiment can execute the subtitle generation method of the present embodiment, and the specific implementation function of the controller 102 is described in the following subtitle generation method. Herein, no further description is given.

[0056] Optionally, the audio data can be played by the communication device or by the wearable device 10. The wearable device 10 can further include earphones, and the user can listen to the voice corresponding to the audio data through the earphones.

[0057] Optionally, the wearable device 10 is in communication connection with the communication device, for example, the wearable device 10 is in Bluetooth connection with the communication device.

[0058] It is conceivable that the wearable device 10 and the communication device can also be connected in communication through wireless local area networks (WLAN) such as Wi-Fi networks, Zigbee, mobile communication networks, global navigation satellite systems (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), and the like.

[0059] In some embodiments, the wearable device 10 includes a head-mounted device; the head-mounted device includes any one of the following: Virtual Reality Glasses, Augmented Reality Glasses, Mixed Reality Glasses, and Artificial Intelligence Glasses.

[0060] It is conceivable to those skilled in the art that the wearable device 10 can also be a smart watch or the like.

[0061] Optionally, the user can wear the head-mounted device, and when listening to the audio data corresponding voice, can see the subtitles corresponding to the audio data.

[0062] Referring to Figure 2 The embodiment of the present application provides a flow chart of a subtitle generation method. The subtitle generation method of the embodiment of the present application can be applied to the wearable device 10 including the display screen 101 and executed by the controller 102. As Figure 2 The subtitle generation method of the embodiment of the present application includes steps S201 to S203.

[0063] S201, receiving audio data sent by a communication device; wherein the audio data is used to play to a target object wearing the wearable device 10.

[0064] Optionally, the audio data sent by the communication device is audio data broadcast by the communication device.

[0065] Optionally, the audio data sent by the communication device includes: when it is detected that the wearable device 10 and the communication device establish a Bluetooth connection, receiving the audio data broadcast by the communication device.

[0066] The audio data can be played by the communication device or by the wearable device 10. The audio data can be pre-stored data of the communication device or the voice of the device user recorded in real time by the communication device as a lecturer.

[0067] S202. Text conversion is performed on the audio data based on a preset subtitle generation pipeline to obtain text information.

[0068] In some embodiments, the audio data is encrypted data. The text conversion is performed on the audio data based on the preset subtitle generation pipeline to obtain the text information, including:

[0069] Decryption information is obtained, wherein the decryption information is obtained based on the physiological feature or the behavior feature.

[0070] The audio data is decrypted based on the decryption information to obtain decrypted audio data.

[0071] The text conversion is performed on the decrypted audio data based on the preset subtitle generation pipeline to obtain the text information.

[0072] The preset subtitle generation pipeline is a tool with audio text conversion function pre-stored in the wearable device 10.

[0073] Optionally, the embodiment of the present application is a subtitle generation method based on encrypted broadcast stream. The encrypted audio data can be received through audio broadcast Auracast, and the audio data is decoded to obtain the text information after being received.

[0074] In some embodiments, the subtitle generation pipeline includes a local machine learning engine. The text conversion is performed on the decrypted audio data based on the preset subtitle generation pipeline to obtain the text information, including:

[0075] Voice extraction is performed on the decrypted audio data to obtain voice feature information.

[0076] The voice feature information is converted into text information by using the local machine learning engine.

[0077] Optionally, the embodiment of the present application can use the local machine learning engine of the wearable device 10 to perform text conversion on the extracted voice to obtain the text information.

[0078] S203. The display screen 101 is controlled to display subtitles corresponding to the text information based on the text information.

[0079] The subtitles corresponding to the text information can be displayed by using a space mapping engine.

[0080] Optionally, the wearable device 10 is a head-mounted device. When the user wears the head-mounted device, the user can see the subtitles in front of the eyes.

[0081] Based on the steps S201 to S203, the subtitle generation method of the embodiment of the application can receive audio data sent by the communication device, without needing to collect environmental sound through the microphone of the wearable device 10, thereby avoiding interference from environmental noise and improving the accuracy of audio-to-text conversion. Then, the preset subtitle generation pipeline is used to perform text conversion based on the audio data to obtain text information, that is, the text conversion can be performed through the pre-stored subtitle generation pipeline without the need to send the audio data to the cloud for text conversion. Further, based on the text information, the display screen 101 is controlled to display subtitles corresponding to the text information for the user to view the subtitles corresponding to the audio data.

[0082] Therefore, the embodiment of the application performs text conversion on audio data without the need to collect environmental sound through the microphone of the wearable device 10 when the audio data is played, thereby improving the accuracy of audio-to-text conversion. Meanwhile, the embodiment of the application does not need to send the audio data to the cloud for processing and then receive the converted data returned by the cloud, thereby avoiding the risk of privacy leakage caused by data transmission of the cloud and improving the efficiency of text conversion.

[0083] Referring to Figure 3 As shown in FIG. 4, the embodiment of the application provides a flowchart for obtaining text information. As shown in FIG. 5, the preset subtitle generation pipeline is used to perform text conversion based on the audio data to obtain text information, including steps S301 to S304. Figure 3

[0084] S301, obtaining decryption information; wherein the decryption information is obtained based on physiological characteristics or behavioral characteristics.

[0085] Optionally, the decryption information can be obtained by scanning a scanning image attached to a desktop or a wall or the like, or the communication device can send the audio data and the decryption information to the wearable device at the same time to obtain the decryption information.

[0086] S302, decrypting the audio data based on the decryption information to obtain decrypted audio data.

[0087] Optionally, the decryption of the audio data can be performed by the AES-128 key verification of the stream decoding module.

[0088] As an example, the sound data is subjected to voice intelligent extraction, and the audio data is decrypted according to the iris, gesture, or brain wave of the lecturer or the user.

[0089] S303, extracting the voice from the decrypted audio data to obtain voice feature information.

[0090] ​Wherein, the decrypted audio data needs to be extracted by human voice, and the voice feature information is obtained, so as to facilitate the text conversion.

[0091] Optionally, the embodiment of the application can obtain the voice feature information through LC3 decoding of the voice separation module and human voice extraction.

[0092] S304, using a local machine learning engine, converting the voice feature information into text information.

[0093] Optionally, the local machine learning engine can be intelligently converted into subtitles and displayed through the display screen as a subtitle generation pipeline.

[0094] In some embodiments, after receiving the audio data sent by the communication device, and before using the preset subtitle generation pipeline to convert the text based on the audio data to obtain the text information, the method further comprises:

[0095] Controlling the wearable device 10 to display language selection information; wherein, the language selection information is used to provide at least two selected languages;

[0096] Obtaining the posture information of the target object; wherein, the posture information includes at least one of the deflection angle and the deflection duration;

[0097] Based on the posture information, determining the target language selected by the target object.

[0098] Optionally, controlling the wearable device 10 to display language selection information includes: displaying language selection information on the display screen; wherein, the language selection information includes at least two voice icons or voice controls.

[0099] Specifically, each voice icon or voice control corresponds to a language, for example: English, Chinese, etc. The user can select the corresponding language by rotating the deflection angle of the head and / or controlling the duration of the deflection angle of the head.

[0100] It is conceivable that the wearable device 10 can also display language selection information through voice playing language selection information, instructing the user to select the required language through the corresponding gesture.

[0101] Optionally, the wearable device 10 includes a posture sensor, such as an IMU (Inertial Measurement Unit), to obtain the deflection angle and deflection duration of the user.

[0102] Optionally, the embodiment of the application can also determine the audio source selected by the target object based on the posture information, and can realize automatic switching of the broadcast stream.

[0103] Correspondingly, the preset subtitle generation pipeline is adopted to perform text conversion based on the audio data to obtain the text information, including: adopting the preset subtitle generation pipeline to perform text conversion based on the audio data and the target language to obtain the text information.

[0104] As an example, the embodiment of the application intelligently selects voice data according to the posture information of the IMU, for example: selecting multiple languages. The head deflection angle and the duration of the deflection angle are obtained, for example: the continuous deflection angle is 30 degrees and the deflection duration is greater than 3s, which indicates that this language or audio source is selected, and after selection, it is converted into corresponding text and displayed in the form of subtitles.

[0105] Through research, in the prior art, manual switching of audio sources in a multi-language scene is inefficient, and the average operation time in a museum scene is 28 seconds / time. The existing subtitle synchronization technology relies on a single audio track processing and cannot adapt to a multi-broadcast stream parallel scene.

[0106] The embodiment of the application can perform text conversion based on audio data based on the target language determined by the user, or determine the language and audio source selected by the target object based on the posture information to realize automatic switching of broadcast streams, and can solve the above technical problems.

[0107] In some embodiments, before receiving the audio data sent by the communication device, the method further includes:

[0108] Obtaining the orientation information of at least one communication device; wherein the orientation information includes at least one of angle information and distance information;

[0109] The wearable device 10 displays the orientation information.

[0110] Optionally, the angle and distance of the orientation information can be anchored to generate a physical grid on the display screen for display. At the same time, the orientation information can also be displayed in the form of a map, and the position of the communication device is marked by displaying the corresponding identification of the orientation information, which can be an article image or other icons with indicating function. The embodiment of the application can also play the orientation information of the corresponding article of each communication device through voice, which is convenient for the user to select the corresponding audio data.

[0111] Optionally, the wearable device 10 displays the orientation information can be in the form of a map, and the position of the communication device is marked by displaying the corresponding identification of the orientation information, which can be an article image or other icons with indicating function. The embodiment of the application can also play the orientation information of the corresponding article of each communication device through voice, which is convenient for the user to select the corresponding audio data.

[0112] As an example, the orientation information includes angle information and distance information, and the display screen can display a 360° image through the orientation information. For example, corresponding to the same distance, the angle is a little lower at 30°, the angle is a little higher at 60°, and the angle at 90° can be in the front of the field of view; corresponding to the same angle, the orientation information corresponds to a small identifier when the distance is a little close, and the orientation information corresponds to a large identifier when the distance is a little far, so that a complete 360° image can be displayed.

[0113] The embodiment of the present application can display the orientation information through the wearable device 10, so that the user knows where the communication device is, each communication device can correspond to an introduction of an article or a speech of a guide, so that the user can select which communication device to approach based on the orientation information to obtain the corresponding audio data.

[0114] In some embodiments, the audio data transmitted by the communication device is received, including: when it is detected that the distance between the target object and the communication device is within the preset distance range, the audio data transmitted by the communication device is received.

[0115] Optionally, the embodiment of the present application can establish a communication connection when the user approaches a certain communication device, so as to transmit the audio data. The preset distance range can be selected and set according to actual conditions.

[0116] Referring to Figure 4 The embodiment of the present application provides another flowchart of a subtitle generation method. As Figure 4 The subtitle generation method of the embodiment of the present application includes steps S401 to S405.

[0117] S401, obtaining orientation information of at least one communication device; wherein the orientation information includes at least one of angle information and distance information.

[0118] Optionally, the embodiment of the present application can obtain the orientation information of the communication device through channel sounding of Bluetooth, GPS (Global Positioning System, Global Positioning System) and other positioning methods.

[0119] S402, controlling the wearable device 10 to display the orientation information.

[0120] Optionally, the wearable device 10 can display the orientation information in the form of a map, and display the identifier corresponding to the orientation information of the communication device to indicate the position, and the identifier can be an article image or other icons with indicating function. The embodiment of the present application can also play the orientation information of each communication device corresponding to the article through the voice mode, so as to facilitate the user to select the corresponding audio data.

[0121] Optionally, the angle and distance of each orientation information can be anchored to generate a physical grid for display on the display screen. Meanwhile, the orientation information can also be displayed in a manner of indicating distance and orientation by the size and position of the image.

[0122] S403, when it is detected that the distance between the target object and the communication device is within the preset distance range, receiving the audio data sent by the communication device.

[0123] Optionally, the embodiment of the present application can establish a communication connection when the user is close to a certain communication device, so as to transmit audio data. The preset distance range can be selected and set according to actual conditions.

[0124] S404, using a preset subtitle generation pipeline, performing text conversion based on the audio data to obtain text information.

[0125] The steps S404 to S405 of the embodiment of the present application are respectively consistent with the principles of steps S202 and S203, and will not be described again.

[0126] S405, based on the text information, controlling the display screen 101 to display subtitles corresponding to the text information.

[0127] As an example, the user is in a museum, and each item has corresponding audio data for explaining the item. For example, there is a vase and a calligraphy and painting near the user, and the vase and the calligraphy and painting correspond to a communication device for explanation or an explainer with a communication device respectively. The user can choose to listen to the explanation of the audio data corresponding to the item according to his own preference, and convert the audio data into subtitles.

[0128] The subtitle generation method of the embodiment of the present application can solve the problems of collecting sound through a microphone and the need for cloud recognition processing. The embodiment of the present application realizes complete separation from the local microphone for subtitle generation, meets the rigid demand for privacy protection, can build an encrypted stream-visual dynamic binding mechanism, and improves the information acquisition efficiency.

[0129] The embodiment of the present application does not need to rely on a traditional microphone for sound pickup, reduces the error rate of audio text recognition, can perform local audio conversion, improves the privacy of the user and different application requirements. Moreover, the embodiment of the present application can perform multi-language intelligent display, instead of traditional secondary processing. According to the display on the display screen based on the orientation information, the position of the required audio data can be obtained, the user is reminded which audio data is the interested data, and the user can make intelligent selection.

[0130] Meanwhile, the embodiment of the present application can select the required language through dynamic header control, without additional operation, data security encryption, decryption is completely completed locally, and the user privacy performance is stronger.

[0131] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0132] Referring to Figure 5 As shown in the figure, the embodiments of the present application provide a structural schematic diagram of a subtitle generation device 50. As Figure 5 As shown in the figure, the subtitle generation device 50 comprises a receiving module 501, a conversion module 502 and a generation module 503.

[0133] The receiving module 501 is configured to receive audio data sent by a communication device; wherein the audio data is used to play to a target object wearing a wearable device 10.

[0134] The conversion module 502 is configured to adopt a preset subtitle generation pipeline, perform text conversion based on the audio data, and obtain text information.

[0135] The generation module 503 is configured to control the display screen 101 to display subtitles corresponding to the text information based on the text information.

[0136] Optionally, the conversion module 502 is configured to obtain decryption information; wherein the decryption information is obtained based on physiological characteristics or behavioral characteristics; based on the decryption information, the audio data is decrypted to obtain decrypted audio data; and the decrypted audio data is subjected to text conversion using a preset subtitle generation pipeline to obtain text information.

[0137] Optionally, the conversion module 502 is configured to perform voice extraction on the decrypted audio data to obtain voice feature information; and the voice feature information is converted into text information using a local machine learning engine.

[0138] Optionally, the subtitle generation device 50 further comprises a selection module, the selection module is configured to control the wearable device 10 to display language selection information; wherein the language selection information is used to provide at least two selectable languages; obtain posture information of the target object; wherein the posture information comprises at least one of a deflection angle and a deflection duration; and determine a target language selected by the target object based on the posture information.

[0139] Optionally, the conversion module 502 is configured to adopt a preset subtitle generation pipeline, perform text conversion based on the audio data and the target language, and obtain text information.

[0140] Optionally, the subtitle generation device 50 further comprises a position display module, the position display module is configured to obtain position information of at least one communication device; wherein the position information comprises at least one of angle information and distance information; and control the wearable device 10 to display each position information.

[0141] Optionally, the receiving module 501 is configured to receive the audio data sent by the communication device when it is detected that the target object is within a preset distance range from the communication device.

[0142] In the application, each module in the subtitle generation device 50 can be a software program module, can be realized by different logical circuits integrated in the processor, or can be realized by multiple distributed processors.

[0143] The subtitle generation device 50 can execute the method provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module in the device of the embodiments of the present application correspond to the steps in the method of the embodiments of the present application. For the detailed functions of each module of the device, refer to the description of the corresponding method in the foregoing description, which will not be repeated here.

[0144] Referring to Figure 6 As shown in the figure, the controller 102 provided by the embodiments of the present application includes a processor 1021, a memory 1022, and a computer program 1023 stored in the memory 1022 and executable on the processor 1021. Figure 6 The processor 1021 executes the computer program to implement the steps of the method of the embodiments of the present application.

[0145] The controller 102 of the embodiments of the present application can include, but is not limited to, the processor 1021 and the memory 1022. Those skilled in the art can understand that Figure 6 The controller 102 is only an example and does not limit the controller 102, and can include more or fewer components than shown, or combine certain components, or different components, for example, can also include an input / output device, a network access device, etc.

[0146] The processor 1021 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0147] The memory 1022 can be an internal storage unit, such as a hard disk or a memory, in some embodiments. The memory 1022 can be a removable / non-removable, volatile / non-volatile computer system storage medium, such as a non-volatile memory, for reading and writing to a non-volatile magnetic medium. The memory 1022 can also be an external storage device in other embodiments, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. The memory 1022 is used to store an operating system, an application program, a boot loader, data, and other programs, such as program codes of a computer program, etc. The memory 1022 can also be used to temporarily store data that has been output or will be output.

[0148] It should be noted that the information interaction, execution process, etc. between the above devices / units, since based on the same concept as the method embodiments of the present application, the specific functions and the technical effects brought about, specific reference can be made to the method embodiments part, and will not be repeated here.

[0149] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the method embodiments, and will not be repeated here.

[0150] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps in each method embodiment.

[0151] The integrated unit described above, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the present application realizes all or part of the processes in the above-mentioned embodiment methods, which can be completed by a computer program instructing relevant hardware. The above-mentioned computer program can be stored in a computer-readable storage medium, and when executed by a processor, can realize the steps of the above-mentioned various method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer-readable medium at least includes any entity or device capable of carrying the computer program code to the device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, such as U disk, mobile hard disk, magnetic disk or optical disk, etc.

[0152] Those skilled in the art can understand that the implementation of all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware, and the program can be stored in a computer-readable storage medium, which, when executed, can include the processes of the above-mentioned various method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (Rom), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc. The storage medium can also include a combination of the above-mentioned types of memories.

[0153] The present application provides a computer program product, which, when running on a processor, causes the processor to execute the steps of the above-mentioned various method embodiments.

[0154] In the above-mentioned embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0155] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0156] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / network device and method can be implemented by other ways. For example, the apparatus / network device embodiments described above are only schematic, for example, the division of the above modules or units is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different parts can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0157] The units described as separate components above can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0158] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A subtitle generation method, characterized in that: Applied to a wearable device including a display screen, the method comprises: Receiving audio data sent by a communication device; wherein the audio data is used to play to a target object wearing the wearable device; Using a preset subtitle generation pipeline, performing text conversion based on the audio data to obtain text information; Based on the text information, the display screen is controlled to display subtitles corresponding to the text information.

2. The subtitle generation method according to claim 1, wherein: The audio data is encrypted data; The method of using a preset subtitle generation pipeline to convert text based on the audio data to obtain text information includes: Obtaining decryption information; wherein the decryption information is obtained based on physiological characteristics or behavioral characteristics; Decrypting the audio data based on the decryption information to obtain decrypted audio data; A preset subtitle generation pipeline is used to convert the decrypted audio data into text to obtain text information.

3. The subtitle generation method according to claim 2, wherein: The subtitle generation pipeline includes a local machine learning engine; The method of using a preset subtitle generation pipeline to convert the decrypted audio data into text to obtain text information includes: Extracting human voice from the decrypted audio data to obtain voice feature information; The local machine learning engine is used to convert the sound feature information into text information.

4. The subtitle generation method according to claim 1, wherein: After receiving the audio data sent by the communication device and before using the preset subtitle generation pipeline to convert the audio data into text and obtain text information, the method further includes: Controlling the wearable device to display language selection information; wherein the language selection information is used to provide at least two languages ​​for selection; Acquiring posture information of the target object; wherein the posture information includes at least one of a deflection angle and a deflection duration; determining a target language selected by the target object based on the posture information; The method of using a preset subtitle generation pipeline to convert text based on the audio data to obtain text information includes: A preset subtitle generation pipeline is used to perform text conversion based on the audio data and the target language to obtain text information.

5. The subtitle generation method according to any one of claims 1 to 4, characterized in that: Before receiving the audio data sent by the communication device, the method further includes: Acquire location information of at least one of the communication devices; wherein the location information includes at least one of angle information and distance information; Control the wearable device to display each of the position information.

6. The subtitle generation method according to claim 5, characterized in that: The receiving audio data sent by the communication device includes: When it is detected that the distance between the target object and the communication device is within a preset distance range, audio data sent by the communication device is received.

7. A controller comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

8. A wearable device, characterized in that: include: A display screen and a controller as claimed in claim 7; The controller is electrically connected to the display screen.

9. The wearable device according to claim 8, wherein: The wearable device includes a head-mounted device; The head-mounted device includes any one of the following: virtual reality VR glasses, augmented reality AR glasses, mixed reality MR glasses, and artificial intelligence AI glasses.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images