A voice recognition display method, device, storage medium and smart glasses

Abstract

This application discloses a voice recognition display method, device, storage medium, and smart glasses, relating to the field of smart wearable technology. The method determines the spatial angle and direction of a sound source by collecting voice information emitted from a sound source in a preset information collection area. It also obtains the facial regions of each communication subject in the display image corresponding to the preset information collection area. Subsequently, the angle and direction of the sound source are matched with the positions of each facial region in the display image to determine the actual communication subject corresponding to the sound source. Thus, the text pattern obtained by converting the voice information emitted by the sound source can be moved to a preset text display area near the facial region of the corresponding communication subject and displayed to the user. When the user uses the smart glasses, the other person's speech can be displayed as text around their face, making it easier for the user to better understand the other person's true meaning and preventing the other person's face from being obscured by the displayed text, thus enhancing the user's sensory experience.

Description

Technical Field

[0001] This application relates to the field of smart wearable technology, and in particular to a voice recognition display method, device, storage medium, and smart glasses. Background Technology

[0002] With the development of science and technology, smart wearable devices have gradually integrated into people's daily lives. Among them, smart glasses, as a common type of smart wearable device, combine the visual assistance function of traditional glasses with modern intelligent virtual reality interaction technology, providing assistance in life and work. For example, when communicating with someone whose language is different, the system can automatically translate what people say into a language the user can understand and then convert it into text for display, allowing the user to understand the true meaning of the other person's expression. However, in reality, the converted text may obscure the facial features of the person being communicated with, making it impossible for the user to see their expression clearly. For some statements that can have different meanings in different contexts, this may lead to ambiguity, causing the user to misunderstand the other person's true intention. Especially when communicating with multiple people, a large area of ​​the image visible to the user is occupied by multiple people. The text generated from the speech of different people may obscure the faces of each person, making it difficult for the user to observe the expressions of the speakers and accurately determine which person's speech the currently displayed text corresponds to. Summary of the Invention

[0003] The main objective of this application is to provide a voice recognition display method, device, storage medium, and smart glasses, aiming to solve the technical problem of how to avoid the displayed text obscuring the face of the person being communicated with when converting the voice of the person being communicated with into text for display through smart glasses.

[0004] To achieve the above objectives, this application provides a voice recognition display method for use in smart glasses. The steps of the voice recognition display method include:

[0005] Collect voice information from multiple communication objects in a preset information collection area, and obtain the sound source angle corresponding to the voice information;

[0006] Collect the display image corresponding to the preset information collection area, and obtain the facial regions corresponding to each of the communication objects in the display image;

[0007] Based on the sound source angle, the target facial region corresponding to the voice information is determined in each of the facial regions;

[0008] Move the text pattern corresponding to the voice information to the preset text display area around the target facial area.

[0009] In one embodiment, the step of collecting voice information emitted by multiple communication objects in a preset information collection area and obtaining the sound source angle corresponding to the voice information includes:

[0010] The voice information emitted by each of the communication objects within the preset information collection area is collected using a microphone array;

[0011] Obtain the maximum voice reception time difference corresponding to the voice information received by each individual microphone in the microphone array;

[0012] The sound source angle is determined based on the maximum voice reception time difference and the preset angular resolution of the microphone array.

[0013] In one embodiment, the step of acquiring the display image corresponding to the preset information acquisition area and obtaining the facial regions corresponding to each of the communication objects in the display image includes:

[0014] The camera captures the display image corresponding to the preset information acquisition area;

[0015] The displayed image is scanned to obtain multiple sets of facial features contained in the displayed image;

[0016] Based on the facial features of each group, the facial regions corresponding to each of the communication objects are determined.

[0017] In one embodiment, the step of determining the target facial region corresponding to the voice information in each of the facial regions based on the sound source angle includes:

[0018] Convert the sound source angle into the corresponding plane angle in the displayed image;

[0019] Based on the plane angle, the sound source region in the displayed image is obtained;

[0020] When one of the facial regions intersects with the sound source region, the facial region intersecting with the sound source region is taken as the target facial region.

[0021] In one embodiment, after the step of obtaining the sound source region in the displayed image based on the plane angle, the method further includes:

[0022] When none of the facial regions intersect with the sound source region, the facial region closest to the sound source region is taken as the target facial region.

[0023] In one embodiment, after the step of moving the text pattern corresponding to the voice information to a preset text display area corresponding to the periphery of the target facial area, the method further includes:

[0024] Detect whether the currently defined preset text display area intersects with other facial areas or other preset text display areas;

[0025] When the preset text display area is detected to intersect with the other facial areas or the other preset text display areas, the text pattern is moved out of the area intersecting with the corresponding facial area or preset text display area.

[0026] In one embodiment, before the step of moving the text pattern corresponding to the voice information to a preset text display area corresponding to the periphery of the target facial area, the method further includes:

[0027] The voice information is recognized to obtain its audio features;

[0028] The language and semantics of the speech information are determined based on the audio features.

[0029] Based on the language and the corresponding preset speech conversion method, the corresponding text pattern is generated according to the semantics.

[0030] In addition, to achieve the above objectives, this application also proposes a control device, which includes: a memory, a processor, and a voice recognition display program stored in the memory and executable on the processor, the voice recognition display program being configured to implement the steps of the voice recognition display method as described above.

[0031] In addition, to achieve the above objectives, this application also proposes a storage medium storing a speech recognition and display program, which, when executed by a processor, implements the steps of the speech recognition and display method described above.

[0032] In addition, to achieve the above objectives, this application also proposes a smart glasses whose processor can run a voice recognition and display program, the voice recognition and display program being configured to implement the steps of the voice recognition and display method described above.

[0033] This application proposes a voice recognition display method, device, storage medium, and smart glasses. The voice recognition display method includes the following steps: acquiring voice information emitted by multiple communication objects in a preset information acquisition area, and obtaining the sound source angle corresponding to the voice information; acquiring a display image corresponding to the preset information acquisition area, and obtaining each facial region corresponding to each of the communication objects in the display image; determining the target facial region corresponding to the voice information in each facial region based on the sound source angle; and moving the text pattern corresponding to the voice information to a preset text display area corresponding to the periphery of the target facial region.

[0034] By collecting voice information from a sound source within a preset information collection area, the spatial angle and direction of the sound source are determined. Furthermore, the facial regions of each communication subject in the displayed image corresponding to the preset information collection area are obtained. Subsequently, the angle and direction of the sound source are matched with the positions of each facial region in the displayed image to determine the actual communication subject corresponding to the sound source. In this way, the text pattern obtained by converting the voice information emitted by the sound source can be moved to a preset text display area near the facial region of the corresponding communication subject and displayed to the user. When the user uses smart glasses, the other person's words can be displayed as text around their face, making it easier for the user to better understand the true meaning expressed, preventing the other person's face from being obscured by the displayed text, and enhancing the user's sensory experience. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating an embodiment of the speech recognition and display method of this application.

[0036] Figure 2 This is a flowchart illustrating Embodiment 2 of the speech recognition display method of this application;

[0037] Figure 3 This is a flowchart illustrating Embodiment 3 of the speech recognition display method of this application;

[0038] Figure 4 This is a schematic diagram of the control device of the hardware operating environment involved in the voice recognition display method of this application.

[0039] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0040] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0041] This application provides a voice recognition and display method applied to smart glasses, as shown in the following embodiments. Figure 1 In this embodiment, the voice recognition and display method includes the following steps:

[0042] Step S10: Collect voice information emitted by multiple communication objects in a preset information collection area, and obtain the sound source angle corresponding to the voice information;

[0043] It should be noted that, in this embodiment, the executing entity can be a controller or control device with information acquisition and processing functions in the smart glasses, specifically a system-on-chip (SOC) chip. In this embodiment, the executing entity can be electrically connected to other functional modules of the smart glasses, thereby acquiring different types of signals, data, or information through these modules. The smart glasses can be electronic glasses with virtual reality interaction functions, and can be any of AR glasses, VR glasses, XR glasses, or other AI glasses.

[0044] It should be understood that, in this embodiment, the smart glasses may be equipped with functional modules such as a voice acquisition module, an image acquisition module, and a display module. The executing entity can acquire the sound emitted from a certain area through the voice acquisition module, that is, acquire voice information; or acquire the image of a certain area through the image acquisition module, and display the acquired image to the user through the display module.

[0045] It should be noted that the preset information collection area refers to the three-dimensional spatial area where the smart glasses can effectively collect sound and light signals. Specifically, it can be an area within a certain angle and distance range corresponding to a certain direction of the smart glasses. The communication object can be a person or robot that communicates with the user verbally; in this case, it can also be understood as a sound source capable of emitting human language.

[0046] It is easy to understand that voice information can be understood as the sounds emitted by various communication partners. The voice acquisition module of smart glasses can be a microphone array, which has multiple microphones placed in different positions. Each microphone acquires the sound emitted from the same sound source at a certain time, so the direction of sound transmission, i.e., the direction of the sound source, can be determined by the time difference in sound acquisition by each microphone. In this embodiment, the specific scenario is that a user wears smart glasses to communicate with multiple communication partners, and a certain area directly in front of the user is the preset information acquisition area. Within the preset information acquisition area, there can be one or more communication partners. At any given moment, at least one communication partner will communicate with the user. At this time, the voice acquisition module can acquire the voice information emitted by one or more communication partners in the preset information acquisition area, and determine the relative direction of the sound source with respect to the smart glasses (or the user) based on the time of receiving the voice information, i.e., the sound source angle mentioned in this document.

[0047] Step S20: Collect the display image corresponding to the preset information collection area, and obtain the facial regions corresponding to each of the communication objects in the display image;

[0048] It should be noted that the facial region refers to the area corresponding to the face of the person being communicated with on the displayed image. In this embodiment, an image of a preset information collection area facing the user can be captured to obtain a planar display image, which is then shown to the user through the display module of the smart glasses. Simultaneously, the displayed image can be scanned to filter out the facial region corresponding to each person being communicated with in the displayed image.

[0049] Step S30: Based on the sound source angle, determine the target facial region corresponding to the voice information in each of the facial regions;

[0050] It is easy to understand that the target facial region refers to the facial region of the person currently emitting the voice message in the displayed image. In this embodiment, the voice message and the displayed image are obtained by the smart glasses body from the sound signal and light signal in the preset information acquisition area, respectively. Therefore, the region corresponding to the obtained sound source angle can be represented in the displayed image. In reality, for a certain voice message, since the sound is emitted through the human mouth, the sound source in the preset information acquisition area should be the mouth of a person being communicated with, or it can be understood as the facial region of a person being communicated with. Based on the above objective facts, the sound source angle corresponding to the voice message can be converted into a pixel region corresponding to a certain angle range in the displayed image. This pixel region should extend to the facial region of a person being communicated with. In this way, it is possible to determine which person being communicated with is emitting the voice message, that is, to determine the target facial region corresponding to the person currently emitting the voice message.

[0051] Step S40: Move the text pattern corresponding to the voice information to the preset text display area around the target facial area.

[0052] It should be noted that, in this embodiment, the smart glasses also have a voice conversion module, which can convert the collected voice information into corresponding text and display it to the user in the form of a graphic through the display module. It can also be superimposed on the display image described above before being displayed to the user.

[0053] It is easy to understand that the preset text display area is a planned area on the displayed image used to display the text corresponding to the voice information, and it can be set at a certain distance from the target facial area. In this embodiment, after the voice information is collected, it can also be converted into a text pattern with corresponding semantics, and then superimposed on the collected display image and displayed to the user. The text pattern can move along with the target facial area in the display image, ensuring that the text pattern is always within the preset text display area close to the target facial area, without obscuring it.

[0054] It is worth noting that in one scenario, the person being communicated with may move while emitting a voice signal. Consequently, the target facial area in the displayed image may also move. Correspondingly, the preset text display area can also move synchronously following the target facial area in the displayed image, based on its direction and distance of movement. Furthermore, the movement of the preset text display area is not limited to following; it can also move freely around the target facial area, or be set by the user to a user-specified movable area (excluding the target facial area) through virtual reality interaction.

[0055] Using the above method, embodiments of this application can convert images of a preset information collection area and the corresponding speech of the communicators into text patterns through smart glasses in scenarios where users are wearing smart glasses and communicating with multiple people. These text patterns are then displayed to the user, making it easier for them to understand the true meaning expressed by the communicators. Simultaneously, the converted text patterns never obscure the facial features of the communicators in the image, allowing the user to perceive the communicators' facial expressions and language, thus assisting in a deeper understanding of the true meaning expressed and enhancing the user's sensory experience.

[0056] This application proposes a voice recognition and display method. The steps of the voice recognition and display method include: acquiring voice information emitted by multiple communication objects in a preset information acquisition area, and obtaining the sound source angle corresponding to the voice information; acquiring a display image corresponding to the preset information acquisition area, and obtaining each facial region corresponding to each of the communication objects in the display image; determining the target facial region corresponding to the voice information in each facial region based on the sound source angle; and moving the text pattern corresponding to the voice information to a preset text display area corresponding to the target facial region. By acquiring the voice information emitted by the sound source in the preset information acquisition area to determine the spatial angle and direction of the sound source, and by acquiring the facial regions of each communication object in the image of the display image corresponding to the preset information acquisition area, the angle and direction of the sound source are matched with the position of each facial region in the display image to determine the actual communication object corresponding to the sound source. In this way, the text pattern obtained by converting the voice information emitted by the sound source can be moved to a preset text display area near the facial region of the corresponding communication object for display to the user. When users wear smart glasses, the words spoken by the other person can be displayed as text around their face, making it easier for users to understand the true meaning of the other person's expression, and also preventing the other person's face from being obscured by the displayed text, thus enhancing the user's sensory experience.

[0057] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 The step of collecting voice information from multiple communication objects in a preset information collection area and obtaining the sound source angle corresponding to the voice information includes:

[0058] Step S11: Collect the voice information emitted by each of the communication objects within the preset information collection area through a microphone array;

[0059] Step S12: Obtain the maximum voice reception time difference corresponding to the voice information received by each individual microphone in the microphone array;

[0060] It should be noted that in this embodiment, the microphone array is the speech recognition module mentioned above. The microphone array may contain multiple single-channel microphones, each capable of capturing speech information emitted within a preset information acquisition area. In specific implementation, the communication object can be considered as a point-like, divergent sound source. Therefore, the distances from which the speech information emitted by the communication object is transmitted to each microphone are different, while the sound transmission rate is almost the same. Consequently, the time it takes for each microphone in the microphone array to acquire speech information differs from one another, and the maximum difference is the maximum speech reception time difference mentioned above.

[0061] Step S13: Determine the sound source angle based on the maximum voice reception time difference and the preset angular resolution of the microphone array.

[0062] It should be noted that the preset angular resolution refers to the unit angle or minimum angle at which the direction of sound transmission can be changed by the microphone array. It is a fixed characteristic generated by the design structure of the microphone array. For example, if the preset angular resolution is 15°, the angle of the sound source that can be identified can be 0°, 15°, 30° or other multiples of 15°.

[0063] It is easy to understand that in this embodiment, the maximum voice reception time difference is necessarily the difference between the shortest time to receive voice information and the worst time to receive voice information, that is, the time difference between the voice information collected by the single microphone closest to the sound source and the single microphone farthest from the sound source. Since the distance between each pair of microphones is a known fixed value, the specific location of the sound source (the object of communication) can be determined by the maximum voice reception time difference corresponding to the two microphones (the single microphone closest to the sound source and the single microphone farthest from the sound source) and the preset angular resolution of the microphone array, thus determining the corresponding sound source angle.

[0064] Further, in this embodiment, the step of acquiring the display image corresponding to the preset information acquisition area and obtaining the facial regions corresponding to each of the communication objects in the display image includes:

[0065] Step S21: Acquire the display image corresponding to the preset information acquisition area using a camera;

[0066] It should be noted that, in this embodiment, the image display module described above can specifically be a camera. The camera can capture the scene within the field of view of the lens (preset information acquisition area) to generate a planar display image.

[0067] Step S22: Scan the displayed image to obtain multiple sets of facial features contained in the displayed image;

[0068] It should be noted that facial features refer to the unique and relatively stable attributes or markers exhibited by the human face, including facial contours, distribution of facial features, and expressions. In this embodiment, an algorithm model capable of facial recognition can be obtained by training on a large number of facial images. This algorithm model is then used to scan the displayed image, thereby identifying all facial features within the image. Furthermore, the algorithm model can also use multi-scale gradient scanning to group all the aforementioned facial features according to the number of people interacting with the image, thus obtaining multiple sets of facial features.

[0069] It is worth noting that in each set of facial features, there is only one set of facial features that can be matched, such as facial contours, facial feature distribution, and facial expressions. That is, a set of facial features corresponds to only one communication partner.

[0070] Step S23: Based on the facial features of each group, determine the facial regions corresponding to each of the communication objects.

[0071] It is easy to understand that, in this embodiment, the facial region corresponding to each communication object in the display image can be determined by the pixel coordinate range of multiple sets of facial features obtained in the display image. It can also be understood as finding the specific location of the face of the communication object in the display image.

[0072] It is worth noting that, in this embodiment, the facial region can specifically be a rectangular region whose length and width dimensions dynamically change according to the size of the human head image or facial image of the corresponding communication object in the displayed image, and each rectangular region contains only all the displayed facial features of one communication object.

[0073] Furthermore, in this embodiment, the step of determining the target facial region corresponding to the voice information in each of the facial regions based on the sound source angle includes:

[0074] Step S31: Convert the sound source angle into the corresponding plane angle in the displayed image;

[0075] Step S32: Based on the plane angle, obtain the sound source region in the displayed image;

[0076] It should be noted that, in this embodiment, the sound source angle can also be approximated as the polar coordinate angle of the sound source (the area emitting voice information) in the polar coordinate system formed with the smart glasses as the origin, and the plane angle of the displayed image can be approximated as the plane angle of the two-dimensional plane coordinate system centered on the smart glasses.

[0077] It is easy to understand that, in this embodiment, the sound source angle can be converted into a plane angle in the displayed image using a coordinate system transformation formula. Then, based on the converted plane angle, the area covered by the sound source (the region emitting the voice information) and the voice information transmission path is divided in the displayed image, i.e., the sound source region mentioned above.

[0078] Step S33: When one of the facial regions intersects with the sound source region, the facial region intersecting with the sound source region is taken as the target facial region.

[0079] It should be noted that, in this embodiment, humans generally communicate by speaking through their mouths. Therefore, it can be assumed that the sound source of the voice information is at least the facial region of one of the communicators. In the displayed image, the sound source region and the facial region of the corresponding communicator should intersect. Therefore, among the several facial regions obtained in the displayed image, the facial region that intersects with the sound source region can be determined as the target facial region that is currently emitting the voice signal.

[0080] Furthermore, in this embodiment, after the step of obtaining the sound source region in the displayed image based on the plane angle, the method further includes:

[0081] Step S34: When none of the facial regions intersect with the sound source region, the facial region closest to the sound source region is taken as the target facial region.

[0082] It is easy to understand that in this embodiment, since the sound source position determined by the sound source angle may differ from the actual sound source position, there may be cases where the sound source area does not intersect or overlap with any facial areas in the displayed image. In this case, the distance from the sound source area to each facial area can be calculated by using the pixels of the displayed image as the unit of measurement, and the facial area with the shortest distance from the sound source area can be selected as the current target facial area.

[0083] It is worth noting that, as another scenario, there are cases where the sound source region intersects with multiple facial regions. In this case, the area of ​​intersection between the sound source region and each facial region can be calculated using the pixels of the displayed image as the unit of measurement. The proportion of the intersection area with the sound source region is then determined by the ratio of the area of ​​each facial region to its corresponding judgment reference value. The facial region with the largest proportion of the intersection area with the sound source region is selected as the current target facial region. The judgment reference value can be adjusted according to the size of each facial region; the larger the size of the corresponding facial region, the larger the required judgment reference value.

[0084] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 After the step of moving the text pattern corresponding to the voice information to the preset text display area corresponding to the periphery of the target facial area, the method further includes:

[0085] Step S51: Detect whether the current preset text display area intersects with other facial areas or other preset text display areas;

[0086] It should be noted that in this embodiment, during actual communication with multiple parties, there may be situations where multiple parties send voice messages. In this scenario, when each party sends a voice message, the voice message is converted into a corresponding text pattern and moved to a preset text display area around the corresponding target facial area in the displayed image. That is, the displayed image shown to the user can simultaneously contain multiple target facial areas and their corresponding text patterns.

[0087] It is easy to understand that in this embodiment, if there are multiple preset text display areas and corresponding multiple target facial areas in the displayed image, the preset text display area corresponding to each target facial area will be detected in turn to see if there is any intersection or overlap with the preset text display areas corresponding to other facial areas or other facial areas.

[0088] Step S52: When it is detected that the preset text display area intersects with the other facial areas or the other preset text display areas, the text pattern is moved out of the area intersecting with the corresponding facial area or preset text display area.

[0089] It is easy to understand that in this embodiment, if there is an intersection or overlap between the preset text display area corresponding to the current target facial area and other facial areas or the preset text display areas corresponding to other facial areas, the pixel coordinates of the preset text display area corresponding to the current target facial area can be adjusted to move it away from the intersecting facial area or the intersecting preset text display area, so that it no longer intersects with other facial areas or preset text display areas, avoiding the text patterns set in the preset text display area from obscuring the text patterns of other facial areas or other preset text display areas. Thus, in a scenario where the user communicates with multiple communication objects simultaneously, it is easier for the user to obtain the voice information and facial expressions sent by each communication object, facilitating the user to understand the true meaning expressed by each communication object in a complex scenario.

[0090] Further, in this embodiment, before the step of moving the text pattern corresponding to the voice information to the preset text display area corresponding to the periphery of the target facial area, the following steps are further included:

[0091] Step S401: Identify the voice information to obtain the audio features of the voice information;

[0092] Step S402: Determine the language and semantics corresponding to the voice information based on the audio features;

[0093] Step S403: Generate the corresponding text pattern according to the semantics based on the language and the corresponding preset voice conversion method.

[0094] It should be noted that the audio features refer to the characteristics of the fundamental frequency, formants, rhythm, etc. of the voice information presented in the frequency domain, which can be extracted from the collected voice information through specific algorithms. The preset voice conversion method refers to the method of converting the voice information with a specific audio feature or the sentence (semantics) corresponding to the voice information into text through a trained voice conversion model.

[0095] It is worth noting that the preset voice conversion method can also include the method of converting the sentences of a certain language into the text of other languages in other countries or regions. For example, converting the Chinese voice information corresponding to "Hello" into the English text "Hello", which can be set actively by the user or automatically selected according to the user's communication habits.

[0096] In this embodiment, audio features can be extracted from the speech information to obtain various language features, such as the sentence, language, and semantics corresponding to the current speech information. The language can include commonly used languages ​​from different countries and regions. The audio features can be compared with a pre-set speech feature library to determine the specific country or region of origin of the current speech information. Simultaneously, based on a preset speech conversion method, the audio features can be directly converted into sentences in the corresponding language. Furthermore, adjustments can be made according to the grammar and other conventions of the corresponding language to appropriately express the semantics in the relevant scenario. Finally, the adjusted sentences are converted into text or text-image, allowing them to be directly generated within a preset text display area of ​​the displayed image for presentation to the user.

[0097] This application provides a control device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the voice recognition display method in the first embodiment described above.

[0098] like Figure 4 As shown, the control device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the control device. The processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the control device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a control device with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0099] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0100] The control device provided in this application, employing the speech recognition and display method described in the above embodiments, solves the technical problem of how to prevent the displayed text from obscuring the face of the person being communicated with when converting the speech of the communicator into text for display through smart glasses. Compared with the prior art, the beneficial effects of the control device provided in this application are the same as those of the speech recognition and display method provided in the above embodiments, and other technical features of this control device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0101] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0102] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0103] Furthermore, this application also proposes a storage medium storing a voice recognition and display program, which, when executed by a processor, implements the steps of the voice recognition and display method described above.

[0104] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0105] The aforementioned computer-readable storage medium may be included in the control device; or it may exist independently and not assembled into the control device.

[0106] The aforementioned computer-readable storage medium carries one or more programs. When the one or more programs are executed by a control device, the control device causes the control device to: acquire voice information emitted by multiple communication objects in a preset information acquisition area and obtain the sound source angle corresponding to the voice information; acquire a display image corresponding to the preset information acquisition area and obtain each facial region corresponding to each of the communication objects in the display image; determine the target facial region corresponding to the voice information in each facial region based on the sound source angle; and move the text pattern corresponding to the voice information to a preset text display area corresponding to the periphery of the target facial region.

[0107] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0110] The storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described speech recognition and display method. This solves the technical problem of how to prevent the displayed text from obscuring the face of the person being communicated with when converting their speech into text using smart glasses. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the speech recognition and display method provided in the above embodiments, and will not be repeated here.

[0111] Furthermore, this application also proposes a smart glasses whose processor can run a voice recognition and display program, which is configured to implement the steps of the voice recognition and display method described above.

[0112] The computer program product provided in this application solves the technical problem of how to avoid the displayed text obscuring the face of the person being communicated with when converting the speech of a communication subject into text for display through smart glasses. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the speech recognition and display method provided in the above embodiments, and will not be repeated here.

[0113] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0114] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0115] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0116] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A voice recognition display method, characterized by, When applied to smart glasses, the voice recognition display method includes the following steps: Collect voice information from multiple communication objects in a preset information collection area, and obtain the sound source angle corresponding to the voice information; Collect the display image corresponding to the preset information collection area, and obtain the facial regions corresponding to each of the communication objects in the display image; Based on the sound source angle, the target facial region corresponding to the voice information is determined in each of the facial regions; Move the text pattern corresponding to the voice information to the preset text display area around the target facial area.

2. The voice recognition display method of claim 1, wherein, The steps of collecting voice information from multiple communication objects in a preset information collection area and obtaining the sound source angle corresponding to the voice information include: The voice information emitted by each of the communication objects within the preset information collection area is collected using a microphone array; Obtain the maximum voice reception time difference corresponding to the voice information received by each individual microphone in the microphone array; The sound source angle is determined based on the maximum voice reception time difference and the preset angular resolution of the microphone array.

3. The voice recognition display method of claim 1, wherein, The step of acquiring the display image corresponding to the preset information acquisition area and obtaining the facial regions corresponding to each of the communication objects in the display image includes: The camera captures the display image corresponding to the preset information acquisition area; The displayed image is scanned to obtain multiple sets of facial features contained in the displayed image; Based on the facial features of each group, the facial regions corresponding to each of the communication objects are determined.

4. The voice recognition display method of claim 1, wherein, The step of determining the target facial region corresponding to the voice information in each of the facial regions based on the sound source angle includes: Convert the sound source angle into the corresponding plane angle in the displayed image; Based on the plane angle, the sound source region in the displayed image is obtained; When one of the facial regions intersects with the sound source region, the facial region intersecting with the sound source region is taken as the target facial region.

5. The voice recognition display method of claim 4, wherein, After the step of obtaining the sound source region in the displayed image based on the plane angle, the method further includes: When none of the facial regions intersect with the sound source region, the facial region closest to the sound source region is taken as the target facial region.

6. The voice recognition display method of claim 1, wherein, After the step of moving the text pattern corresponding to the voice information to the preset text display area around the target facial area, the method further includes: Detect whether the currently defined preset text display area intersects with other facial areas or other preset text display areas; When it is detected that the preset text display area intersects with the other facial areas or the other preset text display areas, the text pattern is moved out of the area where it intersects with the corresponding facial area or preset text display area.

7. The voice recognition display method of claim 1, wherein, Before the step of moving the text pattern corresponding to the voice information to the preset text display area corresponding to the perimeter of the target facial area, the method further includes: The voice information is recognized to obtain its audio features; The language and semantics of the speech information are determined based on the audio features. Based on the language and the corresponding preset speech conversion method, the corresponding text pattern is generated according to the semantics.

8. A control device characterized by comprising: include: A memory, a processor, and a speech recognition display program stored on the memory and executable on the processor, the speech recognition display program being configured to implement the steps of the speech recognition display method as described in any one of claims 1 to 7.

9. A storage medium, characterized in that, The storage medium stores a voice recognition and display program, which, when executed by a processor, implements the steps of the voice recognition and display method as described in any one of claims 1 to 7.

10. A type of smart glasses, characterized in that, The processor of the smart glasses can run a voice recognition display program, which is configured to implement the steps of the voice recognition display method as described in any one of claims 1 to 7.

Images