A real-time live broadcast and face recognition system based on smart AR glasses

Abstract

The present invention discloses a real-time live broadcast and face recognition system based on smart AR glasses, which belongs to the field of face recognition technology, and comprises a video image acquisition module, a user mobile terminal, a face recognition module, a lens capture and tracking module, an audio acquisition module, a wireless transmission module and smart AR glasses; wherein the user mobile terminal comprises a selection input module; the smart AR glasses comprise a wireless communication module, a central processing module, a sound playback module, an image projection module, an image display module and a power supply module; the present invention is provided with a face recognition module and a lens capture and tracking module, which can realize real-time lens switching through face recognition and capture and tracking, obtain a game picture of a face recognition object, and then send the collected audio information to the smart AR glasses together, so that fans in the back seats of the stadium can experience the on-site atmosphere and watch the performance of their favorite stars at close range, thereby helping to improve the on-site fans' viewing experience.

Description

Technical Field

[0001] The present invention relates to the field of face recognition technology, and in particular to a real-time live broadcast and face recognition system based on smart AR glasses. Background Art

[0002] After searching, Chinese patent number CN111783674A discloses a face recognition method and system based on AR glasses. Although the invention can switch working modes according to different usage environments, its application scenarios are limited and cannot be applied to real-time live broadcast scenarios. As the world's largest sport, football is sought after and favored by tens of thousands of people. The World Cup held every four years is even more crowded, filling the stadiums that can accommodate tens of thousands of people. However, the huge stadiums also bring some discomfort to some fans. Due to the popularity of the event, most fans can only choose to buy seats at the back of the stadium. As a result, they can only watch the live broadcast of football matches through the central large screen, which leads to a poor viewing experience. In addition, they cannot watch the wonderful performances of their favorite stars in real time at close range, which makes them very entangled in wanting to experience the atmosphere of the scene and watch the performances of their favorite stars at close range. With the continuous development of technology, smart AR glasses have been developed. Combined with face recognition, it can solve the above problems well, which is conducive to improving the viewing experience of fans at the scene. Therefore, it is particularly important to invent a real-time live broadcast and face recognition system based on smart AR glasses.

[0003] The existing face recognition system based on smart AR glasses has limited application scenarios. It cannot perform player face recognition and camera capture and tracking in real-time football broadcast scenarios, and thus cannot meet the desire of fans in the back seats of stadiums to watch their favorite stars up close, which can easily reduce the viewing experience of fans in the back seats of stadiums. To this end, we propose a real-time live broadcast and face recognition system based on smart AR glasses. Summary of the invention

[0004] The purpose of the present invention is to solve the defects in the prior art and to propose a real-time live broadcast and face recognition system based on smart AR glasses.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A real-time live broadcast and face recognition system based on smart AR glasses, comprising a video image acquisition module, a user mobile terminal, a face recognition module, a lens capture and tracking module, an audio acquisition module, a wireless transmission module and smart AR glasses;

[0007] Among them, the user mobile terminal includes a selection input module; the smart AR glasses include a wireless communication module, a central processing module, a sound playback module, an image projection module, an image display module and a power supply module; the smart AR glasses and the face recognition module are both wirelessly connected to the smart AR glasses.

[0008] Furthermore, the video image acquisition module is specifically a number of camera positions around the football field, which is used to obtain real-time game images of all players on the football field; the user mobile terminal is specifically an electronic communication device, which is used to establish a connection with the smart AR glasses and the face recognition module; the selection input module is specifically an APP, which is used for users to select face recognition objects according to their preferences.

[0009] Furthermore, the face recognition module is used to perform face recognition detection on the face recognition object according to the neural network and in combination with the game screen to obtain a specific recognition target. The specific recognition process is as follows:

[0010] S1: Image acquisition, acquiring images of all players on the football field;

[0011] S2: Image preprocessing: grayscale conversion and image noise removal are performed on the images of all players on the football field, and pixel value features are extracted to obtain a sample feature set of player faces;

[0012] S3: construct a classifier, input the player face sample feature set as input source, and obtain a face recognition model;

[0013] S4: Finally, the face recognition object selected by the user according to his / her preferences is input into the face recognition model to obtain a specific recognition target.

[0014] Furthermore, the lens capture and tracking module is used to capture and track the game screen of the face recognition object according to the specific recognition target, and the specific process is as follows:

[0015] SS1: Get specific identification target;

[0016] SS2: According to the specific identification target, several camera positions are tracked to obtain the tracking result;

[0017] SS3: Perform real-time camera switching based on the tracking results to obtain the game screen of the face recognition object.

[0018] Furthermore, the audio acquisition module is used to obtain audio information on and off the football field in real time, and the audio information includes commentary sound, player sound and referee sound; the wireless transmission module is used to modulate and demodulate the obtained audio information and the game screen of the face recognition object, and wirelessly transmit it to the smart AR glasses.

[0019] Furthermore, the wireless communication module is used to receive the modulated and demodulated audio information and the game screen of the face recognition object, and send them to the central processing module; the central processing module is used to perform analog-to-digital conversion on the audio information and the game screen of the face recognition object, and send the audio information and the game screen of the face recognition object to the sound playback module and the image projection module respectively.

[0020] Furthermore, the sound playback module is specifically a speaker for playing audio information externally; the image projection module is used to project the game screen of the face recognition object onto the image display module; and the image display module is used to display the game screen of the face recognition object to the user.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. The real-time live broadcast and face recognition system based on smart AR glasses is provided with a face recognition module and a lens capture and tracking module. The face recognition module obtains a face recognition model through a neural network, so that the target can be identified and determined for the selected face recognition object; in addition, the lens capture and tracking module performs target capture and tracking on several camera positions through specific recognition targets to obtain tracking results, and then performs real-time lens switching according to the tracking results, which is conducive to real-time acquisition of the game screen of the face recognition object;

[0023] 2. The real-time live broadcast and face recognition system based on smart AR glasses is provided with smart AR glasses, which include a wireless communication module, a central processing module, a sound playback module, an image projection module, an image display module and a power supply module. The smart AR glasses perform analog-to-digital conversion and projection display on the collected audio information and the game images of the face recognition objects, so that fans in the back seats of the stadium can experience the on-site atmosphere and watch the performance of their favorite stars up close, which is conducive to improving the on-site fans' viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0025] Figure 1 This is a schematic diagram of the overall structure of a real-time live broadcast and face recognition system based on smart AR glasses proposed by the present invention. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0027] In the description of the present invention, it is necessary to understand that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0028] Example 1

[0029] Reference Figure 1 , a real-time live broadcast and face recognition system based on smart AR glasses, including a video image acquisition module, a user mobile terminal, a face recognition module, a lens capture and tracking module, an audio acquisition module, a wireless transmission module and smart AR glasses;

[0030] Among them, the user mobile terminal includes a selection input module; the smart AR glasses include a wireless communication module, a central processing module, a sound playback module, an image projection module, an image display module and a power supply module; the smart AR glasses and the face recognition module are both wirelessly connected to the smart AR glasses.

[0031] The video image acquisition module specifically refers to several camera positions around the football field, which are used to obtain real-time game images of all players on the football field; the user mobile terminal specifically refers to an electronic communication device, which is used to establish a connection with the smart AR glasses and the face recognition module; the selection input module specifically refers to an APP, which is used by users to select face recognition objects according to their preferences.

[0032] The face recognition module is used to perform face recognition detection on the face recognition object based on the neural network and combined with the game screen to obtain the specific recognition target. The specific recognition process is as follows:

[0033] S1: Image acquisition, acquiring images of all players on the football field;

[0034] S2: Image preprocessing: grayscale conversion and image noise removal are performed on the images of all players on the football field, and pixel value features are extracted to obtain a sample feature set of player faces;

[0035] S3: construct a classifier, input the player face sample feature set as input source, and obtain a face recognition model;

[0036] S4: Finally, the face recognition object selected by the user according to his / her preferences is input into the face recognition model to obtain a specific recognition target.

[0037] The lens capture and tracking module is used to capture and track the game screen of the face recognition object according to the specific recognition target. The specific process is as follows:

[0038] SS1: Get specific identification target;

[0039] SS2: According to the specific identification target, several camera positions are tracked to obtain the tracking result;

[0040] SS3: Perform real-time camera switching based on the tracking results to obtain the game screen of the face recognition object.

[0041] Specifically, the real-time live broadcast and face recognition system based on smart AR glasses is equipped with a face recognition module and a lens capture and tracking module. The face recognition module obtains a face recognition model through a neural network, so that target recognition and determination can be performed for the selected face recognition object.

[0042] Example 2

[0043] A real-time live broadcast and face recognition system based on smart AR glasses also includes an audio acquisition module, which is used to obtain audio information on and off the football field in real time. The audio information includes commentary sound, player sound and referee sound; the wireless transmission module is used to modulate and demodulate the acquired audio information and the game picture of the face recognition object, and wirelessly transmit them to the smart AR glasses.

[0044] The wireless communication module is used to receive the modulated and demodulated audio information and the game screen of the face recognition object, and send them to the central processing module; the central processing module is used to perform analog-to-digital conversion on the audio information and the game screen of the face recognition object, and send the audio information and the game screen of the face recognition object to the sound playback module and the image projection module respectively.

[0045] The sound playback module is specifically a speaker, which is used to play audio information externally; the image projection module is used to project the game screen of the face recognition object onto the image display module; the image display module is used to feedback the game screen of the face recognition object to the user. The smart AR glasses perform analog-to-digital conversion and projection display on the collected audio information and the game screen of the face recognition object, so that fans in the back seats of the stadium can experience the live atmosphere and watch the performance of their favorite stars up close, which is conducive to improving the viewing experience of fans on site.

[0046] The working principle and use process of the present invention: The real-time live broadcast and face recognition system based on smart AR glasses needs to wear smart AR glasses when in use, and then the video image acquisition module will obtain the game pictures of all players on the football field in real time; at this time, the user can establish a connection with the smart AR glasses and the face recognition module through the electronic communication device, and then select the face recognition object through the selection input module; after the face recognition object is selected, the face recognition module will perform face recognition detection on the face recognition object according to the neural network and combined with the game picture, and the specific process is as follows: the first step is image acquisition, obtaining the images of all players on the football field; the second step is Image preprocessing: convert the grayscale of all the players' images on the football field and remove the image noise, extract the pixel value features, and obtain the sample feature set of the players' faces; the third step: build a classifier, input the sample feature set of the players' faces as the input source, and obtain the face recognition model; the fourth step: finally input the face recognition object selected by the user according to his or her hobbies into the face recognition model to obtain the specific recognition target; then the lens capture and tracking module will capture and track the game screen of the face recognition object according to the specific recognition target; at the same time, the audio acquisition module will obtain the audio information on and off the football field in real time; then the wireless transmission module will obtain the acquired The audio information and the game picture of the face recognition object are modulated and demodulated, and wirelessly transmitted to the smart AR glasses; then the wireless communication module receives the modulated and demodulated audio information and the game picture of the face recognition object, and sends them to the central processing module; then the central processing module performs analog-to-digital conversion on the audio information and the game picture of the face recognition object, and sends the audio information and the game picture of the face recognition object to the sound playback module and the image projection module respectively; then the sound playback module plays the audio information externally, and at the same time the image projection module projects the game picture of the face recognition object onto the image display module, and finally the image display module Feedback and display the game picture of the face recognition object to the user; the present invention is provided with a face recognition module and a lens capture and tracking module, wherein the face recognition module performs player face recognition according to user preference selection to obtain a recognition target, and the lens capture and tracking module performs target capture and tracking on several camera positions according to the recognition target to obtain a tracking result, and then performs real-time lens switching according to the tracking result to obtain the game picture of the face recognition object, and then sends the collected audio information to the smart AR glasses together, so that fans in the back seats of the stadium can experience the on-site atmosphere and watch the performance of their favorite stars at close range, thereby helping to improve the on-site fans' viewing experience.

[0047] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A real-time live broadcast and face recognition system based on smart AR glasses, characterized in that: It includes video image acquisition module, user mobile terminal, face recognition module, lens capture and tracking module, audio acquisition module, wireless transmission module and smart AR glasses; The user mobile terminal includes a selection input module; the smart AR glasses include a wireless communication module, a central processing module, a sound playback module, an image projection module, an image display module and a power supply module; the smart AR glasses and the face recognition module are both wirelessly connected to the smart AR glasses; The video image acquisition module is specifically a number of camera positions around the football field, which is used to obtain the game images of all players on the football field in real time; the user mobile terminal is specifically an electronic communication device, which is used to establish a connection with the smart AR glasses and the face recognition module; the selection input module is specifically an APP, which is used for users to select face recognition objects according to their preferences; The face recognition module is used to perform face recognition detection on the face recognition object according to the neural network and in combination with the game screen to obtain a specific recognition target. The specific recognition process is as follows: S1: Image acquisition, acquiring images of all players on the football field; S2: Image preprocessing: grayscale conversion and image noise removal are performed on the images of all players on the football field, and pixel value features are extracted to obtain a sample feature set of player faces; S3: construct a classifier, input the player face sample feature set as input source, and obtain a face recognition model; S4: Finally, the face recognition object selected by the user according to his / her preferences is input into the face recognition model to obtain a specific recognition target; The lens capture and tracking module is used to capture and track the game screen of the face recognition object according to the specific recognition target, and the specific process is as follows: SS1: Get specific identification target; SS2: According to the specific identification target, several camera positions are tracked to obtain the tracking result; SS3: Real-time camera switching is performed based on the tracking results to obtain the game screen of the face recognition object; The audio acquisition module is used to obtain audio information on and off the football field in real time, and the audio information includes commentary sound, player sound and referee sound; the wireless transmission module is used to modulate and demodulate the obtained audio information and the game picture of the face recognition object, and wirelessly transmit them to the smart AR glasses; The wireless communication module is used to receive the modulated and demodulated audio information and the game picture of the face recognition object, and send them to the central processing module; the central processing module is used to perform analog-to-digital conversion on the audio information and the game picture of the face recognition object, and send the audio information and the game picture of the face recognition object to the sound playback module and the image projection module respectively; The sound playback module is specifically a speaker, which is used to play audio information externally; the image projection module is used to project the game screen of the face recognition object onto the image display module; the image display module is used to feedback and display the game screen of the face recognition object to the user.

Images