Touch screen display device

Abstract

The present invention discloses a touch screen display device, comprising a touch panel, a touch panel driving circuit, a central processing unit, an LED backlight driver, and an LED backlight lamp. The touch display device is characterized in that: the touch display device also includes a color block recognition device, an eye tracking device, and an AI recognition device. The color block recognition device, the eye tracking device, and the AI ​​recognition device are all electrically connected to the central processing unit module and are used for AI adjustment of subtitle position and color when the touch screen display device plays a video. The color block recognition device is connected to the LED backlight lamp. When sensing a large area of ​​the same color of the LED backlight lamp, the color will be recognized and a signal will be transmitted to the central processing unit module. The eye tracking device also includes an ultra-wide-angle front camera, an infrared light source, an image processor, and a viewpoint calculation core. The ultra-wide-angle front camera, the infrared light source, the image processor, and the viewpoint calculation core are all arranged in sequence directly above the touch screen display device. The present invention has the characteristics of strong practicality and intelligence.

Description

Technical Field

[0001] The present invention relates to the technical field of display equipment, and in particular to a touch screen display device. Background Art

[0002] As the market position of smart phones and tablet computers continues to rise, the touch screen industry has been driven to become a sunrise industry. As more and more people like to use touch screen display devices to watch video movies, the traditional method is to add subtitles fixedly at the bottom center of the video screen. This method is simple and efficient. However, due to the richness and diversity of video content, subtitles may block the part of the video that the audience is interested in. And because there is usually continuity between video screens, this occlusion is often continuous, thus affecting the viewing experience. Based on various circumstances, people are in urgent need of a touch screen display device that can display subtitles of video content in a dynamic form, effectively avoid video content occlusion, and improve video playback effects and user viewing experience.

[0003] Therefore, it is necessary to design a touch screen display device that is practical and intelligent. Summary of the invention

[0004] The object of the present invention is to provide a touch screen display device to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a touch screen display device, including a touch panel, a touch panel driving circuit, a central processing unit, an LED backlight driver, and an LED backlight, characterized in that: the touch display device also includes a color block recognition device, an eye tracking device and an AI recognition device, and the color block recognition device, the eye tracking device and the AI ​​recognition device are all electrically connected to the central processing unit module, and are used for AI adjustment of subtitle position and color when the touch screen display device plays a video.

[0006] According to the above technical solution, the color block recognition device is connected to the LED backlight, and when it senses a large area of ​​the same color of the LED backlight, it will recognize the color and quickly send a signal to the central processing unit module.

[0007] According to the above technical solution, the eye tracking device also includes an ultra-wide-angle front camera, an infrared light source, an image processor and a viewpoint computing core. The ultra-wide-angle front camera, infrared light source, image processor and viewpoint computing core are all arranged in sequence directly above the touch screen display device. Compared with ordinary lenses, the ultra-wide-angle lens has a shorter focal length and a larger field of view. Even if the user is very close to the screen, the facial information can still be collected and the human eye can be locked. The high-speed operation of the image processor and the viewpoint computing core can achieve high precision, low latency and a large field of view, so that users have a very good experience when watching videos.

[0008] According to the above technical solution, the ultra-wide-angle front camera collects facial images and captures the eye information of the user's head through the collection and tracking of infrared light sources, image processors and viewpoint calculation core components, and extracts the features of the captured image and obtains the position of the gaze point through precise calculation, and finally sends it to the central processing unit with the lowest delay.

[0009] According to the above technical solution, the ultra-wide-angle front camera can quickly and accurately capture the user's facial information and lock the position of the eyes. The image processor cooperates with the algorithm optimization at the software level and then transmits the signal to the LED backlight driver, so that the eye tracking device can sense the eye position and feed it back to the touch pad through the central processor.

[0010] According to the above technical solution, the central processing unit module still needs to receive the signals sensed by the color block recognition device and the eye tracking device to perform calculations when processing the operations of the original touch screen display device, and then combine with the AI ​​recognition device to realize dynamic subtitles; users no longer have to worry about not being able to concentrate on watching the plot pictures while watching subtitles. This solution allows the subtitles to be presented according to the position of the user's eyes.

[0011] The central processing unit module can combine the signals sent by the color block recognition device and the eye tracking device at the same time, so that the subtitles can be displayed in a large area of ​​color blocks and can also be displayed at a suitable position near the eye focus point according to the movement of the user's eyeballs when watching the plot. The specific selection process is as follows:

[0012] When the eye focus and the color block tracked by the eye tracking device are not in the same place, the central processing unit will give priority to processing the signal of the eye tracking device to intelligently identify the position where the eyes are locked on the screen, and then use this position as the center point to find the large area color block identified by the color block recognition device at the place closest to the center point to present the subtitles. The color of the color block identified by the color block recognition device is uploaded to the central processing unit, and the central processing unit will intelligently change the color of the subtitles according to the color block color, and transmit the signal to the LED backlight driver, thereby achieving a high contrast between the subtitles and the color of the color block behind the subtitles; combined with the position of the eye focus and the large area color block, the subtitles appearing on the screen can follow the position of the eye and avoid unnecessary occlusion. The more humanized design changes the color of the subtitles as the color block behind the subtitles changes, forming a strong contrast with the picture, and successfully solves the embarrassment of traditional subtitles and background colors being the same and unclear.

[0013] According to the above technical solution, the artificial intelligence recognition function of the AI ​​recognition device can recognize shots of character dialogues. When there are shots of character dialogues in the plot, the AI ​​will recognize the dialogue signals based on the up and down jumping and closing of the characters' lips, and then the central processor will transmit the signals to the LED backlight driver according to the set mode to present personalized subtitles such as bubble subtitles on the screen.

[0014] According to the above technical solution, the AI ​​recognition device divides the length and width of the screen into four equal parts, and the entire screen is divided into sixteen small rectangular grids. At this time, the AI ​​recognition device will intelligently identify and distinguish the scene lens mode signal according to the size of the characters or main objects in the plot:

[0015] a. When a person or thing occupies more than 12 blocks on the screen, AI recognition will determine that it is a close-up shot;

[0016] b. When a person or thing occupies six to twelve squares on the screen, AI recognition will determine that this is a close-up shot;

[0017] c. When the image of an individual person or object occupies less than six squares on the screen, AI recognition will determine that this is a long shot.

[0018] According to the above technical solution, the AI ​​recognition device will transmit the recognized different lens modes to the central processor and then connect to the LED backlight driver and the LED backlight to output corresponding personalized subtitles. The specific presentation method is:

[0019] a. When AI recognizes that this is a close-up shot, since the main image occupies a large proportion of the screen, the subtitles will be directly presented in the large color block area in the main image according to the focus position locked by the eyeball; with the color block area signal feedback from the color block recognition device, the subtitles can be evenly arranged in the large color block area;

[0020] b. When AI recognizes that this is a close-up shot, the main content generally occupies a moderate proportion of the screen, and the subtitles will be presented at the closest position to the screen focus outside the main character or object based on the focus position of the eyeball;

[0021] c. When AI recognizes that this is a long shot, the main content usually occupies a smaller proportion of the screen, and a large amount of blank area will be left on the screen. Generally, users will watch the entire video content. At this time, subtitles will appear outside the main object and in a large color block in the central area of ​​the eye focus.

[0022] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0023] (1) By providing an eye tracking device, when the user watches the video on the touch screen display device, the eye tracking device will quickly lock the user's eye information and make corresponding feedback, and the subtitles will always appear flexibly where the user's naked eyes see;

[0024] (2) A color block recognition device is provided to sense the large color block area and the color of the color block on the screen, and then transmit it to the LED light through the central processing unit, so as to present high-contrast subtitles in the large color block area as much as possible;

[0025] (3) An AI recognition device is provided to be able to recognize different lens images such as dialogue shots, close-up shots, and long shots, and make corresponding subtitle mode switches. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0027] Figure 1 It is a schematic diagram of the overall structure of the present invention; DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] See also Figure 1 , the present invention provides a technical solution: a touch screen display device, including a touch panel, a touch panel driving circuit, a central processing unit, an LED backlight driver, and an LED backlight, characterized in that: the touch display device also includes a color block recognition device, an eye tracking device, and an AI recognition device, and the color block recognition device, the eye tracking device, and the AI ​​recognition device are all electrically connected to the central processing unit module, and are used for AI adjustment of subtitle position and color when the touch screen display device plays a video;

[0030] The color block recognition device is connected to the LED backlight, and when it senses a large area of ​​the same color on the LED backlight, it will recognize the color and quickly send a signal to the central processing unit module;

[0031] The eye tracking device also includes an ultra-wide-angle front camera, an infrared light source, an image processor and a viewpoint computing core. The ultra-wide-angle front camera, the infrared light source, the image processor and the viewpoint computing core are all arranged in sequence directly above the touch screen display device. Compared with ordinary lenses, the ultra-wide-angle lens has a shorter focal length and a larger field of view, so that even when the user is very close to the screen, the face information can still be collected and the eyes can be locked. With the high-speed operation of the image processor and the viewpoint computing core, high precision, low latency and a large field of view can be achieved, so that the user has a very good experience when watching videos;

[0032] The ultra-wide-angle front camera collects facial images and uses infrared light sources, image processors, and viewpoint computing core components to collect and track the user's eye information. The features of the captured image are extracted and accurately calculated to obtain the location of the gaze point, which is finally sent to the central processing unit with the lowest latency.

[0033] The ultra-wide-angle front camera can quickly and accurately capture the user's facial information and lock the position of the eyes. The image processor cooperates with the algorithm optimization at the software level and then transmits the signal to the LED backlight driver, so that the eye tracking device can sense the eye position and feedback it to the touchpad through the central processor;

[0034] The CPU module still needs to receive the signals sensed by the color block recognition device and the eye tracking device to perform calculations when processing the operation of the original touch screen display device, and then combine with the AI ​​recognition device to realize dynamic subtitles; users no longer have to worry about not being able to concentrate on watching the plot screen while watching the subtitles. This solution allows the subtitles to be displayed according to the position of the user's eyeballs;

[0035] The central processing unit module can combine the signals sent by the color block recognition device and the eye tracking device at the same time, so that the subtitles can be displayed in a large area of ​​color blocks and can also be displayed at a suitable position near the eye focus point according to the movement of the user's eyeballs when watching the plot. The specific selection process is as follows:

[0036] When the eye focus and the color block tracked by the eye tracking device are not in the same place, the central processor will give priority to processing the signal of the eye tracking device to intelligently identify the position where the eyes are locked on the screen, and then use this position as the center point to find the large area color block identified by the color block recognition device at the place closest to the center point to present the subtitles. The color of the color block identified by the color block recognition device is uploaded to the central processor, and the central processor will intelligently change the color of the subtitles according to the color of the color block, and transmit the signal to the LED backlight driver, thereby achieving a high contrast between the subtitles and the color of the color block behind the subtitles; combined with the position of the eyeball focus and the large area color block, the subtitles appearing on the screen can follow the position of the eyeball and avoid unnecessary occlusion. The more humanized design changes the color of the subtitles as the color block behind the subtitles changes, forming a strong contrast with the picture, and successfully solves the embarrassment of traditional subtitles and background colors being the same and unclear;

[0037] The AI ​​recognition function of the AI ​​recognition device can identify the dialogue shots of the characters. When the characters are talking in the plot, the AI ​​will recognize the dialogue signal based on the ups and downs of the characters' lips and send it back to the central processor. Then the central processor transmits it to the LED backlight driver according to the set mode to display personalized subtitles such as bubble subtitles on the screen.

[0038] The AI ​​recognition device divides the length and width of the screen into four equal parts, and the entire screen is divided into sixteen small rectangular grids. At this time, the AI ​​recognition device will intelligently identify and distinguish the scene lens mode signal according to the size of the characters or main objects in the plot:

[0039] a. When a person or thing occupies more than 12 blocks on the screen, AI recognition will determine that it is a close-up shot;

[0040] b. When a person or thing occupies six to twelve squares on the screen, AI recognition will determine that this is a close-up shot;

[0041] c. When the image of an individual person or object occupies less than six squares on the screen, AI recognition will determine that this is a long shot;

[0042] The AI ​​recognition device will transmit the recognized different lens modes to the central processor and then connect to the LED backlight driver and the LED backlight to output the corresponding personalized subtitles. The specific presentation method is as follows:

[0043] a. When AI recognizes that this is a close-up shot, since the main image occupies a large proportion of the screen, the subtitles will be directly presented in the large color block area in the main image according to the focus position locked by the eyeball; with the color block area signal feedback from the color block recognition device, the subtitles can be evenly arranged in the large color block area;

[0044] b. When AI recognizes that this is a close-up shot, the main content generally occupies a moderate proportion of the screen, and the subtitles will be presented at the closest position to the screen focus outside the main character or object based on the focus position of the eyeball;

[0045] c. When AI recognizes that this is a long shot, the main content usually occupies a smaller proportion of the screen, and a large amount of blank area will be left on the screen. Generally, users will watch the entire video content. At this time, subtitles will appear outside the main object and in a large color block in the central area of ​​the eye focus.

[0046] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0047] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A touch screen display device, comprising a touch panel, a touch panel driving circuit, a central processing unit, an LED backlight driver, and an LED backlight, characterized in that: The touch display device also includes a color block recognition device, an eye tracking device and an AI recognition device, wherein the color block recognition device, the eye tracking device and the AI ​​recognition device are all electrically connected to the central processing unit module and are used for AI adjustment of subtitle position and color when the touch screen display device plays a video; The color block recognition device is connected to the LED backlight, and when it senses a large area of ​​the same color in the LED backlight, it will recognize the color and quickly send a signal to the central processing unit module; The eye tracking device further comprises an ultra-wide-angle front camera, an infrared light source, an image processor and a viewpoint calculation core, wherein the ultra-wide-angle front camera, the infrared light source, the image processor and the viewpoint calculation core are all arranged in sequence directly above the touch screen display device; The ultra-wide-angle front camera collects facial images and tracks the user's eye information through the infrared light source, image processor and viewpoint computing core components, extracts features from the captured image and calculates the position of the gaze point accurately, and finally sends it to the central processing unit with the lowest delay; The ultra-wide-angle front camera can quickly and accurately capture the user's facial information and lock the position of the eyes. The image processor cooperates with the algorithm optimization at the software level, and then transmits the signal to the LED backlight driver, so that the eye tracking device senses the eye position and feeds it back to the touch panel through the central processor; The central processing unit module still needs to receive the signals sensed by the color block recognition device and the eye tracking device to perform calculations when processing the operation of the original touch screen display device, and then combine with the AI ​​recognition device to realize dynamic subtitles; the central processing unit module can combine the signals simultaneously sent by the color block recognition device and the eye tracking device, so that the subtitles can be displayed in a large area of ​​color blocks and can also be displayed in a suitable position near the eye focal point according to the rotation of the user's eyeballs when watching the plot. The specific selection process is as follows: when the eye focus and the color block tracked by the eye tracking device are not in the same place, the central processing unit will give priority to processing the signal of the eye tracking device to intelligently identify the position where the eye is locked on the screen, and then use this position as the center point to find the large area of ​​color blocks recognized by the color block recognition device at the place closest to the center point to present the subtitles, and the color of the color block recognized by the color block recognition device is uploaded to the central processing unit, and the central processing unit will intelligently change the color of the subtitles according to the color block color, and transmit the signal to the LED backlight driver, thereby also achieving a high contrast between the subtitles and the color of the color block behind the subtitles; The AI ​​recognition function of the AI ​​recognition device recognizes the shots of character dialogues. When the characters in the plot are talking, the AI ​​recognizes the shots according to the up and down jumping and opening and closing of the characters' lips and sends a dialogue signal to the central processor. Then the central processor transmits it to the LED backlight driver according to the set mode and displays personalized subtitles on the screen; The AI ​​recognition device divides the length and width of the screen into four equal parts, and the entire screen is divided into sixteen small rectangular grids. At this time, the AI ​​recognition device will intelligently identify and distinguish the scene lens mode signal according to the size of the characters or main objects in the plot: a. When a person or thing occupies more than 12 blocks on the screen, AI recognition will determine that it is a close-up shot; b. When a person or thing occupies six to twelve squares on the screen, AI recognition will determine that this is a close-up shot; c. When the image of an individual person or object occupies less than six squares on the screen, AI recognition will determine that this is a long shot; The AI ​​recognition device will transmit the recognized different lens modes to the central processor and then connect to the LED backlight driver and the LED backlight to output corresponding personalized subtitles. The specific presentation method is: a. When AI recognizes that this is a close-up shot, the subtitles will be directly displayed in the large color block area in the main image according to the focus position locked by the eyeball; with the color block area signal feedback from the color block recognition device, the subtitles will be evenly arranged in the large color block area; b. When AI identifies a close-up shot, the subtitles will be displayed at the closest position to the screen focus outside the main character or object based on the focus position of the eyeball; c. When AI recognizes that this is a long shot, a large amount of blank area will be left on the screen, and users will watch the entire video content. At this time, subtitles will appear outside the main subject and in a large color block in the central area of ​​the eye focus.

Images