Intelligent interactive display method and device for liquid crystal display screen

Abstract

An intelligent interactive display method and device for liquid crystal display screens, comprising the steps of: acquiring real-time video frame data of a liquid crystal splicing display screen; performing convolution calculation on the nth frame data of the current real-time video to extract features, classifying and identifying the video frame image after the pooling operation, acquiring audio data in real time, adjusting the classified and identified video image according to the audio data command, calculating the update degree matrix value in real time, and calculating the change area P. n The degree of change of W k , the degree of change value W k The data is transmitted to the LCD screen for display to prompt the operator to make screen adjustment reference. The update matrix value and change degree value calculation method of the present invention can significantly improve the selection rate of the significant area and greatly reduce image distortion.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an intelligent interactive display method and device for a liquid crystal display screen. Background Art

[0002] In today's modern world of high-speed, centralized information transmission, people constantly acquire a variety of information from the outside world, both actively and passively. Actively, this comes from work records or daily entertainment; passively, from media advertisements and outdoor broadcasts. Research shows that the primary way people acquire information is through vision, with over 60% of information acquired visually. Consequently, research into visual psychology has been ongoing to ensure effective information dissemination. Consequently, large-screen multimedia presentations are in demand because they offer a wider range of information dissemination, high-resolution visual presentations deliver more precise information, and realistic, concrete visual information is more easily accepted and understood. These technologies are widely used in workplace meetings, urban surveillance, outdoor advertising, telemedicine, and other fields. This high-capacity information dissemination is driven by the rapid development of computer technology. As computer processing speeds and storage capacities increase, high-resolution multimedia presentations and virtual reality visualizations, which require enormous computational resources, are gradually maturing. Furthermore, the development of network technology has permeated our daily lives. Increasingly high-speed, even wireless, network transmission is removing the geographical limitations of mobile information dissemination.

[0003] However, traditional large LCD splicing screens have low interaction efficiency and high costs, failing to meet the demands of intelligent display. Furthermore, existing video interaction is inaccurate and inconvenient, making fast, convenient, and efficient transmission essential. Furthermore, traditional video adjustments cannot display the degree of adjustment in real time, nor can they effectively utilize the size of the adjusted data. Therefore, a device capable of real-time, fast, and intelligent display adjustment is urgently needed to improve the user experience. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides an intelligent interactive display method and device for liquid crystal display screens. According to the configuration of the entire process structure and the setting of the feature extraction method, the present invention only performs size transformation on specific areas, greatly reducing the amount of transmitted data and significantly improving data adjustment efficiency. In addition, the calculation method of the change value and other methods of the present application can significantly improve the selection rate of significant areas and greatly reduce image distortion. This is achieved by the following methods:

[0005] An intelligent interactive display method for a liquid crystal display screen includes the following steps: obtaining real-time video frame data of a liquid crystal splicing display screen; performing convolution calculation on the nth frame data of the current real-time video to extract features, performing pooling operation, and classifying and identifying the video frame image after the pooling operation; the convolution operation is as follows:

[0006]

[0007] Where m = 0, 1, 2...M-1, r = 0, 1, 2...R-1, m and r are horizontal and vertical coordinates, f(m, r) is the input image, and g(xm, yr) is the convolution kernel;

[0008] Acquire audio data in real time and adjust the classified and recognized video images based on the audio data;

[0009] Real-time calculation of the updated degree matrix value K n , K n =α(H n -H n-1 )+β(S n -S n-1 )+λ(I n -I n-1 )

[0010] Among them, H n -H n-1 Represents the hue difference between adjacent video frames, S n -S n-1 Indicates the saturation difference between adjacent video frames, I n -I n-1 Represents the brightness difference between adjacent video frames, α, β, and λ are the hue, saturation, and brightness weights respectively;

[0011] The video image identified by the adjustment classification includes zooming in, zooming out, stretching, and shrinking;

[0012] Traverse and update the degree matrix value K n , get the updated degree matrix value K n and the changing area P n ;

[0013] Calculate the change area P n The degree of change of W k :

[0014]

[0015] in, Indicates P n The total grayscale value difference of adjacent video frames in the region, θ n -θ n-1 Indicates P nThe difference between the maximum and minimum grayscale values of the region, Indicates the total grayscale value of the LCD screen. Indicates P n Average gray value of the region, h k Indicates P n The ratio of the area to the size of the LCD screen, h is the ratio setting threshold, if h k -h≥0, then f(h k -h)=1; if h k -h<0, then f(h k -h)=0;

[0016] The change value W k The information is transmitted to the LCD screen for display to prompt the operator to make screen adjustment reference.

[0017] Preferably, the grayscale value acquisition step includes: n Grayscale processing is performed, and the grayscale threshold is selected according to the maximum inter-class variance method OSTU.

[0018] Preferably, the classifying and identifying the video frame images after the pooling operation includes identifying and displaying flowers, animals, and trees.

[0019] Preferably, before performing convolution calculation on the nth frame data of the current real-time video to extract features, the method further includes preprocessing the video image and filtering and denoising the video image information.

[0020] The present invention also discloses an intelligent interactive display device for a liquid crystal display screen, comprising an acquisition module: acquiring real-time video frame data of the liquid crystal splicing display screen;

[0021] The feature extraction module performs convolution calculation on the nth frame data of the current real-time video to extract features and perform pooling operations, and classifies and recognizes the video frame images after the pooling operation; the convolution operation is as follows:

[0022]

[0023] Where m = 0, 1, 2...M-1, r = 0, 1, 2...R-1, m and r are horizontal and vertical coordinates, f(m, r) is the input image, and g(xm, yr) is the convolution kernel;

[0024] The audio acquisition module acquires audio data in real time and adjusts the classified and recognized video images according to the audio data;

[0025] Update degree matrix calculation module, real-time calculation of update degree matrix value K n , K n =α(H n -Hn-1 )+β(S n -S n-1 )+λ(I n -I n-1 )

[0026] Among them, H n -H n-1 Represents the hue difference between adjacent video frames, S n -S n-1 Indicates the saturation difference between adjacent video frames, I n -I n-1 Represents the brightness difference between adjacent video frames, α, β, and λ are the hue, saturation, and brightness weights respectively;

[0027] The video image identified by the adjustment classification includes zooming in, zooming out, stretching, and shrinking;

[0028] Traverse and update the degree matrix value K n , get the updated degree matrix value K n and the changing area P n ;

[0029] Transformation calculation module, calculate the change area P n The degree of change of W k :

[0030]

[0031] in, Indicates P n The total grayscale value difference of adjacent video frames in the region, θ n -θ n-1 Indicates P n The difference between the maximum and minimum grayscale values of the region, Indicates the total grayscale value of the LCD screen. Indicates P n Average gray value of the region, h k Indicates P n The ratio of the area to the size of the LCD screen, h is the ratio setting threshold, if h k -h≥0, then f(h k -h)=1; if h k -h<0, then f(h k -h)=0;

[0032] Display module, the change value W k The information is transmitted to the LCD screen for display to prompt the operator to make screen adjustment reference.

[0033] Preferably, the grayscale value acquisition step includes: nGrayscale processing is performed, and the grayscale threshold is selected according to the maximum inter-class variance method OSTU.

[0034] Preferably, the classifying and identifying the video frame images after the pooling operation includes identifying and displaying flowers, animals, and trees.

[0035] Preferably, before performing convolution calculation on the nth frame data of the current real-time video to extract features, the method further includes preprocessing the video image and filtering and denoising the video image information.

[0036] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0037] The present invention provides an intelligent interactive display method and device for a liquid crystal display screen, by setting the coherence value in a convolutional neural network. , greatly enhancing the recognition accuracy of features in the feature extraction process. In addition, by acquiring audio data in real time and adjusting the classified and identified video images according to the audio data, the adjustment efficiency is greatly enhanced by adjusting the specific images in specific areas. The creative proposal of the calculation of the update degree matrix value and the degree of change value realizes the accurate recognition of the focus area and flexible adjustment. The present invention realizes data transmission only for areas where the video image has changed significantly, while retaining the original image in areas where no significant change has occurred, greatly reducing the amount of transmitted data, and significantly improving the data transmission efficiency, and greatly accelerating the image data selection speed. The image change degree calculation method of the present application can significantly improve the selection rate of significant areas and greatly reduce image distortion. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of the intelligent interactive display method and device for liquid crystal display screens of the present invention;

[0039] Figure 2 Schematic diagram of the pooling operation using average pooling operation in the present invention;

[0040] Figure 3 Schematic diagram of the maximum pooling operation adopted in the pooling operation of the present invention. DETAILED DESCRIPTION

[0041] Those skilled in the art will understand that, as mentioned in the background art, traditional LCD screens have low operating efficiency, large video data transmission volume, and high cost, which cannot meet the needs of intelligence; and the fast, convenient, and efficient transmission of large LCD splicing screens for intelligent operation is currently very necessary. In addition, traditional video transmission transmits all data and cannot effectively utilize the structural characteristics of the transmitted data; therefore, a method that can operate in real time, quickly, and reduce the amount of invalid data has become an urgent need to improve the user experience. In order to make the above-mentioned purposes, features, and beneficial effects of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0042] Example 1:

[0043] An intelligent interactive display method for a liquid crystal display screen, such as Figure 1 As shown, the method includes the following steps: obtaining real-time video frame data of the liquid crystal splicing display screen, which is usually obtained by using a CCD industrial camera or a high-resolution camera; performing convolution calculation on the nth frame data of the current real-time video to extract features, perform pooling operation, and classify and recognize the video frame image after the pooling operation; generally, a convolutional neural network is used for feature extraction, and there are also artificial neural networks or machine learning or SVM machine learning algorithms for feature extraction. In some embodiments, a multi-classification model Adaboost is used for feature extraction, or a pyramid is used; in some embodiments, the pooling operation adopts average pooling, such as Figure 2 As shown in the figure, the specific average pooling operation is to take the average value of each block (such as 2x2) as an element value of the next layer. Therefore, when returning, the loss (or gradient) of each element of the next layer is divided by the size of the block (such as 2x2=4) and then distributed to each element of the block. This is because the loss comes from each element of the block.

[0044] In some embodiments, the pooling operation uses a maximum pooling operation, such as Figure 3 As shown in the figure, max pooling takes the maximum value of each block as the value of an element in the next layer. Therefore, the loss of the next element is derived only from this maximum value, and the gradient update also only updates this maximum value, with the gradient of other values being 0. Therefore, max pooling needs to record the location of the maximum value in the forward propagation, namely max_id. This is also one of the differences between max pooling and average pooling.

[0045] When using a convolutional neural network, the convolution operation is as follows:

[0046]

[0047] Where m = 0, 1, 2...M-1, r = 0, 1, 2...R-1, m and r are horizontal and vertical coordinates, f(m, r) is the input image, and g(xm, yr) is the convolution kernel;

[0048] Acquire audio data in real time, for example, including performing relevant operations on features of the acquired audio data, such as "acquire and stretch a person or animal in a video image," "acquire and shrink a human pose in a video image," "acquire and enlarge a flower image in a video image," "acquire and shrink a house image in a video image," or "acquire and enlarge a lake or river image in a video image." Adjust the classified and identified video image based on the audio data;

[0049] Real-time calculation of the updated degree matrix value K n , K n =α(H n -H n-1 )+β(S n -S n-1 )+λ(I n -I n-1 )

[0050] Among them, H n -H n-1 Represents the hue difference between adjacent video frames, S n -S n-1 Indicates the saturation difference between adjacent video frames, I n -I n-1 Represents the brightness difference between adjacent video frames, α, β, and λ are the hue, saturation, and brightness weights respectively;

[0051] The video image identified by the adjustment classification includes zooming in, zooming out, stretching, and shrinking;

[0052] For example, zooming in and out includes zooming in and out in equal proportions or according to the golden ratio, and slimming down means lengthening the image while shrinking the waist to form an image and data format with the same slender proportions as a person.

[0053] Traverse and update the degree matrix value K n , get the updated degree matrix value K n and the changing area P n ;

[0054] Calculate the change area P n The degree of change of W k :

[0055]

[0056] in, Indicates P nThe total grayscale value difference of adjacent video frames in the region, θ n -θ n-1 Indicates P n The difference between the maximum and minimum grayscale values of the region, Indicates the total grayscale value of the LCD screen. Indicates P n Average gray value of the region, h k Indicates P n The ratio of the area to the size of the LCD screen, h is the ratio setting threshold, if h k -h≥0, then f(h k -h)=1; if h k -h<0, then f(h k -h)=0;

[0057] The change value W k The information is transmitted to the LCD screen for display to prompt the operator to make screen adjustment reference.

[0058] In this embodiment, the grayscale value acquisition step includes: n Grayscale processing is performed, and the grayscale threshold is selected according to the maximum inter-class variance method OSTU.

[0059] For example, in some embodiments, depending on the image presentation type, a threshold method or a histogram method is also used for acquisition. For example, when the image color is dark, the histogram method is used, and when the image color distribution range is wide, the threshold method is used for acquisition.

[0060] In this embodiment, the classification and recognition of the video frame images after the pooling operation includes the recognition and display of flowers, animals, and trees.

[0061] In this embodiment, before performing convolution calculation on the nth frame data of the current real-time video to extract features, the process also includes preprocessing the video image and filtering and denoising the video image information.

[0062] Example 2:

[0063] The present invention also discloses an intelligent interactive display device for a liquid crystal display screen, comprising an acquisition module for acquiring real-time video frame data of a liquid crystal splicing display screen; the acquisition is performed using a common CCD industrial camera or a high-resolution camera.

[0064] The feature extraction module generally uses a convolutional neural network for feature extraction. There are also artificial neural networks, machine learning, or SVM machine learning algorithms for feature extraction. In some embodiments, a multi-classification model Adaboost is used for feature extraction, or a pyramid is used. Convolution calculation is performed on the nth frame data of the current real-time video to extract features, and a pooling operation is performed. The video frame image after the pooling operation is classified and recognized. The convolution operation is as follows:

[0065]

[0066] Where m = 0, 1, 2...M-1, r = 0, 1, 2...R-1, m and r are horizontal and vertical coordinates, f(m, r) is the input image, and g(xm, yr) is the convolution kernel;

[0067] The audio acquisition module acquires audio data in real time and adjusts the classified and identified video images based on the audio data; in some real-time examples, the sound is recognized by acquiring the timbre, volume, etc. of the audio data.

[0068] Update degree matrix calculation module, real-time calculation of update degree matrix value K n , K n =α(H n -H n-1 )+β(S n -S n-1 )+λ(I n -I n-1 )

[0069] Among them, H n -H n-1 Represents the hue difference between adjacent video frames, S n -S n-1 Indicates the saturation difference between adjacent video frames, I n -I n-1 Represents the brightness difference between adjacent video frames, α, β, and λ are the hue, saturation, and brightness weights respectively;

[0070] The video image identified by the adjustment classification includes zooming in, zooming out, stretching, and shrinking;

[0071] Traverse and update the degree matrix value K n , get the updated degree matrix value K n and the changing area P n ;

[0072] Transformation calculation module, calculate the change area P n The degree of change of W k :

[0073]

[0074] in, Indicates P n The total grayscale value difference of adjacent video frames in the region, θ n -θ n-1 Indicates P n The difference between the maximum and minimum grayscale values of the region, Indicates the total grayscale value of the LCD screen. Indicates P n Average gray value of the region, h k Indicates P n The ratio of the area to the size of the LCD screen, h is the ratio setting threshold, if h k -h≥0, then f(h k -h)=1; if h k -h<0, then f(h k -h)=0;

[0075] Display module, the change value W k The information is transmitted to the LCD display screen for display to prompt the operator to make screen adjustment reference. In some embodiments, the display module uses multiple 4*4 or 8*8 or 16*16 or 32*32 square spliced LCD splicing screens; or a rectangular LCD splicing screen.

[0076] In this embodiment, the grayscale value acquisition step includes: n Grayscale processing is performed, and the grayscale threshold is selected according to the maximum inter-class variance method OSTU.

[0077] In this embodiment, the classification and recognition of the video frame images after the pooling operation includes the recognition and display of flowers, animals, and trees. In some embodiments, it also includes the recognition of people, postures, faces, rivers, seas, lakes, and houses.

[0078] In this embodiment, before performing convolution calculation on the nth frame data of the current real-time video to extract features, the process also includes preprocessing the video image and filtering and denoising the video image information.

[0079] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0080] The present invention provides an intelligent interactive display method and device for a liquid crystal display screen, by setting the coherence value in a convolutional neural network. , greatly enhancing the recognition accuracy of features in the feature extraction process. In addition, by acquiring audio data in real time and adjusting the classified and identified video images according to the audio data, the adjustment efficiency is greatly enhanced by adjusting the specific images in specific areas. The creative proposal of the calculation of the update degree matrix value and the degree of change value realizes the accurate recognition of the focus area and flexible adjustment. The present invention realizes data transmission only for areas where the video image has changed significantly, while retaining the original image in areas where no significant change has occurred, greatly reducing the amount of transmitted data, and significantly improving the data transmission efficiency, and greatly accelerating the image data selection speed. The image change degree calculation method of the present application can significantly improve the selection rate of significant areas and greatly reduce image distortion.

[0081] In particular, the method of calculating the degree of change value of the video frame image area in the present application greatly speeds up the image data selection speed; and the image change degree calculation method of the present application can significantly improve the selection rate of significant areas and greatly reduce image distortion.

[0082] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products, and therefore the present application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0083] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. An intelligent interactive display method for a liquid crystal display screen, characterized in that: The method comprises the following steps: obtaining real-time video frame data of a liquid crystal splicing display screen; performing convolution calculation on the nth frame data of the current real-time video to perform feature extraction and pooling operations, and classifying and identifying the video frame image after the pooling operation; The convolution operation is as follows: Where m = 0, 1, 2...M-1, r = 0, 1, 2...R-1, m and r are horizontal and vertical coordinates, f(m, r) is the input image, and g(xm, yr) is the convolution kernel; Acquire audio data in real time and adjust the classified and recognized video images based on the audio data; Real-time calculation of the updated degree matrix value K n , K n =α(H n -H n-1 )+β(S n -S n-1 )+λ(I n -I n-1 ) Among them, H n -H n-1 Represents the hue difference between adjacent video frames, S n -S n-1 Indicates the saturation difference between adjacent video frames, I n -I n-1 Represents the brightness difference between adjacent video frames, α, β, and λ are the hue, saturation, and brightness weights respectively; The video image identified by the adjustment classification includes zooming in, zooming out, stretching, and shrinking; Traverse and update the degree matrix value K n , get the updated degree matrix value K n and the changing area P n ; Calculate the change area P n The degree of change of W k : in, Indicates P n The total grayscale value difference of adjacent video frames in the region, θ n -θ n-1 Indicates P n The difference between the maximum and minimum grayscale values of the region, Indicates the total grayscale value of the LCD screen. Indicates P n Average gray value of the region, h k Indicates P n The ratio of the area to the size of the LCD screen, h is the ratio setting threshold, if h k -h≥0, then f(h k -h)=1; if h k -h<0, then f(h k -h)=0; The change value W k The information is transmitted to the LCD screen for display to prompt the operator to make screen adjustment reference.

2. The intelligent interactive display method for a liquid crystal display screen according to claim 1, characterized in that: The grayscale value acquisition step includes: n Grayscale processing is performed, and the grayscale threshold is selected according to the maximum inter-class variance method OSTU.

3. The intelligent interactive display method for a liquid crystal display screen according to claim 1, characterized in that: The video frame images after the pooling operation are classified and recognized, including the recognition and display of flowers, animals, and trees.

4. The intelligent interactive display method for a liquid crystal display screen according to claim 2, characterized in that: Before performing convolution calculation on the nth frame data of the current real-time video to extract features, the method also includes preprocessing the video image and filtering and denoising the video image information.

5. An intelligent interactive display device for a liquid crystal display screen, characterized in that: Including acquisition module: acquiring real-time video frame data of LCD splicing display screen; The feature extraction module performs convolution calculation on the nth frame data of the current real-time video to extract features and perform pooling operations, and classifies and recognizes the video frame images after the pooling operation; the convolution operation is as follows: Where m = 0, 1, 2...M-1, r = 0, 1, 2...R-1, m and r are horizontal and vertical coordinates, f(m, r) is the input image, and g(xm, yr) is the convolution kernel; The audio acquisition module acquires audio data in real time and adjusts the classified and recognized video images according to the audio data; Update degree matrix calculation module, real-time calculation of update degree matrix value K n , K n =α(H n -H n-1 )+β(S n -S n-1 )+λ(I n -I n-1 ) Among them, H n -H n-1 Represents the hue difference between adjacent video frames, S n -S n-1 Indicates the saturation difference between adjacent video frames, I n -I n-1 Represents the brightness difference between adjacent video frames, α, β, and λ are the hue, saturation, and brightness weights respectively; The video image identified by the adjustment classification includes zooming in, zooming out, stretching, and shrinking; Traverse and update the degree matrix value K n , get the updated degree matrix value K n and the changing area P n ; Transformation value calculation module, calculates the change area P n The degree of change of W k : in, Indicates P n The total grayscale value difference of adjacent video frames in the region, θ n -θ n-1 Indicates P n The difference between the maximum and minimum grayscale values of the region, Indicates the total grayscale value of the LCD screen. Indicates P n Average gray value of the region, h k Indicates P n The ratio of the area to the size of the LCD screen, h is the ratio setting threshold, if h k -h≥0, then f(h k -h)=1; if h k -h<0, then f(h k -h)=0; Display module, the change value W k The information is transmitted to the LCD screen for display to prompt the operator to make screen adjustment reference.

6. The intelligent interactive display device for a liquid crystal display screen according to claim 5, characterized in that: The grayscale value acquisition step includes: n Grayscale processing is performed, and the grayscale threshold is selected according to the maximum inter-class variance method OSTU.

7. The intelligent interactive display device for a liquid crystal display screen according to claim 5, characterized in that: The video frame images after the pooling operation are classified and recognized, including the recognition and display of flowers, animals, and trees.

8. The intelligent interactive display device for a liquid crystal display screen according to claim 5, characterized in that: Before performing convolution calculation on the nth frame data of the current real-time video to extract features, the method also includes preprocessing the video image and filtering and denoising the video image information.

Images