A visual signal processing system and method for projection equipment based on smart home

Abstract

The present invention discloses a visual signal processing system and method for a projection device based on a smart home, and relates to the technical field of smart homes. A user authorizes to obtain smart home data and environmental data; based on the environmental data, a reference projection brightness of a projection device is determined; data of content to be played is obtained, and brightness data of a picture of the content to be played is determined; based on the brightness data of the picture of the content to be played, brightness compensation is performed on the reference projection brightness of the projection device; according to the brightness compensation result, the projection device is controlled to project a picture according to a set brightness; the brightness of the projection device is adjusted based on the current ambient light intensity, the average brightness of the content to be played is analyzed, and the brightness of the projection device is compensated; when there are many dark scenes and the contrast of the projection device is limited, the brightness of the projection device is automatically increased, so that the user can obtain details of the dark scene without manual operation by the user, thereby improving the user's use experience.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a projection device visual signal processing system and method based on a smart home. Background Art

[0002] Smart home, also known as smart residence or smart home, refers to connecting various devices and services in the home through the integration of advanced information and communication technologies to improve the convenience, comfort, safety and efficiency of living. The projection equipment in smart home usually has intelligent functions and can work with other smart devices in the home to provide a more convenient and personalized viewing experience. At present, the brightness of the projection equipment is adjusted according to the ambient light intensity. In a dark room, a lower brightness is used, and in a bright environment, a higher brightness is used to overcome the interference of ambient light. However, the playback content itself will also affect the user's viewing effect. When there are more dark scenes in the playback content, the projection device will use a lower brightness in a dark room. Due to the limited contrast of the projection device, the user will not be able to obtain the details of the dark scene. Therefore, how to control the brightness of the projection device and improve the user's viewing experience has become an urgent problem to be solved. Summary of the invention

[0003] The purpose of the present invention is to provide a visual signal processing system and method for projection equipment based on smart home to solve the problems raised in the above background technology.

[0004] In one aspect of the present invention, a method for processing visual signals of a projection device based on a smart home is provided, comprising:

[0005] S11, user authorizes access to smart home data and environmental data;

[0006] S12, determining a reference projection brightness of the projection device based on the environmental data;

[0007] S13, obtaining data of the content to be played, and determining the brightness data of the picture of the content to be played;

[0008] S14, performing brightness compensation on the reference projection brightness of the projection device based on the brightness data of the picture of the content to be played;

[0009] S15, according to the brightness compensation result, controlling the projection device to project the picture according to the set brightness.

[0010] In step S12, determining the reference projection brightness of the projection device based on the environmental data further includes the following steps:

[0011] Detect the light intensity of the current environment; convert the light intensity into an electrical signal; when the projection device has a built-in processor, transmit the electrical signal to the built-in processor, otherwise transmit the electrical signal to the computer in the smart home module; analyze the ambient light data through the built-in software or algorithm to determine the projection brightness of the projection device and obtain the reference projection brightness; when using the built-in processor, the built-in processor feeds back the reference projection brightness to the computer in the smart home module.

[0012] In step S13, the step of obtaining the data of the content to be played and determining the screen brightness data of the content to be played further comprises the following steps:

[0013] S31, a computer in the smart home module obtains image frame data of the content to be played, and converts the image data of each frame into a linear RGB color space;

[0014] S32, for each pixel of each frame image, calculate the brightness value, ,in is the red channel value of the jth pixel of the i-th frame image, is the green channel value of the jth pixel of the i-th frame image, is the blue channel value of the jth pixel of the i-th frame image, is the brightness value of the jth pixel in the i-th frame image;

[0015] S33, let the maximum brightness of all pixels in the i-th frame image be , calculate the relative brightness of the jth pixel in the i-th frame image , , calculate the average brightness of the i-th frame image , , n is the number of image pixels; calculate the average brightness of all image frames to obtain the brightness data of the content to be played.

[0016] Whether an image is a bright scene or a dark scene depends on the average brightness of the image, not the maximum brightness. For example, the brightness of stars in a night scene is very high, but the night scene picture still gives people the feeling of a dark scene because the average brightness of the night scene picture is low. The higher the average brightness, the lower the sensitivity threshold of the human eye to contrast. The lower the ADL of the picture, the more black and dark colors there are, the more sensitive the human eye is to contrast, and the greater the impact of contrast, while the contrast of the projection device is limited. In order to see the details of the dark scene clearly, the user will manually increase the brightness on the basis of the automatic brightness adjustment of the projection device. For example, when using a mobile phone in a dark room at night, the result of automatic brightness adjustment is to adjust the brightness of the mobile phone screen to the lowest, and the user often needs to manually increase the screen brightness to obtain picture information.

[0017] In step S14, the brightness compensation for the reference projection brightness of the projection device based on the brightness data of the content to be played also includes the following steps:

[0018] Historical smart home data is obtained, wherein the historical smart home data includes video data watched by a user through a device, device brightness manually adjusted during viewing, device contrast, and device brightness automatically adjusted by the device, wherein the device brightness manually adjusted during viewing is determined according to the peak brightness of the device and the brightness percentage set by the user, and the device contrast data is determined according to the device model, and an average brightness of the video is determined based on the video data, and the average brightness of the video, the device brightness automatically adjusted by the device, and the device contrast are used as inputs, and the device brightness manually adjusted during viewing is used as output to train a neural network model; the average brightness of all current image frames, the baseline projection brightness of the projection device, and the projection device contrast are input into the neural network model to obtain the projection brightness of all image frames after brightness compensation.

[0019] Historical data can be obtained through terminal devices such as mobile phones, tablets and computers, not limited to projectors. The neural network model is trained based on the device's automatic adjustment results, device contrast information and user manual adjustment results.

[0020] In step S15, controlling the projection device to project a picture according to the set brightness according to the brightness compensation result also includes the following steps:

[0021] S51, setting the projection brightness of the image frame after brightness compensation to be the first brightness, dividing the image frame into scenes according to the first brightness, and except for the first and last scenes, dividing each of the other scenes into three parts, namely, a first transition part, a stable part, and a second transition part; the first and last scenes are divided into two parts; the first scene only includes the stable part and the second transition part, and the last scene only includes the first transition part and the stable part;

[0022] S52, determine the average brightness and standard deviation of each scene, replace the projection brightness of the stable part of each scene with the average brightness of the scene, and after replacement, smooth the projection brightness of each frame of each scene, so that Represents the projection brightness before smoothing, represents the projection brightness after smoothing, then , for The projection brightness of the next frame, the next two frames, and the next m frames before image smoothing, for The projection brightness of the previous frame, the previous two frames, ... the previous m frames before smoothing, where m is the smoothing window; is the smoothing coefficient;

[0023] S53, controlling the projection device to play the content according to the smoothed brightness.

[0024] The average brightness of the image frame of the content to be played changes according to the scene. In the same scene, the average brightness is similar, and the average brightness in different scenes is quite different. At the same time, in order to prevent the sudden change in the brightness of the projection device when the scene is switched from affecting the user, the first brightness is smoothed to reduce the amplitude of the projection brightness change. To prevent the projection brightness from changing frequently, the projection brightness of the stable part is set at the average value, and the first transition part and the second transition part are adjusted according to the smoothed result.

[0025] Since the ambient light intensity does not change drastically, the control target of the projection brightness can be input into the projection device after the analysis of the content to be played is completed, without the need for subsequent analysis.

[0026] In step S51, dividing the image frame into scenes according to the first brightness further includes the following steps:

[0027] S61, let k = 1, create an empty set , add the first brightness of the first two frames of the content to be played to the set middle;

[0028] S62, calculate the set before adding the new element Variance of internal elements;

[0029] S63, adding the first brightness of the first frame that is not added to the set as a new element to the set middle;

[0030] S64, calculate the set after adding the new element If the variance increase is less than the threshold, the new element is retained in the set. If the variance increase is not less than the threshold, the new element is not retained in the set. In the set, add one to the value of k and create a new empty set. , add the first brightness of the first two frames that are not added to the set to the set , returning to step S62; after the first brightness of all image frames are added to the set, the scene division is completed;

[0031] S65: The image frames with the first brightness in the same set belong to the same scene, and the number of scenes is the same as the number of sets.

[0032] In step S51, dividing the image frame into scenes according to the first brightness further includes the following steps:

[0033] Calculate the mean of the first brightness of the image frame in each scene , start judging from the first brightness of the image frame on the left side of the scene. If the absolute error between the first brightness of the image frame and the mean is not less than the set value, the image frame is added to the first transition part, and the judgment is repeated until the absolute error is less than the set value; start judging from the first brightness of the image frame on the right side of the scene. If the absolute error between the first brightness of the image frame and the mean is not less than the set value, the image frame is added to the second transition part, and the judgment is repeated until the absolute error is less than the set value; the remaining image frames are stable parts; for the first scene, the first transition part judgment is not performed; for the last scene, the second transition part judgment is not performed.

[0034] In another aspect of the present invention, a visual signal processing system for a projection device based on a smart home is provided, comprising: a data acquisition module, a smart home module and a data storage module; the output end of the data acquisition module is connected to the input end of the smart home module and the data storage module, so as to obtain ambient light intensity data; the smart home module and the data storage module are interconnected, and the projection brightness of the projection device is determined based on the ambient light intensity and information of content to be played; the data storage module is used to store historical data of light intensity, specification information of the projection device and historical data of projection brightness.

[0035] The smart home module also includes a playback content analysis unit, a reference projection brightness determination unit, a projection brightness compensation unit, a scene division unit and a projection control unit; the playback content analysis unit is used to obtain image frame data of the content to be played, convert the image data of each frame into a linear RGB color space, and determine the average brightness of the image of each frame; the reference projection brightness determination unit determines the reference projection brightness of the projection device based on the ambient light intensity; the projection brightness compensation unit obtains the projection brightness after brightness compensation based on the average brightness of the content to be played; the scene division unit divides the content to be played into scenes based on the average brightness; the projection control unit smoothes the projection brightness after brightness compensation based on the divided scenes, and controls the projection device to project according to the set brightness. The data acquisition module also includes a communication unit and an ambient light acquisition unit; the communication unit is used to obtain automatically generated projection brightness information from the projection device; the ambient light acquisition unit is used to obtain ambient light intensity data.

[0036] Compared with the prior art, the beneficial effects achieved by the present invention are: adjusting the brightness of the projection device based on the current ambient light intensity, analyzing the average brightness of the content to be played, compensating for the brightness of the projection device, and automatically increasing the brightness of the projection device when there are many dark scenes and the contrast of the projection device is limited, so that the user can obtain the details of the dark scene without manual operation, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0038] Figure 1 It is a structural schematic diagram of a visual signal processing system of a projection device based on a smart home according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] In the embodiments of the present invention, please refer to Figure 1 , provides a projection device visual signal processing system based on smart home, including: a data acquisition module, a smart home module and a data storage module; the output end of the data acquisition module is connected with the input end of the smart home module and the data storage module, and is used to obtain ambient light intensity data; the smart home module and the data storage module are interconnected, and the projection brightness of the projection device is determined based on the ambient light intensity and information of the content to be played; the data storage module is used to store historical data of light intensity, specification information of the projection device and historical data of projection brightness.

[0041] The smart home module also includes a playback content analysis unit, a reference projection brightness determination unit, a projection brightness compensation unit, a scene division unit and a projection control unit; the playback content analysis unit is used to obtain image frame data of the content to be played, convert the image data of each frame into a linear RGB color space, and determine the average brightness of the image of each frame; the reference projection brightness determination unit determines the reference projection brightness of the projection device based on the ambient light intensity; the projection brightness compensation unit obtains the projection brightness after brightness compensation based on the average brightness of the content to be played; the scene division unit divides the content to be played into scenes based on the average brightness; the projection control unit smoothes the projection brightness after brightness compensation based on the divided scenes, and controls the projection device to project according to the set brightness. The data acquisition module also includes a communication unit and an ambient light acquisition unit; the communication unit is used to obtain automatically generated projection brightness information from the projection device; the ambient light acquisition unit is used to obtain ambient light intensity data.

[0042] In an embodiment of the present invention, a method for processing visual signals of a projection device based on a smart home is provided, comprising:

[0043] S11, user authorizes access to smart home data and environmental data;

[0044] S12, determining a reference projection brightness of the projection device based on the environmental data;

[0045] Detect the light intensity of the current environment; convert the light intensity into an electrical signal; when the projection device has a built-in processor, transmit the electrical signal to the built-in processor, otherwise transmit the electrical signal to the computer in the smart home module; analyze the ambient light data through the built-in software or algorithm to determine the projection brightness of the projection device and obtain the reference projection brightness; when using the built-in processor, the built-in processor feeds back the reference projection brightness to the computer in the smart home module.

[0046] S13, obtaining data of the content to be played and determining the brightness data of the content to be played, specifically including steps S31 to S33;

[0047] S31, a computer in the smart home module obtains image frame data of the content to be played, and converts the image data of each frame into a linear RGB color space;

[0048] S32, for each pixel of each frame image, calculate the brightness value, ,in is the red channel value of the jth pixel of the i-th frame image, is the green channel value of the jth pixel of the i-th frame image, is the blue channel value of the jth pixel of the i-th frame image, is the brightness value of the jth pixel in the i-th frame image;

[0049] S33, let the maximum brightness of all pixels in the i-th frame image be , calculate the relative brightness of the jth pixel in the i-th frame image , , calculate the average brightness of the i-th frame image , , n is the number of pixels in the image; the average brightness of all image frames is calculated to obtain the brightness data of the content to be played.

[0050] S14, based on the brightness data of the content to be played, brightness compensation is performed on the reference projection brightness of the projection device:

[0051] Historical smart home data is obtained, wherein the historical smart home data includes video data watched by a user through a device, device brightness manually adjusted during viewing, device contrast, and device brightness automatically adjusted by the device, wherein the device brightness manually adjusted during viewing is determined according to the peak brightness of the device and the brightness percentage set by the user, and the device contrast data is determined according to the device model, and an average brightness of the video is determined based on the video data, and the average brightness of the video, the device brightness automatically adjusted by the device, and the device contrast are used as inputs, and the device brightness manually adjusted during viewing is used as output to train a neural network model; the average brightness of all current image frames, the baseline projection brightness of the projection device, and the projection device contrast are input into the neural network model to obtain the projection brightness of all image frames after brightness compensation.

[0052] S15, according to the brightness compensation result, controlling the projection device to project the picture according to the set brightness, specifically including steps S51 to S53:

[0053] First, the scene is divided according to steps S61 to S65:

[0054] S61, let k = 1, create an empty set , add the first brightness of the first two frames of the content to be played to the set middle;

[0055] S62, calculate the set before adding the new element Variance of internal elements;

[0056] S63, adding the first brightness of the first frame that is not added to the set as a new element to the set middle;

[0057] S64, calculate the set after adding the new element If the variance increase is less than the threshold, the new element is retained in the set. If the variance increase is not less than the threshold, the new element is not retained in the set. In the set, add one to the value of k and create a new empty set. , add the first brightness of the first two frames that are not added to the set to the set , returning to step S62; after the first brightness of all image frames are added to the set, the scene division is completed;

[0058] Optionally, the threshold of the variance increase value may be taken as 2 times the standard deviation of the variance;

[0059] S65: The image frames with the first brightness in the same set belong to the same scene, and the number of scenes is the same as the number of sets.

[0060] Then divide each scene into 3 parts:

[0061] Calculate the mean of the first brightness of the image frame in each scene , start judging from the first brightness of the image frame on the left side of the scene. If the absolute error between the first brightness of the image frame and the mean is not less than the set value, the image frame is added to the first transition part, and the judgment is repeated until the absolute error is less than the set value; start judging from the first brightness of the image frame on the right side of the scene. If the absolute error between the first brightness of the image frame and the mean is not less than the set value, the image frame is added to the second transition part, and the judgment is repeated until the absolute error is less than the set value; the remaining image frames are stable parts; for the first scene, the first transition part judgment is not performed; for the last scene, the second transition part judgment is not performed.

[0062] Optionally, the set value is set to a standard deviation of the first brightness of the image frame;

[0063] Finally, control the projection device to project the image according to the set brightness:

[0064] S51, setting the projection brightness of the image frame after brightness compensation to be the first brightness, dividing the image frame into scenes according to the first brightness, and except for the first and last scenes, dividing each of the other scenes into three parts, namely, a first transition part, a stable part, and a second transition part; the first and last scenes are divided into two parts; the first scene only includes the stable part and the second transition part, and the last scene only includes the first transition part and the stable part;

[0065] S52, determine the average brightness and standard deviation of each scene, replace the projection brightness of the stable part of each scene with the average brightness of the scene, and after replacement, smooth the projection brightness of each frame of each scene, so that Represents the projection brightness before smoothing, represents the projection brightness after smoothing, then , for The projection brightness of the next frame, the next two frames, and the next m frames before image smoothing, for The projection brightness of the previous frame, the previous two frames, ... the previous m frames before smoothing, where m is the smoothing window; is the smoothing coefficient;

[0066] Optionally, set m to 2, then , if there is a missing image frame before or after, the coefficient is set to zero; for the first image frame, Since there is no data of the first two frames, and Set to 0.

[0067] S53, controlling the projection device to play the content according to the smoothed brightness.

[0068] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0069] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for processing visual signals of a projection device based on smart home, characterized in that: The following steps are involved: S11, user authorizes access to smart home data and environmental data; S12, determining a reference projection brightness of the projection device based on the environmental data; S13, obtaining data of the content to be played, and determining the brightness data of the content to be played; including steps S31 to S33: S31, a computer in the smart home module obtains image frame data of the content to be played, and converts the image data of each frame into a linear RGB color space; S32, for each pixel of each frame image, calculate the brightness value, ,in is the red channel value of the jth pixel of the i-th frame image, is the green channel value of the jth pixel of the i-th frame image, is the blue channel value of the jth pixel of the i-th frame image, is the brightness value of the jth pixel in the i-th frame image; S33, let the maximum brightness of all pixels in the i-th frame image be , calculate the relative brightness of the jth pixel in the i-th frame image , , calculate the average brightness of the i-th frame image , , n is the number of pixels in the image; calculate the average brightness of all image frames to obtain the brightness data of the content to be played; S14, based on the brightness data of the content to be played, brightness compensation is performed on the reference projection brightness of the projection device; comprising the following steps: Acquire historical smart home data, the historical smart home data including video data watched by a user through a device, device brightness manually adjusted during viewing, device contrast, and device brightness automatically adjusted by the device, the device brightness manually adjusted during viewing is determined according to the peak brightness of the device and the brightness percentage set by the user, the device contrast data is determined according to the device model, the average brightness of the video is determined based on the video data, the average brightness of the video, the device brightness automatically adjusted by the device, and the device contrast are used as inputs, and the device brightness manually adjusted during viewing is used as output to train a neural network model; the average brightness of all current image frames, the baseline projection brightness of the projection device, and the projection device contrast are input into the neural network model to obtain the projection brightness of all image frames after brightness compensation; S15, controlling the projection device to project the image according to the set brightness according to the brightness compensation result, including steps S51 to S53; S51, setting the projection brightness of the image frame after brightness compensation to be the first brightness, dividing the image frame into scenes according to the first brightness, and except for the first and last scenes, dividing each of the other scenes into three parts, namely, a first transition part, a stable part, and a second transition part; the first and last scenes are divided into two parts; the first scene only includes the stable part and the second transition part, and the last scene only includes the first transition part and the stable part; S52, determine the average brightness and standard deviation of each scene, replace the projection brightness of the stable part of each scene with the average brightness of the scene, and after replacement, smooth the projection brightness of each frame of each scene, so that Represents the projection brightness before smoothing, represents the projection brightness after smoothing, then , for The projection brightness of the next frame, the next two frames, and the next m frames before image smoothing, for The projection brightness of the previous frame, the previous two frames, ... the previous m frames before smoothing, where m is the smoothing window; is the smoothing coefficient; S53, controlling the projection device to play the content according to the smoothed brightness.

2. According to the method for processing visual signals of projection equipment based on smart home according to claim 1, it is characterized in that: In step S12, determining the reference projection brightness of the projection device based on the environmental data further includes the following steps: Detect the light intensity of the current environment; convert the light intensity into an electrical signal; when the projection device has a built-in processor, transmit the electrical signal to the built-in processor, otherwise transmit the electrical signal to the computer in the smart home module; analyze the ambient light data through the built-in software or algorithm to determine the projection brightness of the projection device and obtain the reference projection brightness; when using the built-in processor, the built-in processor feeds back the reference projection brightness to the computer in the smart home module.

3. According to the method for processing visual signals of projection equipment based on smart home in claim 2, it is characterized in that: In step S51, dividing the image frame into scenes according to the first brightness further includes the following steps: S61, let k = 1, create an empty set , add the first brightness of the first two frames of the content to be played to the set middle; S62, calculate the set before adding the new element Variance of internal elements; S63, adding the first brightness of the first frame that is not added to the set as a new element to the set middle; S64, calculate the set after adding the new element If the variance increase is less than the threshold, the new element is retained in the set. If the variance increase is not less than the threshold, the new element is not retained in the set. In the set, add one to the value of k and create a new empty set. , add the first brightness of the first two frames that are not added to the set to the set , returning to step S62; after the first brightness of all image frames are added to the set, the scene division is completed; S65: The image frames with the first brightness in the same set belong to the same scene, and the number of scenes is the same as the number of sets.

4. According to the method for processing visual signals of projection equipment based on smart home according to claim 3, it is characterized in that: In step S51, dividing the image frame into scenes according to the first brightness further includes the following steps: Calculate the mean of the first brightness of the image frame in each scene , start judging from the first brightness of the image frame on the left side of the scene. If the absolute error between the first brightness of the image frame and the mean is not less than the set value, the image frame is added to the first transition part, and the judgment is repeated until the absolute error is less than the set value; start judging from the first brightness of the image frame on the right side of the scene. If the absolute error between the first brightness of the image frame and the mean is not less than the set value, the image frame is added to the second transition part, and the judgment is repeated until the absolute error is less than the set value; the remaining image frames are stable parts; for the first scene, the first transition part judgment is not performed; for the last scene, the second transition part judgment is not performed.

Images