A display processing method and display device

Abstract

This application provides a display processing method and a display device; the method includes: obtaining a target instruction; in response to the target instruction, adjusting the pixels of a target region in at least two frames of a first image to obtain corresponding image data; the pixel values ​​of the target regions in adjacent frames of the image data satisfy a change condition; outputting and displaying based on the image data, such that the target region of a second image contains identification information; the second image is obtained by a first electronic device acquiring an image based on the output display result of the display device.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a display processing method and a display device. Background Technology

[0002] Currently, many software solutions exist that embed different frequency domain (transform domain) algorithms into the graphics card. These solutions add invisible digital watermarks to the display data to provide invisibility protection. The embedded watermarks are invisible during normal display, but can be extracted using specific algorithms after taking a photo or screenshot for purposes such as tracing the source of illegally leaked information. However, if a user extracts the text information from a photo or screenshot and re-renders the image background, the watermark will be lost in the re-rendered image. Furthermore, the watermark will not be displayed on cameras without extraction algorithms. Therefore, the above methods offer poor protection for display data. Summary of the Invention

[0003] This application provides a display processing method, a display device, a computer-readable storage medium, and a computer program product.

[0004] The technical solution of this application embodiment is implemented as follows: In a first aspect, embodiments of this application provide a display processing method applied to a target intelligent agent in an electronic device, the method comprising: Obtain the target instruction; In response to the target instruction, the pixels of the target region in at least two frames of the first image are adjusted to obtain corresponding image data; the pixel values ​​of the target region in adjacent frames of the image data satisfy the change condition. The image data is output and displayed so that the target area of ​​the second image contains identification information; the second image is obtained by the first electronic device from the image acquisition of the output display result of the display device.

[0005] Secondly, embodiments of this application provide a display device, which includes: a display panel and a control module; wherein, The control module is used to obtain a target instruction; in response to the target instruction, adjust the pixels of the target region in at least two frames of the first image to obtain corresponding image data; the pixel values ​​of the target region in adjacent frames of the image data satisfy the change condition; output and display based on the image data, so that the target region of the second image has identification information; the second image is obtained by the first electronic device from the image acquisition of the output display result of the display panel.

[0006] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the display processing method provided in embodiments of this application when executed by a processor.

[0007] Fourthly, embodiments of this application provide a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the display processing method provided in embodiments of this application. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating a display processing method provided in an embodiment of this application. Figure 1 ; Figure 2 This is a flowchart illustrating a display processing method provided in an embodiment of this application. Figure 2 ; Figure 3 This is a flowchart illustrating a display processing method provided in an embodiment of this application. Figure 3 ; Figure 4 This is a flowchart illustrating a display processing method provided in an embodiment of this application. Figure 4 ; Figure 5 This is a flowchart illustrating a display processing method provided in an embodiment of this application. Figure 5 ; Figure 6 This is a flowchart illustrating a display processing method provided in an embodiment of this application. Figure 6 ; Figure 7 This is a flowchart illustrating a display processing method provided in an embodiment of this application. Figure 7 ; Figure 8 This is a flowchart illustrating a display processing method provided in an embodiment of this application. Figure 8 ; Figure 9 This is a schematic diagram of the structure of a display device provided in an embodiment of this application. Figure 1 ; Figure 10 This is a schematic diagram of the structure of a display device provided in an embodiment of this application. Figure 2 ; Figure 11 This is a schematic diagram illustrating a photograph with a watermark provided in an embodiment of this application; Figure 12 This is a schematic diagram of a display device displaying raw data according to an embodiment of this application; Figure 13 This is a schematic diagram illustrating a display device displaying actual watermark data according to an embodiment of this application; Figure 14 This is a flowchart of a watermark generation method provided in an embodiment of this application; Figure 15 This is a flowchart of a dynamic refresh rate adjustment scheme provided in an embodiment of this application; Figure 16 This is a logic block diagram of a display processing device provided in an embodiment of this application. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0010] To address the issues of easily removed invisible watermarks and weak data protection in related technologies, this application provides a display processing method applied to a display device. Exemplarily, the display device includes, but is not limited to, smart screens, OLED displays, VA (vertical alignment) displays, various IPS displays, etc., and the implementation form of this application is not fixed or limited.

[0011] In this embodiment, the display device can adjust its pixels after receiving a target instruction to obtain image data that meets the change conditions, and then display the image data. When the first electronic device collects the output display result of the display device, the second image obtained can directly display observable identification information, reducing the strong dependence on the computing power of the camera. For explicit identification information, it is impossible to eliminate it through conventional image processing technology. When the second image is leaked, the source can be traced directly through the identification information displayed in the collected image. Since the identification information can be observed when the image is collected, it can serve as a reminder to the user of the first electronic device to prevent leakage, thereby greatly improving the protection of the image data output by the display device. In addition, this method does not modify the original image data itself. Therefore, the original image data will not be affected when circulating internally. The identification information of one display device will not appear on another display device, so there will be no reading difficulties. To a certain extent, this also ensures the convenience of the above method in practical scenarios.

[0012] The technical solution of this application will be described in detail below with reference to embodiments, such as... Figure 1 As shown, Figure 1 A flowchart illustrating a display processing method provided in this application embodiment. Figure 1This display processing method can be implemented through steps S101 to S103: Step S101: Obtain the target instruction.

[0013] In the embodiments of this application, the target instruction can be used to cause the display device to enter a protected display mode, and to overlay display identification information when outputting to the display device. The target instruction can be automatically generated by the display device when it is powered on, or it can be generated by the main control device connected to the display device and sent to the display device.

[0014] Step S102: In response to the target instruction, the pixels of the target region in at least two frames of the first image are adjusted to obtain the corresponding image data; the pixel values ​​of the target region in adjacent frames of the image data meet the change condition.

[0015] In the embodiments of this application, when the display device receives the first image, it can adjust the pixels of the target area in at least two frames of the first image, and after the adjustment is completed, the corresponding image data is obtained.

[0016] For display devices lacking independent display capabilities, the first image refers to the display image sent by the main control device to the display device. For example, when the main control device receives an instruction to open a Word document, it will generate a corresponding display image based on the Word document's rendering data (such as a preset image like "Opening file" and a display image containing document data, possibly with a display ratio). The main control device will continue to output the current last frame display image until it receives a new instruction. For display devices with independent display capabilities, the first image can be generated by the display device itself.

[0017] In addition, for image data, the pixel values ​​of the target area in adjacent frames must meet a change condition. Pixel values ​​can refer to grayscale or luminance values. The change condition can be that the average pixel values ​​of the target area in adjacent frames equal the original pixel values ​​of the target area before adjustment. Furthermore, for a single frame image, the change in pixel values ​​is usually set to a small value to avoid excessive power consumption when the display device outputs the adjusted image data.

[0018] It should be noted that regardless of whether the pixel value being adjusted is a grayscale value or a brightness value, the adjusted image data is only used for display on the display device. The only difference is the specific implementation method (for example, the implementation method of grayscale value adjustment is to modify the grayscale value of the target area, while the implementation method of brightness value adjustment is to adjust the driving voltage of the physical pixel), and it will not affect the original image data of the first image itself.

[0019] Step S103: Output display based on image data, so that the target area of ​​the second image contains identification information; the second image is obtained by the first electronic device from the image acquisition of the output display result of the display device.

[0020] In the embodiments of this application, the display device can output image data obtained after adjusting the display pixels. In this way, when the user uses the first electronic device to capture the image of the output display result of the display device, the target area of ​​the captured second image can contain identification information.

[0021] If the display device's refresh rate is greater than 60Hz (where 60Hz is the lowest refresh rate at which the human eye cannot perceive screen cutoffs when the image changes), such as 99Hz, 100Hz, 120Hz, or 144Hz, then when the user observes the display's output, the human eye cannot capture the changing characteristics of adjacent frames at high refresh rates in real time. Therefore, in this case, the user will not observe the display information. This significantly reduces the impact of the display device outputting display information on the user's observation of the original image, further improving the user experience.

[0022] It should be noted that when outputting display data, the display device may not accumulate the display images sent by the main control device. Instead, it can adjust and output each frame as it receives it. Upon receiving the next frame, it adjusts the next frame's display image according to the change conditions of the previous frame and outputs it, repeating this process. This allows the display device to continuously output multiple frames of the first image that meet the change conditions. Alternatively, the display device may not accumulate the display images it generates, but instead adjust and output each frame as it generates it.

[0023] Based on the above embodiments disclosed in this application, the display device can adjust the pixels of the display device after receiving the target instruction to obtain image data that meets the change conditions, and display the image data accordingly. When the first electronic device is used to collect the output display result of the display device, the observable identification information can be directly displayed in the second image obtained, reducing the strong dependence on the computing power of the camera. For explicit identification information, it is impossible to eliminate it through conventional image processing technology. When the second image is leaked, the source can be traced directly through the identification information displayed in the collected image. Since the identification information can be observed when the image is collected, it can serve as a reminder to the user of the first electronic device to prevent leakage, thereby greatly improving the protection of the image data output by the display device. In addition, this method does not modify the original image data itself. Therefore, the original image data will not be affected when circulating internally. The identification information of one display device will not appear on another display device, so there will be no problem of difficulty in reading. To a certain extent, it also ensures the convenience of the above method in practical scenarios.

[0024] In some embodiments, such as Figure 2 As shown, Figure 2 A flowchart illustrating a display processing method provided in this application embodiment. Figure 2 Step S102 can be achieved through steps S121 and S122: Step S121: Determine the location information of the target region in at least two frames of the first image.

[0025] In the embodiments of this application, the identification information may not exist in all areas of the screen displayed on the display device, but only in a portion of the screen, i.e., the target area. For example, depending on the size of the identification information, the target area can be arranged with a sub-display area of ​​the identification information at regular intervals within the display area of ​​the display device. The display angle of the identification information can also be limited. These parameters can be preset and written into the display device, or sent to the display device by the main control device.

[0026] When the display device begins processing the first image, it can read or receive the location information of the target area sent by the main control device from within the display device. This location information is unrelated to the first image itself, but the identification information displayed at the corresponding location will be superimposed on the first image.

[0027] Step S122: Based on the location information of the target area and the pixel change value, adjust the pixel value of the target area to obtain the corresponding image data; wherein, the pixel value adjustment direction of adjacent frames in the image data is opposite.

[0028] In the embodiments of this application, the pixel change value is an inherent attribute of the identification information, representing the difference between the pixel value of the identification information in the target area and the pixel value of the first image. Based on the aforementioned location information and pixel change value, the pixel value of the target area can be adjusted. For a single frame image, the adjustment direction can be either adding or subtracting the aforementioned pixel change value from the pixel value of each pixel in the target area simultaneously. For adjacent frame images, if the adjustment direction of the pixel value of the target area in the first frame image is adding the pixel change value, then the adjustment direction of the pixel value of the target area in the second frame image is opposite to that of the first frame image, i.e., subtracting the pixel change value, to maintain the opposite adjustment directions of the pixel values ​​in adjacent frame images in the image data.

[0029] Based on the embodiments disclosed in this application, the pixel values ​​of the target region can be adjusted in the target region of at least two frames of the first image, which can reduce the number of pixel adjustments and improve the accuracy of pixel adjustments to a certain extent.

[0030] In some embodiments, such as Figure 3 As shown, Figure 3 A flowchart illustrating a display processing method provided in this application embodiment. Figure 3 Step S121 can be achieved through steps S1211 to S1213: Step S1211: Obtain the identification display data.

[0031] In embodiments of this application, the identification display data may include at least identification information, identification display angle, and identification size. The identification information characterizes the specific appearance of the displayed identification; for example, the identification information may be an employee number, name, or other specific string. The identification information may be preset and stored in the display device or in the main control device used to control the display device. When the display device needs to output the identification information, it can retrieve the identification display data from the display device's storage space or receive the identification display data sent by the main control device.

[0032] Step S1212: Generate identifier mask information based on the identifier display data.

[0033] In the embodiments of this application, the identifier mask information refers to an identifier mask image. The identifier mask information may include the relative positions, display angles, display areas, and display quantities of various identifier information. This information can be integrated into an identifier mask image for overlay display with a first image.

[0034] Step S1213: Determine the location information of the target region in the first image based on the identifier mask information.

[0035] In the embodiments of this application, since the identification mask information can be fully represented by the identification mask image, the location information of the target area in the first image can be obtained based on the identification mask information, that is, the area in the first image whose pixel value needs to be adjusted can be determined.

[0036] In one possible embodiment, the mask position in the identifier mask information can also be made random, that is, the same randomly generated position adjustment parameter is applied to all mask positions. In this way, the position information of the target area in the first image can be obtained through the generated position adjustment parameter and the identifier mask information, that is, "mask position information + position adjustment parameter = target area position information".

[0037] Step S1211 includes one of the following: Step S21: Obtain identity information, input the identity information into the first model, and generate corresponding identification display data.

[0038] In the embodiments of this application, the first model can be any type of encoder with fixed parameters (e.g., convolutional neural network encoder, Transformer encoder, hash value, etc.) used to perform targeted encoding of the user's identity information. The identity encoding data obtained after targeting is the corresponding identifier display data.

[0039] Step S22: Obtain identity information and read the corresponding identifier display data from the preset library; The identity information comes from the first electronic device, the second electronic device (such as the main control device mentioned above), or the display device; the display device is used to display the screen of the second electronic device.

[0040] In embodiments of this application, designated identifier display data can also be directly assigned to each user. For example, the identifier display data for user 1 is "A", the identifier display data for user 2 is "B", and so on. When the display device obtains the user's identity information from the first electronic device, the second electronic device, or the display device, it can directly read the identifier display data matching the identity information from a preset library storing the designated identifier display data.

[0041] The first electronic device can be a user's mobile phone, computer, camera, or other similar device; the second electronic device is an office device used by the user and connected to the display device. Both the first and second electronic devices can store the user's identity information or receive user input; the display device can pre-store the user's identity information. This allows the display device to retrieve the relevant identity information when needed.

[0042] Based on the above embodiments disclosed in this application, unique identification information matching the user can be generated according to the user's identity information, which can improve the matching degree between the identification information and the user to a certain extent and improve the traceability of the final acquired image.

[0043] In some embodiments, such as Figure 4 As shown, Figure 4 A flowchart illustrating a display processing method provided in this application embodiment. Figure 4 The above display processing method further includes at least one of the following: Step S104: Generate multiple compensation frame images of at least two first images, and adjust the pixels of the target region in the at least two first images and multiple compensation frame images respectively to obtain the corresponding image data.

[0044] In embodiments of this application, the display device can also perform frame compensation processing on the received image. When the display device receives the first frame image, it can generate multiple compensated frame images by copying the frame image. When the display device receives the next frame image of the previously received image, it can continue to generate multiple compensated frame images by copying the second frame image.

[0045] After generating the compensation frame images, the display device can adjust the pixels of the target region in the two first frames and multiple compensation frame images respectively. The adjustment targets are as follows: For the first frame image and the next compensation frame image after the first frame image, the above-mentioned change conditions must be met; for multiple compensation frame images between the first frame image and the second frame image, the above-mentioned change conditions must be met simultaneously; for the last compensation frame image between the first frame image and the second frame image, the above-mentioned change conditions must be met simultaneously.

[0046] In one possible embodiment, only one compensation frame image may be generated between adjacent frame images.

[0047] For example, if only one compensation frame image is generated between adjacent frame images, the adjustment direction of the pixels in the target region of the first image of the adjacent frame is the same, while the adjustment direction of the pixels in the target region of the compensation frame image is opposite to that of the pixels in the first image of the adjacent frame. In the case of multiple frames, the adjustment direction can be assigned to multiple compensation frame images with reference to the above adjustment target.

[0048] Step S105: Send an adjustment signal to the second electronic device to increase the frame rate of the output first image; the display device is used to display the image output by the second electronic device.

[0049] In the embodiments of this application, the frame rate is the desired frequency at which a bitmap image, in units of frames, appears continuously on the display. The refresh rate refers to the retrace frequency of the display device's screen, and can also be expressed as the desired frequency at which the display device outputs the displayed image. The actual frequency at which the display device outputs the image is determined by the smaller of the frame rate and the refresh rate. For example, if the frame rate is 30 FPS and the refresh rate is 60 Hz, then the actual output frequency of the display device is 30 Hz; if the frame rate is 99 FPS and the refresh rate is 60 Hz, then the actual output frequency of the display device is 60 Hz.

[0050] In the embodiments of this application, the display device can also utilize a second electronic device (including a main control device) to increase the frame rate of the output first image, achieving the same technical effect as frame compensation processing. First, the second electronic device attempts to perform a handshake operation with the display device for high frame rate display. If the handshake is successful, the display device can send a handshake success signal to the second electronic device. This allows the second electronic device to increase the frame rate of the output first image, outputting it at a target frame rate higher than the original frame rate. In this process, the handshake success signal can be considered an adjustment signal. Alternatively, upon confirming a successful handshake, the display device can generate an adjustment signal in response and send it to the second electronic device, similarly enabling the second electronic device to increase the frame rate of the output first image. Clearly, the display device is used to display the first image output by the second electronic device. It should be noted that the above applies to displays without independent display capabilities. For displays with independent display capabilities, the frame rate of their own image processor can also be increased.

[0051] Step S106: Output and display the image data according to the target refresh rate; the target refresh rate is greater than the predetermined refresh rate.

[0052] In embodiments of this application, the display device can also actively increase its refresh rate and output image data at a target refresh rate after the refresh rate is increased. Here, the predetermined refresh rate refers to the refresh rate of the display device before the refresh rate is increased.

[0053] Based on the above embodiments disclosed in this application, the display device can increase the frame rate or refresh rate of the output display image by at least one of three methods, which can reduce the occurrence of users observing the identification information with their naked eyes to a certain extent, and can improve the security and traceability of the output display content while ensuring the user's experience of using the display device.

[0054] In some embodiments, such as Figure 5 As shown, Figure 5 A flowchart illustrating a display processing method provided in this application embodiment. Figure 5 The above display processing method may further include: Step S107: Obtain the frame rate and refresh rate.

[0055] In the embodiments of this application, frame rate refers to the frequency at which the image processor outputs images. The image processor can be a GPU (for a display projected from a main control device or a display with independent display capabilities) or a scaler (for a display without independent display capabilities). The display device can calculate the frame rate of the image processor based on the time difference between receiving adjacent frames when it receives multiple frames from the image processor; alternatively, it can directly obtain the frame rate data sent by the image processor.

[0056] In addition, the refresh rate parameters of the display panel of the display device are usually stored in the cache of the display device after the settings are completed (the second electronic device will also store them at the same time), and the display device can read them when necessary (such as high frame rate handshake).

[0057] Step S108: If the refresh rate is greater than the frame rate, generate at least two frames of the first image, or send an adjustment signal to the second electronic device.

[0058] In the embodiments of this application, if the refresh rate is greater than the frame rate, it means that the display device cannot switch the displayed image at every rewind time, i.e., the actual output frequency of the display device will be lower than the refresh rate. In this case, in order to increase the actual output frequency of the display device, if the display device does not have autonomous display capability, it can generate multiple compensation frame images of at least two first images.

[0059] If the display device has autonomous display capabilities, it can send an adjustment signal to the second electronic device so that the second electronic device can increase the frame rate of its output image after receiving the adjustment signal.

[0060] Based on the above embodiments disclosed in this application, when the refresh rate of the display device is higher than the frame rate of the image processor, multiple compensation frame images can be generated for frame compensation or an adjustment signal can be sent to the second electronic device to enable the second electronic device to increase the frame rate of its output image. To a certain extent, this can increase the frequency of the actual output display image of the display device and reduce the occurrence of users observing identification information.

[0061] In some embodiments, such as Figure 6 As shown, Figure 6 A flowchart illustrating a display processing method provided in this application embodiment. Figure 6 Step S101 includes: Step S111: Obtain an environmental image.

[0062] In embodiments of this application, the display device can capture environmental images around the display device via a camera connected to the control board of the display device or via a camera connected to a second electronic device. The environmental image typically refers to an image of the space facing the display panel of the display device, and may have a certain shooting angle.

[0063] Step S112: Input the environmental image into the second model. If the environmental image represents an intention to acquire images for the display device or an image acquisition behavior for the display device, trigger the target instruction.

[0064] In the embodiments of this application, a second model can be used to perform entity recognition or behavior recognition on environmental images. If the environmental image represents an image acquisition intent targeting a display device or an image acquisition behavior targeting only a display device, such as recognizing a camera, mobile phone camera, or other shooting device, or recognizing a shooting posture, a target instruction can be triggered so that the display device can respond to the target instruction and perform subsequent actions.

[0065] The second model can be a deep learning-based entity recognition or behavior recognition model, which can be trained to enable it to recognize entities or behaviors.

[0066] Based on the above embodiments disclosed in this application, the environmental image in front of the display device can be identified, and when an image acquisition intention or image acquisition behavior is detected for the display device, a target instruction is triggered to add identification information to the image output by the display device. To a certain extent, this can improve the matching degree between the display identification information output by the display device and the user behavior, and improve the accuracy of the display identification information output by the display device.

[0067] In some embodiments, such as Figure 7 As shown, Figure 7 A flowchart illustrating a display processing method provided in this application embodiment. Figure 7 Step S1212 includes: Step S31: Determine the span of the identifier pixels on the display area of ​​the display panel to display the identifier display data.

[0068] In the embodiments of this application, the identifier pixel span refers to the number of pixels in the length and width directions of the area occupied by the identifier display data. In simpler terms, the identifier pixel span is the resolution of the identifier. For example, if pixels are represented by "px", then the identifier pixel span could be "100px × 20px".

[0069] The display panel can be a liquid crystal display panel, a virtual display panel (such as augmented reality (AR) display), or other types of display panels. The display area is the largest area of ​​the display panel that can output and display images.

[0070] Step S32: If the pixel span of the identifier meets the preset identifier length information, determine the maximum number of identifier display data to be displayed based on the identifier display data and the resolution data of the display area.

[0071] In the embodiments of this application, the preset identifier length information is either a specified identifier length information allocated by the second electronic device to the display device or a specified identifier length information stored internally by the display device. Considering the specificity of the identifier information, the specified identifier length information can be a length range.

[0072] The maximum number of signs that can be displayed refers to the maximum number of signs that can exist in the display area of ​​the display panel, which is determined by both the sign display data and the resolution data. For example, if the distance between signs in the sign display data is 100px, and the sign pixel span is 100px × 20px, and the resolution data of the display area is (10 × 540px) × (10 × 320px), then the maximum number of signs that can be displayed is 28 × 27 = 756. Here, "28" refers to the number of signs displayed in the length direction of the display area (first, add the distance between signs to the number of pixels in the length direction of the sign pixel span, then divide the length data of the display area by the sum of the above additions and round up), and "27" refers to the number of signs displayed in the width direction of the display area. If the signs have a certain display angle, the calculation method is similar; it only needs to be calculated separately for the display area corresponding to each sign display direction.

[0073] Step S33: Generate identifier mask information based on the maximum number of displays and the identifier display data.

[0074] In the embodiments of this application, the number of identifiers included in the identifier mask information can be the maximum number of displays, or it can be the number of displays obtained by subtracting one in both the length and width directions. This number of displays can be calculated from the maximum number of displays. When the identifier mask information contains the maximum number of identifiers to be displayed, its identifier mask information is usually unique; if the identifier mask information contains the aforementioned number of identifiers to be displayed, then the identifier mask information is not unique, and the position of the vertex identifier can be randomly determined between the vertex position and the identifier interval position of the display area.

[0075] Based on the above embodiments disclosed in this application, the maximum number of identifiers that the display device can display can be determined according to the identifier pixel span, and identifier mask information can be generated according to the maximum number of identifiers and the identifier display data. To a certain extent, this can improve the matching degree between the identifier mask information and the current display device and improve the accuracy of the identifier mask information.

[0076] In some embodiments, such as Figure 8 As shown, Figure 8A flowchart illustrating a display processing method provided in this application embodiment. Figure 8 Step S102 includes: Step S121: Obtain the pixel brightness value of each pixel in the target region in at least two frames of the first image.

[0077] In the embodiments of this application, the pixel brightness value of a pixel can be adjusted to achieve pixel adjustment. For this purpose, since the identification information will only appear in the target area, the pixel brightness value of each pixel in the target area in at least two frames of the first image can be obtained first.

[0078] Step S122: Based on the brightness change value corresponding to the identification information, adjust the brightness value of each pixel respectively to obtain the corresponding image data; wherein, the adjustment direction of the pixel brightness value of adjacent frames in the image data is opposite.

[0079] In the embodiments of this application, specifically for display devices equipped with a backlight module, the transmittance of the pixels is adjusted. By controlling the transmittance, the degree to which light from the backlight module passes through the pixels is controlled, thereby adjusting the pixel value. For display devices without a backlight module, the driving voltage of the pixels can be directly adjusted to adjust the pixel brightness value.

[0080] Among them, the brightness change value can correspond to the driving voltage change value or the transmittance change value, and the specific value is determined by the properties of the display device itself.

[0081] Based on the above embodiments disclosed in this application, only the brightness value of the pixels can be adjusted, thereby displaying the mark by making the brightness of the target area and the surrounding area significantly different. This can avoid the problem of the original display information disappearing due to the adjustment of grayscale values ​​to a certain extent, and can improve the accuracy and reliability of image data to a certain extent.

[0082] Figure 9 A schematic diagram of the structure of a display device provided in this application embodiment. Figure 1 The display device 90 includes: a display panel 91 and a control module 92; wherein, The control module 92 is used to obtain target instructions; in response to the target instructions, adjust the pixels of the target area in at least two frames of the first image to obtain corresponding image data; the pixel values ​​of the target area in adjacent frames of the image data meet the change conditions; output and display based on the image data, so that the target area of ​​the second image has identification information; the second image is obtained by the first electronic device from the image acquisition of the output display result of the display panel.

[0083] In some embodiments, such as Figure 10 As shown, Figure 10A schematic diagram of the structure of a display device provided in this application embodiment. Figure 2 The control module 92 includes an image processor 921 and a timing controller 922, wherein: An image processor is used to generate multiple compensation frame images of at least two first images, and to adjust the pixels of the target region in the at least two first images and the multiple compensation frame images respectively to obtain corresponding image data; wherein the image frame rate of the image data is greater than a predetermined frame rate; The timing controller is used to output and display image data according to the current refresh rate or according to the target refresh rate; the target refresh rate is the current refresh rate after the target refresh rate is increased.

[0084] In some embodiments, the control module 92 includes an image processor 921 and a timing controller 922, wherein: An image processor for generating at least two first images, or generating multiple compensated frame images of at least two first images; A timing controller is used to adjust the pixels of the target region in at least two first frames of images to obtain corresponding image data, or to adjust the pixels of the target region in at least two first frames and multiple compensation frames to obtain corresponding image data; output and display the image data according to the target refresh rate; the target refresh rate is greater than the predetermined refresh rate.

[0085] The following describes the application of the display processing method provided in the embodiments of this application in a real-world scenario.

[0086] With the explosive growth of digital content, information security faces multiple challenges. The leakage of critical information can cause enormous commercial losses. Monitors, as crucial display units in both the commercial and consumer markets for content creators, bear the responsibility of displaying and transmitting information. Currently, many software solutions exist that embed different frequency domain (converted domain) algorithms into graphics cards to achieve invisible digital watermarking. Digital watermarking is an information hiding technology that embeds specific information (user information, specific identifiers, etc.) into the displayed content. The embedded information is invisible during normal display, but can be extracted through specific algorithms after taking a photo or screenshot for purposes such as tracing the source of illegally leaked information. This solution proposes a hardware solution for invisible watermarking based on spatial domain algorithms integrated into the monitor. When a user takes a photo of the monitor, a watermark containing multiple information, including identifiers or user information, is displayed in the photo. This allows the photographer to be aware of the risk of being traced before sending the photo, achieving the goal of prevention before the fact. The key difference between this solution and current software-based digital watermarking is that the spatial domain algorithm does not require complex mathematical transformations, has low computational requirements, and is suitable for integration into passive display products such as monitors that lack computing units or have limited computing power. Furthermore, the watermark information is visually visible on the photograph without the need for decoding or extraction after taking the picture (e.g., ...). Figure 11 As shown, the watermark can be observed in the photo when the phone is taking a picture, reducing the strong dependence on the computing power of the camera. However, this solution also has limitations due to the refresh rate. For example, at a low refresh rate (below 120Hz), watermark embedding may cause visible flicker or the watermark may become visible to the naked eye, thus affecting the normal display effect. On the other hand, if the display maintains a high refresh rate for a long time, it will contradict the purpose of reducing power consumption to achieve energy saving. Therefore, to balance information security and ESG objectives, this solution further proposes a method of adjusting the refresh rate and embedding the watermark based on electronic devices with photo-taking posture recognition capabilities, such as AI cameras or AI human sensors.

[0087] Regarding the aforementioned spatial domain-based stealth watermarking hardware solution, the core points of this solution are as follows: Overlay watermarking uses brightness jittering of overlapping areas in the original frame image to form the watermark outline. The method of embedding the watermark image by fluctuating the brightness of the original image is as follows: Let the brightness of this pixel in the original frame be A (display data is...). Figure 12 (Integrated watermark location jitter changes) The brightness of the watermark in the previous frame then changes to... (Data in other areas of the image is not processed), the brightness of the watermark area in the next frame is changed to... (At this point, the actual displayed data is all...) Figure 13 However, due to the persistence of vision, the average brightness of the watermark area remains at A within a unit of time that the human eye can distinguish, thus not affecting normal display, and the human eye cannot see the watermark. When taking a photo, however, the exposure speed is higher than that of the human eye, allowing a single frame to be captured at a higher brightness. or The image is captured to create a watermark effect on the photo. As mentioned in the background, the advantage of the invisible watermark hardware solution based on spatial domain algorithms is that when a user takes a picture of the display, there is no need to decode and extract the watermark information after taking the picture. The watermark, which includes a logo or user information, is presented in the photo. This allows the photographer to be aware of the risk of being traced before sending the photo, thus achieving the purpose of prevention.

[0088] Furthermore, to address the issue that the high refresh rate required for invisible watermarking compromises energy efficiency for displays, this solution proposes using AI cameras or AI human sensors with gesture recognition capabilities to dynamically adjust the refresh rate and watermark embedding action. The AI ​​camera or sensor is pre-trained for gesture recognition. When a user is detected taking a picture of the screen, the camera or sensor transmits the data to the display scaler via USB. The scaler outputs a high refresh rate and retrieves the watermark data from its storage unit, embedding it into the original frame data before displaying it on the panel. Alternatively, the scaler triggers a command, causing the logic board to output a high refresh rate and retrieve the watermark data from the panel's storage unit for embedding. Both methods are acceptable. When the AI ​​camera or AI human sensor does not detect any taking action, the display operates at a low refresh rate (24Hz) or a normal refresh rate (60Hz), eliminating the need for embedding an invisible watermark to meet energy-saving requirements.

[0089] Watermark generation scheme description: like Figure 14 As shown, Figure 14 A flowchart illustrating the watermark generation method provided in this application embodiment includes the following steps: Step S1401: The user turns on the monitor; Step S1402: The PC transmits user information to the scaler; Step S1403: The scaler generates watermark data based on user information; Step S1404: Calculate the watermark length; Step S1405: Process the watermark; Step S1406: Determine if the watermark length is met; If yes, proceed to step S1407; otherwise, re-execute step S1404.

[0090] Step S1407: Calculate the number of watermarks; Step S1408: Embed the watermark position; Step S1409: Integrate with the original frame data; Step S1410: Output data with invisible watermark.

[0091] Specifically, when the user turns on the monitor, the PC transmits user information to the scaler via USB protocol. The scaler generates a watermark data table based on the user information, calculates the watermark length, and processes the watermark. If the processed watermark meets the watermark length limit, the next step is to calculate the number of watermarks to be embedded. If it does not meet the watermark length limit, the process is repeated until the watermark length limit is met. Subsequently, as described in the above-mentioned disclosure points, the watermark position is embedded by varying the brightness of the original image, and integrated with the original frame data. That is, the brightness at the watermark location in the previous frame is changed to... (Data in other areas of the image is not processed), the brightness of the watermark area in the next frame is changed to... Then, the data with an invisible watermark is output to the panel for display.

[0092] Description of a dynamic refresh rate adjustment solution based on AI camera / human sensor recognition: like Figure 15 As shown, Figure 15 A flowchart illustrating a dynamic refresh rate adjustment scheme based on AI Camera / Human sensor recognition, provided in this application embodiment, includes the following steps: Step S1501: The user turns on the monitor; Step S1502: The AI ​​Camera / AI Human Sensor determines whether there is a photo-taking activity in front of the screen; Step S1503: The display shows data at the normal refresh rate and the original frame. Step S1504: Notify the scaler, which will execute the watermark generation algorithm and increase the refresh rate. In step S1505, the display performs a high refresh rate and displays data with watermark integration.

[0093] Specifically, the aforementioned invisible watermarking scheme, when the display operates at a low refresh rate, may result in visible flickering or the watermark becoming visible to the naked eye due to the long frame time. However, prolonged operation at a high refresh rate leads to high power consumption, which is not energy-efficient. Therefore, a dynamic refresh rate adjustment scheme based on AI Camera / Human sensor recognition is proposed. The display is equipped with an AI camera or AI human sensor that has been pre-trained for recognizing photographic poses. After the user turns on the display, the AI ​​Camera / Human Sensor determines whether there is a photographing action in front of the screen. If no photographing action is detected, the display displays data at the normal refresh rate and with the original frame, without needing to generate a watermark at the scaler. When photographing action is detected, the scaler is notified via USB protocol to execute the watermark generation and integration algorithm and simultaneously increase the refresh rate to 120Hz or higher. The display then operates at a high refresh rate and displays data with the integrated watermark.

[0094] It should be noted that, in the embodiments of this application, if the above-described display processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0095] This application provides another electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0096] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium can be transient or non-transient.

[0097] This application provides a computer program including computer-readable code. When the computer-readable code runs in an electronic device, the processor in the electronic device executes some or all of the steps in the above-described method.

[0098] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0099] Figure 16 A logic block diagram of a display processing device provided in this application embodiment, the display processing device 1600 including: Acquisition module 1601 is used to obtain the target instruction; The adjustment module 1602 is used to adjust the pixels of the target region in at least two frames of the first image in response to the target instruction, so as to obtain the corresponding image data; the pixel values ​​of the target region in the adjacent frames of the image data meet the change conditions. The first display module 1603 is used to output and display based on image data, so that the target area of ​​the second image contains identification information; the second image is obtained by the first electronic device from the image acquisition of the output display result of the display device.

[0100] In some embodiments, the adjustment module 1602 includes: The first determining submodule is used to determine the location information of the target region in at least two frames of the first image; The first adjustment submodule is used to adjust the pixel values ​​of the target area based on the location information and pixel change values ​​of the target area to obtain the corresponding image data; wherein the pixel values ​​of adjacent frames in the image data are adjusted in opposite directions.

[0101] In some embodiments, the first determining submodule includes: The acquisition unit is used to obtain the identification display data; The generation unit is used to generate identifier mask information based on the identifier display data; The determining unit is used to determine the location information of the target region in the first image based on the identifier mask information; The acquisition unit includes one of the following: The first acquisition subunit is used to obtain identity information, input the identity information into the first model, and generate corresponding identification display data; The second acquisition subunit is used to obtain identity information and read the corresponding identifier display data from the preset library; The identity information comes from a first electronic device, a second electronic device, or a display device; the display device is used to display the screen of the second electronic device.

[0102] In some embodiments, the device 1600 further includes at least one of the following: The generation module is used to generate multiple compensation frame images of at least two first images, and to adjust the pixels of the target region in the at least two first images and multiple compensation frame images to obtain the corresponding image data. The transmitting module is used to send an adjustment signal to the second electronic device, causing the second electronic device to increase the frame rate of the output first image; the display device is used to display the image output by the second electronic device; The second display module is used to output and display image data according to the target refresh rate; the target refresh rate is greater than the predetermined refresh rate.

[0103] In some embodiments, the generation module includes: The first acquisition submodule is used to acquire frame rate and refresh rate; The generation submodule is used to generate at least two frames of the first image or send an adjustment signal to the second electronic device if the refresh rate is greater than the frame rate.

[0104] In some embodiments, the acquisition module 1601 includes: The second acquisition submodule is used to obtain environmental images; The trigger submodule 1602 is used to input the environmental image into the second model. If the environmental image represents an image acquisition intention or an image acquisition behavior for the display device, the target instruction is triggered.

[0105] In some embodiments, the generating unit includes: The first determining subunit is used to determine the span of the identifier pixels on the display area of ​​the display panel; The second determining subunit is used to determine the maximum number of displayable identifier data based on the identifier display data and the resolution data of the display area if the identifier pixel span meets the preset identifier length information. The generation sub-unit is used to generate identifier mask information based on the maximum number of displays and the identifier display data.

[0106] In some embodiments, the adjustment module 1602 includes: The third acquisition submodule is used to acquire the pixel brightness values ​​of each pixel in the target region in at least two frames of the first image; The second adjustment submodule is used to adjust the brightness value of each pixel based on the brightness change value corresponding to the identification information to obtain the corresponding image data; wherein, the adjustment direction of the pixel brightness value of adjacent frames in the image data is opposite.

[0107] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0108] This application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the display processing method as described in any of the above embodiments.

[0109] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0110] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0111] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0112] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0113] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0115] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0116] Furthermore, in the various embodiments of this application, all functional units can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0117] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0118] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A display processing method, applied to a display device, comprising: Obtain the target instruction; In response to the target instruction, the pixels of the target region in at least two frames of the first image are adjusted to obtain the corresponding image data; The pixel values ​​of the target region in adjacent frames of the image data satisfy the change condition; The image data is output and displayed so that the target area of ​​the second image contains identification information; the second image is obtained by the first electronic device from the image acquisition of the output display result of the display device.

2. The method according to claim 1, wherein adjusting the pixels of the target region in at least two frames of the first image to obtain corresponding image data includes: Determine the location information of the target region in the at least two frames of the first image; Based on the location information and pixel change values ​​of the target region, the pixel values ​​of the target region are adjusted to obtain corresponding image data; wherein, the pixel values ​​of adjacent frames in the image data are adjusted in opposite directions.

3. The method according to claim 2, wherein determining the location information of the target region in the at least two frames of the first image includes: Obtain the identification display data; Generate identifier mask information based on the identifier display data; Based on the identifier mask information, the location information of the target region in the first image is determined; The obtained identification display data includes one of the following: Obtain identity information, input the identity information into the first model, and generate corresponding identifier display data; Obtain identity information and read the corresponding identifier display data from the pre-set library; The identity information comes from the first electronic device, the second electronic device, or the display device; the display device is used to display the screen of the second electronic device.

4. The method according to claim 1, further comprising at least one of the following: Generate multiple compensation frame images of the at least two first images, and adjust the pixels of the target region in the at least two first images and the multiple compensation frame images respectively to obtain the corresponding image data; An adjustment signal is sent to the second electronic device, causing the second electronic device to increase the frame rate at which the first image is output; the display device is used to display the image output by the second electronic device; The image data is output and displayed according to a target refresh rate, which is greater than a predetermined refresh rate.

5. The method according to claim 4, further comprising: Get the frame rate and refresh rate; If the refresh rate is greater than the frame rate, generate multiple compensation frame images of the at least two first images or send an adjustment signal to the second electronic device.

6. The method according to any one of claims 1 to 5, wherein obtaining the target instruction comprises: Obtain environmental images; The environmental image is input into the second model. If the environmental image indicates an intention to capture images for the display device or an image capture behavior for the display device, a target instruction is triggered.

7. The method according to claim 3, wherein generating the identifier mask information based on the identifier display data includes: Determine the pixel span of the identifier display data on the display area of ​​the display panel; If the pixel span of the identifier meets the preset identifier length information, the maximum number of identifier display data to be displayed is determined based on the identifier display data and the resolution data of the display area; The identifier mask information is generated based on the maximum number of displays and the identifier display data.

8. The method according to any one of claims 1 to 5, wherein adjusting the pixels of the target region in at least two frames of the first image to obtain corresponding image data comprises: Obtain the pixel brightness values ​​of each pixel in the target region from at least two frames of the first image; Based on the brightness change value corresponding to the identification information, the brightness value of each pixel is adjusted to obtain the corresponding image data; wherein, the adjustment direction of the pixel brightness value of adjacent frames in the image data is opposite.

9. A display device, the display device comprising: Display panel and control module; among which... The control module is used to obtain a target instruction; in response to the target instruction, adjust the pixels of the target region in at least two frames of the first image to obtain corresponding image data; the pixel values ​​of the target region in adjacent frames of the image data satisfy the change condition; output and display based on the image data, so that the target region of the second image has identification information; the second image is obtained by the first electronic device from the image acquisition of the output display result of the display panel.

10. The device according to claim 9, wherein the control module comprises an image processor and a timing controller, wherein: The image processor is configured to generate multiple compensation frame images of the at least two first images, and adjust the pixels of the target region in the at least two first images and the multiple compensation frame images respectively to obtain corresponding image data; wherein, the image frame rate of the image data is greater than a predetermined frame rate; The timing controller is used to output and display the image data according to the current refresh rate, or according to a target refresh rate; the target refresh rate is the current refresh rate after the target refresh rate has been increased.

11. The device according to claim 9, wherein the control module comprises an image processor and a timing controller, wherein: The image processor is used to generate the at least two first images, or to generate multiple compensation frame images of the at least two first images; The timing controller is used to adjust the pixels of the target region in the at least two first images to obtain corresponding image data, or to adjust the pixels of the target region in the at least two first images and the plurality of compensation frame images respectively to obtain corresponding image data. The image data is output and displayed according to a target refresh rate, which is greater than a predetermined refresh rate.

Images