A method, device and equipment for determining an area of interest on a video display interface

Abstract

The application provides a method, device and equipment for determining an interest region on a video display interface. The method on the server side comprises the following steps: acquiring code block division information of at least one video coding block of a video picture displayed on the video display interface; transmitting the code block division information to an eyeball tracking device; acquiring a plurality of macular reflex intensities fed back by the eyeball tracking device according to the code block division information; and determining an interest region of a user on the video picture according to the plurality of macular reflex intensities. The above scheme of the application can realize accurate determination of the interest region on the video display interface.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video processing, in particular to a method, device and equipment for determining an interest region on a video display interface. BACKGROUND

[0002] In machine vision and image processing, an image to be processed is outlined in a box, a circle, an ellipse, an irregular polygon, etc. to form a region of interest (ROI). Various operators and functions are commonly used on traditional machine vision software to obtain the ROI and perform the next step of image processing. The ROI can intelligently encode the video according to customer requirements, optimize the video encoding performance without loss of image quality, and ultimately reduce the network bandwidth occupancy and storage space.

[0003] A general video encoder encodes (compresses) and transmits the entire frame, which puts pressure on network bandwidth and video storage. The ROI is generally determined according to the position of the mouse pointer or the position of the recognized face. The ROI region is compressed using a higher code rate and quantization level, while the region outside the ROI is compressed using a lower code rate and quantization quality, which reduces the transmission bandwidth without affecting the user experience too much.

[0004] At present, the ROI is mainly used for monitoring or video conference scenarios, and the method of determining the ROI according to the position of the mouse pointer or the position of the recognized face is generally adopted. However, this method cannot be used when there is no mouse pointer or face. Some methods use neural networks or AI to predict the position of the ROI, but these methods have high complexity and large processing delay, and are not suitable for wireless networks and mobile devices. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a method, device and equipment for determining an interest region on a video display interface. The present application realizes accurate determination of the interest region on the video display interface.

[0006] To solve the above technical problems, the technical solutions of the present application are as follows:

[0007] A method for determining an interest region on a video display interface is applied to a server, and the method comprises the following steps:

[0008] Obtaining code block division information of at least one video coding block of a video picture displayed on the video display interface;

[0009] Transmitting the code block division information to an eye tracking device;

[0010] acquiring a plurality of macular light reflection intensities fed back by the eye tracking device according to the code block division information;

[0011] determining an interest region of the user on the video picture according to the plurality of macular light reflection intensities.

[0012] Optionally, the macular light reflection intensity is obtained by setting a light source at a position of a video coding block, and the light emitted by the light source is obtained after macular light reflection of the user's eyeball.

[0013] Optionally, the acquiring the plurality of macular light reflection intensities fed back by the eye tracking device according to the code block division information comprises:

[0014] acquiring a plurality of macular light reflection intensities fed back by the eye tracking device according to light emitted by a light source at a position of each of the at least one video coding block after macular light reflection of the user's eyeball, wherein one video coding block is provided with one first light source, and one video coding block corresponds to one macular light reflection intensity.

[0015] Optionally, the acquiring the plurality of macular light reflection intensities fed back by the eye tracking device according to the code block division information comprises:

[0016] acquiring a plurality of macular light reflection intensities fed back by the eye tracking device according to light emitted by a light source at a position of each of the at least one video coding block after macular light reflection of the user's eyeball, wherein one video coding block is provided with one first light source, and one video coding block corresponds to one macular light reflection intensity.

[0017] Optionally, the determining the interest region of the user on the video picture according to the plurality of macular light reflection intensities comprises:

[0018] determining a video coding block with the strongest light reflection intensity in the plurality of macular light reflection intensities as the interest region of the user on the video picture.

[0019] Optionally, the method for determining the interest region on the video display interface further comprises:

[0020] encoding and processing a video picture of the interest region according to a first encoding code rate to obtain a first encoding result;

[0021] transmitting the first encoding result to the video display interface for display.

[0022] Optionally, the method for determining the interest region on the video display interface further comprises:

[0023] dividing the rest of the video coding blocks on the video picture except the interest region into at least one quality level coding block region according to the distance from the user's eyeball to the rest of the video coding blocks;

[0024] configuring a corresponding second coding rate for the at least one quality level coding block region respectively; the second coding rate is less than the first coding rate;

[0025] encoding the video data according to the second coding rate corresponding to each quality level coding block region to obtain a second encoding result;

[0026] transmitting the second encoding result to the video display interface for display.

[0027] Optionally, the method for determining the interest region on the video display interface further comprises:

[0028] when the interest region information returned by the eyeball tracking device cannot be received, determining the interest region according to the motion vector in the video coding block corresponding to the last interest region.

[0029] Optionally, the method for determining the interest region on the video display interface further comprises:

[0030] when the motion vector in the video coding block cannot be obtained, comparing the current video picture and the video picture reporting the last interest region to select the video coding block with the largest change degree as the interest region of the current video picture.

[0031] Embodiments of the present application also provide a method for determining the interest region on the video display interface, applied to an eyeball tracking device, the method comprising:

[0032] receiving the code block division information of at least one video coding block of the video picture displayed on the video display interface sent by a server;

[0033] obtaining a plurality of macular light reflection intensities according to the code block division information;

[0034] reporting the plurality of macular light reflection intensities to the server, so that the server determines the interest region of the user on the video picture according to the plurality of macular light reflection intensities.

[0035] Optionally, obtaining a plurality of macular light reflection intensities according to the code block division information comprises:

[0036] setting a light source at the position of the video coding block according to the code block division information, and obtaining a plurality of macular light reflection intensities after the light emitted by the light source is reflected by the macula of the user's eyeball.

[0037] Optionally, according to the code block division information, a light source is arranged at a position of the video coding block, and a plurality of macular reflex intensities are obtained after light emitted by the light source is reflected by a macula of an eyeball of a user.

[0038] According to the code block division information, a light source is arranged at a position of each of the at least one video coding block, and a plurality of macular reflex intensities are obtained after light emitted by the light source is reflected by a macula of an eyeball of a user.

[0039] Optionally, according to the code block division information, a light source is arranged at a position of the video coding block, and a plurality of macular reflex intensities are obtained after light emitted by the light source is reflected by a macula of an eyeball of a user.

[0040] According to the code block division information, a reference video coding block is selected from the at least one video coding block, a light source is arranged at a position of the reference video coding block, and a reference macular reflex intensity is obtained after light emitted by the light source is reflected by a macula of an eyeball of a user; a preset number of video coding blocks around the reference video coding block are tracked to obtain a preset number of tracking macular reflex intensities; and the plurality of macular reflex intensities include the reference macular reflex intensity and the preset number of tracking macular reflex intensities.

[0041] Embodiments of the present application also provide a device for determining an area of interest on a video display interface, applied to a server, the device comprising:

[0042] a obtaining module, configured to obtain code block division information of at least one video coding block of a video picture displayed on a video display interface;

[0043] a transmitting module, configured to transmit the code block division information to an eyeball tracking device;

[0044] a processing module, configured to obtain a plurality of macular reflex intensities fed back by the eyeball tracking device according to the code block division information, and determine an area of interest of a user on the video picture according to the plurality of macular reflex intensities.

[0045] Embodiments of the present application also provide a device for determining an area of interest on a video display interface, applied to an eyeball tracking device, the device comprising:

[0046] a transceiving module, configured to receive code block division information of at least one video coding block of a video picture displayed on a video display interface sent by a server;

[0047] The processing module is configured to obtain a plurality of macular light reflection intensities according to the code block division information, and report the plurality of macular light reflection intensities to a server, so that the server determines an interest region of a user on the video picture according to the plurality of macular light reflection intensities.

[0048] Embodiments of the present application also provide a computing device, comprising a processor and a memory storing a computer program, the computer program being executed by the processor to perform the method described above.

[0049] Embodiments of the present application also provide a computer readable storage medium storing instructions, the instructions being executed on a computer to make the computer perform the method described above.

[0050] The above scheme of the present application has at least the following beneficial effects:

[0051] The above scheme of the present application has at least the following beneficial effects: BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is a method flow chart for determining an interest region on a video display interface on the server side of an embodiment of the present application;

[0053] Figure 2 is a schematic diagram of three quality video code blocks on a video display interface of an embodiment of the present application;

[0054] Figure 3 is a method flow chart for determining an interest region on a video display interface on the server side of an embodiment of the present application;

[0055] Figure 4 is a module schematic diagram of a device for determining an interest region on a video display interface on the server side of an embodiment of the present application;

[0056] Figure 5 is a module schematic diagram of a device for determining an interest region on a video display interface on the server side of an embodiment of the present application; DETAILED DESCRIPTION

[0057] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0058] As shown in Figure 1 An embodiment of the present disclosure provides a method for determining a region of interest on a video display interface, applied to a server, the method comprising:

[0059] Step 11, obtaining code block partition information of at least one video coding block of a video picture displayed on the video display interface;

[0060] Step 12, transmitting the code block partition information to an eyeball tracking device;

[0061] Step 13, obtaining a plurality of macular light reflection intensities fed back by the eyeball tracking device according to the code block partition information;

[0062] Step 14, determining a region of interest of a user on the video picture according to the plurality of macular light reflection intensities.

[0063] This embodiment of the present disclosure, by obtaining code block partition information of at least one video coding block of a video picture displayed on the video display interface; transmitting the code block partition information to an eyeball tracking device; obtaining a plurality of macular light reflection intensities determined by the eyeball tracking device according to the code block partition information; and determining a region of interest of a user on the video picture according to the plurality of macular light reflection intensities. The present disclosure realizes accurate determination of the region of interest on the video display interface, and can further provide high-quality coding rate for the region of interest, thereby ensuring video display effect and improving user experience.

[0064] Here, the server sends video data to the display device, and the video encoder on the server side divides the video data according to a preset division rule to obtain a plurality of video coding blocks, each of which has the same size, such as 8*8 or 16*16.

[0065] For example, when a user watches a video, the user can wear an eyeball tracking device, and the server transmits the division result of the video coding block, i.e., the code block partition information, to the eyeball tracking device (such as VR glasses, a camera, an infrared probe, etc.).

[0066] Further, the macular reflection detection frequency (less than or equal to the frame rate) is configured to an eye tracking device (a device worn by a user when watching a video, used to track the area covered by the line of sight of the eyeball), and the eye tracking device determines the mapping relationship between the display device and the video coding block according to the code block division information. Here, the display device can be the display screen of the eye tracking device, or the display screen of another video display device.

[0067] In an optional embodiment of the present application, in step 13, the macular reflection intensity is obtained by setting a light source at the position of the video coding block, and the light emitted by the light source is obtained after the macular reflection of the eyeball of the user.

[0068] For example, each video coding block corresponds to a display area of the display screen of the eye tracking device. There are various implementation manners for setting the light source at the position of the video coding block, for example, specifically, the following manners can be used: 1) a light source of the embodiment of the present application is added to the backlight source of the display screen, and each light source is set corresponding to one display area, for example, corresponding to the center of the display area. 2) Some pixels with self-luminous function are set on the display screen, for example, one or more self-luminous pixel units are set at the center of the pixel area corresponding to each display area. Optionally, the light emitted by the light source is invisible light, such as infrared light.

[0069] In the embodiment of the present application, a receiving device can also be set to measure the reflected light of the light emitted by the light source, that is, to measure the reflected light with the same wavelength as the light source. The light source and the receiving device can be integrally set.

[0070] In this embodiment, the macula of the eyeball of the user is a yellow small area about 3.5 mm on the temporal side of the optic disc of the eyeball, and the central depression is the fovea. The fovea is the most sensitive area of the retina in vision (color discrimination and resolution).

[0071] In a specific implementation scheme, step 13 can include:

[0072] In step 131, a plurality of macular reflection intensities are obtained, which are fed back by the eye tracking device according to the light emitted by the light source at the position of each video coding block in the at least one video coding block after the macular reflection of the eyeball of the user. Wherein, a first light source is set at the position of a video coding block, and one video coding block corresponds to one macular reflection intensity.

[0073] In this embodiment, a light source is arranged at the position of each video coding block of the display interface of the display device, and the light emitted by the light source is reflected by the eyeball to obtain the macular (fovea) reflection intensity. In this way, the light source at the position of each video coding block can obtain multiple macular reflection intensities after being reflected by the eyeball, and the video coding block with the strongest macular reflection intensity is the line of sight, which is referred to as the focal video coding block, i.e., the region of interest.

[0074] In another specific implementation, step 13 can include:

[0075] In step 132, a reference video coding block is selected from the at least one video coding block by the eyeball tracking device, a light source is arranged at the position of the reference video coding block, the light emitted by the light source is reflected by the macula of the eyeball to obtain a reference macular reflection intensity, and the macular reflections of a preset number of video coding blocks around the reference video coding block are tracked to obtain a preset number of tracking macular reflection intensities. The multiple macular reflection intensities include the reference macular reflection intensity and the preset number of tracking macular reflection intensities.

[0076] In this embodiment, the reference video coding block is displayed on the display device, the user is required to directly view the reference video coding block, and the light source is arranged to emit light (such as infrared light) to the eyeball at the position of the reference video coding block to obtain the reference macular reflection. Subsequently, the light source is arranged to traverse the macular reflections of multiple video coding blocks around the reference point to obtain a preset number of tracking macular reflection intensities.

[0077] In an optional embodiment of the present application, step 14 can include:

[0078] The video coding block with the strongest reflection intensity in the multiple macular reflection intensities is determined as the region of interest of the user on the video picture.

[0079] In this embodiment, the video coding block with the strongest reflection intensity in the multiple macular reflection intensities, i.e., the region directly viewed by the line of sight of the user, is determined as the region of interest of the user on the video picture. The video coding block corresponding to this region is referred to as the highest-quality coding block.

[0080] Further, the field of view covered by the video coding blocks around the video coding block corresponding to the region of interest can be obtained according to the distance from the screen to the eyeball in combination with the FOV (usually 30 degrees) recognizable by the human eye, and the video coding blocks are referred to as high-quality coding blocks. The remaining coding blocks are all ordinary-quality coding blocks.

[0081] For example, the video coding block corresponding to the region of interest is the highest-quality coding block, and the video coding blocks around the highest-quality coding block are high-quality coding blocks. Figure 2As shown, the video coding block 1 is a focal point video coding block, i.e. an area of interest, which is a highest quality coding block, a plurality of video coding blocks 2 surrounding the video coding block 1 in the figure are high quality coding blocks, and the remaining video coding blocks 3 in the figure except the video coding blocks 1 and 2 are normal quality coding blocks.

[0082] In an optional embodiment of the present application, the method for determining the area of interest on the video display interface can further include, on the basis of the steps 11 to 14 described above:

[0083] Step 15, encoding the video picture of the area of interest according to a first coding rate to obtain a first coding result;

[0084] Step 16, transmitting the first coding result to the video display interface for display.

[0085] In this embodiment, the video picture of the area of interest is encoded according to the first coding rate to ensure smooth playing of the video picture of the area of interest of the user.

[0086] Further, the method for determining the area of interest on the video display interface can further include:

[0087] Step 17, dividing the remaining video coding blocks on the video picture except the area of interest into at least one quality level block region according to the distance from the eyeball of the user to the remaining video coding blocks; here, optionally, the distance from the eyeball of the user to the remaining video coding blocks can be the distance from the eyeball of the user to the center point of the remaining video coding blocks;

[0088] Step 18, configuring a corresponding second coding rate for the at least one quality level block region respectively; the second coding rate is less than the first coding rate;

[0089] Step 19, encoding the video data according to the second coding rate corresponding to each quality level block region to obtain a second coding result;

[0090] Step 20, transmitting the second coding result to the video display interface for display.

[0091] In the specific implementation of this embodiment, for example, Figure 2 As shown, the first coding rate corresponding to the video coding block 1 is greater than the coding rate corresponding to the plurality of video coding blocks 2; the coding rate corresponding to the video coding block 2 is greater than the coding rate corresponding to the video coding block 3. In this way, the video coding block of the area of interest in the direct view of the eyeball can use high quality compression to improve the experience of the user; at the same time, the remaining coding blocks can use lower quality compression means to reduce the network bandwidth and the coding and decoding delay without much degradation of the user experience.

[0092] In an optional embodiment of the present application, the method for determining the region of interest on the video display interface further comprises:

[0093] In step 21, when the region of interest information returned by the eye tracking device cannot be received, the region of interest is determined according to the motion vector in the video coding block corresponding to the last region of interest.

[0094] The method for determining the region of interest in this embodiment ensures the smoothness of video playing.

[0095] In an optional embodiment of the present application, the method for determining the region of interest on the video display interface further comprises:

[0096] When the motion vector in the video coding block cannot be obtained, the video coding block with the largest change degree is selected as the region of interest of the current video picture by comparing the current video picture with the video picture in which the last reported region of interest is located. Thus, the smoothness of video playing is ensured.

[0097] In this embodiment, the eye tracking device encodes and sends the ROI information (hereinafter collectively referred to as dist) to the video encoder of the server, wherein the ROI information includes the highest quality coding block, the high quality coding block, the normal quality coding block and the distance from the screen to the eyeball. Here, the coding block covered by the field of view can be obtained according to the distance from the screen to the eyeball dist combined with the normal sensitivity FOV (usually 10 degrees) of the human eye, which is referred to as the highest quality coding block; the coding block covered by the field of view can be obtained according to the distance from the screen to the eyeball dist combined with the recognizable FOV (usually 30 degrees) of the human eye, which is referred to as the high quality coding block; and the rest of the coding blocks are normal quality coding blocks.

[0098] After the video encoder receives the ROI information, the video encoder encodes according to the three quality coding levels mentioned above and outputs the code stream.

[0099] When the video encoder cannot receive the ROI information returned by the eye tracking device, the focus coding block in the last reported ROI information is extracted, the motion vector in the focus coding block is used to determine the current focus coding block, and the highest quality coding block and the normal quality coding block are determined by using the last reported dist and FOV.

[0100] If the motion vector cannot be extracted from the coding block (for example, when the image difference is extremely large), the coding block with the largest change degree is selected as the focus coding block by comparing the current image with the image in which the last reported ROI information is located.

[0101] The above embodiment of the present application acquires the ROI of the user by using the line-of-sight tracking, transmits the ROI and the non-ROI at different encoding code rates, simultaneously acquires the change of the ROI data by the eye tracking module, uses high-quality compression for the encoding blocks in the range near the direct view of the eye, and uses low-quality compression for the remaining encoding blocks, so that the quality of the viewed picture is ensured, the user experience is ensured, the bandwidth demand in the transmission process is greatly reduced, and the requirement of the hardware of the VR device is reduced.

[0102] As shown in Figure 3 The embodiment of the present application also provides a method for determining the ROI on the video display interface, which is applied to the eye tracking device, and the method comprises the following steps:

[0103] Step 31: receiving the code block division information of at least one video encoding block of the video picture displayed on the video display interface sent by the server;

[0104] Step 32: obtaining a plurality of macular light reflection intensities according to the code block division information;

[0105] Step 33: reporting the plurality of macular light reflection intensities to the server, so that the server determines the ROI of the user on the video picture according to the plurality of macular light reflection intensities.

[0106] Optionally, the step of obtaining a plurality of macular light reflection intensities according to the code block division information comprises the following steps:

[0107] According to the code block division information, a light source is arranged at the position of the video encoding block, and a plurality of macular light reflection intensities are obtained after the light emitted by the light source is reflected by the macula of the eye of the user.

[0108] Optionally, the step of obtaining a plurality of macular light reflection intensities according to the code block division information, comprises the following steps:

[0109] According to the code block division information, a light source is arranged at the position of each video encoding block in the at least one video encoding block, and a plurality of macular light reflection intensities are obtained after the light emitted by the light source is reflected by the macula of the eye of the user; wherein one first light source is arranged at the position of one video encoding block, and one video encoding block corresponds to one macular light reflection intensity.

[0110] Optionally, the step of obtaining a plurality of macular light reflection intensities according to the code block division information, comprises the following steps:

[0111] According to the code block division information, a reference video coding block is selected from the at least one video coding block, a light source is arranged at a position of the reference video coding block, and light emitted by the light source is reflected by a macula of an eyeball of a user to obtain a reference macular light intensity; and a preset number of video coding blocks around the reference video coding block track macular light reflected by the macula of the eyeball of the user to obtain a preset number of tracking macular light intensities; the plurality of macular light intensities include the reference macular light intensity and the preset number of tracking macular light intensities.

[0112] It should be noted that the embodiment is a method on the eyeball tracking device side corresponding to the method on the server side, and all implementation manners of the method on the server side are applicable to the embodiment of the method on the eyeball tracking device side, and the same technical effects can be achieved.

[0113] As shown in Figure 4 The embodiment of the application further provides a device 40 for determining an interest region on a video display interface, applied to a server, and the device 40 comprises:

[0114] a first obtaining module 41 configured to obtain code block division information of at least one video coding block of a video picture displayed on the video display interface;

[0115] a transmission module 42 configured to transmit the code block division information to an eyeball tracking device;

[0116] a first processing module 43 configured to obtain a plurality of macular light intensities fed back by the eyeball tracking device according to the code block division information, and determine an interest region of a user on the video picture according to the plurality of macular light intensities.

[0117] Optionally, the macular light intensity is obtained by arranging a light source at a position of a video coding block, and light emitted by the light source is reflected by a macula of an eyeball of a user to obtain the macular light intensity.

[0118] Optionally, the obtaining of the plurality of macular light intensities fed back by the eyeball tracking device according to the code block division information comprises:

[0119] obtaining a plurality of macular light intensities fed back by the eyeball tracking device according to light emitted by a light source at a position of each video coding block in the at least one video coding block and reflected by a macula of an eyeball of a user; wherein one video coding block is arranged with one first light source, and one video coding block corresponds to one macular light intensity.

[0120] Optionally, the obtaining of the plurality of macular light intensities fed back by the eyeball tracking device according to the code block division information comprises:

[0121] The eye tracking device selects a reference video coding block in the at least one video coding block, a light source is arranged at a position of the reference video coding block, and the light source emits light which, after being reflected by the macula of the user's eyeball, obtains a reference macular reflection intensity; and a preset number of video coding blocks around the reference video coding block track the macular reflection of the user's eyeball to obtain a preset number of tracking macular reflection intensities; the plurality of macular reflection intensities include the reference macular reflection intensity and the preset number of tracking macular reflection intensities.

[0122] Optionally, the interest region of the user on the video picture is determined according to the plurality of macular reflection intensities, including:

[0123] The video coding block with the strongest macular reflection intensity in the plurality of macular reflection intensities is determined as the interest region of the user on the video picture.

[0124] Optionally, the first processing module 43 is further configured to: encode the video picture of the interest region according to a first encoding code rate to obtain a first encoding result;

[0125] The first encoding result is transmitted to the video display interface for display.

[0126] Optionally, the first processing module 43 is further configured to: divide the remaining video coding blocks on the video picture except the interest region into at least one quality level code block region according to distances from the user's eyeball to the remaining video coding blocks;

[0127] The at least one quality level code block region is respectively configured with a corresponding second encoding code rate; the second encoding code rate is less than the first encoding code rate;

[0128] Video data is encoded according to the second encoding code rate corresponding to each quality level code block region to obtain a second encoding result;

[0129] The second encoding result is transmitted to the video display interface for display.

[0130] Optionally, the first processing module 43 is further configured to: when the interest region information returned by the eye tracking device cannot be received, determine the interest region according to a motion vector in a video coding block corresponding to the last interest region.

[0131] Optionally, the first processing module 43 is further configured to: when the motion vector in the video coding block cannot be obtained, compare a current video picture and a video picture in which the last reported interest region is located, and select a video coding block with the largest change degree as the interest region of the current video picture.

[0132] It should be noted that all the implementation manners in the method embodiments are applicable to the device embodiments, and the same technical effects can be achieved.

[0133] As shown in Figure 5 The embodiment of the present application also provides a device 50 for determining an interest area on a video display interface, which is applied to an eyeball tracking device, and the device 50 comprises:

[0134] a transceiving module 51, configured to receive code block division information of at least one video coding block of a video picture displayed on the video display interface sent by a server;

[0135] a second processing module 52, configured to obtain a plurality of macular light reflection intensities according to the code block division information, and report the plurality of macular light reflection intensities to the server, so that the server determines an interest area of a user on the video picture according to the plurality of macular light reflection intensities.

[0136] Optionally, the obtaining of the plurality of macular light reflection intensities according to the code block division information comprises:

[0137] setting a light source at a position of the video coding block according to the code block division information, and obtaining the plurality of macular light reflection intensities after light emitted by the light source is reflected by a macula of an eyeball of the user.

[0138] Optionally, the obtaining of the plurality of macular light reflection intensities according to the code block division information, comprises:

[0139] setting a light source at a position of each of the at least one video coding block according to the code block division information, and obtaining the plurality of macular light reflection intensities after light emitted by the light source is reflected by a macula of an eyeball of the user; wherein one first light source is set at a position of one video coding block, and one video coding block corresponds to one macular light reflection intensity.

[0140] Optionally, the obtaining of the plurality of macular light reflection intensities according to the code block division information, comprises:

[0141] selecting a reference video coding block from the at least one video coding block according to the code block division information, setting a light source at a position of the reference video coding block, obtaining a reference macular light reflection intensity after light emitted by the light source is reflected by a macula of an eyeball of the user, tracking macular light reflection of a preset number of video coding blocks around the reference video coding block to obtain a preset number of tracking macular light reflection intensities, and the plurality of macular light reflection intensities comprise the reference macular light reflection intensity and the preset number of tracking macular light reflection intensities.

[0142] It should be noted that all the implementation manners in the method embodiments described above are applicable to the device embodiments, and the same technical effects can be achieved.

[0143] The embodiments of the present application also provide a computing device, comprising a processor and a memory storing a computer program, when the computer program is run by the processor, the method described above is executed. All the implementation manners in the method embodiments described above are applicable to this embodiment, and the same technical effects can be achieved.

[0144] The embodiments of the present application also provide a computer readable storage medium, comprising instructions, when the instructions are run on a computer, the computer executes the method described above. All the implementation manners in the method embodiments described above are applicable to this embodiment, and the same technical effects can be achieved.

[0145] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software mode depends on the specific application and design constraints of the technical solutions. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0146] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0147] In the embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented by other ways. For example, the device embodiments described above are only schematic, and the division of the units is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0148] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0149] In addition, each functional unit in various embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0150] If the functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a number of instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, and various program code storage media.

[0151] In addition, it should be noted that in the device and method of the present application, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombination should be considered as equivalent solutions of the present application. And the steps of executing the above series of processes can naturally be executed in time sequence according to the order of description, but it is not necessary to be executed in time sequence. Some steps can be executed in parallel or independently of each other. It can be understood by those skilled in the art that all or any steps or components of the method and device of the present application can be realized in hardware, firmware, software or their combination in any computing device (including processor, storage medium, etc.) or network of computing devices, which can be realized by those skilled in the art with their basic programming skills after reading the description of the present application.

[0152] Therefore, the purpose of the present application can also be realized by running a program or a group of programs on any computing device. The computing device can be a commonly known general-purpose device. Therefore, the purpose of the present application can also be realized only by providing a program product containing program code for realizing the method or device. That is, such a program product also constitutes the present application, and the storage medium storing such a program product also constitutes the present application. Obviously, the storage medium can be any known storage medium or any storage medium developed in the future. It should be noted that in the device and method of the present application, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombination should be considered as equivalent solutions of the present application. And the steps of executing the above series of processes can naturally be executed in time sequence according to the order of description, but it is not necessary to be executed in time sequence. Some steps can be executed in parallel or independently of each other.

[0153] The above describes the preferred embodiments of the present application. It should be noted that, for those skilled in the art, several improvements and refinements can be made without departing from the principles of the present application, and these improvements and refinements should also be considered as falling within the scope of the present application.

Claims

1. A method for determining a region of interest on a video display interface, characterized in that, Applied to a server, the method comprises: Obtaining code block division information of at least one video coding block of a video picture displayed on a video display interface; Transmitting the code block division information to an eyeball tracking device; Obtaining multiple macular reflex intensities fed back by the eyeball tracking device according to the code block division information; Determining an interest region of a user on the video picture according to the multiple macular reflex intensities; The macular reflex intensity is obtained by setting a light source at the position of the video coding block, and the light emitted by the light source is reflected by the macula of the eyeball of the user; Obtaining multiple macular reflex intensities fed back by the eyeball tracking device according to the light emitted by the light source at the position of each video coding block in the at least one video coding block after being reflected by the macula of the eyeball of the user; wherein a first light source is set at the position of each video coding block, and one video coding block corresponds to one macular reflex intensity. Determining the video coding block with the strongest reflex intensity in the multiple macular reflex intensities as the interest region of the user on the video picture.

2. The method of claim 1, wherein, Further comprising: Encoding and processing the video picture of the interest region according to a first encoding code rate to obtain a first encoding result; 3. The method of claim 1, wherein, Transmitting the first encoding result to the video display interface for display. Further comprising: Dividing the remaining video coding blocks on the video picture except the interest region into code block regions of at least one quality level according to the distance from the eyeball of the user to the remaining video coding blocks; 4. The method of claim 3, wherein, Configuring a corresponding second encoding code rate for the code block regions of the at least one quality level, respectively; The second encoding code rate is smaller than the first encoding code rate; Encoding and processing the video data according to the second encoding code rate corresponding to each quality level of the code block region to obtain a second encoding result; Transmitting the second encoding result to the video display interface for display. Further comprising: When the interest region information returned by the eyeball tracking device cannot be received, determining the interest region according to the motion vector in the video coding block corresponding to the last interest region.

5. The method of claim 1, wherein, Further comprising: When the motion vector in the video coding block cannot be obtained, comparing the current video picture and the video picture of the last reported interest region to select the video coding block with the largest change degree as the interest region of the current video picture.

6. The method of claim 5, wherein, Applied to an eyeball tracking device, the method comprises: Receiving code block division information of at least one video coding block of a video picture displayed on a video display interface sent by a server; 7. A method for determining a region of interest on a video display interface, characterized in that, Obtaining multiple macular reflex intensities according to the code block division information; Reporting the multiple macular reflex intensities to the server, so that the server determines an interest region of a user on the video picture according to the multiple macular reflex intensities; Obtaining multiple macular reflex intensities according to the light emitted by the light source at the position of each video coding block in the at least one video coding block after being reflected by the macula of the eyeball of the user; ​ ​ According to the code block division information, a light source is arranged at a position of a video coding block, and a plurality of macular reflection intensities are obtained after light emitted by the light source is reflected by a macula of an eyeball of a user; According to the code block division information, a light source is arranged at a position of a video coding block, and a plurality of macular reflection intensities are obtained after light emitted by the light source is reflected by a macula of an eyeball of a user; According to the code block division information, a light source is arranged at a position of a video coding block, and a plurality of macular reflection intensities are obtained after light emitted by the light source is reflected by a macula of an eyeball of a user; 8. A device for determining a region of interest on a video display interface, characterized in that, The device is applied to a server and includes: A code block division information acquisition module is configured to acquire code block division information of at least one video coding block of a video picture displayed on a video display interface; A transmission module is configured to transmit the code block division information to an eyeball tracking device; A processing module is configured to acquire a plurality of macular reflection intensities fed back by the eyeball tracking device according to the code block division information, and determine an interest region of a user on the video picture according to the plurality of macular reflection intensities. The macular reflection intensities are obtained after light emitted by a light source arranged at a position of a video coding block is reflected by a macula of an eyeball of a user. The eyeball tracking device feeds back the plurality of macular reflection intensities according to light emitted by a light source arranged at a position of each of the at least one video coding block and reflected by a macula of an eyeball of a user. The device is applied to an eyeball tracking device and includes:

9. A device for determining a region of interest on a video display interface, characterized in that, A transceiver module is configured to receive code block division information of at least one video coding block of a video picture displayed on a video display interface sent by a server; A processing module is configured to obtain a plurality of macular reflection intensities according to the code block division information, and report the plurality of macular reflection intensities to the server, so that the server determines an interest region of a user on the video picture according to the plurality of macular reflection intensities. The macular reflection intensities are obtained after light emitted by a light source arranged at a position of a video coding block is reflected by a macula of an eyeball of a user. The macular reflection intensities are obtained after light emitted by a light source arranged at a position of a video coding block is reflected by a macula of an eyeball of a user. ​ ​ According to the code block division information, a light source is arranged at a position of each of the at least one video coding block, and light emitted by the light source is reflected by a macula of an eyeball of a user to obtain a plurality of macular reflection intensities; wherein one first light source is arranged at the position of each of the video coding blocks, and one video coding block corresponds to one macular reflection intensity.

10. A computing device, comprising: The method comprises: A processor and a memory storing a computer program, wherein the computer program is run by the processor to execute the method according to any one of claims 1 to 6 or the method according to claim 7.

11. A computer readable storage medium, characterized in that, A storage storing instructions, wherein the instructions are run on a computer to make the computer execute the method according to any one of claims 1 to 6 or the method according to claim 7.

Images