Picture information frame supplementing method of handheld device and handheld device

Abstract

The application provides a frame filling method for picture information of a handheld device and the handheld device, and solves the technical problem of frame drop phenomenon of game video of the handheld device game in the prior art, which affects the overall fluency of the game video and reduces the experience of the user. The frame filling method for picture information of the handheld device provided by the application, when a user experiences a game on the handheld device, when the rendering frame rate of the picture information is less than the screen refresh rate, the motion vector information of a transition frame picture is determined according to the first two frame pictures of the current frame picture information and influencing factors, the motion vector information of the transition frame picture is rendered, the transition frame picture information is generated, and the transition frame picture information is inserted between the current frame picture information and the next frame picture information, so that the frame filling of the picture information seen by the user is realized, the probability of dizziness of the user when experiencing the game is reduced, the overall fluency of the video is improved, and the experience of the user is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of electronic technology, in particular to a frame filling method for picture information of a handheld device and the handheld device. BACKGROUND

[0002] Game consoles are favored by the majority of young people due to convenient portability and strong entertainment. The existing game consoles are either full-key control without the fun of arcade free control or electronic joysticks. During the process of playing the game console, the user is mainly affected by the handheld device host, the display screen and the like as the carrier of the game console. The display screen is used to display game video. When the rendering frame rate of the game video content is different from the refresh rate of the display screen, the game video will appear frame dropping phenomenon, thereby affecting the overall smoothness of the video, causing the user to have a dizziness phenomenon and reducing the user experience. SUMMARY

[0003] Therefore, the present application provides a frame filling method for picture information of a handheld device and the handheld device, which solves the technical problem that the game video of the handheld device game appears frame dropping phenomenon, affects the overall smoothness of the game video and reduces the user experience.

[0004] As a first aspect of the present application, the present application provides a frame filling method for picture information of a handheld device, the handheld device comprising a handheld device host, a control handle in communication connection with the handheld device host and a display screen, wherein the frame filling method for picture information of the handheld device comprises: acquiring a screen refresh rate of the display screen; acquiring a rendering frame rate of picture information displayed by the display screen; when the rendering frame rate of the picture information is less than the screen refresh rate, determining motion vector information of a transition frame picture according to the first two frame picture information of the current frame picture information displayed by the display screen and an influencing factor, wherein the first two frame picture information of the current frame picture information comprises first picture information and second picture information; rendering the motion vector information of the transition frame picture to generate transition frame picture information; and inserting the transition frame picture information between the current frame picture information and next frame picture information.

[0005] In an embodiment of the present application, the motion vector information of the transition frame is determined according to the first two frame information of the current frame information displayed by the display screen and the influencing factors, comprising: determining a first influencing factor which influences the change of display content information of the first frame information and the second frame information according to the first frame information and the second frame information; extracting a plurality of first sub-frame information and a plurality of second sub-frame information from the first frame information and the second frame information respectively; determining the first motion vector information of the transition frame according to the plurality of first sub-frame information and the plurality of second sub-frame information; and determining the motion vector information of the transition frame according to the first motion vector information of the transition frame and the first influencing factor.

[0006] In an embodiment of the present application, the determination of the first motion vector information of the transition frame according to the plurality of first sub-frame information and the plurality of second sub-frame information comprises: matching the plurality of first sub-frame information with the plurality of second sub-frame information to generate at least one group of matching information, each group of the matching information comprising a first sub-frame information and a second sub-frame information matched successfully with the first sub-frame information; determining the motion vector information of the first transition frame sub-frame information according to the first sub-frame information and the second sub-frame information in the group of matching information; and determining the first motion vector information of the transition frame according to the motion vector information of at least one of the first transition frame sub-frame information.

[0007] In an embodiment of the present application, when the first influencing factor comprises a control handle, the movement of the control handle can control the action of the game subject of the user in the frame information; wherein the determination of the motion vector information of the transition frame according to the first motion vector information of the transition frame and the first influencing factor comprises: obtaining the spatial positioning information of the control handle; and determining the motion vector information of the transition frame according to the first motion vector information of the transition frame and the spatial positioning information of the control handle.

[0008] In an embodiment of the present application, when the first influencing factor comprises a handheld device host; wherein the determination of the motion vector information of the transition frame according to the first motion vector information of the transition frame and the first influencing factor comprises: obtaining the spatial positioning information of the handheld device host; and determining the motion vector information of the transition frame according to the first motion vector information of the transition frame and the spatial positioning information of the handheld device host.

[0009] In an embodiment of the present application, when the first influencing factor comprises a handheld device user, wherein the motion vector information of the transition frame is determined according to the first motion vector information of the transition frame picture and the first influencing factor, comprising: obtaining motion trajectory information of the handheld device user; and determining the motion vector information of the transition frame picture according to the first motion vector information of the transition frame picture and the motion trajectory information of the handheld device user.

[0010] In an embodiment of the present application, the first influencing factor causing the display content information of the first picture information and the second picture information to change is determined according to the first picture information and the second picture information, comprising: determining a change attribute of the display content information of the first picture information and the second picture information; and determining the first influencing factor causing the display content information of the first picture information and the second picture information to change according to the change attribute and a preset control rule.

[0011] In an embodiment of the present application, the motion vector information of the transition frame picture is determined according to the first two picture information of the current frame picture information displayed by the display screen and the influencing factor, further comprising: determining a second influencing factor causing the first picture information and the second picture information to change in non-display content information according to a preset control rule; determining a change feature of the second influencing factor; wherein the motion vector information of the transition frame is determined according to the first motion vector information of the transition frame picture and the first influencing factor, comprising: determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture, the first influencing factor and the change feature of the second influencing factor.

[0012] In an embodiment of the present application, when the change feature of the second influencing factor comprises a posture of a handheld device user, the change of the posture of the user can control the change of the posture of a virtual avatar of the user in the picture information; the motion vector information of the transition frame is determined according to the first motion vector information of the transition frame picture, the first influencing factor and the change feature of the second influencing factor, comprising: determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture, the first influencing factor and the posture of the handheld device user.

[0013] As a second aspect of the present application, the present application also provides a handheld device, comprising: a handheld device host; a display screen, which is in communication connection with the handheld device host; a control handle, which is in communication connection with the handheld device host; and a frame filling controller, which is in communication connection with the control handle, the display screen and the handheld device host respectively; wherein the frame filling controller comprises: a judging module, which is used to acquire a screen refresh rate of the display screen and a rendering frame rate of picture information displayed by the display screen, and judge whether the rendering frame rate of the picture information is less than the screen refresh rate; a control chip, which is in communication connection with the judging module, and is used to determine motion vector information of a transition frame picture according to previous two frame picture information and influencing factors of the current frame picture information displayed by the display screen when the rendering frame rate of the picture information is less than the screen refresh rate; and an image rendering module, which is used to render the motion vector information of the transition frame picture to generate transition frame picture information; and the control chip is further used to insert the transition frame picture information between the current frame picture information and next frame picture information.

[0014] In an embodiment of the present application, the control chip comprises: a first influencing factor determining module, which is used to determine a first influencing factor that causes display content information of the previous two frame picture information to change according to the previous two frame picture information of the current frame picture information displayed by the display screen; a vector information determining unit, which comprises: a first calculating module, which is used to extract a plurality of first sub-picture information and a plurality of second sub-picture information from the first picture information and the second picture information respectively; determine first motion vector information of a transition frame picture according to the plurality of first sub-picture information and the plurality of second sub-picture information; an integration module, which is used to determine motion vector information of the transition frame according to the first motion vector information of the transition frame picture and the first influencing factor, and transmit the motion vector information of the transition frame picture to the image rendering module; and an interpolation module, which is in communication connection with the image rendering module, and is used to receive the transition frame picture information transmitted by the image rendering module, and insert the transition frame picture information between the current frame picture information and the next frame picture information.

[0015] In an embodiment of the present application, the first computing module comprises: a matching module configured to match the first sub-picture information with the second sub-picture information to generate at least one matching information group, each of the matching information groups comprising the first sub-picture information and the second sub-picture information matched with the first sub-picture information; a determining module configured to determine the motion vector information of the first transition frame sub-picture information according to the first sub-picture information and the second sub-picture information in the matching information group; and determine the first motion vector information of the transition frame picture information according to the motion vector information of the first transition frame sub-picture information.

[0016] In an embodiment of the present application, the handheld device further comprises: a first inertial sensor disposed on the control handle, the first inertial sensor being configured to detect the IMU data of the control handle, and the control handle being provided with an infrared lamp; a first camera disposed on the handheld device host, the first camera being configured to capture the infrared lamp on the control handle to obtain a space image of a space where the control handle is located; the vector information determining unit further comprises: a second computing module, the second computing module being in communication connection with the first influence factor determining module, the first camera and the first inertial sensor respectively, the second computing module being configured to, when the first influence factor comprises the control handle, calculate the spatial positioning information of the control handle according to the space image of the space where the control handle is located and the IMU data of the control handle; and the integration module is in communication connection with the first computing module and the second computing module respectively, the integration module being configured to determine the motion vector information of the transition frame picture according to the first motion vector information of the transition frame picture and the spatial positioning information of the control handle.

[0017] In an embodiment of the present application, the handheld device further comprises: a second inertial sensor arranged on the handheld device host, the second inertial sensor being configured to detect IMU data of the handheld device host; and a second camera arranged on the handheld device host, the second camera being configured to capture a space in which the handheld device host is located to obtain a space image of the space in which the handheld device host is located; wherein the vector information determination unit further comprises: a third calculation module, which is in communication connection with the first influence factor determination module, the integration module, the second inertial sensor, and the second camera, respectively, and is configured to calculate spatial positioning information of the handheld device host according to the IMU data of the handheld device host and the space image of the space in which the handheld device host is located when the influence factor further comprises the handheld device host; and wherein the integration module is in communication connection with the first calculation module and the third calculation module, respectively, and is configured to determine the motion vector information of the transition frame picture according to the first motion vector information of the transition frame picture and the spatial positioning information of the handheld device host when the first influence factor comprises the handheld device host.

[0018] In an embodiment of the present application, the handheld device is provided with a third camera, the third camera being configured to capture motion trail information of a user of the handheld device; wherein the vector information determination unit further comprises: a fourth calculation module, which is in communication connection with the first influence factor determination module and the third camera, and is configured to obtain the motion trail information of the user of the handheld device captured by the third camera when the first influence factor comprises the user of the handheld device; and wherein the integration module is in communication connection with the fourth calculation module and the first calculation module, respectively, and is configured to determine the motion vector information of the transition frame picture according to the first motion vector information of the transition frame picture and the motion trail information of the user of the handheld device when the first influence factor comprises the user of the handheld device.

[0019] In an embodiment of the present application, the control chip further comprises a second influence factor determination module configured to determine a second influence factor affecting the change in the content of the first picture information and the second picture information according to a preset control rule; the vector information determination unit further comprises a fifth calculation module configured to determine the second influence factor affecting the change in the non-display content information of the first picture information and the second picture information according to a preset control rule, and determine the change characteristics of the second influence factor; the integration module is in communication connection with the first calculation module, the first influence factor determination module and the fifth calculation module, and is configured to determine the motion vector information of the transition frame according to the first motion vector information of the transition frame picture, the first influence factor and the change characteristics of the second influence factor.

[0020] The frame filling method for picture information of a handheld device provided by the present application can be used when a user experiences a game on a handheld device (such as a game console, a mobile phone or a tablet computer). When the rendering frame rate of the picture information is less than the screen refresh rate, the motion vector information of a transition frame picture is determined according to the previous two frames of picture information and the influence factors of the current frame of picture information, the motion vector information of the transition frame picture is rendered, the transition frame picture information is generated, and the transition frame picture information is inserted between the current frame of picture information and the next frame of picture information. Thus, the picture information seen by the user is filled, the probability of dizziness of the user when experiencing the game is reduced, the overall fluency of the video is improved, and the experience of the user is improved. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout the figures. The accompanying drawings are used to provide a further understanding of the embodiments of the present application, form a part of the specification, and are used together with the specification to explain the present application. The accompanying drawings do not limit the present application, and are merely used to explain the present application. In the drawings, the same reference numerals refer to the same components or steps throughout the figures.

[0022] Figure 1 Fig. 1 shows a flowchart of a frame filling method for picture information of a handheld device according to an embodiment of the present application;

[0023] Figure 2 Fig. 2 shows a flowchart of a frame filling method for picture information of a handheld device according to another embodiment of the present application;

[0024] Figure 3 Fig. 3 shows a flowchart of a frame filling method for picture information of a handheld device according to another embodiment of the present application;

[0025] Figure 4 Fig. 4 shows a working principle diagram of a handheld device according to an embodiment of the present application.

[0026] Figure 5 Fig. 1 shows a schematic diagram of a handheld device according to an embodiment of the application;

[0027] Figure 6 Fig. 1 shows a schematic diagram of a handheld device according to an embodiment of the application;

[0028] Figure 7 Fig. 1 shows a schematic diagram of a handheld device according to an embodiment of the application;

[0029] Figure 8 Fig. 1 shows a schematic diagram of a handheld device according to an embodiment of the application;

[0030] Figure 9 Fig. 1 shows a schematic diagram of a handheld device according to an embodiment of the application;

[0031] Figure 10 Fig. 1 shows a schematic diagram of a handheld device according to an embodiment of the application;

[0032] Figure 11 Fig. 1 shows a schematic diagram of a handheld device according to an embodiment of the application. DETAILED DESCRIPTION

[0033] In the description of the application, "a plurality" means at least two, for example two, three, etc., unless expressly specified otherwise. All directional references, such as, up, down, right, left, front, back, top, bottom, etc., are in relation to the particular orientation of the figure in which the embodiment is shown, and are used only to illustrate relative positioning between components, movement, etc. and are not intended to limit the scope of the application. In addition, the terms "comprises", "comprising", "includes", "including" and the like can be used herein. Such terms are intended to cover non-exclusive inclusions, such that comprising can optionally include additional steps, units, components, etc. without deviating from the scope of the application.

[0034] Further, reference being made herein to "an embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the application. The appearances of the phrase "in an embodiment" in various places in the specification are not necessarily referring to the same embodiment, nor are they necessarily all directed to the same embodiment, or are they necessarily all mutually exclusive or alternative embodiments. It is expressly understood that the embodiments described herein are merely examples from a whole class of comparable embodiments which are claimed as falling within the scope of the application, and that structural, architectural or conceptual changes can be made without departing from the scope of the application.

[0035] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all the other embodiments obtained by those skilled in the art without creative efforts belong to the scope of the present application.

[0036] Exemplary frame padding method

[0037] Figure 1 Fig. 1 is a flow diagram of a method for frame interpolation of picture information of a handheld device according to an embodiment of the present application. The handheld device comprises a handheld device host, a control handle in communication connection with the handheld device host, and a display screen, wherein the display screen is used to display game video. The types of the display screen include the following three cases:

[0038] (1) Head-mounted display:

[0039] The head-mounted display comprises a helmet and a display screen fixed on the helmet. The user can see the picture information based on technology on the display screen by wearing the helmet, so as to experience a game with stronger immersion and more realistic experience.

[0040] (2) The display screen is arranged on the handheld device host. For example, when the handheld device is a game console, the display screen is arranged on the game console. When the handheld device is a mobile phone or a tablet computer, the display screen is the display screen of the mobile phone or the tablet computer. The user can experience the game by downloading related software on the mobile phone or the tablet computer.

[0041] (3) External display screen:

[0042] The handheld device is used as a game host of the game, but the game video is displayed on the display screen of the non-handheld device, that is, the external display screen, such as a television.

[0043] Based on the above-mentioned handheld device, the present application provides a method for frame interpolation of picture information of a handheld device, as shown in Fig. 1. The method for frame interpolation of picture information of the handheld device comprises the following steps: Figure 1

[0044] Step S101: obtaining the screen refresh rate of the display screen;

[0045] The screen refresh rate of the display screen is an attribute feature of the display screen, which can be obtained by querying the parameters of the display screen.

[0046] Step S102: obtaining the rendering frame rate of the picture information displayed by the display screen;

[0047] ​The rendering frame rate of screen information refers to the frame rate at which real-world information and game information are rendered to generate screen information.

[0048] Step S103: Determine whether the rendering frame rate of the screen information is less than the screen refresh rate;

[0049] When the judgment result of step S103 is yes, that is, the rendering frame rate of the screen information is less than the screen refresh rate, the screen information seen by the user will show frame drops, causing the user to feel dizzy. Therefore, it is necessary to fill in the dropped frames, that is, to execute steps S104-S106.

[0050] Step S104: Determine the motion vector information of the transition frame based on the previous two frames of the current frame displayed on the screen and the influencing factors;

[0051] Specifically, influencing factors refer to factors that can affect changes in the information displayed on the screen. These factors include those that affect changes in the displayed content of the screen and those that affect changes in the non-display content of the screen.

[0052] Step S105: Render the motion vector information of the transition frame to generate transition frame information; and

[0053] Step S106: Insert the transition frame information between the current frame information and the next frame information.

[0054] The present invention provides a frame interpolation method for handheld device screen information. When a user is playing a game on a handheld device (such as a handheld game console, mobile phone, or tablet computer), and the rendering frame rate of the screen information is less than the screen refresh rate, the motion vector information of the transition frame is determined based on the previous two frames of the current frame and influencing factors. The motion vector information of the transition frame is then rendered to generate transition frame information, which is then inserted between the current frame and the next frame. This achieves frame interpolation of the screen information seen by the user, reducing the probability of dizziness during gameplay, improving the overall smoothness of the video, and thus enhancing the user experience.

[0055] In one embodiment of the present invention, such as Figure 2 As shown, the preceding two frames of the current frame information include the first frame information and the second frame information; wherein, step S104 (determining the motion vector information of the transition frame based on the preceding two frames of the current frame information displayed on the screen and influencing factors) specifically includes the following steps:

[0056] Step S1041: determining a first influencing factor that influences the change of the display content information of the first picture information and the second picture information according to the first picture information and the second picture information;

[0057] Specifically, the device used by the user to experience the game is different, and the influencing factor that influences the change of the display content information in the first picture information and the second picture information is also different, that is, the first influencing factor is related to the device used by the user.

[0058] For example, when the device used by the user to experience the game is a control handle, the display content information in the game can be controlled to change by controlling the control handle, at this time, the first influencing factor is the control handle.

[0059] Specifically, the specific steps of determining the first influencing factor in step S1041 can include the following steps:

[0060] Step S1: determining a change attribute of the display content information of the first picture information and the second picture information;

[0061] The change attribute refers to the embodiment of the change of the first picture information and the second picture information, and the change attribute includes but is not limited to: overall change of the picture and content change of the picture. For example, the overall change of the picture can include that the second picture information is reduced or enlarged relative to the first picture information; the second picture information is moved left, right, forward or backward relative to the first picture information. The content change of the picture refers to that the position of the reference object in the picture changes, for example, the position of the reference object in the second picture information moves to the upper left corner relative to the position in the first picture information.

[0062] Step S2: determining a first influencing factor of the change of the display content information of the first picture information and the second picture information according to the change attribute and a preset control rule.

[0063] When the change attribute of the change of the first picture information and the second picture information is determined in step S10421, the influencing factor of the change of the first picture information and the second picture information is determined according to the change attribute and a preset control rule.

[0064] For example, when the change attribute of the change of the first picture information and the second picture information is that the position of the reference object in the second picture information moves to the upper left corner relative to the position in the first picture information. The reason for controlling the position change of the reference object is that the user operates the control handle to move to the upper left corner, and the preset control rule of the game device is that the control handle moving to the upper left corner can control the reference object to move to the upper left corner. Therefore, according to the preset control rule, it can be determined that the control handle moving to the upper left corner is the first influencing factor.

[0065] It should be understood that the first influencing factor of the change of the display content information of the first picture information and the second picture information is different when the user uses different devices to experience the game.

[0066] Step S1042: extracting a plurality of first sub-picture information and a plurality of second sub-picture information from the first picture information and the second picture information respectively;

[0067] Specifically, the plurality of first sub-picture information and the plurality of second sub-picture information can be selected in the first picture information and the second picture information respectively based on the detection frame.

[0068] Step S1043: determining the first motion vector information of the transition frame picture according to the plurality of first sub-picture information and the plurality of second sub-picture information;

[0069] That is, the first motion vector information of the transition frame picture is determined according to the first sub-picture information and the second sub-picture information, that is, the first motion vector information of the transition frame picture is determined simply from the picture information of the displayed video. The specific determination method can adopt the method described below, that is, step S1043 can specifically include the following steps:

[0070] (I): matching the plurality of first sub-picture information with the plurality of second sub-picture information to generate at least one group of matching information, each group of matching information including a first sub-picture information and a second sub-picture information matched successfully with the first sub-picture information;

[0071] (II): determining the motion vector information of the first transition frame sub-picture information according to the first sub-picture information and the second sub-picture information in the matching information group; and

[0072] (III): determining the first motion vector information of the transition frame picture information according to the motion vector information of the at least one first transition frame sub-picture information.

[0073] Step S1044: determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture and the first influencing factor.

[0074] In an embodiment of the present application, as described above, when the first influencing factor that causes the first picture information and the second picture information to change is determined in step S1041, the device used by the user to experience the game is different, the first influencing factor that causes the first picture information and the second picture information to change is also different, and then the manner of determining the motion vector information of the transition frame in step S1044 (determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture and the first influencing factor) is also different. The following will be described in detail according to the device used by the user to experience the game.

[0075] (1) When the first influencing factor is a control handle, the movement of the control handle can control the action of the game subject of the user in the picture information.

[0076] Specifically, step S1044 (determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture and the first influencing factor) includes the following steps.

[0077] Step S10: Obtain the spatial positioning information of the control handle.

[0078] Specifically, the manner of obtaining the spatial positioning information of the control handle can include: installing a first inertial sensor and an infrared lamp on the control handle, the first inertial sensor can detect the IMU data (for example, 3DOF data) of the control handle; setting a first camera device on the handheld device host, the first camera device is used to shoot the infrared lamp on the control handle to obtain the spatial image of the space where the control handle is located; the 6DOF data of the control handle can be determined according to the IMU data of the control handle and the spatial image of the space where the control handle is located, and the spatial positioning information of the control handle includes the 6DOF data.

[0079] Step S11: Determine the motion vector information of the transition frame picture according to the first motion vector information of the transition frame picture and the spatial positioning information of the control handle.

[0080] (2) When the first influencing factor is a handheld device host, step S1044 (determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture and the first influencing factor) includes the following steps.

[0081] Step S20: Obtain the spatial positioning information of the handheld device host.

[0082] When the handheld device host and the control handle are arranged separately, for example, the handheld device host is fixed at any position in the space, and the game video is displayed on the TV set, the user can watch the game video on the TV set by operating the control handle, in this case, the handheld device host can be moved, when it is determined that the first picture and the second picture change because the handheld device host moves from left to right in the space according to the analysis of the first picture and the second picture, the spatial positioning information of the handheld device host is acquired at this time.

[0083] Specifically, the acquisition method of the spatial positioning information of the handheld device host can be that a second inertial sensor is arranged on the handheld device host, the second inertial sensor can detect the IMU data (for example, 3DOF data) of the handheld device host, and the spatial positioning information of the handheld device host includes the IMU data.

[0084] Specifically, the inertial sensor (Inertial Measurement Unit, IMU for short) is a device for measuring the three-axis attitude angle (or angular rate) and acceleration of an object. Generally, an IMU includes three single-axis accelerometers and three single-axis gyroscopes. The accelerometer detects the acceleration signal of the object on the independent three-axis of the carrier coordinate system, and the gyroscope detects the angular velocity signal of the carrier relative to the navigation coordinate system. Therefore, the inertial sensor can measure the angular velocity and acceleration of an object in three-dimensional space and calculate the attitude of the object, for example, calculate the rotational degrees of freedom of the object. The rotational degrees of freedom refer to the three position-related degrees of freedom of up, down, front, back and left and right. The IMU data is the result data detected by the inertial sensor, that is, the angular velocity and acceleration data of an object in three-dimensional space detected by the inertial sensor. Therefore, the first inertial sensor arranged on the control handle can detect the IMU data of the control handle, and the attitude of the control handle can be calculated based on the IMU data of the control handle, for example, the rotational degrees of freedom of the control handle. The rotational degrees of freedom refer to the three position-related degrees of freedom of up, down, front, back and left and right.

[0085] Optionally, a first camera can also be arranged on the handheld device host, the first camera can take pictures of the surrounding environment where the handheld device host is located, and the 6DOF data of the handheld device host can be determined according to the pictures taken by the first camera and the IMU data of the handheld device host detected by the first inertial sensor. The spatial positioning information of the handheld device host includes the 6DOF data of the handheld device host.

[0086] Step S21: determining the motion vector information of the transition frame picture according to the first motion vector information of the transition frame picture and the spatial positioning information of the handheld device host.

[0087] When the spatial positioning information of the handheld device host is determined in step S20, the motion vector information of the transition frame picture is determined according to the spatial positioning information of the handheld device host, the first picture information and the second picture information. That is, in the estimated transition frame picture, the spatial positioning information of the handheld device host is also referred to in addition to the first picture information and the second picture information, so that the estimation accuracy is improved.

[0088] (3) When the influencing factor is the handheld device user, S1044 (determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture and the first influencing factor) specifically includes the following steps:

[0089] Step S30: acquiring the motion trajectory information of the handheld device user;

[0090] For example, when the user B experiences a game on the handheld device, but the game video is displayed on the display screen of the helmet by wearing the helmet, when the first picture and the second picture change in the display content information, the performance form can be that the motion of the user in the real environment causes the motion of the corresponding game character in the video picture.

[0091] Step S31: determining the motion vector information of the transition frame picture according to the first motion vector information of the transition frame picture and the motion trajectory information of the handheld device user.

[0092] For example, when the motion trajectory information of the handheld device user is determined in step S30, the motion vector information of the picture corresponding to the detection frame in the transition frame picture is determined according to the head motion trajectory information of the handheld device user, the first picture information and the second picture information.

[0093] The above (1), (2), and (3) respectively describe the specific manner of determining the motion vector information of the transition frame according to the first motion vector information of the transition frame and the motion trajectory information of the user of the handheld device when three different first influencing factors affecting the first picture information and the second picture information cause the display content information to change. However, in actual application, the first influencing factor causing the first picture information and the second picture information to change can be only one first influencing factor, or can be more than two first influencing factors. When the first influencing factor causing the first picture information and the second picture information to change includes multiple factors, the motion vector information in the transition frame is respectively determined according to each first influencing factor, the first picture information, and the second picture information, and then the motion vector information of the picture obtained according to each first influencing factor is integrated to finally generate the motion vector information of the picture in the transition frame. For example, when the influencing factors affecting the first picture information and the second picture information to change include the user of the handheld device and the host of the handheld device, the motion vector information in the transition frame is respectively determined by the manner (2) and the manner (3) when determining the motion vector information of the transition frame, and then the motion vector information of the transition frame obtained by (2) and (3) is integrated to finally generate the motion vector information of the transition frame.

[0094] In an embodiment of the present application, as shown in Figure 3 the first two picture information of the current frame picture information includes the first frame picture information and the second frame picture information; wherein, the step S104 (determining the motion vector information of the transition frame according to the first two picture information of the current frame picture information displayed on the display screen and the influencing factors) specifically includes the following steps:

[0095] Step S1045: determining the second influencing factor affecting the first picture information and the second picture information to change in non-display content information according to a preset control rule.

[0096] The first picture information and the second picture information changing in non-display content information means that the display content does not change, for example, the actual display content in the two frame picture information does not change, but visually, the picture information is close to or far away from the eyes of the user.

[0097] The preset control rule is a preset control mode. For example, when user A experiences a game on a handheld device, and a game video is displayed on a display screen of the handheld device, the input instruction corresponding to the approaching or moving away control instruction can be an instruction input or operated by the user on the handheld device. When user B experiences the game on the handheld device, but the game video is displayed on a display screen of a helmet by wearing the helmet, the input instruction corresponding to the approaching or moving away control instruction can be an input instruction generated by shaking the head, for example, shaking the head to the left is approaching, and shaking the head to the right is moving away.

[0098] Step S1046: determining a change feature of the second influence factor;

[0099] The change feature refers to a specific manifestation of non-display content information change of the first picture information and the second picture information. For example, when user B experiences the game on the handheld device, but the game video is displayed on the display screen of the helmet by wearing the helmet, when the first picture information and the second picture information change in the manifestation of non-display content information, the manifestation is that the picture is enlarged, and the influence factor of the picture being enlarged is that the head of the user moves forward, and therefore, the motion vector information in the transition frame picture is determined according to the head movement track of the user, the first picture information, and the second picture information.

[0100] The step S1044 (determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture and the first influence factor) includes:

[0101] Step S1047: determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture, the first influence factor, and the change feature of the second influence factor.

[0102] Since in actual application, the first picture information and the second picture information mostly change in display content information and non-display content information at the same time, when the motion vector information is determined, the change features of the first influence factor and the second influence factor are comprehensively combined to determine the motion vector information of the transition frame.

[0103] Specifically, when the change feature of the second influence factor includes the posture of the user of the handheld device, the change of the posture of the user can control the change of the posture of the virtual avatar of the user in the picture information.

[0104] The step S1047 (determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture, the first influence factor, and the change feature of the second influence factor) can specifically include the following steps:

[0105] Step S10471: determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture, the first influence factor, and the posture of the user of the handheld device.

[0106] Exemplary handheld device

[0107] As a second aspect of the present application, the present application also provides a handheld device, such as Figure 4 as shown in the figure, the handheld device comprises:

[0108] a handheld device host 100;

[0109] a display screen 200, which is in communication connection with the handheld device host 100, and is used for displaying game video;

[0110] a control handle 300, which is in communication connection with the handheld device host 100; and

[0111] a frame interpolation controller 400, which is in communication connection with the control handle 300 and the handheld device host 100 respectively;

[0112] wherein the frame interpolation controller 400 comprises:

[0113] a judging module 401, which is used for acquiring the screen refresh rate of the display screen and the rendering frame rate of the picture information displayed by the display screen, and judging whether the rendering frame rate of the picture information is less than the screen refresh rate;

[0114] a control chip 402, which is in communication connection with the judging module 401, and is used for determining the motion vector information of the transition frame picture according to the previous two frame picture information of the current frame picture information displayed by the display screen and the influence factor when the rendering frame rate of the picture information is less than the screen refresh rate;

[0115] an image rendering module 403, which is in communication connection with the control chip 402, and is used for rendering the motion vector information of the transition frame picture, generating the transition frame picture information, and transmitting the transition frame picture information to the control chip 402;

[0116] the control chip 402 is further used for inserting the transition frame picture information between the current frame picture information and the next frame picture information.

[0117] The application provides a frame filling controller of picture information of a handheld device. When a user experiences a game on the handheld device (for example, a game console, a mobile phone, or a tablet computer), when a rendering frame rate of the picture information is less than a screen refresh rate, motion vector information of a transition frame picture is determined according to the first two frame pictures of the current frame picture information, the motion vector information of the transition frame picture is rendered, transition frame picture information is generated, and the transition frame picture information is inserted between the current frame picture information and the next frame picture information, so that the picture information seen by the user is filled, the probability that the user feels dizzy when experiencing the game is reduced, the overall fluency of the video is improved, and the experience of the user is improved.

[0118] Optionally, the types of the display screen include the following three cases.

[0119] (1) a helmet display device:

[0120] The helmet display device includes a helmet and a display screen fixed on the helmet. The user can see the picture information of the game on the display screen by wearing the helmet, so that the user can experience a game with stronger immersion and more realistic experience.

[0121] (2) the display screen is arranged on a handheld device host. For example, when the handheld device is a game console, the display screen is arranged on the game console. When the handheld device is a mobile phone or a tablet computer, the display screen is a display screen of the mobile phone or the tablet computer. The user can experience the game by downloading related software on the mobile phone or the tablet computer.

[0122] (3) an external display screen:

[0123] The handheld device is a game host of the game, but the video is displayed on a display screen of a non-handheld device, and is displayed on an external display screen, for example, a television.

[0124] In an embodiment of the application, as shown in FIG. 4, the control chip 402 includes: Figure 5 a first influence factor determination module 4021 configured to determine a first influence factor of display content information of the first two frame pictures according to the first two frame pictures of the current frame picture information displayed on the display screen.

[0125]

[0126] ​The vector information determining unit 4022 comprises a first calculating module 40221 and an integrating module 40222. The first calculating module 40221 is configured to extract a plurality of first sub-picture information and a plurality of second sub-picture information from the first picture information and the second picture information respectively, and determine first motion vector information of a transition frame picture according to the plurality of first sub-picture information and the plurality of second sub-picture information. The integrating module 40222 is configured to determine motion vector information of the transition frame according to the first motion vector information of the transition frame picture and the first influence factor, and transmit the motion vector information of the transition frame picture to the image rendering module 403.

[0127] The interpolation module 4023 is in communication connection with the image rendering module 403. The interpolation module 4023 is configured to receive the transition frame picture information transmitted by the image rendering module 403, and insert the transition frame picture information between the current frame picture information and the next frame picture information.

[0128] In an embodiment of the present application, as shown in Figure 6 The first calculating module 40221 comprises:

[0129] The matching module 40223 is configured to match the plurality of first sub-picture information with the plurality of second sub-picture information, and generate at least one set of matching information group. Each set of the matching information group comprises first sub-picture information and second sub-picture information matched successfully with the first sub-picture information.

[0130] The determining module 40224 is configured to determine motion vector information of first transition frame sub-picture information according to the first sub-picture information and the second sub-picture information in the matching information group, and determine first motion vector information of the transition frame picture information according to the motion vector information of at least one first transition frame sub-picture information.

[0131] In an embodiment of the present application, as shown in Figure 7As shown, the handheld device further comprises: a first inertial sensor 500 arranged on the handheld device host 100, wherein the first inertial sensor 500 is configured to detect IMU data of the handheld device host 100; and a first camera 600 configured to capture a space where the handheld device host 100 is located to obtain a space image of the space where the handheld device host 100 is located; at this time, the vector information determination unit 40222 further comprises a second calculation module 2, which is in communication connection with the first camera 600, the first inertial sensor 500, and the first influence factor determination module 4021 respectively, and when the first influence factor determination module 4021 determines that the first influence factor affecting the changes of the first picture information and the second picture information is the handheld device host 100, the spatial positioning information of the handheld device host 100 is obtained; and the motion vector information in the transition frame picture is determined according to the spatial positioning information of the handheld device host 100, the first picture information, and the second picture information. That is, the integration module is in communication connection with the first calculation module 40221 and the second calculation module 2 respectively, and the integration module 40222 is configured to determine the motion vector information of the transition frame picture according to the first motion vector information of the transition frame picture and the spatial positioning information of the control handle.

[0132] Specifically, when the handheld device host 100 and the control handle 300 are arranged separately, that is, the handheld device host 100 can be placed at any position in a space, and the display screen 200 is a display (including a television or a general display) arranged in the space, the user can realize the control of the game by operating the control handle and view the game video on the display arranged in the space, when the first influence factor determination module determines that the first influence factor affecting the changes of the first picture information and the second picture information is the handheld device host 100, the second calculation module 2 receives the IMU data of the handheld device host 100 transmitted by the first inertial sensor 500, and receives the space image of the space where the handheld device host 100 is located captured by the first camera 600, and calculates the IMU data of the handheld device host 100 and the space image of the space where the handheld device host 100 is located to generate the spatial positioning information of the handheld device host 100, and finally determines the motion vector information in the transition frame picture according to the spatial positioning information of the handheld device host 100, the first picture information, and the second picture information.

[0133] In an embodiment of the present application, as shown in Figure 8 the handheld device further comprises: a second inertial sensor 700 arranged on the handheld device host 100, wherein the second inertial sensor 700 is configured to detect IMU data of the handheld device host 100; and a second camera 800 arranged on the handheld device host 100, wherein the second camera 800 is configured to capture a space where the handheld device host 100 is located to obtain a space image of the space where the handheld device host 100 is located;

[0134] The vector information determination unit 4022 further comprises:

[0135] The third calculation module 3 is in communication connection with the first influence factor determination module 4021, the integration module 40222, the second inertial sensor 700 and the second camera 800 respectively, and is configured to calculate the spatial positioning information of the handheld device host 100 according to the IMU data of the handheld device host 100 and the spatial image of the space where the handheld device host 100 is located when the influence factor further comprises the handheld device host 100;

[0136] The integration module 40222 is in communication connection with the first calculation module 40221 and the third calculation module 3 respectively, and is configured to determine the motion vector information of the transition frame picture according to the first motion vector information of the transition frame picture and the spatial positioning information of the handheld device host when the first influence factor comprises the handheld device host 100.

[0137] In an embodiment of the present application, as shown in Figure 9 A third camera 801 is arranged on the handheld device, and the third camera 801 is configured to capture the motion trajectory information of the user of the handheld device; the vector information determination unit 4022 further comprises: a fourth calculation module 4 in communication connection with the first influence factor determination module 4021 and the third camera 801, and configured to capture the motion trajectory information of the user of the handheld device captured by the third camera 801 when the first influence factor comprises the user of the handheld device; the integration module 40222 is in communication connection with the fourth calculation module 4 and the first calculation module 40221 respectively, and is configured to determine the motion vector information of the transition frame picture according to the first motion vector information of the transition frame picture and the motion trajectory information of the user of the handheld device when the first influence factor comprises the user of the handheld device.

[0138] In an embodiment of the present application, as shown in Figure 10 The control chip 402 further comprises:

[0139] The second influence factor determination module 404 is configured to determine a second influence factor affecting the change of the display content information of the first picture information and the second picture information according to a preset control rule;

[0140] The vector information determination unit 4022 further comprises:

[0141] The fifth calculation module 5 is configured to determine a second influence factor affecting the change of the display content information of the first picture information and the second picture information according to a preset control rule, and determine the change characteristics of the second influence factor;

[0142] The integration module 40222 is communicatively connected to the first calculation module 40221, the first influencing factor determination module 4021, and the fifth calculation module 5, respectively. The integration module 40222 is used to determine the motion vector information of the transition frame based on the change characteristics of the first motion vector information, the first influencing factor, and the second influencing factor of the transition frame.

[0143] Figures 7-9 The diagram illustrates three different methods for estimating motion vector information based on varying influencing factors when a handheld device user experiences a game using different handheld devices. However, in practical applications, the changes between the first and second frames in the aforementioned set of images can be influenced by only one factor, or by two or more factors. Therefore, as... Figure 8 As shown, the vector information determination unit 40222 further includes an integration module 4, which is communicatively connected to at least one of the first calculation module 1, the second calculation module 2, and the third calculation module 3. The integration module 4 is used to determine the motion vector information of the transition frame based on multiple motion vector information from multiple sets of frames. Specifically, when there are multiple influencing factors affecting the change of the first and second frame information in a set of frames, the motion vector information of the frame corresponding to the detection box in the transition frame is determined based on each influencing factor, the first frame information, and the second frame information. Then, the motion vector information of the frames obtained based on each influencing factor is integrated to finally generate the motion vector information of the transition frame.

[0144] Exemplary electronic device

[0145] Below, for reference Figure 11 To describe an electronic device according to an embodiment of the present invention. Figure 11 The diagram shown is a structural schematic of an electronic device provided in an embodiment of the present invention.

[0146] like Figure 11 As shown, the electronic device 900 includes one or more processors 901 and memory 902.

[0147] The processor 901 may be a central processing unit (CPU) or other form of processing unit with data processing and / or information execution capabilities, and may control other components in the electronic device 900 to perform desired functions.

[0148] The memory 901 can include one or more computer program products that can include various forms of computer-readable storage media having stored thereon one or more computer program instructions. The computer-readable storage media can include, for example, volatile memory, such as RAM and / or cache, and / or non-volatile memory, such as ROM, hard disk drives, solid state drives, and / or the like. The computer program instructions can be executed by the processor 901 to implement the above-described methods of frame interpolation of picture information of a handheld device, or other desired functions.

[0149] In one example, the electronic device 900 can further include an input device 903 and an output device 904, which are interconnected to each other through a bus system and / or other forms of connection mechanisms (not shown).

[0150] The input device 903 can include, for example, a keyboard, a mouse, and / or the like.

[0151] The output device 904 can output various information to the outside. The output device 904 can include, for example, a display, a communication network and a remote output device connected thereto, and / or the like.

[0152] Of course, in order to simplify, Figure 11 Only some of the components of the electronic device 900 related to the present application are shown in FIG. 9, and components such as buses, input / output interfaces, and the like are omitted. In addition, the electronic device 900 can further include any other appropriate components according to a specific application.

[0153] In addition to the above-described methods and devices, embodiments of the present application can also be a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform steps of the above-described methods of frame interpolation of picture information of a handheld device according to various embodiments of the present application.

[0154] The computer program product can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, and / or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote cloud device or server.

[0155] In addition, an embodiment of the present application can also be a computer-readable storage medium, having stored thereon computer program information, which, when executed by a processor, causes the processor to perform the steps of the method of frame interpolation of picture information of a handheld device according to various embodiments of the present application.

[0156] The computer-readable storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable storage medium or a computer-readable signal medium. The computer-readable storage medium can include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage medium include a communication connection, a portable disc, a hard disc, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0157] The above describes the basic principles of the present application in combination with specific embodiments, but it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are only examples and are not limiting, and these advantages, benefits, effects, etc. cannot be considered as necessary for each embodiment of the present application. In addition, the above specific details are only for the purpose of example and understanding, and are not limiting, and the above details do not limit the present application to the above specific details.

[0158] The block diagrams of the devices, apparatuses, equipment, systems involved in the present application are only illustrative examples and are not intended to require or imply a connection, arrangement, configuration as shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have", etc. are open-ended words, mean "include but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0159] It should also be noted that in the apparatus, equipment and method of the present application, each component or step can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of the present application.

[0160] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0161] The above description is merely illustrative of the application and not limiting. Modifications of the application will occur to those skilled in the art upon reading the description of the application. The scope of the application should be determined, not with reference to the above description, but should be given to the open ends of the claims below.

Claims

1. A method for frame interpolation of a picture information of a hand-held device, characterized in that, The handheld device comprises a handheld device host, a control handle in communication connection with the handheld device host, and a display screen, wherein a picture information frame filling method of the handheld device comprises: acquiring a screen refresh rate of the display screen; acquiring a rendering frame rate of picture information displayed by the display screen; when the rendering frame rate of the picture information is less than the screen refresh rate, determining motion vector information of a transition frame according to previous two frame picture information and influencing factors of current frame picture information displayed by the display screen, wherein the previous two frame picture information of the current frame picture information comprises first picture information and second picture information; determining a first influencing factor of display content information of the first picture information and the second picture information which changes according to the first picture information and the second picture information; determining a second influencing factor of non-display content information of the first picture information and the second picture information which changes according to a preset control rule, and determining a change characteristic of the second influencing factor; extracting a plurality of first sub-picture information and a plurality of second sub-picture information from the first picture information and the second picture information respectively; determining first motion vector information of the transition frame according to the plurality of first sub-picture information and the plurality of second sub-picture information; and determining motion vector information of the transition frame according to the first motion vector information of the transition frame, the first influencing factor, and the change characteristic of the second influencing factor; rendering the motion vector information of the transition frame to generate transition frame picture information; and inserting the transition frame picture information between the current frame picture information and next frame picture information.

2. The method of claim 1, wherein the frame rate of the picture information is increased by the hand-held device. The determining of the first motion vector information of the transition frame according to the plurality of first sub-picture information and the plurality of second sub-picture information comprises: matching the plurality of first sub-picture information and the plurality of second sub-picture information to generate at least one group of matching information, each group of the matching information comprising first sub-picture information and second sub-picture information matched successfully with the first sub-picture information; determining motion vector information of first transition frame sub-picture information according to the first sub-picture information and the second sub-picture information in the group of matching information; and determining the first motion vector information of the transition frame picture information according to the motion vector information of at least one first transition frame sub-picture information.

3. The method of claim 1, wherein the frame rate of the picture information is increased by the hand-held device. When the first influencing factor comprises a control handle, movement of the control handle can control actions of a game subject of a user in picture information; wherein the determining of the motion vector information of the transition frame according to the first motion vector information of the transition frame and the first influencing factor comprises: acquiring spatial positioning information of the control handle; and determining the motion vector information of the transition frame according to the first motion vector information of the transition frame and the spatial positioning information of the control handle.

4. The method of claim 1, wherein the frame rate of the picture information is increased by the hand-held device. When the first influencing factor comprises a handheld device host; wherein the determining of the motion vector information of the transition frame according to the first motion vector information of the transition frame and the first influencing factor comprises: acquire spatial positioning information of the handheld device host; and determine the motion vector information of the transition frame picture according to the first motion vector information of the transition frame picture and the spatial positioning information of the handheld device host.

5. The method of claim 1, wherein the frame rate of the picture information is increased by the hand-held device. When the first influencing factor includes a handheld device user, wherein determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture and the first influencing factor includes: acquire motion trajectory information of the handheld device user; and determine the motion vector information of the transition frame picture according to the first motion vector information of the transition frame picture and the motion trajectory information of the handheld device user.

6. The method of claim 1, wherein the frame rate of the display is increased by a factor of 2. determine the first influencing factor that causes the display content information of the first picture information and the second picture information to change according to the first picture information and the second picture information, including: determine a change attribute of the display content information of the first picture information and the second picture information; and determine the first influencing factor that causes the display content information of the first picture information and the second picture information to change according to the change attribute and a preset control rule.

7. The method of claim 1, wherein the frame rate of the display is increased by a factor of 2. When the change feature of the second influencing factor includes a posture of the handheld device user, the change of the posture of the user can control the change of the posture of the virtual avatar of the user in the picture information. determine the motion vector information of the transition frame according to the first motion vector information of the transition frame picture, the first influencing factor, and the change feature of the second influencing factor, including: determine the motion vector information of the transition frame according to the first motion vector information of the transition frame picture, the first influencing factor, and the posture of the handheld device user.

8. A hand-held device, characterized by including: a handheld device host; a display screen, which is in communication connection with the handheld device host; a control handle, which is in communication connection with the handheld device host; and a frame interpolation controller, which is in communication connection with the control handle, the display screen, and the handheld device host respectively; wherein the frame interpolation controller includes: a judgment module, which is used to acquire a screen refresh rate of the display screen and a rendering frame rate of picture information displayed by the display screen, and to judge whether the rendering frame rate of the picture information is less than the screen refresh rate; a control chip, which is in communication connection with the judgment module, and is used to determine the motion vector information of a transition frame picture according to the previous two frame picture information and influencing factors of the current frame picture information displayed by the display screen when the rendering frame rate of the picture information is less than the screen refresh rate; wherein the previous two frame picture information of the current frame picture information includes first picture information and second picture information; a first influencing factor determination module, which is used to determine the first influencing factor that causes the display content information of the previous two frame picture information to change according to the previous two frame picture information of the current frame picture information displayed by the display screen; a second influencing factor determination module, which is used to determine the second influencing factor that causes the picture content of the first picture information and the second picture information to change according to a preset control rule; The vector information determination unit comprises a first calculation module and an integration module, the first calculation module is used for extracting a plurality of first sub-picture information and a plurality of second sub-picture information from the first picture information and the second picture information respectively; determining first motion vector information of a transition frame picture according to the plurality of first sub-picture information and the plurality of second sub-picture information; the integration module is used for determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture and the first influencing factor, and transmitting the motion vector information of the transition frame picture to the image rendering module; wherein the vector information determination unit further comprises a fifth calculation module, which is used for determining a second influencing factor of non-display content information affecting the first picture information and the second picture information according to a preset control rule, and determining the change characteristics of the second influencing factor; and The integration module is in communication connection with the first calculation module, the first influencing factor determination module and the fifth calculation module respectively, and is used for determining the motion vector information of the transition frame according to the first motion vector information of the transition frame picture, the first influencing factor and the change characteristics of the second influencing factor; The interpolation module is in communication connection with the image rendering module, and is used for receiving the transition frame picture information transmitted by the image rendering module and inserting the transition frame picture information between the current frame picture information and the next frame picture information; The image rendering module is used for rendering the motion vector information of the transition frame picture to generate transition frame picture information; The control chip is also used for inserting the transition frame picture information between the current frame picture information and the next frame picture information.

9. The hand-held device of claim 8, wherein, The first calculation module comprises: A matching module matches the plurality of first sub-picture information with the plurality of second sub-picture information to generate at least one matching information group, each matching information group comprises first sub-picture information and second sub-picture information matched successfully with the first sub-picture information; A determination module is used for determining the motion vector information of the first transition frame sub-picture information according to the first sub-picture information and the second sub-picture information in the matching information group, and determining the first motion vector information of the transition frame picture according to the motion vector information of at least one first transition frame sub-picture information.

10. The handheld device of claim 8, wherein, The handheld device further comprises a first inertial sensor arranged on the control handle, the first inertial sensor is used for detecting IMU data of the control handle, and an infrared lamp is arranged on the control handle; A first camera is arranged on the handheld device host, and the first camera is used for shooting the infrared lamp on the control handle to obtain a space image of a space where the control handle is located; The vector information determination unit further comprises: A second computing module, which is in communication connection with the first influence factor determining module, the first camera and the first inertial sensor respectively, and is configured to calculate the spatial positioning information of the control handle according to the spatial image of the space where the control handle is located and the IMU data of the control handle when the first influence factor includes the control handle; The integration module is in communication connection with the first computing module and the second computing module respectively, and is configured to determine the motion vector information of the transition frame according to the first motion vector information of the transition frame and the spatial positioning information of the control handle.

11. The handheld device of claim 8, wherein, The handheld device further comprises: A second inertial sensor arranged on the handheld device host, which is configured to detect the IMU data of the handheld device host; A second camera arranged on the handheld device host, which is configured to take a picture of the space where the handheld device host is located to obtain the spatial image of the space where the handheld device host is located; The vector information determining unit further comprises: A third computing module, which is in communication connection with the first influence factor determining module, the integration module, the second inertial sensor and the second camera respectively, and is configured to calculate the spatial positioning information of the handheld device host according to the IMU data of the handheld device host and the spatial image of the space where the handheld device host is located when the influence factor further includes the handheld device host; The integration module is in communication connection with the first computing module and the third computing module respectively, and is configured to determine the motion vector information of the transition frame according to the first motion vector information of the transition frame and the spatial positioning information of the handheld device host when the first influence factor includes the handheld device host.

12. The handheld device of claim 8, wherein, The handheld device is provided with a third camera, which is configured to take a picture of the motion trail information of the user of the handheld device; The vector information determining unit further comprises: A fourth computing module, which is in communication connection with the first influence factor determining module and the third camera, and is configured to obtain the motion trail information of the user of the handheld device taken by the third camera when the first influence factor includes the user of the handheld device; The integration module is in communication connection with the fourth computing module and the first computing module respectively, and is configured to determine the motion vector information of the transition frame according to the first motion vector information of the transition frame and the motion trail information of the user of the handheld device when the first influence factor includes the user of the handheld device.

Images