A control method and electronic device

Abstract

The application provides a control method, which comprises: collecting image data of a virtual object in a virtual scene with continuous time sequence; determining head posture data, eye feature data and a first line-of-sight angle parameter of the virtual object in a first coordinate system based on the image data; determining a second line-of-sight angle parameter of the virtual object based on the head posture data and the eye feature data; adjusting the first line-of-sight angle parameter based on the second line-of-sight angle parameter to obtain a third line-of-sight angle parameter; and controlling eyeball movement of the virtual object based on the third line-of-sight angle parameter. Meanwhile, the application also provides an electronic device.

Description

TECHNICAL FIELD

[0001] The present application relates to a control method and an electronic device. BACKGROUND

[0002] The realism of the visual action of a virtual object in a virtual scene in three-dimensional space is closely related to the quality of data collection and feature labeling, but the accuracy of feature labeling is currently low, resulting in poor realism of the virtual object's line of sight. SUMMARY

[0003] Therefore, the embodiments of the present application aim to provide a control method and an electronic device.

[0004] To achieve the above-mentioned purpose, the technical solution of the present application is as follows:

[0005] According to an aspect of the present application, a control method is provided, which comprises:

[0006] Collecting image data of a virtual object in a virtual scene with continuous time sequence;

[0007] Determining head pose data, eye feature data and a first line of sight angle parameter of the virtual object in a first coordinate system based on the image data;

[0008] Determining a second line of sight angle parameter of the virtual object based on the head pose data and the eye feature data;

[0009] Adjusting the first line of sight angle parameter based on the second line of sight angle parameter to obtain a third line of sight angle parameter;

[0010] Controlling the eyeball movement of the virtual object based on the third line of sight angle parameter.

[0011] In the above-mentioned solution, the determination of the second line of sight angle parameter of the virtual object based on the head pose data and the eye feature data comprises:

[0012] Determining eye socket pose data of the virtual object based on the head pose data;

[0013] Determining eyeball position parameters and eye corner opening angle parameters of the virtual object based on the eye feature data;

[0014] Determining the second line of sight angle parameter of the virtual object in the first coordinate system based on the eye socket pose data, the eyeball position parameters and the eye corner opening angle parameters.

[0015] In the scheme, the second line-of-sight angle parameter of the virtual object in the first coordinate system is determined based on the eye socket posture data, the eyeball position parameter, and the eye corner opening and closing angle parameter, including:

[0016] The first offset parameter of the virtual object in the first coordinate system is determined based on the eye socket posture data;

[0017] The second offset parameter of the virtual object in the first coordinate system is determined based on the eyeball position parameter and the eye corner opening and closing angle parameter;

[0018] The second line-of-sight angle parameter of the virtual object in the first coordinate system is determined based on the first offset parameter and the second offset parameter.

[0019] In the scheme, the first line-of-sight angle parameter is adjusted based on the second line-of-sight angle parameter to obtain a third line-of-sight angle parameter, including:

[0020] The first tilt angle parameter and the first yaw angle parameter corresponding to the second line-of-sight angle parameter are determined;

[0021] The second tilt angle parameter and the second yaw angle parameter corresponding to the second line-of-sight angle parameter are adjusted based on the first tilt angle parameter and the first yaw angle parameter to obtain the third line-of-sight angle parameter.

[0022] In the scheme, the second tilt angle parameter and the second yaw angle parameter corresponding to the second line-of-sight angle parameter are adjusted based on the first tilt angle parameter and the first yaw angle parameter to obtain the third line-of-sight angle parameter, including one of:

[0023] The second tilt angle parameter and the second yaw angle parameter corresponding to the second line-of-sight angle parameter are replaced by the first tilt angle parameter and the first yaw angle parameter to obtain the third line-of-sight angle parameter;

[0024] The first tilt angle parameter and the first yaw angle parameter are compared with the second tilt angle parameter and the second yaw angle parameter corresponding to the second line-of-sight angle parameter, and the invalid parameter in the second tilt angle parameter and the second yaw angle parameter is determined according to the comparison result; the invalid parameter is deleted to obtain the third line-of-sight angle parameter.

[0025] In the scheme, the first line-of-sight angle parameter is adjusted based on the second line-of-sight angle parameter to obtain a third line-of-sight angle parameter, further including:

[0026] The second line-of-sight angle parameter is compared with a fourth line-of-sight angle parameter corresponding to a real object in a real space;

[0027] correct the second line-of-sight angle parameter according to the comparison result;

[0028] adjust the first line-of-sight angle parameter based on the corrected second line-of-sight angle parameter to obtain a third line-of-sight angle parameter.

[0029] In the above solution, the second line-of-sight angle parameter includes a plurality of pixel point parameters; and the parameter correction of the second line-of-sight angle parameter according to the comparison result includes:

[0030] determining, according to the comparison result, a parameter error between an angle parameter corresponding to each pixel point in the second line-of-sight angle parameter and the fourth line-of-sight angle parameter;

[0031] deleting a target sub-parameter in the second line-of-sight angle parameter with a parameter error greater than or equal to a parameter threshold.

[0032] According to another aspect of the present application, an electronic device is provided, comprising:

[0033] an acquisition unit configured to acquire image data of a virtual object with continuous time sequence in a virtual scene;

[0034] a determination unit configured to determine, based on the image data, head posture data, eye feature data and a first line-of-sight angle parameter of the virtual object in a first coordinate system, and to determine, based on the head posture data and the eye feature data, a second line-of-sight angle parameter of the virtual object;

[0035] an adjustment unit configured to adjust the first line-of-sight angle parameter based on the second line-of-sight angle parameter to obtain a third line-of-sight angle parameter;

[0036] a control unit configured to control the eyeball movement of the virtual object based on the third line-of-sight angle parameter.

[0037] In the above solution, the determination unit is specifically configured to determine, based on the head posture data, eye socket posture data of the virtual object; determine, based on the eye feature data, eyeball position parameters and eye corner opening angle parameters of the virtual object; and determine, based on the eye socket posture data, the eyeball position parameters and the eye corner opening angle parameters, the second line-of-sight angle parameter of the virtual object in the first coordinate system.

[0038] According to a third aspect of the present application, an electronic device is provided, comprising a processor and a memory for storing a computer program capable of running on the processor,

[0039] When the processor runs the computer program, the processor performs the steps of the control method according to any one of the above solutions.

[0040] The control method and the electronic device provided in the application are a scheme for converting the shooting of a camera on a real scene or a real person into the shooting of a virtual scene or a virtual object. Since the high-fidelity scene or the high-fidelity person modeling constructed by computer simulation has very accurate geometric dimensions, the feature labeling of the virtual object is realized by collecting the image data of the virtual object in the virtual scene with continuous time sequence, the feature labeling accuracy is improved, and the line-of-sight action of the virtual object is more real. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 It is a flow implementation schematic diagram of the control method in the application;

[0042] Figure 2 It is a structural composition schematic diagram of the electronic device in the application Figure 1 ;

[0043] Figure 3 It is a structural composition schematic diagram of the electronic device in the application Figure 2 . DETAILED DESCRIPTION

[0044] To make the purpose, technical scheme and advantages of the application clearer, the technical scheme in the embodiments of the application will be described clearly and completely in combination with the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application. In the case of no conflict, the embodiments in the application and the features in the embodiments can be combined with each other at will. The steps shown in the flowchart of the drawings can be executed in a computer system such as a group of computer executable instructions. Moreover, although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0045] The technical scheme of the application will be further described in detail in combination with the drawings and specific embodiments in the specification.

[0046] Figure 1 It is a flow implementation schematic diagram of the control method in the application, which can be applied to an electronic device, such as a mobile phone, a computer, a tablet device, a game console, a virtual reality, an augmented reality, a mixed reality, a near-eye display device and the like. The method comprises:

[0047] Step 101, collecting image data of a virtual object in a virtual scene with continuous time sequence;

[0048] In the present application, the electronic device can be connected with the webcam in a wired or wireless manner, and through the connection, the virtual object can be used to collect image data with continuous time sequence in the virtual scene.

[0049] Here, the virtual object can be a high-fidelity person or a high-fidelity animal presented by the electronic device, and the virtual scene can be a high-fidelity scene presented by the electronic device.

[0050] For example, when a user uses a mobile phone to play a game, the mobile phone is connected with the webcam, and the high-fidelity digital person in the high-fidelity game scene can be used to collect image data by the webcam, so as to obtain scene data and action data with continuous time sequence of the high-fidelity digital person in the high-fidelity game scene. The scene data and action data with continuous time sequence can be used as training data of an artificial intelligence (AI) model in a real scene.

[0051] In step 102, head pose data, eye feature data, and a first line-of-sight angle parameter of the virtual object in a first coordinate system are determined based on the image data.

[0052] Here, since the line of sight of the eyeball of a real person changes with the position of the target object when the real person looks at the target object in a real scene, when the electronic device collects image data of the virtual object in a virtual scene by using the webcam and determines the viewing direction of the virtual object in a three-dimensional space by using the image data, not only the head pose of the virtual object but also the eye pose of the virtual object needs to be determined. Based on this, the electronic device can randomly extract part of the image data, and determine the head pose data, eye feature data, and first line-of-sight angle parameter of the virtual object in the first coordinate system based on the part of the image data.

[0053] Here, since the virtual object is a high-fidelity object fabricated by the electronic device, the geometric information between the limbs of the virtual object, the geometric information of the five features of the head, the geometric information of the pupil and iris of the eyeball, the skin color of the virtual object, and the pupil color are known, and the running relationship of the eyeball rotation or the head movement of the virtual object can be determined based on the known geometric information. Based on the running relationship, the head pose data, eye feature data, and first line-of-sight angle parameter of the virtual object in the first coordinate system can be determined.

[0054] For example, the two shoulders of the virtual object can be used as the coordinate origin, the angle parameter of the head relative to the two shoulders can be determined, and based on the angle parameter, the head pose data of the virtual object can be determined.

[0055] Here, the eye feature data includes, but is not limited to, a position parameter of eyeballs in eye sockets, and an opening angle parameter of eye corners.

[0056] Based on the head pose data and the eye feature data, a first line-of-sight angle parameter of the virtual object in the first coordinate system can be determined.

[0057] Here, the first coordinate system can refer to a coordinate system of the virtual object in a virtual scene space, such as a face coordinate system centered on a face of the virtual object.

[0058] At step 103, a second line-of-sight angle parameter of the virtual object is determined based on the head pose data and the eye feature data.

[0059] Here, after obtaining the head pose data and the eye feature data of the virtual object, the electronic device can determine eye socket pose data of the virtual object based on the head pose data; determine eyeball position parameters and eye corner opening angle parameters of the virtual object based on the eye feature data; and determine a second line-of-sight angle parameter of the virtual object in the first coordinate system based on the eye socket pose data, the eyeball position parameters, and the eye corner opening angle parameters.

[0060] Here, since the geometric positions of the facial features of the virtual object relative to the head are fixed and unchangeable, the electronic device can directly determine the head pose data as the eye socket pose data.

[0061] Here, since the geometric information of the eyeballs, pupils, and irises of the virtual object is fixed and unchangeable, when determining the eyeball position parameters and the eye corner opening angle parameters of the virtual object based on the eye feature data, the electronic device can obtain the geometric information of the eyeballs, pupils, and irises of the virtual object, and determine the eyeball position parameters and the eye corner opening angle parameters based on the geometric information of the eyeballs, pupils, and irises.

[0062] In this application, when determining the second line-of-sight angle parameter of the virtual object in the first coordinate system based on the eye socket pose data, the eyeball position parameters, and the eye corner opening angle parameters, the electronic device can determine a first offset parameter of the virtual object in the first coordinate system based on the eye socket pose data; determine a second offset parameter of the virtual object in the first coordinate system based on the eyeball position parameters and the eye corner opening angle parameters; and determine the second line-of-sight angle parameter of the virtual object in the first coordinate system based on the first offset parameter and the second offset parameter.

[0063] Here, the electronic device can take the head of the virtual object as the coordinate origin under the first coordinate system, or can take the shoulders of the virtual object as the coordinate origin under the first coordinate system. Taking the head of the virtual object as the coordinate origin as an example, if the eye socket posture data represents that the virtual object deviates 20 degrees to the right from the coordinate origin, the first offset parameter can be recorded as positive 20 degrees. If the eye position parameter and the eye corner opening angle parameter represent that the virtual object deviates 45 degrees to the left from the coordinate origin, the second offset parameter can be recorded as negative 45 degrees. Thus, the second line-of-sight angle parameter of the virtual object under the first coordinate system can be determined based on the first offset parameter and the second offset parameter.

[0064] In step 104, the first line-of-sight angle parameter is adjusted based on the second line-of-sight angle parameter to obtain a third line-of-sight angle parameter.

[0065] In this application, the electronic device can input the eye feature data of the virtual object as input data into the convolutional neural network CNN to obtain the first tilt angle parameter and the first yaw angle parameter corresponding to the second line-of-sight angle parameter; and then adjust the second tilt angle parameter and the second yaw angle parameter corresponding to the first line-of-sight angle parameter based on the first tilt angle parameter and the first yaw angle parameter to obtain the third line-of-sight angle parameter.

[0066] Here, the second tilt angle parameter and the second yaw angle parameter corresponding to the first line-of-sight angle parameter are the original tilt angle and the original yaw angle of the virtual object under the first line-of-sight angle parameter.

[0067] In an implementation manner, when the electronic device adjusts the second tilt angle parameter and the second yaw angle parameter corresponding to the first line-of-sight angle parameter based on the first tilt angle parameter and the first yaw angle parameter to obtain the third line-of-sight angle parameter, the first tilt angle parameter and the first yaw angle parameter can be used to replace the second tilt angle parameter and the second yaw angle parameter corresponding to the first line-of-sight angle parameter to obtain the third line-of-sight angle parameter.

[0068] In another implementation manner, when the electronic device adjusts the second tilt angle parameter and the second yaw angle parameter corresponding to the first line-of-sight angle parameter based on the first tilt angle parameter and the first yaw angle parameter to obtain the third line-of-sight angle parameter, the first tilt angle parameter and the first yaw angle parameter can be compared with the second tilt angle parameter and the second yaw angle parameter corresponding to the first line-of-sight angle parameter to obtain a comparison result, and the invalid parameter in the second tilt angle parameter and the second yaw angle parameter is determined according to the comparison result; and the invalid parameter in the second tilt angle parameter and the second yaw angle parameter is deleted to obtain the third line-of-sight angle parameter.

[0069] In the present application, when the electronic device adjusts the first line-of-sight angle parameter to obtain the third line-of-sight angle parameter according to the second line-of-sight angle parameter, the electronic device can further compare the second line-of-sight angle parameter with a fourth line-of-sight angle parameter corresponding to a real object in real space; perform parameter correction on the second line-of-sight angle parameter according to a comparison result; and adjust the first line-of-sight angle parameter based on the corrected second line-of-sight angle parameter to obtain the third line-of-sight angle parameter.

[0070] In the present application, the second line-of-sight angle parameter includes a plurality of pixel point parameters. When the electronic device performs parameter correction on the second line-of-sight angle parameter according to the comparison result, the electronic device can determine, according to the comparison result, a parameter error between an angle parameter corresponding to each pixel point in the second line-of-sight angle parameter and the fourth line-of-sight angle parameter; and delete a target sub-parameter in the second line-of-sight angle parameter whose parameter error is greater than or equal to a parameter threshold.

[0071] Here, when the electronic device adjusts the first line-of-sight angle parameter based on the corrected second line-of-sight angle parameter to obtain the third line-of-sight angle parameter, the electronic device can directly replace the first line-of-sight angle parameter with the corrected second line-of-sight angle parameter, so as to take the corrected second line-of-sight angle parameter as the third line-of-sight angle parameter.

[0072] Step 105: controlling the eyeball movement of the virtual object based on the third line-of-sight angle parameter.

[0073] In the present application, after obtaining the third line-of-sight angle parameter, the electronic device can label the line of sight of the virtual object in the virtual scene based on the third line-of-sight angle parameter, and control the eye movement of the virtual object by using the labeled third line-of-sight angle parameter.

[0074] Here, the line-of-sight labeling can be performed by using an automatic labeling method, a manual labeling method, or a combination of the automatic and manual labeling methods, which is not limited herein.

[0075] In the present application, the shooting of the camera for the real scene or for the real person is converted into the shooting of the high-fidelity scene or the high-fidelity virtual person. Since the high-fidelity person and the high-fidelity scene are high-fidelity scene models or high-fidelity person models constructed by a computer, the geometric information thereof is very accurate. Therefore, the line-of-sight parameter of the high-fidelity person in the three-dimensional space obtained by using the geometric information can improve the line-of-sight labeling accuracy and reduce the calculation complexity.

[0076] Figure 2 Structure of the electronic device in the present application Figure 1 The electronic device can be a mobile phone, a game console, a computer, a tablet device, an AR, a VR, an MR, or the like near-eye display device, such asFigure 2 The electronic device includes:

[0077] The acquisition unit 201 is configured to acquire image data of a virtual object having continuous time sequences in a virtual scene.

[0078] The determination unit 202 is configured to determine head pose data, eye feature data, and a first line-of-sight angle parameter of the virtual object in a first coordinate system based on the image data, and determine a second line-of-sight angle parameter of the virtual object based on the head pose data and the eye feature data.

[0079] The adjustment unit 203 is configured to adjust the first line-of-sight angle parameter based on the second line-of-sight angle parameter to obtain a third line-of-sight angle parameter.

[0080] The control unit 204 is configured to control eyeball movement of the virtual object based on the third line-of-sight angle parameter.

[0081] In a preferred implementation, the determination unit 202 is specifically configured to determine eye socket pose data of the virtual object based on the head pose data, determine eyeball position parameters and eye corner opening angle parameters of the virtual object based on the eye feature data, and determine the second line-of-sight angle parameter of the virtual object in the first coordinate system based on the eye socket pose data, the eyeball position parameters, and the eye corner opening angle parameters.

[0082] In a preferred implementation, the determination unit 202 is specifically configured to determine a first offset parameter of the virtual object in the first coordinate system based on the eye socket pose data, determine a second offset parameter of the virtual object in the first coordinate system based on the eyeball position parameters and the eye corner opening angle parameters, and determine the second line-of-sight angle parameter of the virtual object in the first coordinate system based on the first offset parameter and the second offset parameter.

[0083] In a preferred implementation, the determination unit 202 is further configured to determine a first tilt angle parameter and a first yaw angle parameter corresponding to the second line-of-sight angle parameter.

[0084] The adjustment unit 203 is specifically configured to adjust a second tilt angle parameter and a second yaw angle parameter corresponding to the first line-of-sight angle parameter based on the first tilt angle parameter and the first yaw angle parameter to obtain the third line-of-sight angle parameter.

[0085] In a preferred implementation, the electronic device further includes a replacement unit 205, a comparison unit 206, and a deletion unit 207. The replacement unit 205 is configured to replace the second tilt angle parameter and the second yaw angle parameter corresponding to the first line-of-sight angle parameter with the first tilt angle parameter and the first yaw angle parameter to obtain the third line-of-sight angle parameter.

[0086] The comparison unit 206 is configured to compare the first tilt angle parameter and the first yaw angle parameter with a second tilt angle parameter and a second yaw angle parameter corresponding to the first line-of-sight angle parameter, to obtain a comparison result.

[0087] The determination unit 202 is further configured to determine an invalid parameter from among the second tilt angle parameter and the second yaw angle parameter according to the comparison result.

[0088] The deletion unit 207 is configured to delete the invalid parameter, to obtain the third line-of-sight angle parameter.

[0089] In a preferred implementation, the electronic device further includes a correction unit 208.

[0090] Specifically, the comparison unit 206 is further configured to compare the second line-of-sight angle parameter with a fourth line-of-sight angle parameter corresponding to a real object in a real space, to obtain a comparison result.

[0091] The correction unit 208 is configured to correct the second line-of-sight angle parameter according to the comparison result.

[0092] The adjustment unit 203 is configured to adjust the first line-of-sight angle parameter based on the corrected second line-of-sight angle parameter, to obtain the third line-of-sight angle parameter.

[0093] In a preferred implementation, the second line-of-sight angle parameter includes a plurality of pixel point parameters.

[0094] The determination unit 202 is further configured to determine a parameter error between an angle parameter corresponding to each pixel point in the second line-of-sight angle parameter and the fourth line-of-sight angle parameter according to the comparison result.

[0095] The deletion unit 207 is further configured to delete a target sub-parameter in the second line-of-sight angle parameter, which has a parameter error greater than or equal to a parameter threshold.

[0096] It should be noted that the electronic device provided in the above embodiments is only exemplified by the division of the above program modules in the description, and in actual applications, the above processes can be completed by different program modules according to needs, that is, the internal structure of the electronic device is divided into different program modules to complete all or part of the above processes. In addition, the electronic device provided in the above embodiments and the process method provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0097] The electronic device provided in the above embodiments and the process method provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0098] The processor is configured to execute the computer program to perform any of the method steps of the control method.

[0099] Figure 3 is a structural component of the electronic device in the present application Figure 2 The electronic device 300 can be a terminal such as a mobile phone, a computer, a digital broadcast terminal, an information transmitting / receiving device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc. Figure 3 The electronic device 300 shown includes at least one processor 301, a memory 302, at least one network interface 304, and a user interface 303. The various components in the electronic device 300 are coupled together by a bus system 305. It can be understood that the bus system 305 is configured to enable communications between the components. The bus system 305 includes, in addition to a data bus, a power bus, a control bus, and a state signal bus. However, for the sake of clarity, only the bus system 305 is shown in Figure 3 various figures.

[0100] The user interface 303 can include a display, a keyboard, a mouse, a trackball, a click wheel, a key, a button, a touchpad, or a touch screen, etc.

[0101] It can be understood that the memory 302 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM can be used, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory 302 described in the embodiments of the present application is intended to include, but is not limited to, these and any other suitable type of memory.

[0102] The memory 302 in the embodiments of the present application is used to store various types of data to support the operation of the electronic device 300. Examples of these data include: any computer programs used for operation on the electronic device 300, such as an operating system 3021 and an application program 3022; contact data; phonebook data; messages; pictures; audio; and the like. The operating system 3021 contains various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application program 3022 can contain various application programs, such as a media player (Media Player), a browser (Browser), and the like, for implementing various application services. The program implementing the method of the embodiments of the present application can be contained in the application program 3022.

[0103] The method disclosed in the embodiments of the present application can be applied in the processor 301 or implemented by the processor 301. The processor 301 can be an integrated circuit chip having a processing capability. In the implementation process, each step of the above method can be completed by the integrated logic circuit or the instruction in the form of software in the processor 301. The processor 301 described above can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, and the like. The processor 301 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, and the like. In combination with the steps of the method disclosed in the embodiments of the present application, the above-mentioned method can be directly embodied as a hardware coding processor to execute, or be executed by a combination of hardware and software modules in the coding processor. The software module can be located in the storage medium, and the storage medium is located in the memory 302. The processor 301 reads the information in the memory 302 and combines the hardware to complete the steps of the above-mentioned method.

[0104] In an exemplary embodiment, the electronic device 300 can be implemented by one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field-Programmable Gate Arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors (Microprocessors), or other electronic elements for executing the aforementioned methods.

[0105] In an exemplary embodiment, the embodiments of the present application further provide a computer readable storage medium, for example, the memory 302 including a computer program, which can be executed by the processor 301 of the electronic device 300 to complete the steps of the aforementioned methods. The computer readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.; or can be various devices including one or any combination of the above memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0106] A computer readable storage medium having a computer program stored thereon, which, when executed by a processor, performs any of the steps of the above processing methods.

[0107] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection between devices or units, which can be electrical, mechanical or other forms.

[0108] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place or distributed on multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0109] The methods disclosed in the several method embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method embodiments.

[0110] The features disclosed in the several product embodiments provided by the present application can be combined arbitrarily without conflict to obtain new product embodiments.

[0111] The features disclosed in the several method or device embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method embodiments or device embodiments.

[0112] The above description is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A control method, the method comprising: acquiring image data of a virtual object having continuous timing in a virtual scene; determining head pose data, eye feature data and a first line-of-sight angle parameter of the virtual object in a first coordinate system based on the image data; the first line-of-sight angle parameter is used to represent an original line-of-sight angle parameter of the virtual object in the first coordinate system; the head pose data is used to determine eye socket pose data of the virtual object; and the eye feature data is used to determine eyeball position parameters and eye corner opening angle parameters of the virtual object; determining a second line-of-sight angle parameter of the virtual object based on the head pose data and the eye feature data; adjusting the first line-of-sight angle parameter based on the second line-of-sight angle parameter to obtain a third line-of-sight angle parameter; controlling eyeball movement of the virtual object based on the third line-of-sight angle parameter.

2. The method of claim 1, wherein, The determining of the second line-of-sight angle parameter of the virtual object based on the head pose data and the eye feature data comprises: determining eye socket pose data of the virtual object based on the head pose data; determining eyeball position parameters and eye corner opening angle parameters of the virtual object based on the eye feature data; determining the second line-of-sight angle parameter of the virtual object in the first coordinate system based on the eye socket pose data, the eyeball position parameters and the eye corner opening angle parameters.

3. The method of claim 2, wherein, The determining of the second line-of-sight angle parameter of the virtual object in the first coordinate system based on the eye socket pose data, the eyeball position parameters and the eye corner opening angle parameters comprises: determining a first offset parameter of the virtual object in the first coordinate system based on the eye socket pose data; determining a second offset parameter of the virtual object in the first coordinate system based on the eyeball position parameters and the eye corner opening angle parameters; determining the second line-of-sight angle parameter of the virtual object in the first coordinate system based on the first offset parameter and the second offset parameter.

4. The method of claim 1, wherein, The adjusting of the first line-of-sight angle parameter based on the second line-of-sight angle parameter to obtain a third line-of-sight angle parameter comprises: determining a first tilt angle parameter and a first yaw angle parameter corresponding to the second line-of-sight angle parameter; adjusting a second tilt angle parameter and a second yaw angle parameter corresponding to the first line-of-sight angle parameter based on the first tilt angle parameter and the first yaw angle parameter to obtain the third line-of-sight angle parameter.

5. The method of claim 4, wherein, The adjusting of the second tilt angle parameter and the second yaw angle parameter corresponding to the first line-of-sight angle parameter based on the first tilt angle parameter and the first yaw angle parameter to obtain the third line-of-sight angle parameter comprises one of: replacing the second tilt angle parameter and the second yaw angle parameter corresponding to the first line-of-sight angle parameter with the first tilt angle parameter and the first yaw angle parameter to obtain the third line-of-sight angle parameter; and comparing the first tilt angle parameter and the first yaw angle parameter with second tilt angle parameter and second yaw angle parameter corresponding to the first line-of-sight angle parameter, determining an invalid parameter in the second tilt angle parameter and the second yaw angle parameter according to a comparison result, and deleting the invalid parameter to obtain the third line-of-sight angle parameter.

6. The method of claim 1, wherein, The adjusting the first line-of-sight angle parameter based on the second line-of-sight angle parameter to obtain the third line-of-sight angle parameter further includes: comparing the second line-of-sight angle parameter with a fourth line-of-sight angle parameter corresponding to a real object in a real space; performing parameter correction on the second line-of-sight angle parameter according to a comparison result; adjusting the first line-of-sight angle parameter based on the corrected second line-of-sight angle parameter to obtain the third line-of-sight angle parameter.

7. The method of claim 6, wherein, The second line-of-sight angle parameter includes a plurality of pixel point parameters. The performing parameter correction on the second line-of-sight angle parameter according to a comparison result includes: determining a parameter error between an angle parameter corresponding to each pixel point in the second line-of-sight angle parameter and the fourth line-of-sight angle parameter according to the comparison result; and deleting a target sub-parameter in the second line-of-sight angle parameter with the parameter error greater than or equal to a parameter threshold. 8.An electronic device, comprising: an acquisition unit configured to acquire image data of a virtual object having continuous time sequences in a virtual scene; a determination unit configured to determine, based on the image data, head posture data, eye feature data, and a first line-of-sight angle parameter of the virtual object in a first coordinate system; the first line-of-sight angle parameter is used to represent an original line-of-sight angle parameter of the virtual object in the first coordinate system; the head posture data is used to determine eye socket posture data of the virtual object; the eye feature data is used to determine eyeball position parameters and eye corner opening angle parameters of the virtual object; and the determination unit is further configured to determine a second line-of-sight angle parameter of the virtual object based on the head posture data and the eye feature data; an adjustment unit configured to adjust the first line-of-sight angle parameter based on the second line-of-sight angle parameter to obtain a third line-of-sight angle parameter; a control unit configured to control movement of eyeballs of the virtual object based on the third line-of-sight angle parameter.

9. The electronic device of claim 8, wherein, The determination unit is specifically configured to determine eye socket posture data of the virtual object based on the head posture data; determine eyeball position parameters and eye corner opening angle parameters of the virtual object based on the eye feature data; and determine the second line-of-sight angle parameter of the virtual object in the first coordinate system based on the eye socket posture data, the eyeball position parameters, and the eye corner opening angle parameters.

10. An electronic device comprising: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor is configured to execute the computer program to perform the steps of the control method according to any one of claims 1 to 7.

Images