Image processing method and device, computer device, readable storage medium and program product

Abstract

The application relates to an image processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: determining a terminal user of a first communication terminal and a gaze focus position on a screen of the first communication terminal at a current time; generating a shooting instruction corresponding to a target real object; sending the shooting instruction to a second communication terminal connected to the first communication terminal in video communication, so that the second communication terminal controls a plurality of target cameras to shoot the target real object to obtain a plurality of shooting images of the target real object; performing image processing on the plurality of shooting images returned by the second communication terminal to obtain a target image of the target real object; and sending the target image to the first communication terminal, so that the first communication terminal displays the target image at the gaze focus position on the screen of the first communication terminal. The method can improve the display effect of an image aimed at by the gaze direction of a terminal user of a communication terminal.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of two-way video technology, real-time interaction and communication between two parties has become increasingly convenient, such as video conferencing and video calls.

[0003] In traditional technology, during two-way video communication, the image captured by one communication terminal is typically displayed on the screen of another communication terminal to achieve real-time interaction and communication. However, on the screen of the communication terminal, the image that the user is looking at may be too small, unclear, or have a poor shooting angle, resulting in a poor display effect of the image that the user is looking at. Summary of the Invention

[0004] Therefore, it is necessary to provide an image processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the display effect of the image in the direction of the user's gaze on the communication terminal, in order to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides an image processing method, comprising:

[0006] Determine the current gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal;

[0007] Generate a shooting command corresponding to the target object; the target object refers to the object on the screen of the first communication terminal that is associated with the gaze focus position;

[0008] The shooting command is sent to a second communication terminal that is connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured on the second communication terminal to shoot the target object and obtain multiple images of the target object;

[0009] The system receives the multiple captured images returned by the second communication terminal, performs image processing on the multiple captured images, and obtains a target image of the target object.

[0010] The target image is sent to the first communication terminal, so that the first communication terminal displays the target image at the eye focus position on the screen of the first communication terminal.

[0011] In one of the embodiments, the determining of the gaze focus position of the terminal user of the first communication terminal at the current time on the screen of the first communication terminal comprises:

[0012] receiving the image of the human eye of the terminal user sent by the first communication terminal; the first communication terminal is configured to capture the image of the human eye of the terminal user watching the screen of the first communication terminal at the current time through the human eye capturing camera;

[0013] identifying the image of the human eye to obtain the gaze direction of the terminal user when watching the screen of the first communication terminal;

[0014] determining the gaze focus position according to the gaze direction.

[0015] In one of the embodiments, before the generation of the shooting instruction corresponding to the target real object, further comprising:

[0016] identifying the image of the gaze focus position on the screen of the first communication terminal to obtain an image recognition result;

[0017] determining the real object associated with the gaze focus position according to the image recognition result;

[0018] taking the real object associated with the gaze focus position as the target real object.

[0019] In one of the embodiments, the determining of the real object associated with the gaze focus position according to the image recognition result comprises:

[0020] in the case that the image recognition result indicates that the real object in the image of the gaze focus position is successfully identified, taking the real object in the image of the gaze focus position as the real object associated with the gaze focus position;

[0021] in the case that the image recognition result indicates that the real object in the image of the gaze focus position is not successfully identified, taking the real object in the image in a preset area centered on the gaze focus position as the real object associated with the gaze focus position.

[0022] In one of the embodiments, the sending of the target image to the first communication terminal comprises:

[0023] in the case that the target real object is the real object in the image of the gaze focus position, sending the target image to the first communication terminal to make the first communication terminal display the target image at the position point of the gaze focus position on the screen of the first communication terminal;

[0024] In a case where the target real object is a real object in an image in a preset area centered on the eye focus position, the target image is sent to the first communication terminal, so that the first communication terminal displays the target image in a preset area centered on the eye focus position on a screen of the first communication terminal.

[0025] In one of the embodiments, the sending of the shooting instruction to the second communication terminal in video communication connection with the first communication terminal comprises:

[0026] The shooting instruction is sent to the second communication terminal in video communication connection with the first communication terminal, so that the second communication terminal adjusts shooting angles of a plurality of target cameras configured by the second communication terminal based on the shooting instruction, obtains a target shooting angle of each target camera for the target real object, and controls the each target camera to focus on and shoot the target real object according to the corresponding target shooting angle, to obtain a plurality of shooting images for the target real object.

[0027] In a second aspect, the present application further provides an image processing device, comprising:

[0028] A position determination module is configured to determine an eye focus position of a terminal user of the first communication terminal on a screen of the first communication terminal at a current time point;

[0029] An instruction generation module is configured to generate a shooting instruction corresponding to a target real object, the target real object representing a real object associated with the eye focus position on the screen of the first communication terminal;

[0030] An instruction sending module is configured to send the shooting instruction to a second communication terminal in video communication connection with the first communication terminal, so that the second communication terminal controls a plurality of target cameras configured by the second communication terminal to shoot the target real object, to obtain a plurality of shooting images for the target real object;

[0031] An image processing module is configured to receive the plurality of shooting images returned by the second communication terminal, and perform image processing on the plurality of shooting images, to obtain a target image for the target real object;

[0032] An image sending module is configured to send the target image to the first communication terminal, so that the first communication terminal displays the target image at the eye focus position on the screen of the first communication terminal.

[0033] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0034] determining a terminal user of a first communication terminal at a current time, and a gaze focus position on a screen of the first communication terminal;

[0035] generating a shooting instruction corresponding to a target object, the target object representing a real object associated with the gaze focus position on the screen of the first communication terminal;

[0036] sending the shooting instruction to a second communication terminal in video communication connection with the first communication terminal, so as to control a plurality of target cameras configured by the second communication terminal to shoot the target object, and obtain a plurality of shooting images for the target object;

[0037] receiving the plurality of shooting images returned by the second communication terminal, and performing image processing on the plurality of shooting images, to obtain a target image for the target object;

[0038] sending the target image to the first communication terminal, so as to display the target image at the gaze focus position on the screen of the first communication terminal.

[0039] In a fourth aspect, the present application further provides a computer readable storage medium, having a computer program stored thereon, the computer program being executed by a processor to implement the following steps:

[0040] determining a terminal user of a first communication terminal at a current time, and a gaze focus position on a screen of the first communication terminal;

[0041] generating a shooting instruction corresponding to a target object, the target object representing a real object associated with the gaze focus position on the screen of the first communication terminal;

[0042] sending the shooting instruction to a second communication terminal in video communication connection with the first communication terminal, so as to control a plurality of target cameras configured by the second communication terminal to shoot the target object, and obtain a plurality of shooting images for the target object;

[0043] receiving the plurality of shooting images returned by the second communication terminal, and performing image processing on the plurality of shooting images, to obtain a target image for the target object;

[0044] sending the target image to the first communication terminal, so as to display the target image at the gaze focus position on the screen of the first communication terminal.

[0045] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, the computer program being executed by a processor to implement the following steps:

[0046] determining a terminal user of the first communication terminal at the current time, and a gaze focus position on a screen of the first communication terminal;

[0047] generating a shooting instruction corresponding to a target object, the target object representing an object associated with the gaze focus position on the screen of the first communication terminal;

[0048] sending the shooting instruction to a second communication terminal in a video communication connection with the first communication terminal, so that the second communication terminal controls a plurality of target cameras configured by the second communication terminal to shoot the target object, and obtains a plurality of shooting images of the target object;

[0049] receiving the plurality of shooting images returned by the second communication terminal, and performing image processing on the plurality of shooting images to obtain a target image of the target object;

[0050] sending the target image to the first communication terminal, so that the first communication terminal displays the target image at the gaze focus position on the screen of the first communication terminal.

[0051] The image processing method, device, computer device, computer readable storage medium and computer program product determine a terminal user of the first communication terminal at the current time, and a gaze focus position on a screen of the first communication terminal. Then, a shooting instruction corresponding to a target object is generated. The target object represents an object associated with the gaze focus position on the screen of the first communication terminal. Then, the shooting instruction is sent to a second communication terminal in a video communication connection with the first communication terminal, so that the second communication terminal controls a plurality of target cameras configured by the second communication terminal to shoot the target object, and obtains a plurality of shooting images of the target object. Finally, the plurality of shooting images returned by the second communication terminal are received, and image processing is performed on the plurality of shooting images to obtain a target image of the target object. The target image is sent to the first communication terminal, so that the first communication terminal displays the target image at the gaze focus position on the screen of the first communication terminal. In this way, according to the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current time, a plurality of shooting images of the target object associated with the gaze focus position are obtained, and image processing is performed on the plurality of shooting images. The obtained target image is sent to the first communication terminal, so that the target image is displayed at the gaze focus position on the screen of the first communication terminal. Thus, the purpose of processing and then presenting the image corresponding to the gaze focus position of the terminal user based on the tracking result of the human eye is achieved, the display effect of the image focused by the terminal user is better, and the display effect of the image targeted by the gaze direction of the terminal user of the communication terminal is improved. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application or the related art. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0053] Figure 1 An application environment diagram of the image processing method in one embodiment;

[0054] Figure 2 A flowchart of the image processing method in one embodiment;

[0055] Figure 3 A flowchart of the step of determining the target object in one embodiment;

[0056] Figure 4 A flowchart of the image processing method in another embodiment;

[0057] Figure 5 A scene diagram of the naked eye 3D (three-dimensional) two-way video call in one embodiment;

[0058] Figure 6 A structural diagram of the communication terminal in one embodiment;

[0059] Figure 7 A diagram of the position A of the screen of the communication terminal a in one embodiment;

[0060] Figure 8 A diagram of the object W on the position A of the screen of the communication terminal a in one embodiment;

[0061] Figure 9 A diagram of the spherical space with the position A as the center and r as the radius on the screen of the communication terminal a in one embodiment;

[0062] Figure 10 A structural diagram of the RGB-D (RGB-Depth, color depth) camera holder in one embodiment;

[0063] Figure 11 A flowchart of the image processing method based on human eye tracking in one embodiment;

[0064] Figure 12 A structural block diagram of the image processing device in one embodiment;

[0065] Figure 13 An internal structural diagram of the computer device in one embodiment. DETAILED DESCRIPTION

[0066] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0067] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0068] At present, the general video conference system does not have the function of tracking the human eyes of the user, and the naked-eye 3D video conference system has the function of tracking the human eyes of the viewer, but the implementation is also based on the tracking result of the human eyes of the viewer, and the 3D picture of the corresponding angle is projected to the viewer to ensure that the viewer sees the correct angle image. It cannot be targeted to process the image according to the tracking result of the human eyes of the viewer, and the image focused by the user's eyes is enlarged, clearer, or better angle displayed to the user. Based on this, the present application provides an image processing method to improve the display effect of the image targeted by the gaze direction of the terminal user of the communication terminal, such as enlarging, making clearer, or better angle displaying the image focused by the user's eyes to the user.

[0069] The image processing method provided by the embodiments of the present application can be applied to the application environment as shown in Figure 1 The server 103 communicates with the first communication terminal 101 and the second communication terminal 102 through the network, respectively. Specifically, refer to Figure 1The server 103 determines the terminal user of the first communication terminal 101 at the current time, the gaze focus position on the screen of the first communication terminal 101, and generates a shooting instruction corresponding to the target real object; the target real object represents a real object on the screen of the first communication terminal associated with the gaze focus position; then the shooting instruction is sent to the second communication terminal 102 in video communication connection with the first communication terminal 101. The second communication terminal 102 controls a plurality of target cameras configured by the second communication terminal 102 to shoot the target real object, obtains a plurality of shooting images for the target real object, and sends the plurality of shooting images to the server 103. The server 103 receives the plurality of shooting images returned by the second communication terminal, and performs image processing on the plurality of shooting images to obtain a target image for the target real object, and then sends the target image to the first communication terminal 101. The first communication terminal 101 displays the target image at the gaze focus position on the screen of the first communication terminal 101. The first communication terminal 101 and the second communication terminal 102 can be, but are not limited to, various personal computers, notebook computers, smart phones and tablet computers. The server 103 can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0070] It should be noted that the present application can be applied to real-time video conference systems, including naked-eye 3D-based video conference systems, non-3D video conference systems, etc., and can also be applied to two-way real-time video application scenarios in non-conference scenarios, etc. The specific application is not limited.

[0071] In order to more clearly illustrate the image processing method provided by the embodiments of the present application, the following is an explanation of some terms:

[0072] RGB: Red, green, and blue are called three primary colors of light, which are represented in English as R (red), G (green), and B (blue).

[0073] RGB camera: An RGB camera, also known as a color camera, can capture the red, green, and blue three main color channels of an image simultaneously, thereby generating a complete color image.

[0074] RGB-D: RGB-D (RGB-Depth) technology is a three-dimensional vision sensing technology that combines RGB color images and depth images. It acquires three-dimensional information of a scene by capturing color images and depth information in the scene.

[0075] 3D: 3Dimensions, which is Chinese for three-dimensional, is a three-dimensional concept, that is, a space composed of X, Y, and Z axes, which is relative to a plane (2D) with only length and width.

[0076] Naked eye 3D display: a kind of visual experience technology without wearing any glasses or helmet and other auxiliary equipment, which allows the audience to directly watch the 3D image with stereoscopic effect with naked eyes.

[0077] In an exemplary embodiment, as shown in Figure 2 , an image processing method is provided, which is applied to the server in Figure 1 for example, and includes the following steps S201 to S205. Among them:

[0078] Step S201, determine the terminal user of the first communication terminal at the current time, and the gaze focus position on the screen of the first communication terminal.

[0079] Among them, the first communication terminal and the second communication terminal refer to two communication terminals participating in two-way video communication, which can specifically refer to two communication terminals participating in real-time video conference (including naked eye 3D-based video conference, non-3D video conference, etc.), or two communication terminals participating in non-video conference two-way real-time video call.

[0080] It should be noted that the present application is applicable to real-time video conference system, including naked eye 3D-based video conference system, non-3D video conference system, etc. For convenience of description, the present application is introduced by naked eye 3D two-way video call system. Figure 5 As shown in , user A and user B use communication terminal a (i.e. first communication terminal) and communication terminal b (i.e. second communication terminal) respectively to achieve the purpose of two-way remote naked eye 3D video call.

[0081] Figure 5 Among them, the structure of the first communication terminal and the second communication terminal is the same, and is configured with multiple RGB-D cameras, a human eye capture camera and a naked eye 3D display screen. For example, referring to Figure 6 , the communication terminal a and the communication terminal b used by A and B are exactly the same; Figure 6 As shown in , the structure of the communication terminal (such as communication terminal a and communication terminal b) is shown in the figure, in which

[0082] 1 refers to RGB-D camera, 2 refers to RGB-D camera, 3 refers to RGB-D camera, 4 refers to RGB-D camera, four cameras cooperate to complete the shooting of RGB, RGB-D pictures and videos; 5 refers to human eye capture camera, which is used to capture the eye movement of the viewer (such as user A and user B), and realize the eye movement tracking function; 6 is naked eye 3D display screen.

[0083] The gaze focus position refers to a gaze focus position of a terminal user of the first communication terminal on a screen of the first communication terminal at the current moment, specifically, a landing position of the gaze of the terminal user of the first communication terminal at the current moment on the screen of the first communication terminal. For example, referring to Figure 7 , the gaze of user A at t0 is focused on position A (x0, y0, z0) of the screen 6 of the communication terminal a, that is, the gaze of user A at t0 lands on position A of the screen 6 of the communication terminal a, indicating that user A is focusing on watching the image at position A at t0.

[0084] Exemplarily, the first communication terminal captures an image of the human eye of the terminal user who is currently watching the screen of the first communication terminal (such as the image of the human eye of user A) through a configured human eye capturing camera, and sends the captured image of the human eye to the server. The server identifies and determines the received image of the human eye based on a human eye tracking algorithm, and obtains the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment, such as Figure 7 position A shown.

[0085] Step S202, generating a shooting instruction corresponding to a target object; the target object refers to an object associated with the gaze focus position on the screen of the first communication terminal.

[0086] The shooting instruction refers to a high-definition shooting instruction, specifically, an instruction to clearly shoot the target object, such as an instruction to clearly shoot Figure 8 the image of the object W shown, Figure 9 an instruction to clearly shoot the image of the object in the spherical space with position A as the center and r as the radius.

[0087] The target object refers to an object associated with the gaze focus position on the screen of the first communication terminal, such as Figure 8 the object W shown, Figure 9 the object in the spherical space with position A as the center and r as the radius.

[0088] Exemplarily, after determining the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment, the server identifies the image of the gaze focus position on the screen of the first communication terminal, obtains the target object associated with the gaze focus position, and generates a shooting instruction corresponding to the target object.

[0089] For example, in the case where the target object is an object in the image of the gaze focus position, the server generates an instruction to clearly shoot the object in the image of the gaze focus position (such as an instruction to clearly shoot Figure 8the image in the preset region centered at the gaze focus position) as the shooting instruction; in the case where the target object is an object in the image in the preset region centered at the gaze focus position, the server generates an instruction (such as a shooting instruction) for shooting the object in the image in the preset region centered at the gaze focus position in a clear manner. Figure 9 the image of the object in the spherical space with the position A as the center and r as the radius.

[0090] In step S203, the shooting instruction is sent to the second communication terminal in video communication connection with the first communication terminal, so that the second communication terminal controls the plurality of target cameras configured by the second communication terminal to shoot the target object, and a plurality of shooting images of the target object are obtained.

[0091] In the embodiment, the first communication terminal is in video communication connection with the second communication terminal, which means that the terminal user of the first communication terminal is in video communication with the terminal user of the second communication terminal. For details, refer to Figure 5 .

[0092] In the embodiment, the target camera is an RGB-D camera, which is used to collect color images and depth information of the target object.

[0093] In the embodiment, before the plurality of target cameras shoot the target object, the shooting angles of the plurality of target cameras need to be adjusted to obtain the best shooting angle of each target camera. Subsequently, each target camera shoots the target object according to the corresponding best shooting angle.

[0094] In the embodiment, the plurality of shooting images of the target object are high-definition shooting images of the target object from different shooting angles.

[0095] For example, the server sends the shooting instruction to the second communication terminal in video communication connection with the first communication terminal. The second communication terminal adjusts the shooting angles of the plurality of target cameras configured by the second communication terminal according to the shooting instruction, and controls each target camera to shoot the target object according to the corresponding best shooting angle, so as to obtain a plurality of shooting images of the target object.

[0096] In step S204, the plurality of shooting images returned by the second communication terminal are received, and image processing is performed on the plurality of shooting images to obtain a target image of the target object.

[0097] In the embodiment, the plurality of shooting images returned by the second communication terminal are obtained based on the plurality of best shooting angles, so that the target image obtained by processing the plurality of shooting images can be better displayed to the user

[0098] The image processing refers to image fusion processing, image magnification processing, image super-resolution processing, etc., and aims to make the target image achieve the effects of magnification and clearer. In this way, according to the human eye tracking result of the viewer, the image can be processed in a targeted manner, the image focused by the user's eyes can be magnified, and the user can be shown more clearly or better angle.

[0099] Exemplarily, the second communication terminal sends the obtained multiple photographed images to the server, and the server performs a series of image processing on the received multiple photographed images to obtain a target image of the target real object. For example, the server performs fusion processing on the multiple photographed images to obtain a fusion image, and then performs magnification processing and / or super-resolution processing on the fusion image to obtain a processed image as the target image of the target real object.

[0100] In step S205, the target image is sent to the first communication terminal, so that the first communication terminal displays the target image at the eye focus position on the screen of the first communication terminal.

[0101] The target image displayed at the eye focus position on the screen of the first communication terminal can be displayed at the position point of the eye focus position on the screen of the first communication terminal, or can be displayed in a preset region centered on the eye focus position on the screen of the first communication terminal.

[0102] Exemplarily, the server sends the target image to the first communication terminal, and the first communication terminal displays the target image at the eye focus position on the screen of the first communication terminal after receiving the target image, such as displaying the target image at position A of the screen 6 of the communication terminal a as shown in FIG. 6, or displaying the target image in the spherical space with position A as the center and r as the radius as shown in FIG. 7. Figure 7 Figure 9 In this way, based on the human eye tracking result of the viewer, the image focused by the viewer's eyes is magnified, or provided more clearly or better angle, so that the viewer can see the real object image that the viewer wants to see more clearly, more clearly, and better angle, and the user experience is improved.

[0103] ​In the image processing method, a terminal user of the first communication terminal at a current time and a gaze focus position on a screen of the first communication terminal are determined first; then a shooting instruction corresponding to a target real object is generated, the target real object being a real object on the screen of the first communication terminal associated with the gaze focus position; the shooting instruction is then sent to a second communication terminal in video communication connection with the first communication terminal, so that the second communication terminal controls a plurality of target cameras configured by the second communication terminal to shoot the target real object to obtain a plurality of shooting images of the target real object; finally, the plurality of shooting images returned by the second communication terminal are received, the plurality of shooting images are processed to obtain a target image of the target real object, and the target image is sent to the first communication terminal, so that the target image is displayed on the gaze focus position on the screen of the first communication terminal. In this way, according to the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current time, a plurality of shooting images of the target real object associated with the gaze focus position are obtained, the plurality of shooting images are processed, and the obtained target image is sent to the first communication terminal to display the target image on the gaze focus position on the screen of the first communication terminal, thereby achieving the purpose of processing the image of the real object corresponding to the gaze focus position of the terminal user based on the tracking result of the human eye and then presenting it to the terminal user, which can improve the display effect of the image corresponding to the gaze direction of the terminal user of the communication terminal.

[0104] In an exemplary embodiment, the step S201 of determining the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current time specifically includes the following contents: receiving the human eye image of the terminal user sent by the first communication terminal; the first communication terminal is configured with a human eye capturing camera to shoot the human eye image of the terminal user watching the screen of the first communication terminal at the current time; the human eye image is identified to obtain the gaze direction of the terminal user watching the screen of the first communication terminal; and the gaze focus position is determined according to the gaze direction.

[0105] The human eye image of the terminal user sent by the first communication terminal is shot by the human eye capturing camera configured by the first communication terminal.

[0106] The human eye image of the terminal user sent by the first communication terminal is shot by the human eye capturing camera configured by the first communication terminal.

[0107] The gaze direction can be determined based on a human eye tracking algorithm.

[0108] The gaze focus position refers to a landing position of a gaze direction of a terminal user of the first communication terminal on the screen of the first communication terminal at the current time, such as Figure 7 the position A shown in FIG. 6.

[0109] Exemplarily, the first communication terminal captures the eyes of the terminal user who is currently watching the screen of the first communication terminal through the configured eye capturing camera, obtains an eye image of the terminal user who is currently watching the screen of the first communication terminal, and then sends the eye image to the server; the server identifies the eye image based on an eye tracking algorithm to obtain the gaze direction of the terminal user when watching the screen of the first communication terminal; finally, the landing position of the gaze direction of the terminal user when watching the screen of the first communication terminal at the current time on the screen of the first communication terminal is identified, and the landing position is confirmed as the gaze focus position, such as Figure 7 the position A shown in FIG. 6.

[0110] For example, at t0, the camera 5 of the communication terminal a captures the gaze direction of user A; according to the gaze direction of user A captured by the camera 5 of the communication terminal a, the system identifies and determines that user A's gaze falls on the position A of the screen 6 of the communication terminal a at t0, as shown in Figure 7 .

[0111] In this embodiment, the eye image of the terminal user who is currently watching the screen of the first communication terminal is captured through the configured eye capturing camera, and the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current time is determined based on the eye image, which is beneficial to subsequent processing of multiple shooting images of the target real object associated with the gaze focus position before the images are presented to the terminal user, thereby improving the display effect of the images targeted by the gaze direction of the terminal user of the communication terminal.

[0112] In one exemplary embodiment, as shown in Figure 3 before the shooting instruction corresponding to the target real object is generated, the step of determining the target real object is further included, specifically including the following steps S301 to S303. Wherein:

[0113] Step S301, identifying the image of the gaze focus position in the screen of the first communication terminal to obtain an image recognition result.

[0114] Step S302, determining the real object associated with the gaze focus position according to the image recognition result.

[0115] Step S303, taking the real object associated with the gaze focus position as the target real object.

[0116] The image recognition result is used to indicate what the object is in the image at the eye focus position, specifically including objects in images at the eye focus position that were successfully identified and objects in images at the eye focus position that were not successfully identified.

[0117] The object associated with the eye focus position is determined based on the image recognition results; different image recognition results correspond to different objects associated with the eye focus position.

[0118] For example, the server acquires an image of the screen of the first communication terminal and extracts the image of the eye focus position from the image. Then, it performs image recognition on the image of the eye focus position using a pre-trained image recognition model to obtain the corresponding image recognition result. Based on the image recognition result, it determines the physical object associated with the eye focus position and finally confirms the physical object associated with the eye focus position as the target physical object.

[0119] In this embodiment, by recognizing the image of the eye focus position on the screen of the first communication terminal, an image recognition result is obtained. Then, based on the image recognition result, the target object associated with the eye focus position is determined, thus achieving the effect of accurately recognizing the target object associated with the eye focus position. This is beneficial for generating shooting instructions corresponding to the target object in the future, and facilitates the control of multiple target cameras configured on the second communication terminal to capture multiple images of the target object.

[0120] In an exemplary embodiment, step S302 above, which determines the physical object associated with the gaze focus position based on the image recognition result, specifically includes the following: if the image recognition result indicates that the physical object in the image at the gaze focus position has been successfully identified, the physical object in the image at the gaze focus position is taken as the physical object associated with the gaze focus position; if the image recognition result indicates that the physical object in the image at the gaze focus position has not been successfully identified, the physical object in the image within a preset area centered on the gaze focus position is taken as the physical object associated with the gaze focus position.

[0121] Here, the objects in the image at the focal point of the gaze refer to the objects contained in the image at that focal point, such as... Figure 8 The actual object W shown.

[0122] The preset area centered on the gaze focus point can refer to a spherical space with the gaze focus point as the center and a preset value as the radius, for example... Figure 9 The spherical space shown is centered at position A and has a radius of r; it can also refer to a pre-defined circular area centered at the eye focus position, or a pre-defined range centered at the eye focus position.

[0123] Exemplarily, in a case where the image recognition result indicates that the real object in the image of the gaze focus position is successfully recognized, the server confirms the real object in the image of the gaze focus position as the real object associated with the gaze focus position, such as the real object W shown in FIG. 1; in a case where the image recognition result indicates that the real object in the image of the gaze focus position is not successfully recognized, the server acquires an image in a preset region centered at the gaze focus position in the screen of the first communication terminal, and identifies the image to obtain a real object in the image in the preset region centered at the gaze focus position, and finally confirms the real object in the image in the preset region centered at the gaze focus position as the real object associated with the gaze focus position, such as the real object in the preset region centered at the gaze focus position shown in FIG. 2. Figure 8 Figure 9

[0124] For example, referring to FIG. 1 and FIG. 2, Figure 11 , the system performs image recognition on the content at the position A (x0, y0, z0) of the screen 6 of the communication terminal a. If the system successfully recognizes that the position A (x0, y0, z0) is an image of the real object W, the system gives an instruction to the communication terminal b to clearly photograph the real object W. If the system does not successfully recognize what the image at the position A (x0, y0, z0) is, the system takes a spherical space on the imaging space of the communication terminal a with the position A (x0, y0, z0) as the center and r as the radius, and gives an instruction to the communication terminal b to clearly photograph the image in the spherical space.

[0125] In this embodiment, the corresponding real object associated with the gaze focus position is determined according to different image recognition results, which facilitates subsequent accurate processing of multiple photographed images of the target real object associated with the gaze focus position before being presented to the terminal user, and is beneficial to improve the image display effect of the gaze focus position.

[0126] In an exemplary embodiment, the step S205 of sending the target image to the first communication terminal specifically includes the following contents: in a case where the target real object is a real object in an image of the gaze focus position, the target image is sent to the first communication terminal, so that the first communication terminal displays the target image at a position point on the screen of the first communication terminal where the gaze focus position is located; in a case where the target real object is a real object in an image in a preset region centered at the gaze focus position, the target image is sent to the first communication terminal, so that the first communication terminal displays the target image in the preset region centered at the gaze focus position on the screen of the first communication terminal.

[0127] ​​Exemplarily, the server sends the target image to the first communication terminal; the first communication terminal determines the position point of the eye focus position on the screen of the first communication terminal in the case that the target real object is the real object in the image with the eye focus position as the focus position, and displays the received target image at the position point of the eye focus position on the screen of the first communication terminal, such as Figure 7 and Figure 8 , the enlarged and clear image of the real object W at position A is displayed. In addition, the first communication terminal determines the preset region with the eye focus position as the center on the screen of the first communication terminal in the case that the target real object is the real object in the image in the preset region with the eye focus position as the center, and displays the received target image in the preset region with the eye focus position as the center on the screen of the first communication terminal, such as Figure 9 , the enlarged and clear image of the real object in the image in the preset region with the eye focus position as the center is displayed in the spherical space with position A (x0, y0, z0) as the spherical center and r as the radius.

[0128] For example, referring to Figure 11 , the system displays the images captured by the camera 1, the camera 2, the camera 3 and the camera 4 of the communication terminal b at time t1 on the screen of the communication terminal a after processing; at this time, the image at position A (x0, y0, z0) looked by the user A of the communication terminal a becomes clear. Alternatively, the system displays the images captured by the camera 1, the camera 2, the camera 3 and the camera 4 of the communication terminal b at time t1 on the screen of the communication terminal a after processing; at this time, the image in the sphere with position A (x0, y0, z0) as the spherical center and r as the radius on the screen 6 of the communication terminal a becomes clear, and the image looked by the user A becomes clear.

[0129] In the embodiment, the target image is sent to the first communication terminal, and the first communication terminal displays the target image at the eye focus position on the screen of the first communication terminal, so that the display effect of the image focused by the user of the terminal is better, and the display effect of the image aimed at by the direction of the eye of the user of the communication terminal is improved.

[0130] In an exemplary embodiment, the step S203 of sending the shooting instruction to the second communication terminal in video communication connection with the first communication terminal specifically includes the following contents: sending the shooting instruction to the second communication terminal in video communication connection with the first communication terminal, so that the second communication terminal adjusts the shooting angle of each target camera based on the shooting instruction, obtains the target shooting angle of each target camera for the target real object, and controls each target camera to focus and shoot the target real object according to the corresponding target shooting angle, to obtain a plurality of shooting images for the target real object.

[0131] The target shooting angle refers to the optimal shooting angle.

[0132] Each target camera corresponds to a target shooting angle for the target real object.

[0133] When adjusting the shooting angle of each target camera of the second communication terminal, the shooting angle can be adjusted through the camera holder of each target camera, or adjusted through other means, which is not limited by the present application.

[0134] The camera holder can be an RGB-D camera holder. Referring to Figure 10 7 is the camera lens, 8 is the pitch rotation axis, and 9 is the left-right rotation axis. The communication terminal b controls the motor through the left-right rotation axis 9 to rotate the RGB-D camera lens 7 to the target position in the left-right direction, and then through the pitch rotation axis 8 to rotate the RGB-D camera lens 7 to the target position in the pitch direction. The two steps rotate the camera lens 7 to the final target position. This holder device ensures that the lenses 7 of the cameras 1, 2, 3, and 4 of the communication terminal b can be rotated to the final target position, so that each camera has the optimal shooting angle.

[0135] Exemplarily, the server sends a shooting instruction to the second communication terminal in video communication connection with the first communication terminal; the second communication terminal drives the camera holder of each target camera configured by the second communication terminal according to the shooting instruction, to adjust the shooting angle of each target camera through the camera holder of each target camera, to obtain the target shooting angle (i.e. the closest shooting angle) of each target camera for the target real object, and then controls each target camera to focus on the target real object according to the corresponding target shooting angle, and to shoot the target real object, to obtain multiple shooting images of the target real object.

[0136] For example, referring to Figure 11The system sends a clear shooting instruction of the real object W to the communication terminal b; the communication terminal b controls the gimbals of the camera 1, the camera 2, the camera 3 and the camera 4 to rotate respectively, so that the camera 1, the camera 2, the camera 3 and the camera 4 have the best shooting angle for shooting the real object W; then, the communication terminal b controls the camera 1, the camera 2, the camera 3 and the camera 4 to automatically focus respectively, and shoots a clear photo of the real object W at the time t1. Alternatively, the system sends a clear shooting instruction of the image in the ball to the communication terminal b; the communication terminal b controls the gimbals of the camera 1, the camera 2, the camera 3 and the camera 4 to rotate respectively, so that the camera 1, the camera 2, the camera 3 and the camera 4 have the best shooting angle for shooting the corresponding real object of the image in the ball; then, the communication terminal b controls the camera 1, the camera 2, the camera 3 and the camera 4 to automatically focus respectively, and shoots a clear photo of the corresponding real object of the image in the ball at the time t1.

[0137] In the embodiment, the shooting instruction is sent to the second communication terminal connected with the first communication terminal in video communication, so that the second communication terminal adjusts the shooting angles of the multiple target cameras configured by the second communication terminal based on the shooting instruction, and controls each target camera to focus and shoot the target real object according to the corresponding target shooting angle, to obtain multiple shooting images of the target real object. In this way, by adjusting the shooting angles and automatic focusing, the clarity of the shooting images can be higher and the angles can be better, which is beneficial to further improving the display effect of the image corresponding to the gaze direction of the terminal user of the communication terminal.

[0138] In one exemplary embodiment, as shown in Figure 4 Another image processing method is provided, which is applied to a server as an example and includes the following steps S401 to S410. Wherein:

[0139] Step S401, receiving a human eye image of a terminal user of a first communication terminal sent by the first communication terminal; the first communication terminal is configured to shoot the human eye image of the terminal user currently watching the screen of the first communication terminal by a human eye capturing camera.

[0140] Step S402, identifying the human eye image to obtain the gaze direction of the terminal user when watching the screen of the first communication terminal.

[0141] Step S403, determining the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal according to the gaze direction.

[0142] Step S404, identifying the image of the gaze focus position on the screen of the first communication terminal to obtain an image recognition result.

[0143] Step S405, according to the image recognition result, determine the real object associated with the gaze focus position.

[0144] Wherein, in the case that the image recognition result indicates that the real object in the image of the gaze focus position is successfully recognized, the server takes the real object in the image of the gaze focus position as the real object associated with the gaze focus position; in the case that the image recognition result indicates that the real object in the image of the gaze focus position is not successfully recognized, the server takes the real object in the image within the preset region centered on the gaze focus position as the real object associated with the gaze focus position.

[0145] Step S406, take the real object associated with the gaze focus position as the target real object.

[0146] Step S407, generate a shooting instruction corresponding to the target real object.

[0147] Step S408, send the shooting instruction to the second communication terminal in video communication connection with the first communication terminal, so that the second communication terminal controls the multiple target cameras configured by the second communication terminal to shoot the target real object, and obtains multiple shooting images of the target real object.

[0148] Wherein, the second communication terminal is configured to adjust the shooting angle of the multiple target cameras configured by the second communication terminal based on the shooting instruction, obtain the target shooting angle of each target camera for the target real object, and control each target camera to focus and shoot the target real object according to the corresponding target shooting angle, so as to obtain multiple shooting images of the target real object.

[0149] Step S409, receive the multiple shooting images returned by the second communication terminal, and perform image processing on the multiple shooting images to obtain a target image of the target real object.

[0150] Step S410, send the target image to the first communication terminal, so that the first communication terminal displays the target image at the gaze focus position on the screen of the first communication terminal.

[0151] Wherein, in the case that the target real object is the real object in the image of the gaze focus position, the server sends the target image to the first communication terminal, so that the first communication terminal displays the target image at the position point of the gaze focus position on the screen of the first communication terminal; in the case that the target real object is the real object in the image within the preset region centered on the gaze focus position, the server sends the target image to the first communication terminal, so that the first communication terminal displays the target image in the preset region centered on the gaze focus position on the screen of the first communication terminal.

[0152] In the image processing method, according to the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment, a plurality of shooting images of the target real object associated with the gaze focus position are acquired, the plurality of shooting images are processed, and the obtained target image is sent to the first communication terminal to display the target image at the gaze focus position on the screen of the first communication terminal, so as to realize the purpose of processing the image of the real object corresponding to the gaze focus position of the terminal user based on the human eye tracking result and then presenting the image to the terminal user, and the display effect of the image focused by the terminal user is better, thereby improving the display effect of the image corresponding to the gaze direction of the terminal user of the communication terminal.

[0153] In order to more clearly illustrate the image processing method provided by the embodiments of the present application, the image processing method is specifically described below with one embodiment. In an exemplary embodiment, referring to Figure 11 The present application also provides an image processing method based on human eye tracking, which magnifies, or provides more clearly, or provides at a more optimal angle, the image focused by the viewer based on the human eye tracking result of the viewer. The specific content is as follows:

[0154] At t0 moment, the camera 5 of the communication terminal a captures the gaze direction of user A. According to the gaze direction of user A captured by the camera 5, the system identifies and determines that the gaze of user A at t0 moment falls on the position A (x0, y0, z0) of the screen 6 of the communication terminal a.

[0155] The system performs image recognition on the content at the position A (x0, y0, z0) of the screen 6 of the communication terminal a, and if the system successfully identifies that the position A (x0, y0, z0) is the image of the real object W, the system gives the instruction of clearly shooting the real object W to the communication terminal b. The communication terminal b controls the gimbals of the camera 1, the camera 2, the camera 3 and the camera 4 to rotate respectively, so that the camera 1, the camera 2, the camera 3 and the camera 4 have the best shooting angle for shooting the real object W. Then, the communication terminal b controls the camera 1, the camera 2, the camera 3 and the camera 4 to focus automatically, and at t1 moment, the clear photos of the real object W are shot respectively. The system processes the images shot by the camera 1, the camera 2, the camera 3 and the camera 4 of the communication terminal b at t1 moment and displays them on the screen of the communication terminal a. At this time, the image at the position A (x0, y0, z0) looked by the gaze of user A of the communication terminal a becomes clear.

[0156] The system performs image recognition on the content at position A(x0, y0, z0) of the screen 6 of the communication terminal a. If the system fails to recognize what the image at position A(x0, y0, z0) is, the system takes a spherical space on the imaging space of the communication terminal a with position A(x0, y0, z0) as the center and r as the radius, and gives an instruction to the communication terminal b to clearly photograph the image in the sphere. The communication terminal b controls the gimbals of the camera 1, the camera 2, the camera 3, and the camera 4 to rotate respectively, so that the camera 1, the camera 2, the camera 3, and the camera 4 have the best shooting angle for photographing the corresponding real object in the sphere. Then, the communication terminal b controls the camera 1, the camera 2, the camera 3, and the camera 4 to automatically focus respectively, and photographs clear photos of the corresponding real object in the sphere at t1. The system displays the images photographed by the camera 1, the camera 2, the camera 3, and the camera 4 of the communication terminal b at t1 on the screen of the communication terminal a. At this time, the image in the sphere with position A(x0, y0, z0) as the center and r as the radius on the screen 6 of the communication terminal a is clear, and the image that the user A wants to see is clear.

[0157] It should be noted that the embodiment of the present application is a naked eye 3D video conference system, but the protection scope of the present application is not limited to the naked eye 3D video conference system, and any application that does not change the substantial content of the present application is within the protection scope.

[0158] It should be noted that in the present application, the camera is focused on the real object when the real object is recognized, and the camera is focused on the sphere space with the line of sight landing point as the center when the real object is not recognized, which is the core invention point of the present application, and any substantial change is within the protection scope of the present application.

[0159] It should be noted that in the embodiment of the present application, the camera is focused on the sphere space with the line of sight landing point as the center when the real object is not recognized, and in the 2D display application system, the camera is focused on the circular area with the line of sight landing point as the center, or the camera is focused on a certain area with the line of sight landing point as the center, and the like, which are application extensions that do not change the core idea of the present application, and are within the protection scope of the present application.

[0160] It should be noted that the camera of the image acquisition end in the embodiment of the present application uses a gimbal to rotate the camera to a suitable angle for shooting, but applications that change the direction of the camera in other ways are also within the protection scope of the present application.

[0161] It should be noted that the camera of the image acquisition end in the embodiment of the present application uses a gimbal, and if the embodiment is simple and does not need to use the gimbal to rotate the camera to a certain angle, or uses other similar devices to achieve the same rotation effect, the method does not change the substantial content of the present application and is also within the protection scope of the present application.

[0162] Compared with a video conference system without using the application, the image processing method based on human eye tracking can make the user focus more, see the real object image more clearly and from a better angle, create a sense of being on the scene for the user, and improve the user experience.

[0163] It should be understood that, although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0164] Based on the same inventive concept, the embodiments of the application also provide an image processing device for implementing the above-mentioned image processing method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more image processing device embodiments provided below can refer to the limitations of the image processing method described above, and will not be repeated here.

[0165] In one exemplary embodiment, as shown in Figure 12 An image processing device is provided, comprising: a position determination module 1210, an instruction generation module 1220, an instruction sending module 1230, an image processing module 1240, and an image sending module 1250, wherein:

[0166] The position determination module 1210 is configured to determine the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current time.

[0167] The instruction generation module 1220 is configured to generate a shooting instruction corresponding to a target real object; the target real object represents a real object associated with the gaze focus position on the screen of the first communication terminal.

[0168] The instruction sending module 1230 is configured to send the shooting instruction to a second communication terminal connected in video communication with the first communication terminal, so that the second communication terminal controls a plurality of target cameras configured by the second communication terminal to shoot the target real object to obtain a plurality of shooting images for the target real object.

[0169] The image processing module 1240 is configured to receive the plurality of captured images returned by the second communication terminal, and perform image processing on the plurality of captured images to obtain a target image of the target real object.

[0170] The image sending module 1250 is configured to send the target image to the first communication terminal, so that the first communication terminal displays the target image at the gaze focus position on the screen of the first communication terminal.

[0171] In an example embodiment, the position determining module 1210 is further configured to receive a human eye image of a terminal user sent by the first communication terminal, and the first communication terminal is configured to capture, by using a human eye capturing camera, a human eye image of a terminal user who is currently watching the screen of the first communication terminal at the current time, and perform identification on the human eye image to obtain a gaze direction of the terminal user when watching the screen of the first communication terminal, and determine the gaze focus position according to the gaze direction.

[0172] In an example embodiment, the image processing apparatus further includes a real object determining module configured to perform identification on an image of the gaze focus position on the screen of the first communication terminal to obtain an image identification result, determine a real object associated with the gaze focus position according to the image identification result, and take the real object associated with the gaze focus position as the target real object.

[0173] In an example embodiment, the real object determining module is further configured to, in a case where the image identification result indicates that the real object in the image of the gaze focus position is successfully identified, take the real object in the image of the gaze focus position as the real object associated with the gaze focus position, and in a case where the image identification result indicates that the real object in the image of the gaze focus position is not successfully identified, take a real object in an image in a preset region centered on the gaze focus position as the real object associated with the gaze focus position.

[0174] In an example embodiment, the image sending module 1250 is further configured to, in a case where the target real object is the real object in the image of the gaze focus position, send the target image to the first communication terminal so that the first communication terminal displays the target image at a position point on the screen of the first communication terminal where the gaze focus position is located, and in a case where the target real object is the real object in the image in the preset region centered on the gaze focus position, send the target image to the first communication terminal so that the first communication terminal displays the target image in the preset region centered on the gaze focus position on the screen of the first communication terminal.

[0175] In an example embodiment, the instruction sending module 1230 is further configured to send the shooting instruction to a second communication terminal in a video communication connection with the first communication terminal, and control the second communication terminal to adjust shooting angles of a plurality of target cameras configured by the second communication terminal based on the shooting instruction, to obtain a target shooting angle of each target camera for the target real object, and to control each target camera to focus and shoot the target real object according to the corresponding target shooting angle, to obtain a plurality of shooting images for the target real object.

[0176] The modules in the image processing apparatus can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in a computer device in software form, so as to be invoked and executed by a processor to perform operations corresponding to the modules.

[0177] In an example embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 13 The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store shooting images, target images, and the like. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement an image processing method.

[0178] Those skilled in the art can understand that Figure 13 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. A specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0179] In an example embodiment, a computer device is also provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0180] In an exemplary embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps of the above method embodiments.

[0181] In an exemplary embodiment, a computer program product is provided, which comprises a computer program. The computer program is executed by a processor to implement the steps of the above method embodiments.

[0182] A person of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the computer program can include the processes of the above method embodiments. Any reference to a memory, a database or other medium in each embodiment provided in the present application can include at least one of a non-volatile memory and a volatile memory. The non-volatile memory can include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical storage, a high-density embedded non-volatile memory, a resistive random access memory (ReRAM), a magnetoresistive random access memory (MRAM), a ferroelectric random access memory (FRAM), a phase change memory (PCM), a graphene memory, etc. The volatile memory can include a random access memory (RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), etc. The database involved in each embodiment provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., and is not limited thereto. The processor involved in each embodiment provided in the present application can be a general processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., and is not limited thereto.

[0183] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, any combination of these technical features is deemed to be within the scope of the present application.

[0184] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Determine the current gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal; Generate a shooting command corresponding to the target object; the target object refers to the object on the screen of the first communication terminal that is associated with the gaze focus position; The shooting command is sent to a second communication terminal that is connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured on the second communication terminal to shoot the target object and obtain multiple images of the target object; The system receives the multiple captured images returned by the second communication terminal, performs image processing on the multiple captured images, and obtains a target image of the target object. The target image is sent to the first communication terminal, so that the first communication terminal displays the target image at the eye focus position on the screen of the first communication terminal.

2. The method according to claim 1, characterized in that, Determining the current gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal includes: The first communication terminal receives an image of the user's eye sent by the first communication terminal; the first communication terminal is configured to capture an image of the user's eye as they are currently viewing the screen of the first communication terminal using a configured eye-capturing camera. The human eye image is identified to obtain the direction of the user's gaze when viewing the screen of the first communication terminal; The focal point of the gaze is determined based on the direction of the gaze.

3. The method according to claim 1, characterized in that, Before generating the shooting instructions corresponding to the target object, it also includes: The image of the eye focus position on the screen of the first communication terminal is identified to obtain the image recognition result; Based on the image recognition results, determine the physical object associated with the gaze focus position; The physical object associated with the focal point of the gaze is taken as the target physical object.

4. The method according to claim 3, characterized in that, The step of determining the physical object associated with the gaze focus position based on the image recognition result includes: If the image recognition result indicates that the object in the image at the gaze focus position has been successfully identified, the object in the image at the gaze focus position shall be regarded as the object associated with the gaze focus position. If the image recognition result indicates that the object in the image at the gaze focus position was not successfully identified, the object in the image within a preset area centered on the gaze focus position will be regarded as the object associated with the gaze focus position.

5. The method according to claim 4, characterized in that, Sending the target image to the first communication terminal includes: If the target object is the object in the image at the focal point of the gaze, the target image is sent to the first communication terminal, so that the first communication terminal displays the target image at the focal point of the gaze on the screen of the first communication terminal. When the target object is an object in an image within a preset area centered on the gaze focus position, the target image is sent to the first communication terminal, so that the first communication terminal displays the target image on its screen within the preset area centered on the gaze focus position.

6. The method according to any one of claims 1 to 5, characterized in that, Sending the shooting command to a second communication terminal connected to the first communication terminal via video communication includes: The shooting command is sent to a second communication terminal connected to the first communication terminal via video communication. Based on the shooting command, the second communication terminal adjusts the shooting angles of multiple target cameras configured on the second communication terminal to obtain the target shooting angle of each target camera for the target object. The second communication terminal then controls each target camera to focus on and shoot the target object according to the corresponding target shooting angle, thereby obtaining multiple images of the target object.

7. An image processing apparatus, characterized in that, The device includes: The location determination module is used to determine the current eye focus position of the terminal user of the first communication terminal on the screen of the first communication terminal. The instruction generation module is used to generate a shooting instruction corresponding to the target object; the target object refers to the object on the screen of the first communication terminal that is associated with the gaze focus position. The instruction sending module is used to send the shooting instruction to a second communication terminal that is connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured on the second communication terminal to shoot the target object and obtain multiple images of the target object; An image processing module is used to receive the multiple captured images returned by the second communication terminal, perform image processing on the multiple captured images, and obtain a target image of the target object. An image sending module is used to send the target image to the first communication terminal, so that the first communication terminal displays the target image at the eye focus position on the screen of the first communication terminal.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images