Image processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to an image processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: determining a sight focusing position of a terminal user of a first communication terminal on a screen of the first communication terminal at the current moment; generating a shooting instruction corresponding to the target object; the shooting instruction is sent to a second communication terminal in video communication connection with the first communication terminal, so that the second communication terminal controls a plurality of configured target cameras to shoot the target object, and a plurality of shot images for the target object are obtained; performing image processing on the plurality of shot images returned by the second communication terminal to obtain a target image for the target object; and sending the target image to the first communication terminal, so that the first communication terminal displays the target image at a sight focusing position on a screen of the first communication terminal. By adopting the method, the display effect of the image corresponding to the sight direction of the terminal user of the communication terminal can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of two-way video technology, real-time interaction and communication between two parties has become more and more convenient, such as video conferencing and video calls.

[0003] In conventional technology, during two-way video communication, the captured image of one communication terminal is generally displayed on the screen of another communication terminal to achieve real-time interaction and communication; however, on the screen of the communication terminal, the image in the direction of the gaze of the terminal user of the communication terminal may be too small, unclear, or have a bad shooting angle, resulting in poor display effect of the image in the direction of the gaze of the terminal user of the communication terminal. Summary of the invention

[0004] Based on this, it is necessary to provide an image processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve the display effect of the image directed by the terminal user of the communication terminal in order to solve the above technical problems.

[0005] In a first aspect, the present application provides an image processing method, comprising:

[0006] Determine a focus position of a terminal user of a first communication terminal on a screen of the first communication terminal at a current moment;

[0007] Generate a shooting instruction corresponding to a target object; the target object refers to a physical object associated with the gaze focus position on the screen of the first communication terminal;

[0008] Sending the shooting instruction to a second communication terminal connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured by the second communication terminal to shoot the target object, thereby obtaining multiple photographed images of the target object;

[0009] receiving the plurality of photographed images returned by the second communication terminal, and performing image processing on the plurality of photographed images to obtain a target image for the target object;

[0010] The target image is sent to the first communication terminal, so that the first communication terminal displays the target image at a focus position on the screen of the first communication terminal.

[0011] In one of the embodiments, determining the gaze focus position of the terminal user of the first communication terminal at the current moment on the screen of the first communication terminal includes:

[0012] receiving a human eye image of the terminal user sent by the first communication terminal; the first communication terminal is used to capture the human eye image of the terminal user who is currently viewing the screen of the first communication terminal through a configured human eye capture camera;

[0013] Recognize the human eye image to obtain the gaze direction of the terminal user when viewing the screen of the first communication terminal;

[0014] The gaze focus position is determined according to the gaze direction.

[0015] In one embodiment, before generating a shooting instruction corresponding to the target object, the method further includes:

[0016] Recognize the image of the gaze focus position on the screen of the first communication terminal to obtain an image recognition result;

[0017] Determining a physical object associated with the gaze focus position according to the image recognition result;

[0018] The physical object associated with the gaze focus position is used as the target physical object.

[0019] In one embodiment, determining the physical object associated with the gaze focus position according to the image recognition result includes:

[0020] When the image recognition result indicates that the real object in the image of the gaze focus position is successfully recognized, the real object in the image of the gaze focus position is used as the real object associated with the gaze focus position;

[0021] When the image recognition result indicates that the real object in the image of the gaze focus position is not successfully recognized, the real object in the image within a preset area centered on the gaze focus position is used as the real object associated with the gaze focus position.

[0022] In one embodiment, sending the target image to the first communication terminal includes:

[0023] In the case where the target object is a physical object in the image of the gaze focus position, the target image is sent to the first communication terminal, so that the first communication terminal displays the target image at the location of the gaze focus position on the screen of the first communication terminal;

[0024] When the target object is a physical object in an image within a preset area centered on the gaze focus position, the target image is sent to the first communication terminal, so that the first communication terminal displays the target image in the preset area centered on the gaze focus position on the screen of the first communication terminal.

[0025] In one embodiment, sending the shooting instruction to a second communication terminal connected to the first communication terminal via video communication includes:

[0026] The shooting instruction is sent to a second communication terminal connected to the first communication terminal by video communication, so that the second communication terminal adjusts the shooting angles of multiple target cameras configured by the second communication terminal based on the shooting instruction, obtains the target shooting angle of each target camera for the target object, and controls each target camera to focus and shoot the target object according to the corresponding target shooting angle, so as to obtain multiple captured images of the target object.

[0027] In a second aspect, the present application further provides an image processing device, comprising:

[0028] A position determination module, used to determine the gaze focus position of a terminal user of a first communication terminal on the screen of the first communication terminal at a current moment;

[0029] An instruction generation module, used to generate a shooting instruction corresponding to a target object; the target object refers to a physical object associated with the gaze focus position on the screen of the first communication terminal;

[0030] An instruction sending module is used to send the shooting instruction to a second communication terminal connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured by the second communication terminal to shoot the target object, thereby obtaining multiple shot images of the target object;

[0031] An image processing module, configured to receive the plurality of photographed images returned by the second communication terminal, and perform image processing on the plurality of photographed images to obtain a target image for the target object;

[0032] The image sending module is used to send the target image to the first communication terminal, so that the first communication terminal displays the target image at a focus position on the screen of the first communication terminal.

[0033] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0034] Determine a focus position of a terminal user of a first communication terminal on a screen of the first communication terminal at a current moment;

[0035] Generate a shooting instruction corresponding to a target object; the target object refers to a physical object associated with the gaze focus position on the screen of the first communication terminal;

[0036] Sending the shooting instruction to a second communication terminal connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured by the second communication terminal to shoot the target object, thereby obtaining multiple photographed images of the target object;

[0037] receiving the plurality of photographed images returned by the second communication terminal, and performing image processing on the plurality of photographed images to obtain a target image for the target object;

[0038] The target image is sent to the first communication terminal, so that the first communication terminal displays the target image at a focus position on the screen of the first communication terminal.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0040] Determine a focus position of a terminal user of a first communication terminal on a screen of the first communication terminal at a current moment;

[0041] Generate a shooting instruction corresponding to a target object; the target object refers to a physical object associated with the gaze focus position on the screen of the first communication terminal;

[0042] Sending the shooting instruction to a second communication terminal connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured by the second communication terminal to shoot the target object, thereby obtaining multiple photographed images of the target object;

[0043] receiving the plurality of photographed images returned by the second communication terminal, and performing image processing on the plurality of photographed images to obtain a target image for the target object;

[0044] The target image is sent to the first communication terminal, so that the first communication terminal displays the target image at a focus position on the screen of the first communication terminal.

[0045] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0046] Determine a focus position of a terminal user of a first communication terminal on a screen of the first communication terminal at a current moment;

[0047] Generate a shooting instruction corresponding to a target object; the target object refers to a physical object associated with the gaze focus position on the screen of the first communication terminal;

[0048] Sending the shooting instruction to a second communication terminal connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured by the second communication terminal to shoot the target object, thereby obtaining multiple photographed images of the target object;

[0049] receiving the plurality of photographed images returned by the second communication terminal, and performing image processing on the plurality of photographed images to obtain a target image for the target object;

[0050] The target image is sent to the first communication terminal, so that the first communication terminal displays the target image at a focus position on the screen of the first communication terminal.

[0051] The above-mentioned image processing method, device, computer equipment, computer-readable storage medium and computer program product first determine the focus position of the terminal user of the first communication terminal at the current moment on the screen of the first communication terminal; then generate a shooting instruction corresponding to the target object; the target object represents the object associated with the focus position on the screen of the first communication terminal; then send the shooting instruction to the second communication terminal connected to the first communication terminal by video communication, so that the second communication terminal controls multiple target cameras configured by the second communication terminal to shoot the target object, and obtain multiple captured images of the target object; finally, receive the multiple captured images returned by the second communication terminal, perform image processing on the multiple captured images, obtain the target image of the target object, and send the target image to the first communication terminal, so that the first communication terminal displays the target image at the focus position on the screen of the first communication terminal. In this way, according to the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment, multiple captured images of the target object associated with the gaze focus position are obtained in a targeted manner, and image processing is performed on the multiple captured images. The obtained target image is then sent to the first communication terminal to display the target image at the gaze focus position on the screen of the first communication terminal, thereby achieving the purpose of processing the image of the real object corresponding to the gaze focus position of the terminal user based on the human eye tracking result and then presenting it to the terminal user, which can make the display effect of the image focused by the terminal user's gaze better, thereby improving the display effect of the image targeted by the gaze direction of the terminal user of the communication terminal. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 A diagram showing an application environment of an image processing method in an embodiment;

[0054] Figure 2 is a schematic flow chart of an image processing method in one embodiment;

[0055] Figure 3 A schematic diagram of a flow chart of steps for determining a target object in one embodiment;

[0056] Figure 4 is a schematic flow chart of an image processing method in another embodiment;

[0057] Figure 5 A scene diagram of a naked-eye 3D (three-dimensional) two-way video call in one embodiment;

[0058] Figure 6 is a schematic diagram of the structure of a communication terminal in an embodiment;

[0059] Figure 7 A schematic diagram of a position A of a screen of a communication terminal a in one embodiment;

[0060] Figure 8 is a schematic diagram of a physical object W at position A on the screen of a communication terminal a in one embodiment;

[0061] Fig. 9 A schematic diagram of a spherical space with position A as the center and r as the radius on the screen of a communication terminal a in one embodiment;

[0062] Fig.10 Schematic diagram of the structure of an RGB-D (RGB-Depth, color depth) camera gimbal in one embodiment;

[0063] Fig.11 A schematic diagram of a flow chart of an image processing method based on human eye tracking in one embodiment;

[0064] Fig.12 is a structural block diagram of an image processing device in one embodiment;

[0065] Fig.13 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0068] At present, ordinary video conferencing systems do not have user-based eye tracking functions. Naked-eye 3D video conferencing systems have viewer eye tracking functions, but they also implement 3D images of corresponding angles to viewers based on viewer eye tracking results to ensure that viewers see images at the correct angles. They cannot process images in a targeted manner based on viewer eye tracking results to enlarge, make clearer, or show the images focused on by the user's eyes to the user at a better angle. Based on this, the present application proposes an image processing method to improve the display effect of the image targeted by the gaze direction of the terminal user of the communication terminal, such as enlarging, making clearer, or showing the images focused on by the user's eyes to the user at a better angle.

[0069] The image processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown in FIG. 1 , the server 103 communicates with the first communication terminal 101 and the second communication terminal 102 through the network. Specifically, refer to Figure 1, the server 103 determines the focus position of the terminal user of the first communication terminal 101 on the screen of the first communication terminal 101 at the current moment, and generates a shooting instruction corresponding to the target object; the target object represents the object associated with the focus position on the screen of the first communication terminal; and then sends the shooting instruction to the second communication terminal 102 connected to the first communication terminal 101 via video communication. The second communication terminal 102 controls multiple target cameras configured by the second communication terminal 102 to shoot the target object, obtain multiple captured images of the target object, and send the multiple captured images to the server 103. The server 103 receives the multiple captured images returned by the second communication terminal, performs image processing on the multiple captured images, obtains the target image of the target object, and then sends the target image to the first communication terminal 101. The first communication terminal 101 displays the target image at the focus position on the screen of the first communication terminal 101. Among them, the first communication terminal 101 and the second communication terminal 102 can be, but are not limited to, various personal computers, laptops, smart phones and tablet computers. The server 103 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0070] It should be noted that the present application can be applied to real-time video conferencing systems, including video conferencing systems based on naked-eye 3D, non-3D video conferencing systems, etc. It can also be applied to two-way real-time video application scenarios in non-conference scenarios, etc., and this application does not make any specific limitations.

[0071] In order to more clearly explain the image processing method provided in the embodiment of the present application, some terms are explained as follows:

[0072] RGB: Red, green, and blue are called the three primary colors of light, which are expressed in English as R (red), G (green), and B (blue).

[0073] RGB camera: RGB camera, also known as color camera, can simultaneously capture the three main color channels of red, green and blue of an image to generate a complete color image.

[0074] RGB-D: RGB-D (RGB-Depth) technology is a three-dimensional visual sensing technology that combines RGB color images and depth images. It obtains three-dimensional information of the scene by collecting color images and depth information in the scene.

[0075] 3D: The abbreviation of 3Dimensions, which means three-dimensional in Chinese. 3D is the concept of space, that is, the space composed of three axes of X, Y, and Z, relative to the plane (2D) which has only length and width.

[0076] Naked-eye 3D display: a visual experience technology that does not require wearing any glasses, helmets or other auxiliary equipment. It allows viewers to watch three-dimensional 3D images directly with their naked eyes.

[0077] In an exemplary embodiment, Figure 2 As shown, an image processing method is provided, which is applied to Figure 1 The server in the example is used to illustrate, including the following steps S201 to S205. Among them:

[0078] Step S201: determining the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment.

[0079] Among them, the first communication terminal and the second communication terminal refer to two communication terminals participating in two-way video communication, specifically, they may refer to two communication terminals participating in a real-time video conference (including a video conference based on naked-eye 3D, a non-3D video conference, etc.), or they may refer to two communication terminals participating in a two-way real-time video call of a non-video conference.

[0080] It should be noted that the present application is applicable to real-time video conferencing systems, including video conferencing systems based on naked-eye 3D, non-3D video conferencing systems, etc. For the convenience of explanation, the present application is introduced as a naked-eye 3D two-way video call system. Figure 5 The figure shows a naked-eye 3D two-way video call scenario, where user A and user B use communication terminal a (i.e., the first communication terminal) and communication terminal b (i.e., the second communication terminal) respectively to achieve the purpose of a remote naked-eye 3D two-way video call between the two.

[0081] The first communication terminal and the second communication terminal have the same structure, both of which are equipped with multiple RGB-D cameras, a human eye capture camera, and a naked eye 3D display. Figure 5 , the communication terminals a and b used by A and B are exactly the same; Figure 6 FIG. 1 is a schematic diagram of the structure of a communication terminal (such as communication terminal a and communication terminal b). Figure 6 In the figure, 1 refers to RGB-D camera, 2 refers to RGB-D camera, 3 refers to RGB-D camera, and 4 refers to RGB-D camera. The four cameras work together to complete the shooting of RGB, RGB-D pictures and videos; 5 refers to the human eye capture camera, which is used to capture the eye movements of the viewer (such as user A and user B) to realize the eye tracking function; 6 is the naked eye 3D display screen.

[0082] The terminal user of the first communication terminal refers to a user currently using the first communication terminal.

[0083] The gaze focus position refers to the gaze position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment, specifically refers to the point where the gaze of the terminal user of the first communication terminal falls on the screen of the first communication terminal at the current moment; for example, refer to Figure 7 , user A's gaze is focused on position A (x0, y0, z0) of the screen 6 of communication terminal a at time t0, that is, user A's gaze falls on position A of the screen 6 of communication terminal a at time t0, indicating that user A is concentrating on watching the image at position A at time t0.

[0084] Exemplarily, the first communication terminal uses a configured eye-capturing camera to capture an eye image of a terminal user who is currently viewing the screen of the first communication terminal (e.g., an eye image of user A), and sends the captured eye image to the server. The server identifies and determines the received eye image based on an eye-tracking algorithm, and obtains the eye focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment, such as Figure 7 Position A shown.

[0085] Step S202, generating a shooting instruction corresponding to the target object; the target object refers to the object associated with the gaze focus position on the screen of the first communication terminal.

[0086] Among them, the shooting command refers to the high-definition shooting command, specifically, the command to clearly shoot the target object, such as Figure 8 Instructions for taking a clear shot of the image of the object W shown Fig. 9 The instructions shown are for an image of a real object in a spherical space with position A as the center and r as the radius.

[0087] The target object refers to the object on the screen of the first communication terminal that is associated with the focus position, such as Figure 8 The actual object shown is W, Fig. 9 The shown figure is the real object in the spherical space with position A as the center and r as the radius.

[0088] Exemplarily, after determining the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment, the server identifies the image of the gaze focus position on the screen of the first communication terminal, obtains the target object associated with the gaze focus position, and generates a shooting instruction corresponding to the target object.

[0089] For example, when the target object is an object in the image at the focus position, the server generates an instruction to clearly capture the object in the image at the focus position (eg, clearly capture the object). Figure 8In the case where the target object is a physical object in the image within the preset area centered on the focus position, the server generates an instruction to clearly capture the physical object in the image within the preset area centered on the focus position (for example, clearly capture Fig. 9 The instructions for generating an image of a real object in a spherical space with position A as the center and r as the radius) are shown.

[0090] Step S203, sending a shooting instruction to a second communication terminal connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured by the second communication terminal to shoot the target object, and obtains multiple captured images of the target object.

[0091] The first communication terminal is connected to the second communication terminal via video communication, indicating that the terminal user of the first communication terminal is communicating with the terminal user of the second communication terminal via video communication. Figure 5 .

[0092] Among them, the target camera refers to an RGB-D camera, which is used to collect color images and depth information of the target object.

[0093] Before multiple target cameras shoot the target object, it is necessary to adjust the shooting angles of the multiple target cameras to obtain the best shooting angle for each target camera; subsequently, each target camera shoots the target object according to the corresponding best shooting angle.

[0094] The multiple photographed images of the target object refer to high-definition photographed images of the target object at different shooting angles.

[0095] Exemplarily, the server sends a shooting instruction to a second communication terminal connected to the first communication terminal by video communication; the second communication terminal adjusts the shooting angles of multiple target cameras configured in the second communication terminal according to the shooting instruction, and controls each target camera to shoot the target object at the corresponding optimal shooting angle, thereby obtaining multiple captured images of the target object.

[0096] Step S204: receiving a plurality of photographed images returned by the second communication terminal, performing image processing on the plurality of photographed images, and obtaining a target image for the target object.

[0097] The multiple images returned by the second communication terminal are obtained based on multiple optimal shooting angles, so that the target image obtained after processing the multiple images can be displayed to the user at a better angle.

[0098] Image processing refers to image fusion processing, image magnification processing, image super-resolution processing, etc., the purpose of which is to magnify and make the target image clearer. In this way, according to the viewer's eye tracking results, the image can be processed in a targeted manner, and the image focused by the user's eyes can be magnified and displayed to the user more clearly or at a better angle.

[0099] Exemplarily, the second communication terminal sends the obtained multiple captured images to the server, and the server performs a series of image processing on the received multiple captured images to obtain a target image for the target object; for example, the server fuses the multiple captured images to obtain a fused image, and then enlarges and / or super-resolution the fused image to obtain a processed image as the target image for the target object.

[0100] Step S205: sending the target image to the first communication terminal, so that the first communication terminal displays the target image at the focus position on the screen of the first communication terminal.

[0101] Among them, displaying the target image at the focus position on the screen of the first communication terminal may refer to displaying the target image at the point where the focus position is located on the screen of the first communication terminal, or may refer to displaying the target image in a preset area centered on the focus position on the screen of the first communication terminal.

[0102] Exemplarily, the server sends the target image to the first communication terminal; after receiving the target image, the first communication terminal displays the target image at a focus position on the screen of the first communication terminal, such as at Figure 7 The target image is displayed at position A of the communication terminal a screen 6 as shown, or at Fig. 9 The target image is displayed in a spherical space with position A as the center and r as the radius. In this way, based on the viewer's eye tracking results, the image focused by the viewer is enlarged, or provided to the viewer in a clearer or better angle, so that the viewer can see the real image he wants to see more clearly, more focused, and at a better angle, creating an immersive experience for the user, thereby improving the user experience.

[0103] In the above-mentioned image processing method, the focus position of the terminal user of the first communication terminal at the current moment on the screen of the first communication terminal is first determined; then a shooting instruction corresponding to the target object is generated; the target object represents the object associated with the focus position on the screen of the first communication terminal; then the shooting instruction is sent to a second communication terminal connected to the first communication terminal by video communication, so that the second communication terminal controls multiple target cameras configured by the second communication terminal to shoot the target object, and obtains multiple captured images of the target object; finally, the multiple captured images returned by the second communication terminal are received, the multiple captured images are processed to obtain a target image for the target object, and the target image is sent to the first communication terminal, so that the first communication terminal displays the target image at the focus position on the screen of the first communication terminal. In this way, according to the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment, multiple captured images of the target object associated with the gaze focus position are obtained in a targeted manner, and image processing is performed on the multiple captured images. The obtained target image is then sent to the first communication terminal to display the target image at the gaze focus position on the screen of the first communication terminal, thereby achieving the purpose of processing the image of the real object corresponding to the gaze focus position of the terminal user based on the human eye tracking result and then presenting it to the terminal user, which can make the display effect of the image focused by the terminal user's gaze better, thereby improving the display effect of the image targeted by the gaze direction of the terminal user of the communication terminal.

[0104] In an exemplary embodiment, the above step S201, determining the gaze focus position of the terminal user of the first communication terminal at the current moment on the screen of the first communication terminal, specifically includes the following contents: receiving the human eye image of the terminal user sent by the first communication terminal; the first communication terminal is configured to capture the human eye image of the terminal user who is currently watching the screen of the first communication terminal through a human eye capture camera; identifying the human eye image to obtain the gaze direction of the terminal user when watching the screen of the first communication terminal; and determining the gaze focus position according to the gaze direction.

[0105] The human eye image of the terminal user sent by the first communication terminal is obtained by taking pictures with a human eye capturing camera configured by the first communication terminal.

[0106] The human eye image of the terminal user sent by the first communication terminal refers to the human eye image of the terminal user who is currently watching the screen of the first communication terminal, captured by a human eye capture camera configured by the first communication terminal.

[0107] The gaze direction may be determined based on an eye tracking algorithm.

[0108] The gaze focus position refers to the position of the gaze direction of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment, for example Figure 7 Position A shown.

[0109] Exemplarily, the first communication terminal uses a configured eye-capturing camera to capture the eyes of a terminal user who is currently viewing the screen of the first communication terminal, obtains an eye image of the terminal user who is currently viewing the screen of the first communication terminal, and then sends the eye image to a server; the server identifies the eye image based on an eye tracking algorithm to obtain the gaze direction of the terminal user when viewing the screen of the first communication terminal; finally, the gaze direction of the terminal user when viewing the screen of the first communication terminal at the current moment is identified, and the landing position on the screen of the first communication terminal is identified, and the landing position is confirmed as the gaze focus position, such as Figure 7 Position A shown.

[0110] For example, at time t0, the camera 5 of the communication terminal a captures the gaze direction of the user A; based on the gaze direction of the user A captured by the camera 5 of the communication terminal a, the system recognizes and determines that the gaze of the user A falls on the position A of the screen 6 of the communication terminal a at time t0. For details, refer to Figure 7 .

[0111] In this embodiment, a human eye capture camera is configured to capture the human eye image of the terminal user who is currently viewing the screen of the first communication terminal, and based on the human eye image, the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal is determined at the current moment, which is conducive to subsequent targeted processing of multiple captured images of the target object associated with the gaze focus position before presenting them to the terminal user, thereby improving the display effect of the image targeted by the gaze direction of the terminal user of the communication terminal.

[0112] In an exemplary embodiment, Figure 3 As shown, before generating the shooting instruction corresponding to the target object, a step of determining the target object is also included, which specifically includes the following steps S301 to S303. Among them:

[0113] Step S301: Recognize the image of the focus position on the screen of the first communication terminal to obtain an image recognition result.

[0114] Step S302: determining the physical object associated with the gaze focus position according to the image recognition result.

[0115] Step S303: taking the physical object associated with the gaze focus position as the target physical object.

[0116] Among them, the image recognition result is used to indicate what the real object in the image at the focus position is, specifically including the real object in the image at the focus position that is successfully identified and the real object in the image at the focus position that is not successfully identified.

[0117] The physical object associated with the gaze focus position is determined based on the image recognition result; different image recognition results correspond to different physical objects associated with the gaze focus position.

[0118] Exemplarily, the server obtains an image of the screen of the first communication terminal, and extracts an image of the gaze focus position from the image, then performs image recognition on the image of the gaze focus position through a pre-trained image recognition model to obtain a corresponding image recognition result, and determines the physical object associated with the gaze focus position based on the image recognition result, and finally confirms the physical object associated with the gaze focus position as the target physical object.

[0119] In this embodiment, an image recognition result is obtained by identifying the image of the focus position on the screen of the first communication terminal, and then the target object associated with the focus position is determined based on the image recognition result, thereby achieving the effect of accurately identifying the target object associated with the focus position, which is beneficial to the subsequent generation of shooting instructions corresponding to the target object, and facilitates the control of multiple target cameras configured in the second communication terminal to obtain multiple images of the target object.

[0120] In an exemplary embodiment, the above step S302, based on the image recognition result, determines the physical object associated with the gaze focus position, and specifically includes the following contents: when the image recognition result indicates that the physical object in the image of the gaze focus position is successfully identified, the physical object in the image of the gaze focus position is used as the physical object associated with the gaze focus position; when the image recognition result indicates that the physical object in the image of the gaze focus position is not successfully identified, the physical object in the image within a preset area centered on the gaze focus position is used as the physical object associated with the gaze focus position.

[0121] The physical object in the image at the focus position refers to the physical object contained in the image at the focus position, such as Figure 8 The actual object W shown.

[0122] The preset area centered on the gaze focus position may refer to a spherical space with the gaze focus position as the sphere center and a preset value as the radius, such as Fig. 9 The spherical space shown has position A as the center and r as the radius; it may also refer to a preset circular area with the gaze focus position as the center, or it may refer to a preset range with the gaze focus position as the center.

[0123] Exemplarily, when the image recognition result indicates that the physical object in the image at the focus position is successfully recognized, the server identifies the physical object in the image at the focus position as the physical object associated with the focus position, such as Figure 8 The physical object W shown; when the image recognition result indicates that the physical object in the image of the gaze focus position is not successfully recognized, the server obtains an image in a preset area centered on the gaze focus position on the screen of the first communication terminal, and recognizes the image to obtain the physical object in the image in the preset area centered on the gaze focus position, and finally confirms the physical object in the image in the preset area centered on the gaze focus position as the physical object associated with the gaze focus position, such as Fig. 9 The shown figure is the real object in the spherical space with position A as the center and r as the radius.

[0124] For example, see Fig.11 , the system performs image recognition on the content at position A (x0, y0, z0) of the screen 6 of communication terminal a. If the system successfully recognizes that the image at position A (x0, y0, z0) is the image of the real object W, the system gives a command to the communication terminal b to clearly shoot the real object W. If the system does not successfully recognize what the image at position A (x0, y0, z0) is, the system takes the position A (x0, y0, z0) as the center of the sphere and r as the radius, and takes a spherical space on the imaging space of communication terminal a. The system gives a command to the communication terminal b to clearly shoot the image in the spherical space.

[0125] In this embodiment, the corresponding physical object associated with the focus position is determined based on different image recognition results, so that multiple captured images of the target physical object associated with the focus position can be accurately processed and then presented to the terminal user, which is beneficial to improving the image display effect of the focus position.

[0126] In an exemplary embodiment, the above step S205, sending the target image to the first communication terminal, specifically includes the following contents: when the target object is a physical object in the image of the focus position, sending the target image to the first communication terminal, so that the first communication terminal displays the target image at the location of the focus position on the screen of the first communication terminal; when the target object is a physical object in the image within a preset area centered on the focus position, sending the target image to the first communication terminal, so that the first communication terminal displays the target image in the preset area centered on the focus position on the screen of the first communication terminal.

[0127] Exemplarily, the server sends the target image to the first communication terminal; when the target object is a physical object in the image of the focus position, the first communication terminal determines the point where the focus position is located on the screen of the first communication terminal, and displays the received target image at the point where the focus position is located on the screen of the first communication terminal, such as reference Figure 7 and Figure 8 , and displays an enlarged and cleared image of the object W at position A. In addition, when the target object is an object in the image within a preset area centered on the eye focus position, the first communication terminal determines the preset area centered on the eye focus position on the screen of the first communication terminal, and displays the received target image in the preset area centered on the eye focus position on the screen of the first communication terminal, such as referring to Fig. 9 In a spherical space with position A (x0, y0, z0) as the center and r as the radius, an enlarged and clear image of the real object in the image within a preset area centered on the gaze focus position is displayed.

[0128] For example, see Fig.11 , the system processes the images captured by camera 1, camera 2, camera 3, and camera 4 of communication terminal b at time t1 and displays them on the screen of communication terminal a; at this time, the image at position A (x0, y0, z0) where user A of communication terminal a is looking becomes clear. Alternatively, the system processes the images captured by camera 1, camera 2, camera 3, and camera 4 of communication terminal b at time t1 and displays them on the screen of communication terminal a; at this time, on the screen 6 of communication terminal a, the image inside the sphere with position A (x0, y0, z0) as the center and r as the radius becomes clear, and the image that user A wants to see becomes clear.

[0129] In this embodiment, the target image is sent to the first communication terminal, so that the first communication terminal displays the target image at the focus position on the screen of the first communication terminal, which can make the display effect of the image focused by the terminal user better, thereby improving the display effect of the image targeted by the terminal user's gaze of the communication terminal.

[0130] In an exemplary embodiment, the above step S203, sending a shooting instruction to a second communication terminal connected to the first communication terminal via video communication, specifically includes the following contents: sending a shooting instruction to the second communication terminal connected to the first communication terminal via video communication, so that the second communication terminal adjusts the shooting angles of multiple target cameras configured in the second communication terminal based on the shooting instruction, obtains a target shooting angle of each target camera for the target object, and controls each target camera to focus and shoot the target object according to the corresponding target shooting angle, so as to obtain multiple captured images of the target object.

[0131] The target shooting angle refers to the optimal shooting angle.

[0132] Each target camera corresponds to a target shooting angle for the target object.

[0133] Among them, when adjusting the shooting angles of multiple target cameras configured by the second communication terminal, adjustment can be made through the camera gimbal of each target camera, or through other methods, which are not specifically limited in this application.

[0134] The camera gimbal may refer to an RGB-D camera gimbal. Fig.10 , 7 is the camera lens, 8 is the pitch rotation axis, which is driven by a motor to rotate, and 9 is the left-right rotation axis, which is driven by a motor to rotate. Communication terminal b controls the motor to rotate the RGB-D camera lens 7 to the target position in the left-right direction through the left-right rotation axis 9, and then rotates the RGB-D camera lens 7 to the target position in the pitch direction through the pitch rotation axis 8. These two steps rotate the camera lens 7 to the final target position. This pan-tilt device ensures that the lenses 7 of camera 1, camera 2, camera 3, and camera 4 of communication terminal b can all be rotated to the final target position, so that each camera has the best shooting angle.

[0135] Exemplarily, the server sends a shooting instruction to a second communication terminal connected to the first communication terminal by video communication; the second communication terminal drives the camera gimbal of each target camera configured by the second communication terminal according to the shooting instruction, so as to adjust the shooting angle of each target camera through the camera gimbal of each target camera, and obtain the target shooting angle (i.e., the closest shooting angle) of each target camera for the target object, and then controls each target camera to focus on the target object according to the corresponding target shooting angle, and shoot the target object, so as to obtain multiple captured images of the target object.

[0136] For example, see Fig.11, the system gives the instruction to the communication terminal b to take a clear picture of the object W; the communication terminal b controls the pan / tilts of its own camera 1, camera 2, camera 3, and camera 4 to rotate respectively, so that the camera 1, camera 2, camera 3, and camera 4 have the best shooting angle to take the object W; then, the communication terminal b controls its own camera 1, camera 2, camera 3, and camera 4 to automatically focus respectively, and take clear pictures of the object W at time t1. Alternatively, the system gives the instruction to take a clear picture of the image inside the ball to the communication terminal b; the communication terminal b controls the pan / tilts of its own camera 1, camera 2, camera 3, and camera 4 to rotate respectively, so that the camera 1, camera 2, camera 3, and camera 4 have the best shooting angle to take the image inside the ball corresponding to the object; then, the communication terminal b controls its own camera 1, camera 2, camera 3, and camera 4 to automatically focus respectively, and take clear pictures of the image inside the ball corresponding to the object at time t1.

[0137] In this embodiment, a shooting instruction is sent to a second communication terminal connected to the first communication terminal for video communication, so that the second communication terminal adjusts the shooting angles of multiple target cameras configured for the second communication terminal based on the shooting instruction, and controls each target camera to focus and shoot the target object according to the corresponding target shooting angle, thereby obtaining multiple captured images of the target object; in this way, by adjusting the shooting angle and automatic focus, the captured images can be made clearer and have a better angle, which is conducive to further improving the display effect of the image directed by the terminal user of the communication terminal.

[0138] In an exemplary embodiment, Figure 4 As shown, another image processing method is provided, which is described by taking the method applied to a server as an example, and includes the following steps S401 to S410. Among them:

[0139] Step S401, receiving a human eye image of a terminal user of the first communication terminal sent by the first communication terminal; the first communication terminal is used to capture the human eye image of the terminal user who is currently watching the screen of the first communication terminal through the configured human eye capture camera.

[0140] Step S402: Recognize the human eye image to obtain the gaze direction of the terminal user when viewing the screen of the first communication terminal.

[0141] Step S403: determining the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment according to the gaze direction.

[0142] Step S404: Recognize the image of the focus position on the screen of the first communication terminal to obtain an image recognition result.

[0143] Step S405: determining the physical object associated with the gaze focus position according to the image recognition result.

[0144] Among them, when the image recognition result indicates that the real object in the image of the focus position is successfully identified, the server uses the real object in the image of the focus position as the physical object associated with the focus position; when the image recognition result indicates that the real object in the image of the focus position is not successfully identified, the server uses the real object in the image within a preset area centered on the focus position as the physical object associated with the focus position.

[0145] Step S406: taking the physical object associated with the gaze focus position as the target physical object.

[0146] Step S407: Generate a shooting instruction corresponding to the target object.

[0147] Step S408, sending a shooting instruction to a second communication terminal connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured in the second communication terminal to shoot the target object, and obtains multiple shot images of the target object.

[0148] Among them, the second communication terminal is used to adjust the shooting angles of multiple target cameras configured by the second communication terminal based on the shooting instruction, obtain the target shooting angle of each target camera for the target object, and control each target camera to focus and shoot the target object according to the corresponding target shooting angle, so as to obtain multiple captured images of the target object.

[0149] Step S409: receiving a plurality of photographed images returned by the second communication terminal, performing image processing on the plurality of photographed images, and obtaining a target image for the target object.

[0150] Step S410: sending the target image to the first communication terminal, so that the first communication terminal displays the target image at a focus position on the screen of the first communication terminal.

[0151] Among them, when the target object is a physical object in the image of the focus position, the server sends the target image to the first communication terminal, so that the first communication terminal displays the target image at the location of the focus position on the screen of the first communication terminal; when the target object is a physical object in the image within a preset area centered on the focus position, the server sends the target image to the first communication terminal, so that the first communication terminal displays the target image in the preset area centered on the focus position on the screen of the first communication terminal.

[0152] In the above-mentioned image processing method, according to the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment, multiple captured images of the target object associated with the gaze focus position are obtained in a targeted manner, and image processing is performed on the multiple captured images. The obtained target image is then sent to the first communication terminal to display the target image at the gaze focus position on the screen of the first communication terminal, thereby achieving the purpose of processing the image of the physical object corresponding to the gaze focus position of the terminal user based on the human eye tracking result and then presenting it to the terminal user, which can make the display effect of the image focused by the terminal user better, thereby improving the display effect of the image targeted by the gaze direction of the terminal user of the communication terminal.

[0153] In order to more clearly illustrate the image processing method provided by the embodiment of the present application, the image processing method is specifically described below with reference to a specific embodiment. Fig.11 This application also proposes an image processing method based on human eye tracking, which, based on the viewer's eye tracking results, enlarges the image that the viewer focuses on, or provides it to the viewer in a clearer or better angle. The specific contents are as follows:

[0154] At time t0, the camera 5 of communication terminal a captures the gaze direction of user A. Based on the gaze direction of user A captured by camera 5, the system identifies and determines that user A's gaze falls on position A (x0, y0, z0) of the screen 6 of communication terminal a at time t0.

[0155] The system performs image recognition on the content at position A (x0, y0, z0) of the screen 6 of communication terminal a. If the system successfully recognizes that the image at position A (x0, y0, z0) is the image of the physical object W, the system gives the instruction to the communication terminal b to clearly shoot the physical object W. Communication terminal b controls the pan / tilts of its own cameras 1, 2, 3, and 4 to rotate respectively, so that cameras 1, 2, 3, and 4 have the best shooting angles to shoot the physical object W. Then, communication terminal b controls its own cameras 1, 2, 3, and 4 to automatically focus respectively, and take clear photos of the physical object W at time t1. The system processes the images taken by cameras 1, 2, 3, and 4 of communication terminal b at time t1 and displays them on the screen of communication terminal a. At this time, the image at position A (x0, y0, z0) where user A of communication terminal a is looking becomes clear.

[0156] The system performs image recognition on the content at position A (x0, y0, z0) of the screen 6 of communication terminal a. If the system fails to successfully identify what the image at position A (x0, y0, z0) is, the system will take position A (x0, y0, z0) as the center of the sphere and r as the radius, and take a spherical space in the imaging space of communication terminal a. The system will give the instruction of clear shooting of the image inside the sphere to communication terminal b. Communication terminal b controls the pan / tilts of its own cameras 1, 2, 3, and 4 to rotate respectively, so that cameras 1, 2, 3, and 4 have the best shooting angle to shoot the corresponding real object of the image inside the sphere. Then, communication terminal b controls its own cameras 1, 2, 3, and 4 to automatically focus respectively, and take clear photos of the corresponding real object of the image inside the sphere at time t1. The system processes the images shot by cameras 1, 2, 3, and 4 of communication terminal b at time t1 and displays them on the screen of communication terminal a. At this time, on the screen 6 of the communication terminal a, the image inside the sphere with the position A (x0, y0, z0) as the center and r as the radius becomes clear, and the image that the user A wants to see becomes clear.

[0157] It should be noted that the embodiment of the present application is a naked-eye 3D video conferencing system, but the protection scope of the present application is not limited to the naked-eye 3D video conferencing system, and any application that does not change the substantive content of the present application is within the scope of protection.

[0158] It should be noted that in the present application, when a real object is identified, the camera is focused on the real object as the center, and when the real object is not identified, the camera is focused in spherical space with the line of sight as the center of the sphere. This is the core invention point of the present application, and any substantial changes are within the scope of protection of the present application.

[0159] It should be noted that, in the embodiment of the present application, when the physical object is not recognized, the spherical space focusing camera is selected with the sight point as the center of the sphere, and in the 2D display application system, the circular area focusing camera is selected with the sight point as the center of the circle, or the camera is selected with a certain area focusing camera with the sight point as the center, etc. These application extensions that do not change the core idea of ​​the present application are all within the protection scope of the present application;

[0160] It should be noted that the camera of the image acquisition end in the embodiment of the present application uses a pan / tilt to rotate the camera to a suitable angle for shooting, but this does not change the substantive content of the present application, and applications that change the direction of the camera in other ways are also within the scope of protection of the present application;

[0161] It should be noted that the camera at the image acquisition end of the embodiment of the present application uses a gimbal. If the embodiment is relatively simple and does not require the use of a gimbal to rotate the camera to a certain angle, or uses other similar devices to achieve the same rotation effect, it does not change the essential content of the present application and is also within the scope of protection of the present application.

[0162] Compared with a video conferencing system that does not use the present application, the above-mentioned image processing method based on human eye tracking can enable users to see the real images they want to see more focused, clearer, and at a better angle, creating an immersive experience for users and improving user experience.

[0163] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0164] Based on the same inventive concept, the embodiment of the present application also provides an image processing device for implementing the above-mentioned image processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image processing device embodiments provided below can refer to the limitations on the image processing method above, and will not be repeated here.

[0165] In an exemplary embodiment, Fig.12 As shown, an image processing device is provided, including: a position determination module 1210, an instruction generation module 1220, an instruction sending module 1230, an image processing module 1240 and an image sending module 1250, wherein:

[0166] The position determination module 1210 is used to determine the gaze focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment.

[0167] The instruction generation module 1220 is used to generate a shooting instruction corresponding to the target object; the target object refers to the object associated with the focus position on the screen of the first communication terminal.

[0168] The instruction sending module 1230 is used to send a shooting instruction to a second communication terminal connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured by the second communication terminal to shoot the target object and obtain multiple captured images of the target object.

[0169] The image processing module 1240 is used to receive a plurality of photographed images returned by the second communication terminal, and perform image processing on the plurality of photographed images to obtain a target image of the target object.

[0170] The image sending module 1250 is used to send the target image to the first communication terminal, so that the first communication terminal displays the target image at the focus position on the screen of the first communication terminal.

[0171] In an exemplary embodiment, the position determination module 1210 is also used to receive a human eye image of a terminal user sent by the first communication terminal; the first communication terminal is used to capture a human eye image of the terminal user who is currently viewing the screen of the first communication terminal through a configured human eye capture camera; the human eye image is identified to obtain the gaze direction of the terminal user when viewing the screen of the first communication terminal; and the gaze focus position is determined based on the gaze direction.

[0172] In an exemplary embodiment, the image processing device also includes a physical object determination module, which is used to identify the image of the gaze focus position on the screen of the first communication terminal to obtain an image recognition result; determine the physical object associated with the gaze focus position based on the image recognition result; and use the physical object associated with the gaze focus position as the target physical object.

[0173] In an exemplary embodiment, the physical object determination module is also used to, when the image recognition result indicates that the physical object in the image at the focus position is successfully identified, use the physical object in the image at the focus position as the physical object associated with the focus position; when the image recognition result indicates that the physical object in the image at the focus position is not successfully identified, use the physical object in the image within a preset area centered on the focus position as the physical object associated with the focus position.

[0174] In an exemplary embodiment, the image sending module 1250 is also used to send the target image to the first communication terminal when the target object is a physical object in the image of the focus position, so that the first communication terminal displays the target image at the location of the focus position on the screen of the first communication terminal; when the target object is a physical object in the image within a preset area centered on the focus position, the target image is sent to the first communication terminal, so that the first communication terminal displays the target image in the preset area centered on the focus position on the screen of the first communication terminal.

[0175] In an exemplary embodiment, the instruction sending module 1230 is also used to send a shooting instruction to a second communication terminal connected to the first communication terminal via video communication, so that the second communication terminal adjusts the shooting angles of multiple target cameras configured in the second communication terminal based on the shooting instruction, obtains a target shooting angle of each target camera for the target object, and controls each target camera to focus and shoot the target object according to the corresponding target shooting angle, thereby obtaining multiple captured images of the target object.

[0176] Each module in the above image processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0177] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.13 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as captured images and target images. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image processing method is implemented.

[0178] Those skilled in the art will understand that Fig.13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0179] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0180] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0181] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0182] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0183] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0184] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: The method comprises: Determine a focus position of a terminal user of a first communication terminal on a screen of the first communication terminal at a current moment; Generate a shooting instruction corresponding to a target object; the target object refers to a physical object associated with the gaze focus position on the screen of the first communication terminal; Sending the shooting instruction to a second communication terminal connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured by the second communication terminal to shoot the target object, thereby obtaining multiple shot images of the target object; receiving the plurality of photographed images returned by the second communication terminal, and performing image processing on the plurality of photographed images to obtain a target image for the target object; The target image is sent to the first communication terminal, so that the first communication terminal displays the target image at a focus position on the screen of the first communication terminal.

2. The method according to claim 1, characterized in that The determining the focus position of the terminal user of the first communication terminal on the screen of the first communication terminal at the current moment includes: receiving a human eye image of the terminal user sent by the first communication terminal; the first communication terminal is used to capture the human eye image of the terminal user who is currently viewing the screen of the first communication terminal through a configured human eye capture camera; Recognize the human eye image to obtain the gaze direction of the terminal user when viewing the screen of the first communication terminal; The gaze focus position is determined according to the gaze direction.

3. The method according to claim 1, characterized in that Before generating the shooting instruction corresponding to the target object, it also includes: Recognize the image of the gaze focus position on the screen of the first communication terminal to obtain an image recognition result; Determining a physical object associated with the gaze focus position according to the image recognition result; The physical object associated with the gaze focus position is used as the target physical object.

4. The method according to claim 3, characterized in that The determining, according to the image recognition result, a physical object associated with the gaze focus position includes: When the image recognition result indicates that the real object in the image at the gaze focus position is successfully recognized, the real object in the image at the gaze focus position is used as the real object associated with the gaze focus position; When the image recognition result indicates that the real object in the image of the gaze focus position is not successfully recognized, the real object in the image within a preset area centered on the gaze focus position is used as the real object associated with the gaze focus position.

5. The method according to claim 4, characterized in that The sending the target image to the first communication terminal includes: In the case where the target object is a physical object in the image of the gaze focus position, the target image is sent to the first communication terminal, so that the first communication terminal displays the target image at a point where the gaze focus position is located on the screen of the first communication terminal; When the target object is a physical object in an image within a preset area centered on the gaze focus position, the target image is sent to the first communication terminal, so that the first communication terminal displays the target image in the preset area centered on the gaze focus position on the screen of the first communication terminal.

6. The method according to any one of claims 1 to 5, characterized in that: The step of sending the shooting instruction to a second communication terminal connected to the first communication terminal via video communication includes: The shooting instruction is sent to a second communication terminal connected to the first communication terminal by video communication, so that the second communication terminal adjusts the shooting angles of multiple target cameras configured by the second communication terminal based on the shooting instruction, obtains the target shooting angle of each target camera for the target object, and controls each target camera to focus and shoot the target object according to the corresponding target shooting angle, so as to obtain multiple captured images of the target object.

7. An image processing device, characterized in that: The device comprises: A position determination module, used to determine the gaze focus position of a terminal user of a first communication terminal on the screen of the first communication terminal at a current moment; An instruction generation module, used to generate a shooting instruction corresponding to a target object; the target object refers to a physical object associated with the gaze focus position on the screen of the first communication terminal; An instruction sending module is used to send the shooting instruction to a second communication terminal connected to the first communication terminal via video communication, so that the second communication terminal controls multiple target cameras configured by the second communication terminal to shoot the target object, thereby obtaining multiple shot images of the target object; An image processing module, configured to receive the plurality of photographed images returned by the second communication terminal, and perform image processing on the plurality of photographed images to obtain a target image for the target object; The image sending module is used to send the target image to the first communication terminal, so that the first communication terminal displays the target image at a focus position on the screen of the first communication terminal.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images