Remote interaction method of camera terminal equipment

Abstract

The invention belongs to the technical field of remote interaction, and particularly relates to a remote interaction method of camera terminal equipment, which comprises the following steps that: S1) a camera terminal acquires a real-time video stream and transmits the real-time video stream to a screen display end; s2) establishing a conversion relation between the two-dimensional image coordinates of the screen display end and the three-dimensional space coordinates of the camera terminal; s3) deriving a three-dimensional space coordinate of the target object according to the conversion relation; and S4) the camera terminal automatically adjusts internal parameters and external parameters of the camera according to the three-dimensional space coordinates. Accurate conversion between two-dimensional coordinates of a screen display end and three-dimensional space coordinates of a target object is realized by using a camera calibration technology and a projection matrix algorithm. And a solid technical basis is provided for improving the imaging definition of the target object.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote interaction, and specifically relates to a remote interaction method for a camera terminal device. Background Art

[0002] With the rapid development of technology, especially in the fields of computer vision and remote interaction, such as remote monitoring, industrial inspection, etc., the demand for target object tracking and display control is increasing day by day. Remote interaction operations require users to be able to select and mark target objects in an intuitive way and be able to smoothly and conveniently view the positions of target objects on the screen. However, in complex application scenarios (such as dynamic scenarios), there are problems with unclear imaging and display of target objects in the imaging diagram of the camera terminal, and the requirements for interactive operation cannot be met.

[0003] On the other hand, due to the limited imaging field of view of the camera, traditional cameras often cannot meet the demand for simultaneous display of multiple objects in the scene. Even if more objects are displayed by zooming in on the image, without precise perspective adjustment and parameter optimization, the target object may still be out of the field of view or the imaging display may be unclear. Although autofocus and automatic adjustment of the camera perspective have been applied in some professional applications, most existing non-standard camera terminal devices still cannot automatically adjust the internal and external parameters (focal length and perspective) of the camera according to the three-dimensional coordinates of the object or the dynamic changes of the scene; usually, they are set manually by the operator, lacking an automated intelligent adjustment mechanism. Summary of the Invention

[0004] To solve the technical problems of unclear imaging and display of the camera terminal in remote interaction and the lack of an automated intelligent adjustment mechanism for the internal and external parameters of the camera, the present invention provides a remote interaction method for a camera terminal device.

[0005] The object of the present invention can be achieved by the following technical solutions:

[0006] A remote interaction method for a camera terminal device includes the following steps:

[0007] S1) The camera terminal collects a real-time video stream and transmits it to the screen display end;

[0008] S2) Establish a conversion relationship between the two-dimensional image coordinates of the screen display end and the three-dimensional space coordinates of the camera terminal;

[0009] S3) The screen display end obtains the two-dimensional image coordinates of the target object in real time and derives the three-dimensional space coordinates of the target object according to the conversion relationship;

[0010] S4) The camera terminal automatically adjusts its internal parameters (image center point, focal length) and external parameters (translation vector, perspective) of the camera according to the three-dimensional space coordinates.

[0011] Preferably, the internal parameters include the center point of the image and the focal length; the external parameters include the translation vector and the viewing angle.

[0012] Preferably, in step S2), the method for establishing the conversion relationship includes:

[0013] S21) Establish the internal parameter matrix of the camera through the internal parameters of the camera, and its expression is:

[0014]

[0015] In the formula: f x and f y are the effective focal lengths of the camera in the horizontal and vertical directions (usually determined by the actual focal length f and the pixel scale factor); c x and c y are the coordinates of the center point of the image; 0 and 1 represent the expansion of homogeneous coordinates;

[0016] S22) Normalize the two-dimensional image coordinates of the screen display end;

[0017]

[0018] In the formula, (x, y) are the two-dimensional image coordinates, (x′, y′) are the normalized two-dimensional image coordinates; K is the internal parameter matrix of the camera;

[0019] S23) Establish the projection matrix between the three-dimensional space coordinates and the two-dimensional image coordinates, and the projection matrix formula is:

[0020]

[0021] In the formula, (X, Y, Z) are the three-dimensional space coordinates; R is the rotation matrix; T is the translation vector.

[0022] Preferably, in step S4), the adjustment method for the internal and external parameters of the camera includes:

[0023] S41) According to the three-dimensional space coordinates of the target object that have been deduced, and the three-dimensional space coordinates of the pre-defined camera, calculate the distance D between the target object and the camera; the calculation formula is:

[0024]

[0025] In the formula, (X, Y, Z) are the three-dimensional space coordinates of the target object; (X c , Y c , Z c ) are the three-dimensional space coordinates of the camera;

[0026] S42) Calculate the focal length from the camera to the target object, that is, for focal length adjustment; its calculation formula is:

[0027] f = k·D;

[0028] Wherein, k is a constant (determined by the optical characteristics of the camera), representing the proportional relationship between the distance and the focal length;

[0029] S43) The viewing angle between the camera and the target object is adjusted by rotating the camera; First, calculate the direction vector from the camera to the target object, and its calculation formula is:

[0030]

[0031] is the direction vector, representing the viewing angle from the camera to the target object;

[0032] S44) Calculate the rotation angle required to rotate the camera from the current direction to the target object direction vector through a rotation algorithm (such as quaternion, Euler angle);

[0033] S45) Adjust the viewing angle of the camera according to the calculated focal length and rotation angle, so that the target object is located at the center of the image and ensure the image is clear.

[0034] Preferably, it further includes step S5): Mark the target object on the screen display end (such as numbers 1, 2, 3, etc.), and record the three-dimensional space coordinates of each mark;

[0035] If the viewing angle of the camera changes, the two-dimensional image coordinates of the target object on the screen display end are deduced in reverse through the projection matrix to ensure that the marked target object is always displayed at the corresponding coordinates or the corresponding edge direction on the screen display end.

[0036] Preferably, it further includes step S6): Calculate the minimum circumscribed circle and the center point of all the marked points, and display the overview image of the target object containing all the marks by adjusting the viewing angle of the camera and reducing the image ratio of the screen display end.

[0037] Preferably, the specific implementation method of step S6) includes:

[0038] S61) Collect the two-dimensional image coordinates of all the marked points, denoted as the coordinate set P;

[0039] P = {(x1, y1), (x2, y2),..., (x n , y n )};

[0040] S62) Use the Welzl algorithm to calculate the minimum circumscribed circle containing all the marked points, and obtain the center coordinates (x r , y r ) and the radius r;

[0041] S63) According to steps S3 - S4), calculate and adjust the internal and external parameters of the camera based on the center coordinates (x r , y r ) so that the center of the circle is located at the center of the image and ensure that the image is clear;

[0042] S64) Calculate the scaling ratio S based on the screen size of the screen display end and the radius r; then scale and center the image to ensure that all marked targets are included in the overview map; the calculation formula for the scaling ratio S is:

[0043]

[0044] In the formula, W is the width of the screen size; H is the length of the screen size, and M is the reserved margin.

[0045] Preferably, the calculation steps of the Welzl algorithm include:

[0046] S621) Take all the coordinate sets P as the boundary set Q;

[0047] S622) For each point in the coordinate set P, randomly select a point and remove it from the coordinate set P;

[0048] S623) Recursively calculate the minimum circumscribed circle of the remaining points;

[0049] S624) If the currently calculated circle already contains the removed point, keep the circle; otherwise, add the point to the boundary set R and recalculate the new circumscribed circle;

[0050] S625) When the boundary set R reaches 3 points, determine the unique circle based on these three points; that is, obtain the center coordinates (x r , y r ) and radius r of the minimum circumscribed circle.

[0051] Advantages of the present invention:

[0052] 1. Utilizing camera calibration technology and projection matrix algorithm, the accurate conversion between the two - dimensional coordinates of the screen display end and the three - dimensional space coordinates of the target object is realized. It provides a solid technical foundation for improving the imaging clarity of the target object.

[0053] 2. By calculating the minimum circumscribed circle of all marked points, and then intelligently adjusting the camera focal length, viewing angle, and image scaling ratio, it ensures that all marked target objects can be clearly and comprehensively displayed on the screen; an automated intelligent adjustment mechanism is realized, improving the user's interaction experience.

[0054] 3. Through standardized coordinate conversion algorithms and viewing angle adjustment methods, it can adapt to different camera hardware and display devices, and has good scalability and universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0056] Figure 1 It is a flowchart of the steps of a remote interaction method for a camera terminal device of the present invention.

[0057] Figure 2 It is a flowchart of step S4) in the remote interaction method for a camera terminal device of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0059] Please refer to Figure 1 - Figure 2 As shown, a remote interaction method for a camera terminal device includes the following steps:

[0060] S1) The camera terminal collects a real-time video stream and transmits it to the screen display end;

[0061] S2) Establish a conversion relationship between the two-dimensional image coordinates of the screen display end and the three-dimensional space coordinates of the camera terminal;

[0062] S3) The screen display end obtains the two-dimensional image coordinates of the target object in real time, and derives the three-dimensional space coordinates of the target object according to the conversion relationship;

[0063] S4) The camera terminal automatically adjusts the internal parameters (image center point, focal length) and external parameters (translation vector, viewing angle) of its camera according to the three-dimensional space coordinates.

[0064] Specifically, the camera terminal collects the video stream of the scene in real time, that is, image data. The camera terminal needs to have a high-definition video stream collection function, and can use a high-definition camera, a network camera, a smart phone or a tablet device; and configure a control device for adjusting its camera external parameters (translation vector, viewing angle, etc.). In addition, the transmission network uses an efficient video stream protocol, such as RTSP or WebRTC, to ensure low-latency transmission of the real-time video stream.

[0065] The screen display terminal receives and displays the video stream; the screen display terminal needs to have interactive operation functions such as screen touch or mouse click, and can adopt a touchable smart screen, a smart phone or a tablet device. The user directly interacts with the screen through touch or mouse click, etc., selects and marks the target object, and provides accurate coordinate input. In addition, an interactive interface needs to be provided, and the program system is set to allow the user to delete or modify the mark to ensure that the system can react to the user operation in real time. The screen display terminal is not only a video stream display terminal, but also requires a system program for in-depth interaction with the camera terminal.

[0066] The system program includes functions such as real-time video display, coordinate conversion, target marking and tracking, full-view display, and user interaction. Through a reasonable program architecture (such as image processing, coordinate transformation, geometric calculation, etc.), the logical operations of the interactive operation are processed in real time.

[0067] Specifically, the implementation process of steps S1 - S4 realizes the interactive operation between the camera terminal and the screen display terminal, and can adjust the focal length and viewing angle of the camera according to the three-dimensional coordinates of the target object, so as to ensure that the target object is always clearly displayed on the screen. This operation involves the precise conversion relationship between three-dimensional space and two-dimensional image coordinates, distance calculation; finally, an accurate numerical control method for adjusting the internal and external parameters (focal length, viewing angle) of the camera is realized.

[0068] Furthermore, the internal parameters include the image center point and the focal length; the external parameters include the translation vector and the viewing angle.

[0069] Specifically, the internal parameters mainly include the focal length of the camera and the image center point, which are used to describe the optical characteristics of the camera and how to map the points in three-dimensional space to the center of the image plane. The external parameters describe the relationship between the camera and the three-dimensional space coordinate system, and usually include the translation vector and the viewing angle (the rotation matrix in a certain direction). The viewing angle determines the shooting direction of the camera, that is, the orientation of the lens; these parameters determine the position and direction of the camera in three-dimensional space.

[0070] Furthermore, in step S2), the method for establishing the conversion relationship includes:

[0071] S21) Establish the internal parameter matrix of the camera through the internal parameters of the camera, and its expression is:

[0072]

[0073] In the formula: f x and f y are the effective focal lengths of the camera in the horizontal and vertical directions (usually determined by the actual focal length f and the pixel scale factor); c x and c y are the coordinates of the principal point of the image; 0 and 1 represent the expansion of homogeneous coordinates;

[0074] S22) Normalize the two-dimensional image coordinates of the screen display end;

[0075]

[0076] In the formula, (x, y) are the two-dimensional image coordinates, and (x′, y′) are the normalized two-dimensional image coordinates; K is the internal parameter matrix of the camera;

[0077] S23) Establish a projection matrix between the three-dimensional space coordinates and the two-dimensional image coordinates. The projection matrix formula is:

[0078]

[0079] In the formula, (X, Y, Z) are the three-dimensional space coordinates; R is the rotation matrix; T is the translation vector.

[0080] Specifically, the internal parameter matrix K of the camera is usually obtained through the camera calibration process. The calibration process includes taking pictures using a calibration board with a known size (such as a chessboard pattern) and identifying the corner points of the calibration board through image processing techniques. Through multiple images at different angles, algorithms such as Zhang Zhengyou calibration method or Tsai calibration method are used; by fitting the relationship between the pixel coordinates of the calibration points and their actual physical positions, the internal parameter matrix can be deduced.

[0081] The normalization process is to eliminate the influence of different image resolutions and focal lengths. The normalized two-dimensional coordinates (x′, y′) will no longer be directly affected by the screen resolution, image size, and focal length. This enables subsequent coordinate conversions to be uniformly processed under images with different scaling ratios, avoiding the differences in images with different scaling ratios.

[0082] The projection matrix is the projection conversion relationship between the three-dimensional space coordinates (X, Y, Z) of the target object in the three-dimensional space and the two-dimensional image coordinates (x, y). It is mainly calculated through the rotation matrix R, and the rotation matrix R can be obtained through the camera pose or through visual SLAM technology.

[0083] Furthermore, in step S4), the adjustment methods for the internal and external parameters of the camera include:

[0084] S41) According to the three-dimensional space coordinates of the target object that have been deduced, and the three-dimensional space coordinates of the camera defined in advance, calculate the distance D between the target object and the camera; the calculation formula is:

[0085]

[0086] In the formula, (X, Y, Z) are the three-dimensional space coordinates of the target object; (X c , Y c , Z c) are the three-dimensional spatial coordinates of the camera;

[0087] S42) Calculate the focal length from the camera to the target object, which is used for focal length adjustment; its calculation formula is:

[0088] f = k·D;

[0089] In the formula, k is a constant (determined by the optical characteristics of the camera), representing the proportional relationship between distance and focal length;

[0090] S43) Adjust the viewing angle between the camera and the target object by rotating the camera; first calculate the direction vector from the camera to the target object, and its calculation formula is:

[0091]

[0092] is the direction vector, representing the viewing angle from the camera to the target object;

[0093] S44) Calculate the rotation angle required to rotate the camera from the current direction to the target object direction vector through a rotation algorithm (such as quaternion, Euler angle);

[0094] S45) Adjust the viewing angle of the camera according to the calculated focal length and rotation angle, so that the target object is located at the center of the image and ensure the image is clear.

[0095] Specifically, the adjustment of the focal length is based on the relative position distance D between the three-dimensional spatial coordinates of the target object and the camera to adjust the focal length. The calculated focal length f will adjust the optical system of the camera to ensure the clarity of the image of the target object.

[0096] In the specific implementation process, to clearly observe the target object and achieve the magnification and reduction of the target object, it is necessary to adjust the alignment and focusing of the target object with the center of the camera viewing angle, that is, rotate and adjust the orientation of the camera, so as to present the target object to the center of the image, which is conducive to the camera focusing and ensures the best clarity of the target object. Among them, the rotation algorithm can be represented by Euler angles, quaternions or rotation axes and angles. Since in the present invention, the adjustment of the viewing angle can adopt the Euler angle method: changing the viewing angle by rotating around the camera coordinate axes, which is simple and practical; it can meet most of the target object positioning requirements. The calculation of the viewing angle (Euler angle) first calculates the direction vector between the target object and the camera; then uses the Euler angle formula to calculate the rotation angle required for the camera to rotate from the current viewing angle to the target object direction; the calculation of the rotation angle ensures the alignment of the camera viewing angle with the target object, and adjusts the orientation of the camera by rotation, so as to present the target object to the center of the image.

[0097] Finally, according to the focal length calculated in step S42), adjust the focal length of the camera lens to make the image of the target object clear. According to the rotation angle calculated in step S44), adjust the rotation angle of the camera to ensure that the target object is always clearly displayed in the image and located at the center of the field of view.

[0098] Furthermore, it also includes step S5): Mark the target object on the screen display end (such as numbers 1, 2, 3, etc.), and record the three-dimensional space coordinates of each mark;

[0099] If the camera view changes, then reverse-derive the two-dimensional image coordinates of the target object on the screen display end through the projection matrix to ensure that the marked target object is always displayed at the corresponding coordinates or corresponding edge directions on the screen display end.

[0100] Specifically, the user selects the target object on the screen through a touch screen or other interaction methods and marks it. For example, by clicking on an object on the touch screen, the system assigns a number (such as 1, 2, 3, etc.) to the object and records the three-dimensional space coordinates (X, Y, Z) of the object and stores them in a database or cache.

[0101] On the screen display end, the system displays the mark number and the corresponding three-dimensional space coordinates on the overlay layer of the video stream through a graphical interface. When the camera view changes, the position of the target object on the screen may change. To ensure that the mark is always displayed at the correct coordinates or edge directions, the system program reverse-derives the two-dimensional image coordinates of the target object on the screen display end through the projection matrix and displays them at the corresponding coordinate positions and corresponding edge directions on the overlay layer of the video stream (if not within the image displayed on the screen).

[0102] Furthermore, it also includes step S6): Calculate the minimum circumscribed circle and the center point of all the marked points, and by adjusting the camera view and reducing the image scale of the screen display end, thus display an overview image of the target object containing all the marks.

[0103] Furthermore, the specific implementation method of step S6) includes:

[0104] S61) Collect the two-dimensional image coordinates of all the marked points, denoted as the coordinate set P;

[0105] P = {(x1, y1), (x2y2),..., (x n , y n )};

[0106] S62) Use the Welzl algorithm to calculate the minimum circumscribed circle containing all the marked points, and obtain the center coordinates (x r , y r ) and the radius r;

[0107] S63) According to steps S3 - S4), calculate and adjust the internal and external parameters of the camera based on the center coordinates (x r , y r ) so that the center of the circle is located at the center of the image and ensure the image is clear;

[0108] S64) Calculate the scaling ratio S based on the screen size of the screen display end and the radius r; then scale and center the display of the image to ensure that all marked targets are included in the overview image; the calculation formula for the scaling ratio S is:

[0109]

[0110] In the formula, W is the width of the screen size; H is the length of the screen size, and M is the reserved margin.

[0111] Specifically, the minimum circumscribed circle is the smallest circle that contains all the marked points. Calculate its center and radius to ensure that all marked target objects are within the camera's field of view, and make it fully displayed by adjusting the image ratio. Use the Welzl algorithm to calculate the minimum circumscribed circle that contains all the marked points. The Welzl algorithm is an effective geometric algorithm for calculating the minimum circumscribed circle of a set of planar points. Then, according to the center of the minimum circumscribed circle, adjust the camera's perspective to ensure that the target object is at the center of the image. According to the center of the target object and the focal length of the camera, adjust the internal parameters (such as focal length) and external parameters (perspective) of the camera in real time.

[0112] Calculate the scaling ratio S based on the radius r of the minimum circumscribed circle and the screen size. By reducing the display area of the image, make the diameter 2r of the minimum circumscribed circle adapt to the screen size while keeping all target objects displayed within the image. Through these steps, the system can precisely control the display of the image, enabling multiple marked target objects to be completely and panoramically displayed on the same screen, enhancing the user's interaction experience.

[0113] Furthermore, the calculation steps of the Welzl algorithm include:

[0114] S621) Take all the coordinate sets P as the boundary set Q;

[0115] S622) For each point in the coordinate set P, randomly select a point and remove it from the coordinate set P;

[0116] S623) Recursively calculate the minimum circumscribed circle of the remaining points;

[0117] S624) If the currently calculated circle already contains the removed point, keep the circle; otherwise, add the point to the boundary set R and recalculate the new circumscribed circle;

[0118] S625) When the boundary set R reaches three points, a unique circle is determined based on these three points (the optimal circle must be determined by at most three boundary points); that is, the center coordinates (x r , y r ) and radius r of the minimum circumscribed circle are obtained.

[0119] In summary, the present invention realizes functions such as the adjustment of the internal and external parameters of the camera, the marking and overview of the target object through the precise three-dimensional coordinate and two-dimensional image coordinate conversion relationship, and realizes an efficient, real-time, and accurate remote interaction method between the camera terminal device and the screen display terminal. Through the standardized coordinate conversion method and the perspective adjustment algorithm, it can also adapt to the interaction between different camera hardware and display devices, and has good universality. It can be widely applied to many fields such as remote monitoring and industrial inspection, providing an efficient and accurate solution.

[0120] In the description of the specific implementation manner, the descriptions referring to terms such as "specifically", "as a more preferred implementation manner", "further", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. The above content is only an example and explanation of the structure of the present invention. Those skilled in the art of the present technology can make various modifications or supplements to the described specific embodiments or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined by the claims of the present invention, they should all fall within the protection scope of the present invention.

Claims

1. A remote interaction method for a camera terminal device, characterized in that: It includes the following steps: S1) The camera terminal collects a real-time video stream and transmits it to the screen display terminal; S2) Establish a conversion relationship between the two-dimensional image coordinates of the screen display terminal and the three-dimensional space coordinates of the camera terminal; S3) The screen display terminal obtains the two-dimensional image coordinates of the target object in real time, and derives the three-dimensional space coordinates of the target object according to the conversion relationship; S4) The camera terminal automatically adjusts the internal parameters and external parameters of its camera according to the three-dimensional space coordinates.

2. The remote interaction method of a camera terminal device according to claim 1, characterized in that: In step S2), the method for establishing the conversion relationship includes: S21) Establish the internal parameter matrix of the camera through the internal parameters of the camera, and its expression is: Where: f x and f y are the effective focal lengths of the camera in the horizontal and vertical directions; c x and c y are the coordinates of the image center point; 0 and 1 represent the extension of homogeneous coordinates; S22) Normalize the two-dimensional image coordinates of the screen display terminal; In the formula, (x, y) are the two-dimensional image coordinates, (x′, y′) are the normalized two-dimensional image coordinates; K is the internal parameter matrix of the camera; S23) Establish a projection matrix between the three-dimensional space coordinates and the two-dimensional image coordinates, and the projection matrix formula is: In the formula, (X, Y, Z) are the three-dimensional space coordinates; R is the rotation matrix; T is the translation vector.

3. The remote interaction method of a camera terminal device according to claim 1, characterized in that: In step S4), the method for adjusting the internal parameters and external parameters of the camera includes: S41) According to the three-dimensional space coordinates of the target object that have been derived, and the pre-defined three-dimensional space coordinates of the camera, calculate the distance D between the target object and the camera; the calculation formula is: where (X, Y, Z) are the three-dimensional spatial coordinates of the target object; (X c , Y c , Z c ) are the three-dimensional spatial coordinates of the camera; S42) Calculate the focal length from the camera to the target object, that is, for focal length adjustment; its calculation formula is: f = k·D; In the formula, k is a constant representing the proportional relationship between the distance and the focal length; S43) The viewing angle between the camera and the target object is adjusted by rotating the camera; first calculate the direction vector from the camera to the target object, and its calculation formula is: is a direction vector, representing the viewing angle from the camera to the target object; S44) Calculate the rotation angle required to rotate the camera from the current direction to the direction vector of the target object through the rotation algorithm; S45) According to the calculated focal length and rotation angle, adjust the viewing angle of the camera so that the target object is located at the center of the image and ensure that the image is clear.

4. The remote interaction method of a camera terminal device according to claim 1, characterized in that: The internal parameters include the image center point and the focal length; the external parameters include the translation vector and the viewing angle.

5. A remote interaction method for a camera terminal device according to any one of claims 1-4, characterized in that: It further includes step S5): Mark the target object on the screen display terminal and record the three-dimensional space coordinates of each mark; If the camera viewing angle changes, the two-dimensional image coordinates of the target object on the screen display terminal are reversely derived through the projection matrix to ensure that the marked target object is always displayed at the corresponding coordinates or the corresponding edge direction on the screen display terminal.

6. The remote interaction method of a camera terminal device according to claim 5, characterized in that: It further includes step S6): Calculate the minimum circumscribed circle and the center point of all the marked points, and display the overview image of the target object containing all the marks by adjusting the camera viewing angle and reducing the image ratio of the screen display terminal.

7. The remote interaction method of a camera terminal device according to claim 6, characterized in that: The specific implementation method of step S6) includes: S61) Collect the two-dimensional image coordinates of all the marked points, denoted as the coordinate set P; P = {(x1, y1), (x2, y2),..., (x n , y n )}; S62) Calculate the minimum circumscribed circle containing all the marked points using the Welzl algorithm to obtain the center coordinates (x r , y r ) and the radius r; S63) According to steps S3 - S4), calculate and adjust the internal and external parameters of the camera based on the center coordinates (x r , y r ) so that the center of the circle is located at the center of the image and ensure that the image is clear; S64) According to the screen size and the radius r of the screen display terminal, calculate the scaling ratio S; then scale and center the display of the image to ensure that all the marked targets are included in the overview image.

8. A remote interaction method for a camera terminal device according to claim 7, characterized in that: The calculation formula of the scaling ratio S is: In the formula, W is the width of the screen size; H is the length of the screen size, and M is the reserved margin.

9. A remote interaction method for a camera terminal device according to claim 7, characterized in that: The calculation steps of the Welzl algorithm include: S621) Take all the coordinate sets P as the boundary set Q; S622) For each point in the coordinate set P, randomly select a point and remove it from the coordinate set P; S623) Recursively calculate the minimum circumscribed circle of the remaining points; S624) If the currently calculated circle already contains the removed point, keep the circle; otherwise, add the point to the boundary set R and recalculate the new circumscribed circle; S625) When the boundary set R reaches three points, a unique circle is determined based on these three points; that is, the center coordinates (x r , y r ) and radius r of the minimum circumscribed circle are obtained.

Images