Image tracking simulation method, device, equipment and storage medium
By constructing a simulated imaging environment in image tracking simulation and using the minimum mean square algorithm LMS for calibration, the problems of low focus measurement accuracy and cumbersome operation in traditional methods are solved, and fast and accurate image tracking simulation is achieved, which improves the simulation accuracy and efficiency.
Patent Information
- Application Number
- CN202110036617.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-12
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-01-12
AI Technical Summary
In traditional image tracking simulation methods, it is difficult to accurately measure the focus of the TV equipment, difficult to align the center of the screen, low calibration accuracy, cumbersome operation and time-consuming and labor-intensive.
By constructing a simulated imaging environment, the distance and angle of the imaging device relative to the projection screen is calibrated using the preset minimum mean square algorithm LMS, the calibration distance and calibration angle are obtained, and image tracking simulation is performed based on these parameters.
It realizes fast and accurate focus positioning, simplifies system calibration work, improves simulation accuracy and efficiency, reduces interference to the external environment, and is simple and convenient to operate.
Smart Images

Figure CN112950677B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image simulation, and in particular to an image tracking simulation method, device, equipment and storage medium. Background Art
[0002] For image tracking simulation experiments, the simplest way to simulate target motion is to generate a virtual observation target through a projection screen, capture the target through a television device and observe its motion; in the experiment, the imaging quality is related to the authenticity of the simulation experiment, and the relative relationship between the observation device and the projection screen affects the accuracy of the observation data output. Therefore, a good imaging environment and calibration of the relative relationship between the observation device and the projection screen are very important; but the observation equipment in an open environment is easily affected by the external environment, which affects the authenticity and accuracy of the simulation experiment; and the imaging focus of the observation device is not explicitly visible, which undoubtedly greatly increases the measurement difficulty for the distance of the focus relative to the screen and the projection position on the screen; in addition, since the simulation experiment needs to be carried out many times at different development stages, the distance between the television observation device and the screen will inevitably change during the experiment.
[0003] The traditional image tracking simulation method is to simulate the target movement through a double-standard slide rail to observe the elevation and azimuth angles; the calibration process of this method takes a long time, and has high requirements for the test site and environment, and the imaging environment is easily disturbed; before the test, the relative position of the imaging device and the imaging screen is obtained by manual measurement, that is, firstly, the relative distance between the TV observation device and the screen is calculated according to the field of view characteristics of the TV observation device, and then the TV observation device and the screen are placed according to the pre-calculated size, and then the center point of the TV observation device is made to coincide with the center point of the screen through repeated manual adjustments; the operation process is not only very cumbersome, but also has low accuracy, and it takes a lot of time and effort before the test. Summary of the invention
[0004] The main purpose of the present invention is to provide an image tracking simulation method, device, equipment and storage medium, aiming to solve the technical problems in the prior art that the traditional image tracking simulation method is difficult to accurately measure the focus of television equipment, it is difficult to align the focus with the center of the screen, the calibration accuracy is difficult to grasp, and the operation process is cumbersome and time-consuming and labor-intensive.
[0005] In a first aspect, the present invention provides an image tracking simulation method, the image tracking simulation method comprising the following steps:
[0006] Constructing a simulated imaging environment;
[0007] In the simulated imaging environment, the distance and angle of the imaging device relative to the projection screen are calibrated according to a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle;
[0008] Image tracking simulation is performed according to the calibration distance and the calibration angle.
[0009] Optionally, the constructing a simulated imaging environment includes:
[0010] The imaging device and the projection screen are placed in a relatively closed imaging environment by means of a box, one side of the box is the projection screen, and the other side of the box is the imaging device fixed by a turntable mechanism;
[0011] The imaging device faces the projection screen, and the projection screen displays an image of the observed object. The box, the imaging device and the projection screen form a simulated imaging environment.
[0012] Optionally, the step of calibrating the distance and angle of the imaging device relative to the projection screen according to a preset least mean square algorithm LMS in the simulated imaging environment to obtain a calibrated distance and a calibrated angle includes:
[0013] Acquiring a preset horizontal distance between an imaging device and a projection screen in the simulated imaging environment;
[0014] Acquiring the relative position and posture information between the imaging device and the observed object, and acquiring the size and shape information of the observed object;
[0015] The distance and angle of the imaging device relative to the projection screen are calibrated according to the preset horizontal distance, the relative position, the posture information and the size and shape information in combination with a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle.
[0016] Optionally, the calibrating the distance and angle of the imaging device relative to the projection screen according to the preset horizontal distance, the relative position, the posture information and the size and shape information in combination with a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle includes:
[0017] Acquire the screen resolution of the projection screen, and determine the physical size of a single pixel according to the screen resolution and the size and shape information;
[0018] Determine the elevation angle and azimuth angle of the imaging device according to the preset horizontal distance, the relative position and the posture information;
[0019] Obtaining a preset state quantity, a preset filter coefficient and an objective function of a preset least mean square algorithm LMS;
[0020] Determine the initial pixel position of the projection point according to the physical size, the elevation angle, the azimuth angle, the preset state quantity, and the preset filter coefficient;
[0021] Determine an error function according to the objective function and the initial pixel position, and determine a gradient vector between the error function and the preset filter coefficient according to the error function;
[0022] Converging the preset filter coefficients through the convergence factor of the LMS and the gradient vector to obtain target filter coefficients;
[0023] The target projection point coordinates are calibrated according to the target filter coefficients, and the calibration distance and calibration angle are determined according to the target projection point coordinates.
[0024] Optionally, the converging the preset filter coefficient by using the convergence factor of the LMS and the gradient vector to obtain the target filter coefficient includes:
[0025] The preset filter coefficient is converged by the convergence factor of the LMS and the gradient vector, and the target filter coefficient is obtained by the following formula:
[0026] ω k+1 =ω k +2μe k x k
[0027] Among them, ω k+1 is the target filter coefficient, ω k is the preset filter coefficient, μ is the convergence factor, 2e k x k is the instantaneous value of the gradient vector estimate.
[0028] Optionally, calibrating the target projection point coordinates according to the target filter coefficient, and determining the calibration distance and calibration angle according to the target projection point coordinates, includes:
[0029] Determine the target projection point coordinates according to the target filter coefficient and the preset state quantity;
[0030] Acquire the observation point coordinates of the imaging device, and determine the current distance and current angle between the two coordinates according to the observation point coordinates and the target projection point coordinates;
[0031] The current distance is used as the calibration distance, and the current angle is used as the calibration angle.
[0032] Optionally, before performing the image tracking simulation according to the calibration distance and the calibration angle, the image tracking simulation method further includes:
[0033] When there is an angle deviation between the imaging device and the projection screen, obtaining the height tilt angle of the imaging device in the height direction and the azimuth tilt angle corresponding to the azimuth;
[0034] Correcting the coordinates of the observation point on the projection screen according to the elevation tilt angle and the azimuth tilt angle;
[0035] Re-determine the calibration distance and calibration angle based on the corrected observation point coordinates.
[0036] In a second aspect, in order to achieve the above-mentioned object, the present invention further proposes an image tracking simulation device, the image tracking simulation device comprising:
[0037] A building module for building a simulated imaging environment;
[0038] A calibration module, used to calibrate the distance and angle of the imaging device relative to the projection screen in the simulated imaging environment according to a preset least mean square algorithm LMS to obtain a calibration distance and a calibration angle;
[0039] A simulation module is used to perform image tracking simulation according to the calibration distance and the calibration angle.
[0040] In the third aspect, to achieve the above-mentioned purpose, the present invention also proposes an image tracking simulation device, which includes: a memory, a processor, and an image tracking simulation program stored in the memory and executable on the processor, wherein the image tracking simulation program is configured to implement the steps of the image tracking simulation method as described above in the claim.
[0041] In a fourth aspect, in order to achieve the above-mentioned purpose, the present invention further proposes a storage medium, on which an image tracking simulation program is stored, and when the image tracking simulation program is executed by a processor, the steps of the image tracking simulation method described above are implemented.
[0042] The image tracking simulation method proposed in the present invention constructs a simulated imaging environment; in the simulated imaging environment, calibrates the distance and angle of an imaging device relative to a projection screen according to a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle; performs image tracking simulation according to the calibrated distance and the calibrated angle; it is not necessary to obtain the distance between the focus of a television device and the projection screen by measuring, and can quickly and accurately locate the position of the imaging device focus projected on the projection screen, greatly simplifies the calibration work of the system, and is simple to operate, simplifies the positioning of the focus of an observation device, and has good isolation from the external environment, making image tracking simulation more convenient and efficient, ensuring simulation accuracy, and improving simulation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 A schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention;
[0044] Figure 2It is a flowchart of the first embodiment of the image tracking simulation method of the present invention;
[0045] Figure 3 A schematic diagram of a flow chart of a second embodiment of an image tracking simulation method of the present invention;
[0046] Figure 4 A schematic diagram of a flow chart of a third embodiment of an image tracking simulation method of the present invention;
[0047] Figure 5 A schematic diagram of a flow chart of a fourth embodiment of an image tracking simulation method of the present invention;
[0048] Figure 6 A diagram showing the relationship between the observation device and the projection screen in the image tracking simulation method of the present invention;
[0049] Figure 7 A schematic diagram of a fifth embodiment of the image tracking simulation method of the present invention;
[0050] Figure 8 A schematic diagram of the angle deviation in the high and low angle directions in the image tracking simulation method of the present invention;
[0051] Fig. 9 It is a functional module diagram of the first embodiment of the image tracking simulation device of the present invention;
[0052] Fig.10 It is a schematic diagram of the imaging structure of the image tracking simulation device of the present invention.
[0053] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0054] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0055] The solution of the embodiment of the present invention is mainly: by constructing a simulated imaging environment; in the simulated imaging environment, calibrating the distance and angle of the imaging device relative to the projection screen according to a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle; performing image tracking simulation according to the calibrated distance and the calibrated angle; it is possible to obtain the distance between the focus of the television device and the projection screen without measuring, and to quickly and accurately locate the position of the focus of the imaging device projected on the projection screen, which greatly simplifies the calibration work of the system, and is simple to operate, simplifies the positioning of the focus of the observation device, and has good isolation from the external environment, making the image tracking simulation more convenient and efficient, ensuring the simulation accuracy, and improving the simulation efficiency, and solving the technical problems in the prior art that the traditional image tracking simulation method is difficult to accurately measure the focus of the television device, it is also difficult to align the focus with the center of the screen, the calibration accuracy is difficult to grasp, and the operation process is cumbersome and time-consuming and labor-intensive.
[0056] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.
[0057] like Figure 1 As shown, the device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory, or it may be a stable memory (Non-Volatile Memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0058] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation of the device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0059] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an image tracking simulation program.
[0060] The device of the present invention calls the image tracking simulation program stored in the memory 1005 through the processor 1001, and performs the following operations:
[0061] Constructing a simulated imaging environment;
[0062] In the simulated imaging environment, the distance and angle of the imaging device relative to the projection screen are calibrated according to a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle;
[0063] Image tracking simulation is performed according to the calibration distance and the calibration angle.
[0064] Further, the processor 1001 may call the image tracking simulation program stored in the memory 1005, and further perform the following operations:
[0065] The imaging device and the projection screen are placed in a relatively closed imaging environment by means of a box, one side of the box is the projection screen, and the other side of the box is the imaging device fixed by a turntable mechanism;
[0066] The imaging device faces the projection screen, and the projection screen displays an image of the observed object. The box, the imaging device and the projection screen form a simulated imaging environment.
[0067] Further, the processor 1001 may call the image tracking simulation program stored in the memory 1005, and further perform the following operations:
[0068] Acquiring a preset horizontal distance between an imaging device and a projection screen in the simulated imaging environment;
[0069] Acquiring the relative position and posture information between the imaging device and the observed object, and acquiring the size and shape information of the observed object;
[0070] The distance and angle of the imaging device relative to the projection screen are calibrated according to the preset horizontal distance, the relative position, the posture information and the size and shape information in combination with a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle.
[0071] Further, the processor 1001 may call the image tracking simulation program stored in the memory 1005, and further perform the following operations:
[0072] Acquire the screen resolution of the projection screen, and determine the physical size of a single pixel according to the screen resolution and the size and shape information;
[0073] Determine the elevation angle and azimuth angle of the imaging device according to the preset horizontal distance, the relative position and the posture information;
[0074] Obtaining a preset state quantity, a preset filter coefficient and an objective function of a preset least mean square algorithm LMS;
[0075] Determine the initial pixel position of the projection point according to the physical size, the elevation angle, the azimuth angle, the preset state quantity, and the preset filter coefficient;
[0076] Determine an error function according to the objective function and the initial pixel position, and determine a gradient vector between the error function and the preset filter coefficient according to the error function;
[0077] Converging the preset filter coefficients through the convergence factor of the LMS and the gradient vector to obtain target filter coefficients;
[0078] The target projection point coordinates are calibrated according to the target filter coefficients, and the calibration distance and calibration angle are determined according to the target projection point coordinates.
[0079] Further, the processor 1001 may call the image tracking simulation program stored in the memory 1005, and further perform the following operations:
[0080] The preset filter coefficient is converged by the convergence factor of the LMS and the gradient vector, and the target filter coefficient is obtained by the following formula:
[0081] ω k+1 =ω k +2μe k x k
[0082] Among them, ω k+1 is the target filter coefficient, ω k is the preset filter coefficient, μ is the convergence factor, 2e k x k is the instantaneous value of the gradient vector estimate.
[0083] Further, the processor 1001 may call the image tracking simulation program stored in the memory 1005, and further perform the following operations:
[0084] Determine the target projection point coordinates according to the target filter coefficient and the preset state quantity;
[0085] Acquire the observation point coordinates of the imaging device, and determine the current distance and current angle between the two coordinates according to the observation point coordinates and the target projection point coordinates;
[0086] The current distance is used as the calibration distance, and the current angle is used as the calibration angle.
[0087] Further, the processor 1001 may call the image tracking simulation program stored in the memory 1005, and further perform the following operations:
[0088] When there is an angle deviation between the imaging device and the projection screen, obtaining the height tilt angle of the imaging device in the height direction and the azimuth tilt angle corresponding to the azimuth;
[0089] Correcting the coordinates of the observation point on the projection screen according to the elevation tilt angle and the azimuth tilt angle;
[0090] Re-determine the calibration distance and calibration angle based on the corrected observation point coordinates.
[0091] This embodiment uses the above scheme to construct a simulated imaging environment; in the simulated imaging environment, the distance and angle of the imaging device relative to the projection screen are calibrated according to the preset least mean square algorithm LMS to obtain the calibration distance and the calibration angle; image tracking simulation is performed according to the calibration distance and the calibration angle; it is possible to obtain the distance between the focus of the television device and the projection screen without measuring, and to quickly and accurately locate the position of the imaging device focus projected on the projection screen, which greatly simplifies the calibration work of the system, and is simple to operate, simplifies the positioning of the focus of the observation device, and has good isolation from the external environment, making image tracking simulation more convenient and efficient, ensuring simulation accuracy, and improving simulation efficiency.
[0092] Based on the above hardware structure, an embodiment of the image tracking simulation method of the present invention is proposed.
[0093] Reference Figure 2 , Figure 2 FIG. 4 is a flow chart of the first embodiment of the image tracking simulation method of the present invention.
[0094] In a first embodiment, the image tracking simulation method comprises the following steps:
[0095] Step S10: construct a simulated imaging environment.
[0096] It should be noted that the simulated imaging environment is a simulated imaging environment that is built through a combination of specific structures or accessories and can isolate interference from the external environment to realize image tracking simulation. The purpose of constructing the simulated imaging environment is to isolate interference factors from the external environment and make image tracking simulation more convenient and efficient. The construction method can be various, and this embodiment does not limit this.
[0097] Step S20: Calibrate the distance and angle of the imaging device relative to the projection screen according to a preset least mean square algorithm LMS in the simulated imaging environment to obtain a calibrated distance and a calibrated angle.
[0098] It should be noted that the preset least mean square algorithm (Least Mean Square, LMS) is a pre-set adaptive least mean square algorithm. LMS can simplify the equipment calibration process, avoid the measurement of the relative position between the focus of the observation device and the projection screen, improve the simulation accuracy, and make the simulation process more convenient and efficient. According to LMS, the distance and angle of the imaging device relative to the projection screen can be calibrated to obtain the calibrated distance and angle, that is, the calibrated distance and the calibrated angle.
[0099] Step S30: performing image tracking simulation according to the calibration distance and the calibration angle.
[0100] It should be noted that the calibration distance and the calibration angle may be used as simulation model parameters to start image tracking simulation, that is, the calibration distance and the calibration angle may be used as control inputs of the simulation model.
[0101] This embodiment uses the above scheme to construct a simulated imaging environment; in the simulated imaging environment, the distance and angle of the imaging device relative to the projection screen are calibrated according to the preset least mean square algorithm LMS to obtain the calibration distance and the calibration angle; image tracking simulation is performed according to the calibration distance and the calibration angle; it is possible to obtain the distance between the focus of the television device and the projection screen without measuring, and to quickly and accurately locate the position of the imaging device focus projected on the projection screen, which greatly simplifies the calibration work of the system, and is simple to operate, simplifies the positioning of the focus of the observation device, and has good isolation from the external environment, making image tracking simulation more convenient and efficient, ensuring simulation accuracy, and improving simulation efficiency.
[0102] Further, Figure 3 FIG. 1 is a flow chart of a second embodiment of the image tracking simulation method of the present invention. Figure 3 As shown, a second embodiment of the image tracking simulation method of the present invention is proposed based on the first embodiment. In this embodiment, the step S10 specifically includes the following steps:
[0103] Step S11, the imaging device and the projection screen are placed in a relatively closed imaging environment through a box, one side of the box is the projection screen, and the other side of the box is the imaging device fixed by a turntable mechanism.
[0104] It should be noted that the external environmental interference can be isolated by the box, and there are imaging equipment and projection screen in the box. One side of the box is the projection screen, and the other side of the box is the imaging equipment fixed by a turntable mechanism. The turntable mechanism is an adjustable bracket structure, which can adjust the distance and angle of the imaging equipment relative to the projection screen.
[0105] Step S12: the imaging device faces the projection screen, the projection screen displays an image of the observed object, and a simulated imaging environment is formed by the box, the imaging device and the projection screen.
[0106] It can be understood that, generally by default, the imaging device is facing the projection screen, and the projection screen displays the image of the observed object. The observed object is the observed target that the imaging device needs to perform image tracking. The box, the imaging device and the projection screen form a simulated imaging environment, and then the position of the imaging device focus projection on the projection screen is quickly and accurately located, which greatly simplifies the calibration work of the system.
[0107] Through the above scheme, this embodiment uses a box to place the imaging device and the projection screen in a relatively closed imaging environment. One side of the box is the projection screen, and the other side of the box is the imaging device fixed by a turntable mechanism; the imaging device is facing the projection screen, and the projection screen displays the image of the observed object. The box, the imaging device and the projection screen form a simulated imaging environment, which can isolate external interference factors, does not need to measure the distance between the focus of the television device and the projection screen, and can quickly and accurately locate the position of the imaging device focus projection on the projection screen, which greatly simplifies the calibration work of the system.
[0108] Further, Figure 4 FIG. 4 is a flow chart of a third embodiment of the image tracking simulation method of the present invention. Figure 4 As shown, a third embodiment of the image tracking simulation method of the present invention is proposed based on the first embodiment. In this embodiment, the step S20 specifically includes the following steps:
[0109] Step S21: acquiring a preset horizontal distance between an imaging device and a projection screen in the simulated imaging environment.
[0110] It should be noted that the preset horizontal distance is a preset default horizontal distance between the imaging device and the projection screen, and the preset horizontal distance is a distance that needs to be calibrated.
[0111] Step S22: obtaining the relative position and posture information between the imaging device and the observed object, and obtaining the size and shape information of the observed object.
[0112] It can be understood that the imaging device and the observed object have a relative position and corresponding posture information, the relative position is the spatial relative distance position of the imaging device and the observed object, the posture information is the information corresponding to the spatial relative angle orientation posture of the imaging device and the observed object, and the size and shape information of the observed object is the size information and shape contour information of the observed object.
[0113] Step S23, calibrating the distance and angle of the imaging device relative to the projection screen according to the preset horizontal distance, the relative position, the posture information and the size and shape information in combination with a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle.
[0114] It should be noted that the preset least mean square algorithm LMS performs correlation operations on parameters such as the preset horizontal distance, the relative position, the posture information and the size and shape information, thereby calibrating the distance and angle of the imaging device relative to the projection screen to obtain the calibrated distance and calibration angle.
[0115] This embodiment adopts the above scheme, by obtaining the preset horizontal distance between the imaging device and the projection screen in the simulated imaging environment; obtaining the relative position and posture information between the imaging device and the observed object, and obtaining the size and shape information of the observed object; calibrating the distance and angle of the imaging device relative to the projection screen according to the preset horizontal distance, the relative position, the posture information and the size and shape information in combination with the preset least mean square algorithm LMS, to obtain the calibrated distance and the calibrated angle; the calibration work of the system can be greatly simplified, and the operation is simple, the positioning of the focus of the observation device is simplified, and the isolation from the external environment is good, so that the image tracking simulation is more convenient and efficient, the simulation accuracy is guaranteed, and the simulation efficiency is improved.
[0116] Further, Figure 5 FIG. 4 is a flow chart of a fourth embodiment of the image tracking simulation method of the present invention. Figure 5 As shown, a fourth embodiment of the image tracking simulation method of the present invention is proposed based on the third embodiment. In this embodiment, the step S23 specifically includes the following steps:
[0117] Step S231: Obtain the screen resolution of the projection screen, and determine the physical size of a single pixel according to the screen resolution and the size and shape information.
[0118] It should be noted that the screen resolution is the screen resolution of the projection screen. The physical size of a single pixel can be determined by combining the size and shape information with the screen resolution. In a specific implementation, the physical size of a single pixel is determined by the following formula according to the screen resolution and the size and shape information:
[0119]
[0120] Among them, k xi is the physical size of a single pixel on the X axis, k yiis the Y-axis physical size of a single pixel, L is the length of the projection screen obtained from the size and shape information, W is the width of the projection screen obtained from the size and shape information, and x imax and imax is the screen resolution.
[0121] Step S232: determining the elevation angle and azimuth angle of the imaging device according to the preset horizontal distance, the relative position and the posture information.
[0122] It can be understood that the elevation angle and azimuth angle of the television device output relative to the projection screen can be determined based on the preset horizontal distance, the relative position and the posture information.
[0123] In the specific implementation, refer to Figure 6 , Figure 6 FIG. 1 is a diagram showing the relationship between the observation device and the projection screen in the image tracking simulation method of the present invention. Figure 6 As shown, let the focus of the TV device be O c , the intersection of its axis and the projection screen is O, and the distance is l. With O as the origin, a plane coordinate system Oxy is established on the projection screen. Let P represent the observation point, and its projections on Ox and Oy are N and M. The TV observation equipment outputs the elevation angle α and the azimuth angle β. Define the elevation angle α = ∠MO c O, azimuth angle β = ∠NO c O, the projection screen is divided into four quadrants using the coordinate system Oxy. The elevation angles in quadrants I and II are positive, and the azimuth angles in quadrants II and III are positive. i x i y i Represents the screen pixel coordinate system.
[0124] Step S233: Obtain a preset state quantity, preset filter coefficient and objective function of a preset least mean square algorithm LMS.
[0125] It should be understood that the preset state quantity is the quantity of different states exhibited by the preset least mean square algorithm LMS. The preset state quantity can be selected from quantities that are inconvenient to measure directly or difficult to measure accurately. The filter coefficient is the coefficient used by the preset least mean square algorithm LMS to filter data. The objective function is the observation pixel position corresponding to the observation point of the observation device.
[0126] Step S234, determining the initial pixel position of the projection point according to the physical size, the elevation angle, the azimuth angle, the preset state quantity, and the preset filter coefficient.
[0127] It can be understood that the initial pixel position can be obtained by the following formula according to the physical size, the elevation angle, the azimuth angle, the preset state quantity, and the preset filter coefficient:
[0128]
[0129] Among them, k xi is the physical size of a single pixel on the X axis, k yi is the physical size of a single pixel on the Y axis, x ip is the X-axis coordinate of the initial pixel position of the observation point, y ip is the Y-axis coordinate of the initial pixel position of the observation point, α is the elevation angle, β is the azimuth angle, l is the preset horizontal distance, and x is the horizontal distance. io is the X-axis coordinate of the pixel corresponding to the origin, y io is the Y-axis coordinate of the pixel corresponding to the coordinate origin, ω is the preset filter coefficient, and x is the preset state quantity.
[0130] In the specific implementation, the above formula is obtained by the following method: Continue to refer to Figure 6 , the altitude and azimuth are output by the observation equipment, and the pixel position of point P (x ip ,y ip ) is read from the screen. The screen size and resolution can be obtained by checking the device properties, i.e., k xi and k yi It can be calculated; the amount to be calibrated is the distance l, the pixel position of the projection point O (x io ,y io )(i.e. the zero position of coordinate Oxy); let the coordinate of point P in Oxy be (x p ,y p ), then:
[0131]
[0132] From the definition of altitude and azimuth, we can know that: Combining these formulas, we can get:
[0133]
[0134] That can be expressed as:
[0135]
[0136] Step S235 , determining an error function according to the objective function and the initial pixel position, and determining a gradient vector between the error function and the preset filter coefficient according to the error function.
[0137] It can be understood that the error function is determined according to the objective function and the initial pixel position by the following formula:
[0138]
[0139] Under stationary conditions, the mean square error can be expressed as:
[0140]
[0141] In the above formula, p = E[d k x k ] represents the correlation vector between the objective function and the state vector; represents the autocorrelation matrix of the state vector. In this paper, the superscript T represents the transpose of a vector or matrix. For the system sampling at time k, Set the objective function to d k According to the error function, the maximum gradient vector gk between the error function and the preset filter coefficient can be determined by the following formula:
[0142]
[0143] Step S236: Converge the preset filter coefficients through the convergence factor of the LMS and the gradient vector to obtain target filter coefficients.
[0144] It can be understood that by converging the preset filter coefficients to obtain the target filter coefficients, the accuracy of the image tracking simulation can be improved.
[0145] Furthermore, the step S236 specifically includes the following steps:
[0146] The preset filter coefficient is converged by the convergence factor of the LMS and the gradient vector, and the target filter coefficient is obtained by the following formula:
[0147] ω k+1 =ω k +2μe k x k
[0148] Among them, ω k+1 is the target filter coefficient, ω k is the preset filter coefficient, μ is the convergence factor, 2e k x k is the instantaneous value of the gradient vector estimate.
[0149] In the specific implementation, the preset state quantity and the initial value of the preset filter coefficient can be randomly selected. The pseudo code of the adaptive LMS method is as follows:
[0150] for k=1:N
[0151]
[0152]
[0153]
[0154]
[0155] By randomly selecting the initial values of ω and x and choosing an appropriate gain convergence coefficient μ, we can get the estimated value of the state quantity x after convergence, that is, the required calibration result; the adaptive LMS has low computational complexity, is easy to converge in a stable environment, can converge unbiasedly to obtain the Wiener solution, and uses finite precision to achieve algorithm stability.
[0156] The above analysis is based on the assumption that the imaging device is facing the projection screen and has no angle deviation. The angle deviation correction method is as shown in the following embodiment.
[0157] It should be understood that, generally, the search can be performed according to the steepest descent rate:
[0158]
[0159] μ represents the convergence factor, Indicates g k The estimated value of g is given by taking the instantaneous value of p and R as k The estimated value of the target filter coefficient is established because the instantaneous value is taken. Taking a convergence factor within a certain range can ensure the convergence of the filter coefficient. The value range of the convergence factor μ is Among them, tr represents the trace of the matrix, which is the sum of the diagonal elements of the trace matrix.
[0160] Step S237, calibrating the target projection point coordinates according to the target filter coefficient, and determining the calibration distance and calibration angle according to the target projection point coordinates.
[0161] It can be understood that the coordinates of the target projection point can be determined by the target coefficient. The target projection point is the coordinates of the projection point of the observation device on the projection screen relative to the measured target. After the target coefficient is determined, it can be obtained according to the calculation method of the initial pixel position.
[0162] Furthermore, the step S237 specifically includes the following steps:
[0163] Determine the target projection point coordinates according to the target filter coefficient and the preset state quantity;
[0164] Acquire the observation point coordinates of the imaging device, and determine the current distance and current angle between the two coordinates according to the observation point coordinates and the target projection point coordinates;
[0165] The current distance is used as the calibration distance, and the current angle is used as the calibration angle.
[0166] It should be understood that the formula for obtaining the initial pixel position is obtained by applying the determined target filter coefficient and the preset state quantity.
[0167]
[0168] It should be noted that the projection point position is generated on the screen, reflecting the motion relationship of the simulation model, so xip and yip are known. In fact, the calibration quantity requires distance 1, focus projection position xi0 and yi0, that is, the final value of the two coefficient omega estimated in the pseudocode. The commonly used method is to take the final coefficient value or the average of the final few as the estimated value, and the target projection point coordinates can be obtained. After obtaining the observation point coordinates of the imaging device, the current distance and current angle between the two coordinates can be determined according to the observation point coordinates and the target projection point coordinates, and then the current distance is used as the calibration distance, and the current angle is used as the calibration angle.
[0169] Through the above scheme, this embodiment obtains the screen resolution of the projection screen, determines the physical size of a single pixel according to the screen resolution and the size and shape information; determines the elevation and azimuth of the imaging device according to the preset horizontal distance, the relative position and the posture information; obtains the preset state quantity, preset filter coefficient and target function of the preset least mean square algorithm LMS; determines the initial pixel position of the projection point according to the physical size, the elevation, the azimuth, the preset state quantity and the preset filter coefficient; determines the error function according to the target function and the initial pixel position, and determines the gradient vector of the error function and the preset filter coefficient according to the error function; converges the preset filter coefficient through the convergence factor of the LMS and the gradient vector to obtain the target filter coefficient; calibrates the target projection point coordinates according to the target filter coefficient, and determines the calibration distance and calibration angle according to the target projection point coordinates, which can greatly simplify the calibration work of the system, is simple to operate, simplifies the positioning of the focus of the observation device, has good isolation from the external environment, makes image tracking simulation more convenient and efficient, ensures simulation accuracy, and improves simulation efficiency.
[0170] Further, Figure 7 FIG. 5 is a flow chart of a fifth embodiment of the image tracking simulation method of the present invention. Figure 7 As shown, a fifth embodiment of the image tracking simulation method of the present invention is proposed based on the first embodiment. In this embodiment, before step S30, the image tracking simulation method further includes the following steps:
[0171] Step S301: when there is an angle deviation between the imaging device and the projection screen, obtaining the height tilt angle of the imaging device in the height direction and the azimuth tilt angle corresponding to the azimuth.
[0172] It is understandable that when there is an angle deviation between the imaging device and the projection screen, the angle deviation can be corrected, and the height tilt angle of the imaging device in the height direction and the azimuth tilt angle corresponding to the azimuth need to be obtained at the current moment.
[0173] Step S302: Correct the coordinates of the observation point on the projection screen according to the elevation tilt angle and the azimuth tilt angle.
[0174] It should be understood that the elevation angle can be adjusted to obtain a new elevation angle through the elevation tilt angle, and the azimuth angle can be adjusted to obtain a new azimuth angle through the azimuth tilt angle. The coordinates of the observation point can be recalculated based on the new elevation angle and azimuth angle, thereby completing the correction of the coordinates of the observation point on the projection screen.
[0175] Step S303: re-determine the calibration distance and calibration angle according to the corrected observation point coordinates.
[0176] It is understandable that a new calibration distance and a new calibration angle can be re-determined according to the corrected observation point coordinates through the method of the above embodiment.
[0177] In the specific implementation, see Figure 8 , Figure 8 Schematic diagram of the angle deviation in the high and low angle directions in the image tracking simulation method of the present invention. Figure 8 As shown, considering the high and low angle directions (the azimuth direction analysis is the same), O c is the focus of the camera, O is the projection point of the focus on the projection screen, the intersection of the camera axis and the projection screen is projected at O′ in the vertical axis, a certain imaging point is P(x, y), the corresponding measurement height angle is α, the camera height tilt angle is Δα, and the geometric relationship is:
[0178] |OP|=l·tan(α+Δα)≈l·tanα+l·Δα+l·Δα·tan 2 α
[0179] Therefore, the two measurement points P i (x i ,y i ) and P j (x j ,y j ) There is a relationship:
[0180] |P i P j |=y j -y i = l(tanα j -tanα i )+lΔα(tan 2 αj -tan 2 α i )
[0181] make Δα can be calculated using the method shown in the first section; similarly, Δβ can be calculated.
[0182] The modified form of the above embodiment with Δα and Δβ is The calibration method with deviation angle can be obtained.
[0183] When the imaging device and the projection screen are placed in the correct position and the calibration error meets the simulation requirements, the angle correction can be ignored to simplify the model. After the calibration is completed, the calibration amount is used as the simulation model parameter to start the image tracking simulation.
[0184] Through the above scheme, this embodiment obtains the height tilt angle of the imaging device in the height direction and the azimuth tilt angle corresponding to the azimuth when there is an angular deviation between the imaging device and the projection screen; corrects the observation point coordinates on the projection screen according to the height tilt angle and the azimuth tilt angle; and redetermines the calibration distance and calibration angle according to the corrected observation point coordinates. When there is an angular deviation, correction can be performed by calculation, thereby effectively ensuring simulation accuracy and improving simulation efficiency.
[0185] Correspondingly, the present invention further provides an image tracking simulation device.
[0186] Reference Fig. 9 , Fig. 9 It is a functional module diagram of the first embodiment of the image tracking simulation device of the present invention.
[0187] In a first embodiment of the image tracking simulation device of the present invention, the image tracking simulation device comprises:
[0188] The construction module 10 is used to construct a simulated imaging environment.
[0189] The calibration module 20 is used to calibrate the distance and angle of the imaging device relative to the projection screen according to a preset least mean square algorithm LMS in the simulated imaging environment to obtain a calibrated distance and a calibrated angle.
[0190] The simulation module 30 is used to perform image tracking simulation according to the calibration distance and the calibration angle.
[0191] The steps implemented by the functional modules of the image tracking simulation device may refer to the various embodiments of the image tracking simulation method of the present invention, and will not be described in detail here.
[0192] In the specific implementation, see Fig.10 , Fig.10FIG. 1 is a schematic diagram of the imaging structure of the image tracking simulation device of the present invention, as shown in FIG. Fig.10 As shown, the box is used to simulate the imaging environment and isolate the interference of the external environment. One side of the box is a projection screen, which generates an image of the observed object, and the other side is fixed with an imaging device, which is fixed on a turntable and can be fine-tuned in azimuth and can move a certain distance; the fixed bracket is used to fix the observation device, which can move the device forward and backward a certain distance, and the posture can be fine-tuned to a certain extent; the image generation computer is used to generate an object motion image on the projection screen according to the relative position and posture information of the observation device and the observed object calculated by the relative motion model, the distance between the observation device and the projection screen, and the size and shape of the observed object; the simulation system includes a simulation system of the observation device and the motion model of the observed object, which receives the imaging information of the imaging device and solves it into the observation angle information as the control input of the model.
[0193] In addition, an embodiment of the present invention further provides a storage medium, on which an image tracking simulation program is stored, and when the image tracking simulation program is executed by a processor, the following operations are implemented:
[0194] Constructing a simulated imaging environment;
[0195] In the simulated imaging environment, the distance and angle of the imaging device relative to the projection screen are calibrated according to a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle;
[0196] Image tracking simulation is performed according to the calibration distance and the calibration angle.
[0197] Furthermore, when the image tracking simulation program is executed by the processor, the following operations are also implemented:
[0198] The imaging device and the projection screen are placed in a relatively closed imaging environment by means of a box, one side of the box is the projection screen, and the other side of the box is the imaging device fixed by a turntable mechanism;
[0199] The imaging device faces the projection screen, and the projection screen displays an image of the observed object. The box, the imaging device and the projection screen form a simulated imaging environment.
[0200] Furthermore, when the image tracking simulation program is executed by the processor, the following operations are also implemented:
[0201] Acquiring a preset horizontal distance between an imaging device and a projection screen in the simulated imaging environment;
[0202] Acquiring the relative position and posture information between the imaging device and the observed object, and acquiring the size and shape information of the observed object;
[0203] The distance and angle of the imaging device relative to the projection screen are calibrated according to the preset horizontal distance, the relative position, the posture information and the size and shape information in combination with a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle.
[0204] Furthermore, when the image tracking simulation program is executed by the processor, the following operations are also implemented:
[0205] Acquire the screen resolution of the projection screen, and determine the physical size of a single pixel according to the screen resolution and the size and shape information;
[0206] Determine the elevation angle and azimuth angle of the imaging device according to the preset horizontal distance, the relative position and the posture information;
[0207] Obtaining a preset state quantity, a preset filter coefficient and an objective function of a preset least mean square algorithm LMS;
[0208] Determine the initial pixel position of the projection point according to the physical size, the elevation angle, the azimuth angle, the preset state quantity, and the preset filter coefficient;
[0209] Determine an error function according to the objective function and the initial pixel position, and determine a gradient vector between the error function and the preset filter coefficient according to the error function;
[0210] Converging the preset filter coefficients through the convergence factor of the LMS and the gradient vector to obtain target filter coefficients;
[0211] The target projection point coordinates are calibrated according to the target filter coefficients, and the calibration distance and calibration angle are determined according to the target projection point coordinates.
[0212] Furthermore, when the image tracking simulation program is executed by the processor, the following operations are also implemented:
[0213] The preset filter coefficient is converged by the convergence factor of the LMS and the gradient vector, and the target filter coefficient is obtained by the following formula:
[0214] ω k+1 =ω k +2μe k x k
[0215] Among them, ω k+1 is the target filter coefficient, ω k is the preset filter coefficient, μ is the convergence factor, 2e k x k is the instantaneous value of the gradient vector estimate.
[0216] Furthermore, when the image tracking simulation program is executed by the processor, the following operations are also implemented:
[0217] Determine the target projection point coordinates according to the target filter coefficient and the preset state quantity;
[0218] Acquire the observation point coordinates of the imaging device, and determine the current distance and current angle between the two coordinates according to the observation point coordinates and the target projection point coordinates;
[0219] The current distance is used as the calibration distance, and the current angle is used as the calibration angle.
[0220] Furthermore, when the image tracking simulation program is executed by the processor, the following operations are also implemented:
[0221] When there is an angle deviation between the imaging device and the projection screen, obtaining the height tilt angle of the imaging device in the height direction and the azimuth tilt angle corresponding to the azimuth;
[0222] Correcting the coordinates of the observation point on the projection screen according to the elevation tilt angle and the azimuth tilt angle;
[0223] Re-determine the calibration distance and calibration angle based on the corrected observation point coordinates.
[0224] This embodiment uses the above scheme to construct a simulated imaging environment; in the simulated imaging environment, the distance and angle of the imaging device relative to the projection screen are calibrated according to the preset least mean square algorithm LMS to obtain the calibration distance and the calibration angle; image tracking simulation is performed according to the calibration distance and the calibration angle; it is possible to obtain the distance between the focus of the television device and the projection screen without measuring, and to quickly and accurately locate the position of the imaging device focus projected on the projection screen, which greatly simplifies the calibration work of the system, and is simple to operate, simplifies the positioning of the focus of the observation device, and has good isolation from the external environment, making image tracking simulation more convenient and efficient, ensuring simulation accuracy, and improving simulation efficiency.
[0225] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0226] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0227] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. An image tracking simulation method, characterized in that: The image tracking simulation method comprises: Constructing a simulated imaging environment; In the simulated imaging environment, the distance and angle of the imaging device relative to the projection screen are calibrated according to a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle; Performing image tracking simulation according to the calibration distance and the calibration angle; The step of calibrating the distance and angle of the imaging device relative to the projection screen in the simulated imaging environment according to a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle includes: Acquiring a preset horizontal distance between an imaging device and a projection screen in the simulated imaging environment; Acquiring the relative position and posture information between the imaging device and the observed object, and acquiring the size and shape information of the observed object; The distance and angle of the imaging device relative to the projection screen are calibrated according to the preset horizontal distance, the relative position, the posture information and the size and shape information in combination with a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle.
2. The image tracking simulation method according to claim 1, characterized in that: The constructing of the simulated imaging environment comprises: The imaging device and the projection screen are placed in a relatively closed imaging environment by means of a box, one side of the box is the projection screen, and the other side of the box is the imaging device fixed by a turntable mechanism; The imaging device faces the projection screen, and the projection screen displays an image of the observed object. The box, the imaging device and the projection screen form a simulated imaging environment.
3. The image tracking simulation method according to claim 1, characterized in that: The method of calibrating the distance and angle of the imaging device relative to the projection screen according to the preset horizontal distance, the relative position, the posture information and the size and shape information in combination with a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle includes: Acquire the screen resolution of the projection screen, and determine the physical size of a single pixel according to the screen resolution and the size and shape information; Determine the elevation angle and azimuth angle of the imaging device according to the preset horizontal distance, the relative position and the posture information; Obtaining a preset state quantity, a preset filter coefficient and an objective function of a preset least mean square algorithm LMS; Determine the initial pixel position of the projection point according to the physical size, the elevation angle, the azimuth angle, the preset state quantity, and the preset filter coefficient; Determine an error function according to the objective function and the initial pixel position, and determine a gradient vector between the error function and the preset filter coefficient according to the error function; Converging the preset filter coefficients through the convergence factor of the LMS and the gradient vector to obtain target filter coefficients; The target projection point coordinates are calibrated according to the target filter coefficients, and the calibration distance and calibration angle are determined according to the target projection point coordinates.
4. The image tracking simulation method according to claim 3, characterized in that: The step of converging the preset filter coefficient by using the convergence factor of the LMS and the gradient vector to obtain the target filter coefficient comprises: The preset filter coefficient is converged by the convergence factor of the LMS and the gradient vector, and the target filter coefficient is obtained by the following formula: oh k+1 =ω k +2μe k x k Among them, ω k+1 is the target filter coefficient, ω k is the preset filter coefficient, μ is the convergence factor, 2e k x k is the instantaneous value of the gradient vector estimate.
5. The image tracking simulation method according to claim 3, characterized in that: The step of calibrating the target projection point coordinates according to the target filter coefficient, and determining the calibration distance and calibration angle according to the target projection point coordinates includes: Determine the target projection point coordinates according to the target filter coefficient and the preset state quantity; Acquire the observation point coordinates of the imaging device, and determine the current distance and current angle between the two coordinates according to the observation point coordinates and the target projection point coordinates; The current distance is used as the calibration distance, and the current angle is used as the calibration angle.
6. The image tracking simulation method according to any one of claims 1 to 5, characterized in that: Before performing the image tracking simulation according to the calibration distance and the calibration angle, the image tracking simulation method further includes: When there is an angle deviation between the imaging device and the projection screen, obtaining the height tilt angle of the imaging device in the height direction and the azimuth tilt angle corresponding to the azimuth; Correcting the coordinates of the observation point on the projection screen according to the elevation tilt angle and the azimuth tilt angle; Re-determine the calibration distance and calibration angle based on the corrected observation point coordinates.
7. An image tracking simulation device, characterized in that: The image tracking simulation device comprises: A building module for building a simulated imaging environment; A calibration module, used to calibrate the distance and angle of the imaging device relative to the projection screen in the simulated imaging environment according to a preset least mean square algorithm LMS to obtain a calibration distance and a calibration angle; A simulation module, used for performing image tracking simulation according to the calibration distance and the calibration angle; The calibration module is also used to obtain a preset horizontal distance between the imaging device and the projection screen in the simulated imaging environment; obtain the relative position and posture information between the imaging device and the observed object, and obtain the size and shape information of the observed object; calibrate the distance and angle of the imaging device relative to the projection screen according to the preset horizontal distance, the relative position, the posture information and the size and shape information in combination with a preset least mean square algorithm LMS to obtain a calibrated distance and a calibrated angle.
8. An image tracking simulation device, characterized in that: The image tracking simulation device comprises: a memory, a processor, and an image tracking simulation program stored in the memory and executable on the processor, wherein the image tracking simulation program is configured to implement the steps of the image tracking simulation method according to any one of claims 1 to 6.
9. A storage medium, characterized in that: An image tracking simulation program is stored on the storage medium, and when the image tracking simulation program is executed by the processor, the steps of the image tracking simulation method according to any one of claims 1 to 6 are implemented.
