Motion Tracking Algorithm Simulation Test Method, System, Device and Storage Medium

Abstract

The present application provides a method, system, device and storage medium for simulating and testing a motion tracking algorithm. The method includes determining a simulated viewing angle range according to the field of view angle of a camera and an observation point; according to the simulated viewing angle range, performing a planar projection mapping on the spherical region of a panoramic video to obtain a camera-captured video of a planar region; and using the camera-captured video to test a motion tracking algorithm to be tested, realizing multiple simulations based on the input of the same panoramic video to generate a simulated competition scene in a virtual environment, and realizing multiple verifications and comparative analyses of the algorithm when assisting in the development, verification and testing of the motion tracking algorithm. To a certain extent, it solves the defects of testing a target tracking algorithm in a real competition scene at present.

Description

Technical Field

[0001] This application belongs to the field of computer vision, and particularly relates to a method, system, device, and storage medium for simulating and testing motion tracking algorithms. Background Art

[0002] In the field of computer vision technology, developing an automated focus capture algorithm based on computer vision requires a large amount of algorithm testing and verification. However, due to the real-time and unique nature of sports competitions, there are problems with image quality loss when using ordinary cameras to shoot the competition and then playing back the video. Especially when the video is captured a second time through the lens to the monitor, the clarity decreases significantly. At the same time, the size of the monitor limits the viewing angle and it is difficult to present the full picture of the competition in all directions.

[0003] In addition, when testing the target tracking algorithm in a real competition scenario, since each competition is unique, if a problem with the algorithm is found, the modified algorithm cannot be verified a second time in the same scenario. The testing of the algorithm cannot control the input to compare the output results, and it is impossible to perfectly compare the advantages and disadvantages of the algorithms.

[0004] Therefore, in the aspect of video intelligent analysis and tracking, it is urgent to simulate the real competition scenario in a virtual environment based on the panoramic video for the motion tracking algorithm simulation to assist in the development, verification, and testing of the motion tracking algorithm.

[0005] The above statements are only used to provide background technical information related to this application. Unless otherwise indicated, the content described in this section is not prior art for other parts of this application. Summary of the Invention

[0006] In view of the above existing technical limitations, this application provides a method, system, device, and storage medium for simulating and testing motion tracking algorithms, which can generate multiple simulated competition scenarios in a virtual environment based on the input of the same 360° panoramic video, and realize multiple verifications and comparative analyses of the algorithm when assisting in the development, verification, and testing of the motion tracking algorithm.

[0007] In addition, this application improves the accuracy and authenticity of algorithm testing through closed-loop control and panoramic video restoration; this application separates the tested algorithm from the test platform, the test server is responsible for data acquisition and pan-tilt control, the algorithm client independently processes algorithm analysis, allows developers to freely integrate different algorithm SDKs, and the ends communicate through Socket to achieve flexible algorithm integration testing, making the platform more flexible and multi-scenario; this application supports multiple operating systems such as iOS and Android through cross-platform design to adapt to various scenario requirements.

[0008] According to the first aspect of the embodiments of this application, a method for simulating and testing a motion tracking algorithm is provided, including:

[0009] Determine the simulated viewing angle range according to the field of view angle of the camera and the observation point;

[0010] According to the simulated viewing angle range, project the spherical area of the panoramic video onto a plane to obtain the camera acquisition video of the plane area; the camera acquisition video is used to test the motion tracking algorithm to be tested.

[0011] In some embodiments of the present application, after obtaining the camera acquisition video of the plane area, it further includes:

[0012] Input the camera acquisition video into the motion tracking algorithm to be tested for analysis; the motion tracking algorithm to be tested is used to identify key motion events in the video or lock on to the target object;

[0013] Generate a pan-tilt control strategy or a new observation point;

[0014] Update the pan-tilt parameters according to the pan-tilt control strategy or the new observation point, and determine the new simulated viewing angle range;

[0015] According to the new camera acquisition video corresponding to the new simulated viewing angle range, further input it into the motion tracking algorithm to be tested for testing until the test ends.

[0016] In some embodiments of the present application, the pan-tilt control strategy includes controlling at least one of the yaw value, pitch value, and roll value of the pan-tilt parameters; the center of the simulated viewing angle range is the observation point.

[0017] In some embodiments of the present application, determining the simulated viewing angle range according to the field of view angle of the camera and the observation point includes:

[0018] Obtain the field of view angle of the camera and the pan-tilt parameters; the pan-tilt parameters include the yaw value, pitch value, and roll value;

[0019] Determine the observation point according to the yaw value, pitch value, and roll value;

[0020] Determine the simulated viewing angle range according to the field of view angle and the observation point.

[0021] In some embodiments of the present application, the camera is a simulated camera and the pan-tilt is a virtual pan-tilt; the field of view angle of the simulated camera is set customarily.

[0022] According to the second aspect of the embodiments of the present application, a motion tracking algorithm simulation test method is provided, which is applied to a server and includes:

[0023] Determine the simulated viewing angle range according to the field of view angle of the camera and the observation point;

[0024] According to the simulated viewing angle range, project the spherical area of the panoramic video onto a plane to obtain the camera acquisition video of the plane area;

[0025] Send the video captured by the camera to the client so that the client can input the video captured by the camera into the motion tracking algorithm to be tested for analysis, and then generate a pan-tilt control strategy or a new observation point; the motion tracking algorithm to be tested is used to identify key motion events in the video or lock on to a target object.

[0026] Update the pan-tilt parameters according to the pan-tilt control strategy or the new observation point, and determine a new simulated viewing angle range.

[0027] Further input the new video captured by the camera corresponding to the new simulated viewing angle range into the motion tracking algorithm to be tested of the target device for testing until the test is completed.

[0028] According to the third aspect of the embodiments of the present application, a method for simulating and testing a motion tracking algorithm is provided, which is applied to a client and includes:

[0029] Receive the video captured by the camera from the server; the server is used to determine a simulated viewing angle range according to the field of view angle of the camera and the observation point; according to the simulated viewing angle range, project and map the spherical area of the panoramic video onto a plane area to obtain the video captured by the camera in the plane area.

[0030] Input the video captured by the camera into the motion tracking algorithm to be tested for analysis; the motion tracking algorithm to be tested is used to identify key motion events in the video or lock on to a target object.

[0031] Generate a pan-tilt control strategy or a new observation point; send it to the server so that the server can update the pan-tilt parameters according to the pan-tilt control strategy or the new observation point, determine a new simulated viewing angle range; and further input the new video captured by the camera corresponding to the new simulated viewing angle range into the motion tracking algorithm to be tested of the target device for testing until the test is completed.

[0032] According to the fourth aspect of the embodiments of the present application, a system for simulating and testing a motion tracking algorithm is provided, including:

[0033] A pan-tilt module, which is used to determine a simulated viewing angle range according to the field of view angle of the camera and the observation point.

[0034] A simulated camera capture module, which is used to map the spherical area of the panoramic video to obtain the video captured by the camera in the plane area according to the simulated viewing angle range; the video captured by the camera is used to test the motion tracking algorithm to be tested.

[0035] In some embodiments of the present application, the system includes a server and a client; the server includes a pan-tilt module and a simulated camera capture module;

[0036] The client includes a detection module; the detection module is equipped with the motion tracking algorithm to be tested.

[0037] A detection module, which is configured to input the video collected by the camera into the motion tracking algorithm to be tested for analysis, identify key motion events in the video or lock on to the target object, and obtain the simulation results of the algorithm.

[0038] In some embodiments of the present application, the client further includes a policy module, and the server further includes a pan-tilt control module;

[0039] The policy module is configured to generate a pan-tilt control policy or a new observation point according to the simulation results of the algorithm;

[0040] The pan-tilt control module is configured to update the pan-tilt parameters according to the pan-tilt control policy or the new observation point, and determine a new simulation viewing angle range;

[0041] The detection module further inputs the new camera-collected video corresponding to the new simulation viewing angle range into the motion tracking algorithm to be tested until the test ends.

[0042] According to the fifth aspect of the embodiments of the present application, there is provided a motion tracking algorithm simulation test device, including: a storage unit, configured to store executable instructions; and a processing unit, configured to be connected to the memory to execute the executable instructions so as to complete the motion tracking algorithm simulation test method.

[0043] According to the sixth aspect of the embodiments of the present application, there is provided a computer-readable storage medium, on which a computer program is stored; the computer program is executed by a processor to implement the motion tracking algorithm simulation test method.

[0044] By using the motion tracking algorithm simulation test method, system, device and medium of the present application, the method includes determining a simulation viewing angle range according to the field of view angle of the camera and the observation point; according to the simulation viewing angle range, performing a planar projection mapping on the spherical area of the panoramic video to obtain the camera-collected video of the planar area; the camera-collected video is used to test the motion tracking algorithm to be tested, realizing multiple simulations based on the input of the same 360° panoramic video to generate a simulated competition scene in a virtual environment, and realizing multiple verifications and comparative analyses of the algorithm when assisting in the development, verification and testing of the motion tracking algorithm. To a certain extent, it solves the defects of the current target tracking algorithm test in a real competition scene.

[0045] By establishing a simulation platform based on a 360° panoramic video, the present application can simulate and restore the 360° panoramic video recording the competition into a competition in the platform, realizing that the input of the same scene is repeatedly used for testing the motion tracking algorithm, flexibly simulating the motion scene captured by the camera, and providing a controllable virtual environment for the development and verification of the motion tracking algorithm.

[0046] Compared with the traditional recording and playback method, the simulation platform of the present application can: 1. Simulate sports competition scenes from multiple angles and scenarios, providing rich training and test data; 2. Evaluate the adaptability of different algorithms in various situations by comparing the results output by different algorithms; 3. Provide higher clarity and flexibility, break through the limitations of monitors and cameras, and achieve panoramic tracking and algorithm verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The schematic embodiments and descriptions thereof of the present application are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0048] Figure 1 FIG. shows a schematic diagram of the steps of a motion tracking algorithm simulation test method according to an embodiment of the present application;

[0049] Figure 2 FIG. shows a schematic diagram of the steps of another motion tracking algorithm simulation test method according to an embodiment of the present application;

[0050] Figure 3 FIG. shows a schematic diagram of the principle of a simulation test closed-loop control according to an embodiment of the present application;

[0051] Figure 4 FIG. shows a schematic diagram of the steps of determining a simulation viewing angle range according to an embodiment of the present application;

[0052] Figure 5 FIG. shows a schematic diagram of the principle of a motion tracking algorithm simulation test method according to an embodiment of the present application;

[0053] Figure 6 FIG. shows a flowchart of the principle of a motion tracking algorithm simulation test method according to an embodiment of the present application;

[0054] Figure 7 FIG. is a schematic diagram of mapping a spherical region to a rectangular region according to an embodiment of the present application;

[0055] Figure 8 FIG. shows a schematic diagram of the structure of a motion tracking algorithm simulation test system according to an embodiment of the present application;

[0056] Figure 9 FIG. shows a schematic diagram of the structure of another motion tracking algorithm simulation test system according to an embodiment of the present application;

[0057] Figure 10 FIG. shows a schematic diagram of the structure of a motion tracking algorithm simulation test device 400 according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] Regarding this application, in the field of computer vision technology, in order to develop a good computer vision-based automated focus capture algorithm, a large amount of panoramic data is required for algorithm testing and verification.

[0059] Currently, when testing the target tracking algorithm in a real competition scenario, since each competition is unique, the algorithm cannot be verified twice in the same scenario. The testing of the algorithm cannot control the input to compare the output results, and it is impossible to perfectly compare the advantages and disadvantages of the algorithms.

[0060] Or when using a pure simulated competition scenario, not only a large amount of data is required, but also complex physical calculations and environmental simulations are involved, increasing the cost and time of development and testing, and it is difficult to simulate a real competition situation. The traditional method of making competition simulations has a huge workload, and the results often deviate from the actual situation, making it difficult to truly reflect the detailed changes during the movement process. This also leads to the relatively limited verification environment for current motion tracking algorithms, lacking flexible and controllable simulation tools to test the performance and accuracy of different algorithms.

[0061] Generally speaking, the existing motion tracking technologies lack a flexible simulation platform for verifying the performance of different algorithms in complex motion scenarios. Simply shooting competition videos with a camera cannot simulate the tracking effects from multiple angles, resulting in being limited by the existing materials and recording devices when developing and optimizing algorithms, and being unable to comprehensively evaluate the algorithm performance.

[0062] Currently, there is a lack of a platform system that can take into account the entire competition scenario video and support zooming in and analyzing local areas. This platform system plays an important role in the implementation, verification, and testing of motion tracking algorithms.

[0063] Based on the above considerations, this application provides a motion tracking algorithm simulation test method, system, device, and medium. The method includes determining the simulated viewing angle range according to the field of view angle of the camera and the observation point; according to the simulated viewing angle range, performing a planar projection mapping on the spherical region of the panoramic video to obtain the camera acquisition video of the planar region; the camera acquisition video is used to test the motion tracking algorithm to be tested, realizing multiple simulations based on the input of the same 360° panoramic video to generate a simulated competition scenario in a virtual environment, and realizing multiple verifications and comparative analyses of the algorithm when assisting in the development, verification, and testing of the motion tracking algorithm. To a certain extent, it solves the defects of currently testing the target tracking algorithm in a real competition scenario.

[0064] Regarding the 360° panoramic video, a 360-degree panoramic camera can monitor and cover the surrounding scene without blind spots, and is equipped with a fish-eye lens or a reflective mirror (such as a parabolic or hyperbolic mirror), or is formed by splicing multiple ordinary lenses facing different directions to form a 360-degree panoramic field of view.

[0065] In addition, this application improves the accuracy and authenticity of algorithm testing through closed-loop control and panoramic video restoration; this application separates the tested algorithm from the test platform. The test server is responsible for data collection and pan-tilt control, and the algorithm client independently processes algorithm analysis, allowing developers to freely integrate different algorithm SDKs. Flexible algorithm integration testing is achieved through Socket communication between the ends, making the platform more flexible and multi-scenario; this application supports multi-operating systems such as iOS and Android through cross-platform design to adapt to various scenario requirements.

[0066] In order to make the technical solutions and advantages in the embodiments of this application clearer, the following further details the exemplary embodiments of this application with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than an exhaustive list of all embodiments. It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other.

[0067] Embodiment 1

[0068] Figure 1 The schematic diagram of the steps of the motion tracking algorithm simulation test method according to the embodiment of this application is shown.

[0069] As Figure 1 shown, a motion tracking algorithm simulation test method according to the embodiment of this application includes:

[0070] S1: Determine the simulated viewing angle range according to the field of view angle of the camera and the observation point.

[0071] S2: According to the simulated viewing angle range, perform a planar projection mapping on the spherical region of the panoramic video to obtain the camera-captured video of the planar region; the camera-captured video is used to test the motion tracking algorithm to be tested.

[0072] Compared with the current situation when testing the target tracking algorithm in a real competition scenario, where secondary verification cannot be performed in the same scenario and the test of the algorithm cannot compare the output results by controlling the input; this application realizes multiple simulations based on the input of the same 360° panoramic video to generate simulated competition scenarios in a virtual environment, and realizes multiple verifications and comparative analyses of the algorithm when assisting in the development, verification, and testing of the motion tracking algorithm. To a certain extent, it solves the defects in the current testing of the target tracking algorithm in a real competition scenario.

[0073] Figure 2 The schematic diagram of the steps of another motion tracking algorithm simulation test method according to the embodiment of this application is shown.

[0074] As Figure 2 shown, after obtaining the camera-captured video of the planar region in S2, it further includes:

[0075] S3: Input the video captured by the camera into the motion tracking algorithm to be tested for analysis; the motion tracking algorithm to be tested is used to identify key motion events in the video or lock on to the target object;

[0076] S4: Generate a pan-tilt control strategy or a new observation point;

[0077] S5: Update the pan-tilt parameters according to the pan-tilt control strategy or the new observation point, and determine the new simulated viewing angle range;

[0078] S6: Further input the new video captured by the camera corresponding to the new simulated viewing angle range into the motion tracking algorithm to be tested until the test ends.

[0079] Figure 3 The schematic diagram of the principle of the simulated test closed-loop control according to the embodiment of the present application is shown.

[0080] As Figure 3 shown, the algorithm result of the present application forms a closed-loop control of "data acquisition → analysis → strategy feedback → adjustment → re-data acquisition" by dynamically adjusting the pan-tilt to change the camera direction or the target point (observation point). Through the closed-loop control and panoramic video restoration, the accuracy and authenticity of the algorithm test are improved, and the technical effect of flexibly controlling the input variables and comprehensively evaluating the algorithm performance is achieved.

[0081] Specifically in implementation, the pan-tilt control strategy includes controlling at least one of the yaw value, pitch value, and roll value of the pan-tilt parameters; the center of the simulated viewing angle range is the observation point.

[0082] Figure 4 The schematic diagram of the steps for determining the simulated viewing angle range according to the embodiment of the present application is shown.

[0083] As Figure 4 shown, wherein, determining the simulated viewing angle range according to the field of view angle of the camera and the observation point includes:

[0084] S11: Obtain the field of view angle of the camera and the pan-tilt parameters; the pan-tilt parameters include the yaw value, pitch value, and roll value;

[0085] S12: Determine the observation point according to the yaw value, pitch value, and roll value;

[0086] S13: Determine the simulated viewing angle range according to the field of view angle and the observation point.

[0087] Preferably, the camera is an analog camera and the pan-tilt is a virtual pan-tilt; the field of view angle of the analog camera is set customarily.

[0088] Regarding a photographing device such as a camera, its field of view angle is an inherent property of the mobile terminal. For example, a mobile phone lens has a fixed field of view angle. This field of view angle can be calculated by a formula or obtained by looking up a table.

[0089] During the execution of this method, the field of view angle of the photographing device can be calculated, or the field of view angles of various models of mobile terminals can be calculated in advance and made into a data table. When this method is executed, the terminal model is obtained by calling a system function, and the field of view angle data corresponding to the model is obtained by looking up the table.

[0090] Regarding the field of view angle FOV (Field of View), it is the angular range that a camera lens can capture a scene, and is divided into horizontal (HFOV), vertical (VFOV), and diagonal (DFOV) field of view angles.

[0091] Its calculation is based on the sensor size and the lens focal length, and the formula is as follows:

[0092] ;

[0093] Among them, the sensor size is the actual physical length of the camera sensor in a certain direction (unit: mm).

[0094] The horizontal field of view angle (HFOV) can be calculated through the horizontal direction sensor size. The vertical field of view angle (VFOV) is calculated through the vertical direction sensor size. The diagonal field of view angle (DFOV) is calculated through the diagonal direction sensor size.

[0095] Regarding the observation point, or the target point, the acquisition point of a photographing device such as a camera, it is the center point of the current camera captured picture. An observation point is determined by three coordinates of y / p / r, that is, an observation point can be determined according to three parameters of yaw, pitch, and roll.

[0096] Regarding the spherical region of the panoramic video in S2, it is converted in real time following the transformation of the observation point. For example, the calculation result of the algorithm causes the pan-tilt parameters to rotate, resulting in the change of the observation point. According to the new observation point, a new camera acquisition video is intercepted and converted in the spherical region.

[0097] When specifically implemented, the camera acquisition video of the plane region obtained by plane projection mapping of the spherical region of the panoramic video in S2 is specifically realized through the following process:

[0098] 1) Input a 360° panoramic video.

[0099] 2) Coordinate transformation, mapping the panoramic pixels of the panoramic video frame into three-dimensional spherical coordinates (xyz).

[0100] First, define the parameters of the virtual camera: Viewpoint direction: determined by the yaw, pitch, and roll angles or directly determined by the viewpoint coordinates. Field of view (FOV): horizontal field of view (e.g., 90°) and vertical field of view (e.g., 60°). Imaging plane: the resolution of the output rectangular video frame (e.g., 1920×1080 pixels).

[0101] First, establish a spherical coordinate system and store the 360° panoramic video in an equidistant cylindrical projection. Each pixel corresponds to the longitude and latitude coordinates (θ, ) of the sphere; the longitude θ is the horizontal direction angle θ∈[0,2π); the latitude is the vertical direction angle, ∈[−π / 2,π / 2].

[0102] Then, convert the spherical coordinates to three-dimensional coordinates (x, y, z) using the following formula:

[0103] ;

[0104] Thus, map the three-dimensional pixels of the panoramic video to a two-dimensional plane.

[0105] 3) Viewpoint adjustment: Based on the y / p / r coordinates of the virtual camera, i.e., yaw, pitch, and roll, apply the rotation matrix to obtain the virtual camera viewpoint (x′y′z′).

[0106] 4) Perspective projection: Calculate the normalized coordinates (u, v) and map them to the output frame of the plane rectangle.

[0107] Project the rotated three-dimensional point (x′y′z′) onto the imaging plane of the virtual camera.

[0108] 5) Distortion correction: If the original video has a fish-eye distortion, correct it first and then project.

[0109] Convert the corrected fish-eye image to an equidistant cylindrical projection format as the input for subsequent spherical projection.

[0110] 6) Finally, output the plane rectangular video frame of the simulated camera viewpoint.

[0111] Utilize OpenGL or CUDA for parallel computing of coordinate transformation and interpolation to improve the processing speed. Generate a rectangular video frame of the specified viewpoint for input testing of computer vision algorithms.

[0112] Thus, by adjusting the observation point parameters, yaw, pitch, roll, and FOV, the shooting effect of a camera at any angle can be simulated. High-precision restoration, combined with a mathematical projection model and an interpolation algorithm, ensures geometric accuracy of the output image. Strong compatibility, supports fisheye distortion correction, and adapts to various panoramic camera inputs.

[0113] In specific implementation, the simulation test method of the motion tracking algorithm of this application can be independently completed by the server or the client of the platform, or can be completed by the cooperation of the server and the client.

[0114] Another simulation test method of the motion tracking algorithm provided by this application, applied to the server, includes the following steps:

[0115] 1) Determine the simulated viewing angle range according to the field of view angle of the camera and the observation point;

[0116] 2) According to the simulated viewing angle range, project the spherical area of the panoramic video onto a plane to obtain the camera-captured video of the plane area;

[0117] 3) Send the camera-captured video to the client so that the client can input the camera-captured video into the motion tracking algorithm to be tested for analysis, and then generate a pan-tilt control strategy or a new observation point; the motion tracking algorithm to be tested is used to identify key motion events in the video or lock the target object;

[0118] 4) Update the pan-tilt parameters according to the pan-tilt control strategy or the new observation point, and determine the new simulated viewing angle range;

[0119] 5) According to the new camera-captured video corresponding to the new simulated viewing angle range, further input it into the motion tracking algorithm to be tested of the target device for testing until the test ends.

[0120] Another simulation test method of the motion tracking algorithm provided by this application, applied to the client, includes the following steps:

[0121] 10) Receive the camera-captured video from the server; the server is used to determine the simulated viewing angle range according to the field of view angle of the camera and the observation point; according to the simulated viewing angle range, project the spherical area of the panoramic video onto a plane to obtain the camera-captured video of the plane area;

[0122] 20) Input the camera-captured video into the motion tracking algorithm to be tested for analysis; the motion tracking algorithm to be tested is used to identify key motion events in the video or lock the target object;

[0123] 30) Generate a pan-tilt control strategy or a new observation point; send it to the server so that the server can update the pan-tilt parameters according to the pan-tilt control strategy or the new observation point, determine a new simulated viewing angle range; and collect videos according to the new cameras corresponding to the new simulated viewing angle range, and further input them into the motion tracking algorithm to be tested of the target device for testing until the test ends. Figure 5 The schematic diagram of the principle of the motion tracking algorithm simulation test method according to the embodiment of the present application is shown.

[0124] As Figure 5 shown, the motion tracking algorithm simulation test method of the present application is carried on two parts, the server 1 and the client 2.

[0125] In the embodiment of the present application, the client 2 is a mobile terminal, such as a smart phone, a tablet computer and other devices, including client Apps using iOS, Android (Linux App, macOS App or Android App). That is, it can be carried on the intelligent pan-tilt for shooting intelligent devices; at the same time, the client is equipped with the algorithm to be tested.

[0126] The client App using iOS or Android can perform corresponding simulation operations after connecting to the server through Socket.

[0127] The server 1 is used for capturing, collecting and controlling: responsible for collecting video information (such as panoramic video or real-time camera video) and receiving control signals from the client, such as the rotation or zoom operation of the camera.

[0128] The client 2 is used for algorithm detection and strategy generation: the client is equipped with the algorithm to be tested, which is used to process video information according to the collected video provided by the server, analyze the content and output the rotation direction or target point, and control the rotation of the camera.

[0129] Specifically, for capture: capture the panoramic video of the competition, obtain the video content of the corresponding area according to the current angle information, and store it in the buffer.

[0130] For algorithm detection: analyze the video content in the buffer through the algorithm to be tested to identify key motion events or target objects. The algorithm can be pre-equipped with multiple motion tracking algorithms to be tested, such as the algorithm to be tested A and the algorithm to be tested B.

[0131] Strategy generation: formulate a motion tracking strategy according to the analysis results, such as determining the rotation direction or focus position of the camera.

[0132] Control: control the pan-tilt or other camera devices according to the strategy to achieve dynamic adjustment of the camera (such as direction rotation, focal length adjustment, etc.).

[0133] It can be seen that the above closed-loop algorithm testing process is completed through mutual communication and collaboration between the server and the client.

[0134] This application uses a dual-module design, splitting the system into two modules: a server (1) and a client (2). These modules communicate via sockets for data transmission and control commands. The server provides a data source (such as panoramic video) and receives control commands from the client. The client processes the data source and returns the results to the server through algorithmic calculations. The server then executes the corresponding operations using the pan / tilt / zoom (PTZ) or other control devices.

[0135] This application implements a closed-loop control process, forming a closed-loop control system through two-way communication between the server and the client: the server simulates video data and transmits it to the client; the client analyzes the video content, formulates a strategy, and issues control instructions; the server adjusts the direction or focal length of the camera according to the instructions; the adjusted camera captures new video content and transmits it to the client again for analysis, forming a complete closed loop.

[0136] The following is a detailed description of the technical solution of the embodiment of the present application.

[0137] Figure 6 Detailed description of the principle flow chart of the motion tracking algorithm simulation test method according to an embodiment of the present application is shown in FIG.

[0138] like Figure 6 As shown, (1) First, the server loads the 360° panoramic video and links the pan / tilt.

[0139] 360° panoramic video uses a 360° camera to capture the full range of a sporting event, ensuring every angle of the match can be recorded and simulated. 360° panoramic video is captured using specialized panoramic cameras, such as fisheye cameras. This type of video can capture the entire field of play from a single point, without blind spots.

[0140] Figure 7 Schematic diagram of mapping a spherical area into a rectangular area according to an embodiment of the present application.

[0141] like Figure 7 As shown, in this embodiment, by mapping the spherical area into a rectangular area, the simulation of the image taken by the real camera is achieved, which provides a basis for subsequent algorithm testing and analysis.

[0142] The 360° panoramic video stored in the buffer is processed using planar projection technology. The spherical region within the determined simulated viewing angle range in the 360-degree video is mapped into a rectangular region as seen by the camera, thereby generating a video that simulates the acquisition by the camera and storing it in the buffer Buffer. The simulated viewing angle range is determined by an observation point (the center point of the simulated camera) and a field of view angle. The observation point is determined according to the yaw angle, pitch angle, and roll angle.

[0143] Thus, a method for obtaining a video content that simulates the shooting of a real camera from a 360° video is realized.

[0144] Then, the pan-tilt head is connected. The server communicates with the pan-tilt head to achieve the effect of controlling the movement of the pan-tilt head according to the test algorithm. Here, the pan-tilt head can be a simulated pan-tilt head and a simulated camera.

[0145] (2) Then, the server establishes a socket connection with the client.

[0146] Among them, socket communication is an abstract interface between the application layer and the transport layer. It identifies the communication endpoints through the IP address and port number, and supports multiple protocols, including but not limited to TCP (Transmission Control Protocol) and UDP (User Datagram Protocol), which are used to achieve inter-process communication between different devices. In socket communication that supports the TCP protocol, the mobile terminal sends a connection request to the test end. After receiving the confirmation connection signal sent by the test end, it returns an ACK confirmation signal to the test end to complete the establishment of the connection with the test end, and then starts to transmit image data. In socket communication that supports the UDP protocol, there is no need to send a connection request. Instead, the address is directly bound to create socket communication for data sending and receiving.

[0147] In this embodiment, the communication connection between the mobile terminal and the test end is realized through socket, which improves the security of image data transmission and realizes the real-time interaction between the mobile terminal and the test end.

[0148] (3) Transmit the data.

[0149] Next, the data collected by the server is encoded and passed to the analysis module of the client through a buffer (Buffer). The analysis module restores the data to the original video data through decoding.

[0150] Among them, the server obtains the buffered video and sends it to the test end, including: saving the buffered video to the buffer, encoding the buffered video and sending it to the test end; the test end decodes the encoded buffered video to obtain the buffered video.

[0151] A buffer is an area used to temporarily store data, mainly for coordinating data transmission between components with different speeds or different timings.

[0152] In this embodiment, by encoding and decoding the buffered video, the transmission efficiency and flexibility are improved, and the reliability and real-time performance of the transmission are enhanced by using the buffer.

[0153] Optionally, the encoding methods include H.264 encoding and H.265 encoding.

[0154] Among them, H.264 and H.265 are two main video coding standards, which achieve high compression efficiency through techniques such as predictive coding, transform coding, and entropy coding, while maintaining relatively high video quality. H.264 supports multiple resolutions and frame rates, and is applicable to various application scenarios from low-resolution mobile devices to high-definition TVs. H.265 supports higher resolutions than H.264 and is suitable for the transmission and storage of ultra-high-definition videos. Its parallel processing architecture can better utilize the computing power of modern multi-core processors to improve the encoding speed, and is applicable to scenarios such as video streaming, video conferencing, and digital TV, especially in scenarios that require high-resolution and high-quality videos.

[0155] In this application, the encoding methods include but are not limited to H.264 encoding and H.265 encoding, and the management and storage of image data are realized by encoding and decoding the buffered video.

[0156] (4) Algorithm analysis and simulation testing.

[0157] Each video frame after encoding and decoding processing is a restored normal video frame. After the video collected by the simulated camera is transmitted to the client, the client will identify it as a normal video image frame through decoding, and then load the recognition and tracking algorithms to be tested for video analysis and detection, such as the algorithm A to be tested and the algorithm B to be tested.

[0158] When loading the target algorithm to be tested for testing as needed, the way to load the algorithm can be to call it by importing the software development kit sdk (Software Development Kit) of the corresponding algorithm in the platform.

[0159] On the simulation platform, the input panoramic video can be processed according to different algorithms at any time.

[0160] Through the control function of the platform, developers can test the performance of the algorithm in a simulated scenario.

[0161] Then, a control strategy is generated based on the operation result of the algorithm, such as pan-tilt control or observation point update, and then transmitted to the server through a communication connection (network / Bluetooth). The server controls the pan-tilt to rotate for a new round of camera video acquisition.

[0162] Finally, the simulation result of the algorithm is determined based on the rotation mode of the pan-tilt and the image situation of the server.

[0163] For example, for the same competition video, when loading Algorithm A, it may be recognized as tracking the target to the left, while Algorithm B may be recognized as tracking to the right; the simulation platform generates corresponding video clips through the simulation environment to verify the accuracy and effect of each algorithm, thereby judging the effectiveness of the algorithm.

[0164] Specifically, the client inputs the buffered video into the motion tracking algorithm A. After analyzing the buffered video, the motion tracking algorithm A simulates and predicts that the player will move to the left and outputs the next predicted position of the player. According to the simulation result of the motion tracking algorithm A, the test end sends a control instruction of "move to the left" to the mobile terminal, and the control instruction includes the specific moving distance and angle. Similarly, when the buffered video is input into the motion tracking algorithm B, after analyzing the buffered video, the motion tracking algorithm B may simulate and predict that the player will move to the right and output the next predicted position of the player. Then, according to the simulation result of the motion tracking algorithm B, the test end will send a control instruction of "move to the right" to the mobile terminal.

[0165] In this embodiment, the client can quickly verify the accuracy of the motion tracking algorithm without relying on the server or real device debugging, providing higher flexibility.

[0166] In this application, the shooting pan-tilt includes but is not limited to common handheld pan-tilts and shooting cameras.

[0167] In this embodiment, feeding back the image captured after dynamically adjusting the shooting pan-tilt to the test end can realize the comparison and verification between the algorithm expectation and the actual effect, providing a basis for subsequent evaluation of the algorithm and optimization of algorithm parameters.

[0168] Optionally, when sending the image captured after dynamically adjusting the shooting pan-tilt to the client for re-analysis, it includes: continuously monitoring the data change of the gyroscope in the shooting pan-tilt; after the data of the gyroscope changes, obtaining the image captured after dynamically adjusting the shooting pan-tilt and sending it to the client for algorithm analysis.

[0169] When a mobile terminal acquires an image, it needs to obtain the image captured after the dynamic adjustment of the shooting gimbal. The image before the dynamic adjustment of the shooting gimbal cannot be used as a test image to analyze the performance of the motion tracking algorithm. Only when the data of the gyroscope changes, that is, when the shooting gimbal rotates or shakes, the acquired image is the image captured after adjustment. If all images are continuously sent, it is necessary to screen out the image frames after dynamic adjustment from a large number of images for analysis, which is inefficient and may miss key frames. Therefore, continuously listening to the change of gyroscope data can accurately capture the moment when the gimbal starts to move and adjust, so as to conduct more targeted analysis.

[0170] In this embodiment, by listening to the change of the gyroscope data in the shooting gimbal, unnecessary data transmission and processing are reduced, and the efficiency is improved.

[0171] In summary, the motion tracking algorithm simulation test method of this application includes determining the simulation view range according to the field of view angle of the camera and the observation point; according to the simulation view range, performing a planar projection mapping on the spherical area of the panoramic video to obtain the camera acquisition video of the planar area; the camera acquisition video is used to test the motion tracking algorithm to be tested, realizing multiple simulation generations of the simulation competition scene in the virtual environment based on the input of the same 360° panoramic video. When assisting in the development, verification, and testing of the motion tracking algorithm, multiple verifications and comparative analyses of the algorithm are realized. To a certain extent, it solves the defects of the current target tracking algorithm test in the real competition scene.

[0172] In addition, this application improves the accuracy and authenticity of the algorithm test through closed-loop control and panoramic video restoration; this application separates the tested algorithm from the test platform. The test server is responsible for data acquisition and gimbal control, and the algorithm client independently processes algorithm analysis, allowing developers to freely integrate different algorithm SDKs. The ends communicate through Socket to achieve flexible algorithm integration testing, and the platform is more flexible and multi-scenario; this application supports multi-operating systems such as iOS and Android through cross-platform design to adapt to various scenario requirements.

[0173] This application can establish a simulation platform based on a 360° panoramic video, and can simulate and restore the 360° panoramic video recording the competition into a competition in the platform, realizing the repeated use of the input of the same scene for testing the motion tracking algorithm, flexibly simulating the motion scene captured by the camera, and providing a controllable virtual environment for the development and verification of the motion tracking algorithm.

[0174] Compared with the traditional recording and playback method, the simulation platform of the present application can: 1. Simulate the motion competition scenes from multiple angles and scenarios, providing rich training and test data; 2. Evaluate the adaptability in various situations by comparing the results output by different algorithms; 3. Provide higher clarity and flexibility, break through the limitations of the display and camera, and achieve panoramic tracking and algorithm verification.

[0175] Embodiment 2

[0176] This embodiment provides a motion tracking algorithm simulation test system. For the details not disclosed in the motion tracking algorithm simulation test system of this embodiment, please refer to the specific implementation content of the motion tracking algorithm simulation test method in other embodiments.

[0177] Figure 8 The structural schematic diagram of the motion tracking algorithm simulation test system according to an embodiment of the present application is shown.

[0178] As Figure 8 shown, the motion tracking algorithm simulation test system includes a pan-tilt module 10 and a simulated camera acquisition module 20.

[0179] The pan-tilt module 10 is used to determine the simulated viewing angle range according to the field of view angle of the camera and the observation point;

[0180] The simulated camera acquisition module 20 is used to map the spherical area of the panoramic video to obtain the camera acquisition video of the planar area according to the simulated viewing angle range; the camera acquisition video is used to test the motion tracking algorithm to be tested.

[0181] Using the motion tracking algorithm simulation test system of the present application realizes the input of the same 360° panoramic video to simulate and generate the simulated competition scenes in the virtual environment multiple times, and realizes the multiple verification and comparative analysis of the algorithm when assisting in the development, verification and test of the motion tracking algorithm. To a certain extent, it solves the defects of the current target tracking algorithm test in the real competition scene.

[0182] Figure 9 The structural schematic diagram of another motion tracking algorithm simulation test system according to an embodiment of the present application is shown.

[0183] As Figure 9 shown, in other preferred embodiments, the system includes a server 1 and a client 2; the server 1 includes a pan-tilt module 10 and a simulated camera acquisition module 20;

[0184] The client 2 includes a detection module 30; the detection module 30 is equipped with the motion tracking algorithm to be tested;

[0185] The detection module 30 is used to input the video collected by the camera into the motion tracking algorithm to be tested for analysis, identify key motion events in the video or lock the target object, and obtain the algorithm simulation results.

[0186] In the preferred implementation, the client 2 further includes a policy module 40, and the server 1 further includes a pan-tilt control module 50.

[0187] The policy module 40 is used to generate a pan-tilt control policy or a new observation point according to the algorithm simulation results.

[0188] The pan-tilt control module 50 is used to update the pan-tilt parameters according to the pan-tilt control policy or the new observation point, and determine the new simulation viewing range.

[0189] The detection module 30 of the client 2 further inputs the new camera-collected video corresponding to the new simulation viewing range into the motion tracking algorithm to be tested until the test ends.

[0190] Thus, the present application realizes that through closed-loop control and panoramic video restoration, the accuracy and authenticity of algorithm testing are improved; in the present application, the tested algorithm is separated from the test platform, the test server is responsible for data collection and pan-tilt control, the algorithm client independently processes algorithm analysis, allows developers to freely integrate different algorithm SDKs, and the ends communicate through Socket to achieve flexible algorithm integration testing, and the platform is more flexible and multi-scenario; the present application supports multiple operating systems such as iOS and Android through cross-platform design to adapt to various scenario requirements.

[0191] Embodiment 3

[0192] This embodiment provides a motion tracking algorithm simulation test device. For the details not disclosed in the motion tracking algorithm simulation test device of this embodiment, please refer to the specific implementation content of the motion tracking algorithm simulation test method or system in other embodiments.

[0193] Figure 10 The structural schematic diagram of the motion tracking algorithm simulation test device 400 according to the embodiment of the present application is shown.

[0194] As Figure 10 shown, the motion tracking algorithm simulation test device 400 includes: a storage unit 402: used to store executable instructions; and a processing unit 401: used to be connected to the storage unit 402 to execute the executable instructions to complete the motion tracking algorithm simulation test method.

[0195] Those skilled in the art can understand, the schematic Figure 10It is only an example of the motion tracking algorithm simulation test device 400, which does not constitute a limitation on the motion tracking algorithm simulation test device 400. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the motion tracking algorithm simulation test device 400 may also include input / output devices, network access devices, buses, etc.

[0196] The so-called processing unit 401 (Central Processing Unit, CPU) may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application-specific integrated circuits (ApplicationSpecific Integrated Circuit, ASIC), field-programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processing unit 401 may also be any conventional processor, etc. The processing unit 401 is the control center of the motion tracking algorithm simulation test device 400, and connects various parts of the entire motion tracking algorithm simulation test device 400 through various interfaces and lines.

[0197] The storage unit 402 can be used to store computer-readable instructions. The processing unit 401 realizes various functions of the motion tracking algorithm simulation test device 400 by running or executing the computer-readable instructions or modules stored in the storage unit 402, and by calling the data stored in the storage unit 402. The storage unit 402 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area may store data created according to the use of the motion tracking algorithm simulation test device 400. In addition, the storage unit 402 may include a hard disk, memory, plug-in hard disk, smart media card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash card (Flash Card), at least one magnetic disk storage device, flash device, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM) or other non-volatile / volatile storage devices.

[0198] When the modules integrated in the motion tracking algorithm simulation test device 400 are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present invention, computer-readable instructions can also be used to instruct relevant hardware to complete. The computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed by a processor, the steps of the above-mentioned various method embodiments can be implemented.

[0199] Embodiment 4

[0200] This embodiment provides a computer-readable storage medium, on which a computer program is stored; the computer program is executed by a processor to implement the motion tracking algorithm simulation test method in other embodiments.

[0201] By using the motion tracking algorithm simulation test device and medium of the present application, including determining the simulated viewing angle range according to the field of view angle of the camera and the observation point; according to the simulated viewing angle range, performing a planar projection mapping on the spherical region of the panoramic video to obtain the camera-captured video of the planar region; the camera-captured video is used to test the motion tracking algorithm to be tested, and it is realized that multiple simulation-generated simulated competition scenes in the virtual environment are generated based on the input of the same 360° panoramic video. When assisting in the development, verification, and testing of the motion tracking algorithm, multiple verifications and comparative analyses of the algorithm are realized. To a certain extent, it solves the defects of the current target tracking algorithm test in the real competition scene.

[0202] In addition, the present application improves the accuracy and authenticity of the algorithm test through closed-loop control and panoramic video restoration; the present application separates the tested algorithm from the test platform. The test server is responsible for data acquisition and pan-tilt control, and the algorithm client independently processes algorithm analysis, allowing developers to freely integrate different algorithm SDKs. The two ends communicate through Socket to achieve flexible algorithm integration testing, and the platform is more flexible and multi-scenario; the present application supports multi-operating systems such as iOS and Android through cross-platform design to adapt to various scenario requirements.

[0203] By establishing a simulation platform based on a 360° panoramic video, the present application can simulate and restore the 360° panoramic video recording the competition into a competition in the platform, realizing that the input of the same scene is repeatedly used for testing the motion tracking algorithm, and flexibly simulating the motion scene captured by the camera, providing a controllable virtual environment for the development and verification of the motion tracking algorithm.

[0204] Compared with the traditional recording and playback method, the simulation platform of the present application can: 1. Simulate the sports competition scenes from multiple angles and scenarios, and provide rich training and test data; 2. Evaluate the adaptability of different algorithms in various situations by comparing the results output by different algorithms; 3. Provide higher clarity and flexibility, break through the limitations of monitors and cameras, and achieve panoramic tracking and algorithm verification.

[0205] Those skilled in the art should understand that the terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "said" used in the present invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0206] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to a determination".

[0207] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concept. Therefore, the appended claims are intended to be interpreted to include the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0208] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

Claims

1. A method for simulating and testing a motion tracking algorithm, characterized in that, Including: Determine the simulated viewing angle range according to the field of view angle of the camera and the observation point; According to the simulated viewing angle range, project the spherical area of the panoramic video onto a plane to obtain the camera-captured video of the plane area; Input the camera-captured video into the motion tracking algorithm to be tested for analysis; the motion tracking algorithm to be tested is used to identify key motion events in the video or lock on to the target object; Generate a pan-tilt control strategy or a new observation point; Update the pan-tilt parameters according to the pan-tilt control strategy or the new observation point, and determine the new simulated viewing angle range; According to the new camera-captured video corresponding to the new simulated viewing angle range, further input it into the motion tracking algorithm to be tested until the test ends.

2. The motion tracking algorithm simulation test method according to claim 1, wherein The pan-tilt control strategy includes controlling at least one of the yaw value, pitch value, and roll value of the pan-tilt parameters; the center of the simulated viewing angle range is the observation point.

3. The motion tracking algorithm simulation test method according to claim 1, characterized in that The determining the simulated viewing angle range according to the field of view angle of the camera and the observation point includes: Obtain the field of view angle of the camera and the pan-tilt parameters; the pan-tilt parameters include the yaw value, pitch value, and roll value; Determine the observation point according to the yaw value, pitch value, and roll value; Determine the simulated viewing angle range according to the field of view angle and the observation point.

4. The motion tracking algorithm simulation test method according to claim 3, characterized in that The camera is a simulated camera, and the pan-tilt is a virtual pan-tilt; the field of view angle of the simulated camera is set customarily.

5. A method for simulating and testing a motion tracking algorithm, characterized in that, Applied to the server side, including: Determine the simulated viewing angle range according to the field of view angle of the camera and the observation point; According to the simulated viewing angle range, project the spherical area of the panoramic video onto a plane to obtain the camera-captured video of the plane area; Send the camera-captured video to the client, so that the client inputs the camera-captured video into the motion tracking algorithm to be tested for analysis, and then generates a pan-tilt control strategy or a new observation point; the motion tracking algorithm to be tested is used to identify key motion events in the video or lock on to the target object; Update the pan-tilt parameters according to the pan-tilt control strategy or the new observation point, and determine the new simulated viewing angle range; According to the new camera-captured video corresponding to the new simulated viewing angle range, further input it into the motion tracking algorithm to be tested of the target device until the test ends.

6. A method for simulating and testing a motion tracking algorithm, characterized in that, Applied to the client side, including: Receive the camera-captured video from the server; the server is used to determine the simulated viewing angle range according to the field of view angle of the camera and the observation point; according to the simulated viewing angle range, project the spherical area of the panoramic video onto a plane to obtain the camera-captured video of the plane area; Input the camera-captured video into the motion tracking algorithm to be tested for analysis; the motion tracking algorithm to be tested is used to identify key motion events in the video or lock on to the target object; Generate a pan-tilt control strategy or a new observation point; send it to the server, so that the server updates the pan-tilt parameters according to the pan-tilt control strategy or the new observation point, determines the new simulated viewing angle range; and according to the new camera-captured video corresponding to the new simulated viewing angle range, further input it into the motion tracking algorithm to be tested of the target device until the test ends.

7. A motion tracking algorithm simulation test system, characterized in that, Including: A pan-tilt module, used to determine the simulated viewing angle range according to the field of view angle of the camera and the observation point; The analog camera acquisition module is configured to map the spherical region of the panoramic video to obtain the camera acquisition video of the planar region according to the analog viewing range; The camera acquisition video is used to test the motion tracking algorithm to be tested; The detection module is configured to input the camera acquisition video into the motion tracking algorithm to be tested for analyzing and identifying key motion events in the video or locking the target object, and obtain the algorithm simulation result; The strategy module is configured to generate a pan-tilt control strategy or a new observation point according to the algorithm simulation result; The pan-tilt control module is configured to update the pan-tilt parameters according to the pan-tilt control strategy or the new observation point, and determine a new analog viewing range; The detection module further inputs the new camera acquisition video corresponding to the new analog viewing range into the motion tracking algorithm to be tested until the test ends.

8. A motion tracking algorithm simulation test device, characterized in that, Comprising: A storage unit for storing executable instructions; And A processing unit for connecting to the memory to execute the executable instructions to complete the method according to any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, A computer program is stored thereon; the computer program is executed by a processor to implement the method according to any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement the method according to any one of claims 1-6.

Images