Target positioning method, device, equipment and storage medium

Abstract

The present application discloses a target positioning method, device, equipment and storage medium, which relates to the field of student standing up tracking technology. The method includes: using an AI character detection algorithm to capture the motion of a standing student, and determining the position information of the standing student in a panoramic camera; calculating the first offset from the standing student to the close-up camera according to the calibration parameters and position information obtained in advance; achieving coarse positioning of the standing student according to the first offset, and obtaining a panoramic image and a close-up image after coarse positioning; calculating the second offset from the standing student to the close-up camera according to the panoramic image and the close-up image, and controlling the close-up camera to the corresponding position based on the second offset to achieve fine positioning. After coarse positioning and fine positioning, the method ensures the accuracy of the close-up camera positioning target, and combines image processing technology and intelligent algorithms to effectively overcome the limitation problem caused by the fixed installation angle between multiple cameras.

Description

Technical Field

[0001] The present application relates to the field of tracking technology, and in particular to a target positioning method, apparatus, device, and storage medium. Background Art

[0002] In modern education, using panoramic, wide-angle, fixed-focus cameras combined with artificial intelligence (AI) technology to capture students standing up to answer questions in class has become an effective means of enhancing interactive teaching and student engagement. However, when using optical zoom close-up cameras to capture specific students in greater detail and clarity, this process presents two major technical challenges: insufficient positioning accuracy—how to accurately identify and lock onto the target student's position within the panoramic image; and the limited relative mounting angles of multiple cameras, which restricts the system's ability to flexibly capture students in different positions. These issues not only affect the quality of the final video or image but also reduce the overall system efficiency and user experience.

[0003] Traditional solutions often rely on manual calibration or rule-based methods to improve positioning accuracy, but these solutions are often difficult to adapt to complex classroom environments and are incapable of handling multiple moving targets. Furthermore, due to the limitations of physical space layout, adjusting the relative angles between multiple cameras to optimize coverage is also very difficult. Therefore, a new solution is urgently needed to overcome these limitations and ensure high-precision target positioning and high-quality image acquisition even in dynamically changing teaching environments. Summary of the Invention

[0004] The main purpose of this application is to provide a target positioning method, device, equipment and storage medium, which aims to solve the technical problems of poor positioning accuracy of optical zoom close-up cameras in locating standing student targets and limited relative installation angles of multiple cameras when using panoramic wide-angle fixed-focus cameras to capture standing student targets through AI technology in educational scenarios.

[0005] To achieve the above objectives, the present application proposes a target positioning method, which is applied to a panoramic fixed-focus camera, wherein the panoramic fixed-focus camera includes a panoramic camera and a close-up camera. The method comprises:

[0006] Using an AI person detection algorithm to capture the motion of a standing student and determine the position information of the standing student in the panoramic camera;

[0007] Calculating a first offset from the standing student to the close-up camera based on pre-acquired calibration parameters and the position information;

[0008] Performing rough positioning of the standing student according to the first offset, and acquiring a panoramic image and a close-up image after rough positioning;

[0009] A second offset from the standing student to the close-up camera is calculated based on the panoramic image and the close-up image after coarse positioning, and the close-up camera is controlled to a corresponding position based on the second offset to complete the fine positioning of the standing student.

[0010] In one embodiment, before the step of calculating the first offset from the standing student to the close-up camera based on the pre-acquired calibration parameters and the position information, the method further includes:

[0011] Performing offline calibration on the panoramic camera and the close-up camera to obtain calibration parameters;

[0012] The offline calibration of the panoramic camera and the close-up camera to obtain calibration parameters includes:

[0013] Through the person detection algorithm, the panoramic camera and the close-up camera synchronously detect the standing students and record the corresponding position information of the standing students into the calibration file.

[0014] In one embodiment, the step of calculating a first offset from the standing student to the close-up camera based on the calibration parameters and the position information includes:

[0015] According to the calibration parameters and position information, the mapping position of the standing student in the panoramic camera in the close-up camera is calculated by the homography transformation formula;

[0016] Based on the mapped position, a first offset from the standing student to the close-up camera is calculated.

[0017] In one embodiment, the step of roughly locating the standing student according to the first offset and acquiring a panoramic image and a close-up image after the rough positioning includes:

[0018] converting the first offset into a first control instruction;

[0019] Based on the first control instruction, the close-up camera is controlled to a corresponding position so that the standing student is located at the center of the screen of the close-up camera, thereby obtaining a panoramic image of the standing student and a close-up image after rough positioning.

[0020] In one embodiment, the step of calculating a second offset from the standing student to the close-up camera based on the panoramic image and the coarsely positioned close-up image, and controlling the close-up camera to a corresponding position based on the second offset to complete the fine positioning of the standing student includes:

[0021] Extracting feature points from the panoramic image and feature points from the close-up image after rough positioning;

[0022] performing feature point matching on the feature points in the panoramic image and the feature points in the roughly positioned close-up image to obtain matching point pairs;

[0023] Calculating a homography matrix of the panoramic image and a homography matrix of the close-up image based on the matching point pairs;

[0024] Calculating a second offset from the standing student to the close-up camera according to the homography matrix of the panoramic image and the homography matrix of the close-up image;

[0025] Calculating a second offset from the standing student to the close-up camera based on the homography matrix of the panoramic image and the homography matrix of the close-up image includes:

[0026] According to the homography matrix of the panoramic image and the homography matrix of the close-up image, the standing pupil in the panoramic camera is homographed to obtain the coordinate position mapped on the close-up picture;

[0027] Based on the coordinate position, calculate the second offset of the standing student to the close-up camera

[0028] The close-up camera is controlled to a corresponding position based on the second offset to complete the detailed positioning of the students standing up.

[0029] In one embodiment, the step of controlling the close-up camera to a corresponding position based on the second offset to complete detailed positioning of the standing student includes:

[0030] Multi-task target detection is performed using a close-up camera to obtain the size of the standing student in the close-up image.

[0031] Determining a preset zoom of the close-up camera based on the size;

[0032] Obtaining a second offset after magnification change according to the second offset and the preset magnification;

[0033] The second offset after the zoom is converted into a second control instruction, and the close-up camera is controlled to a corresponding position based on the second control instruction to complete the detailed positioning of the standing student.

[0034] In one embodiment, after the step of controlling the close-up camera to a corresponding position based on the second offset to complete the detailed positioning of the standing student, the method further includes:

[0035] Use a close-up camera to identify the standing student in the center of the close-up frame;

[0036] Detecting the standing students using target detection technology to obtain scale information of the standing students;

[0037] Based on the scale information, the current close-up picture is composed to obtain a double-dashed line close-up picture.

[0038] In addition, to achieve the above objectives, the present application also proposes a target positioning device, which includes:

[0039] A capture module is used to capture the motion of a standing student using an AI person detection algorithm and determine the position information of the standing student in the panoramic camera;

[0040] A calculation module, comprising a first calculation unit, a second calculation unit, and a third calculation unit:

[0041] The first calculation unit is configured to calculate a first offset from the standing student to the close-up camera based on pre-acquired calibration parameters and the position information;

[0042] The second calculation unit is configured to perform a rough positioning of the standing student according to the first offset, and obtain a panoramic image and a close-up image after the rough positioning;

[0043] The third calculation unit is used to calculate a second offset from the standing student to the close-up camera based on the panoramic image and the close-up image after coarse positioning, and control the close-up camera to a corresponding position based on the second offset to complete the fine positioning of the standing student.

[0044] In addition, to achieve the above-mentioned purpose, the present application also proposes a target positioning device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the target positioning method as described above.

[0045] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the target positioning method described above are implemented.

[0046] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the target positioning method described above.

[0047] This application provides a target positioning method, apparatus, device, and storage medium. The method is applied to a panoramic fixed-focus camera, which includes a panoramic camera and a close-up camera. The method includes: using an AI person detection algorithm to capture the motion of a standing student and determine the position information of the standing student in the panoramic camera; calculating a first offset from the standing student to the close-up camera based on pre-acquired calibration parameters and position information; performing coarse positioning of the standing student based on the first offset, obtaining a panoramic image and a close-up image after coarse positioning; calculating a second offset from the standing student to the close-up camera based on the panoramic image and the close-up image after coarse positioning, and controlling the close-up camera to a corresponding position based on the second offset to complete fine positioning of the standing student. After coarse and fine positioning, this method ensures the accuracy of the close-up camera's target positioning. Furthermore, by combining advanced image processing technology and intelligent algorithms, it effectively overcomes the limitations caused by the fixed installation angles between multiple cameras, allowing the system to flexibly adapt even in complex and changing environments, achieving high-precision target tracking and positioning functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0050] Figure 1 A flowchart of the first embodiment of the target positioning method of the present application is provided;

[0051] Figure 2 A schematic diagram of the calibration method for a panoramic camera and a close-up camera provided in Example 1 of the present application;

[0052] Figure 3 A schematic diagram of the offset angle required for the first coarse positioning of the close-up camera provided in Example 1 of the present application;

[0053] Figure 4 A schematic diagram of the target position of the close-up camera after coarse positioning provided in Example 1 of the present application;

[0054] Figure 5 This is a schematic diagram of feature point matching provided in Example 1 of the present application;

[0055] Figure 6 A flowchart of the second embodiment of the target positioning method of this application is provided;

[0056] Figure 7 A schematic diagram of the target position of the close-up camera after fine positioning provided in Example 2 of the present application;

[0057] Figure 8 A schematic diagram of a simplified flow chart of the target positioning method provided in Example 1 of the present application;

[0058] Figure 9 This is a schematic diagram of the module structure of the target positioning device according to an embodiment of the present application;

[0059] Figure 10 Schematic diagram of the device structure of the hardware operating environment involved in the target positioning method in the embodiment of the present application.

[0060] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0061] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0062] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0063] The main solution of the embodiment of the present application is: using an AI character detection algorithm to capture the motion of a standing student and determine the position information of the standing student in the panoramic camera; calculating a first offset from the standing student to the close-up camera based on pre-acquired calibration parameters and position information; performing coarse positioning of the standing student based on the first offset, and obtaining a panoramic image and a close-up image after coarse positioning; calculating a second offset from the standing student to the close-up camera based on the panoramic image and the close-up image after coarse positioning, and controlling the close-up camera to the corresponding position based on the second offset to complete the fine positioning of the standing student.

[0064] The field of multi-camera positioning encompasses a variety of techniques and methods used to ensure precise spatial alignment of image data acquired from different viewpoints and sensors. With technological advancements, multi-camera positioning methods continue to evolve, from traditional image processing algorithms to innovative deep learning-based techniques, with researchers pursuing even higher levels of positioning accuracy and speed.

[0065] Traditional methods often rely on manually extracted feature matching, such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Robust Features), which have been widely used in remote sensing image processing. However, these methods, which rely on manually designed features, have limitations when processing complex and variable remote sensing data, especially when faced with significant geometric distortion and grayscale differences between different sensors. To address this, researchers have proposed structure-based registration methods, such as the Directional Phase Dense Feature (DFOP)-based multi-sensor remote sensing image registration method. This method constructs a three-dimensional structural feature descriptor that is invariant to illumination and contrast to capture the geometric structure distribution of the image. This effectively resists grayscale differences between multi-sensor images and improves matching accuracy.

[0066] In recent years, the rise of deep learning technology has brought new breakthroughs in multi-camera localization methods. The powerful feature extraction capabilities of deep neural networks enable deep learning-based registration methods to automatically learn more discriminative and robust feature representations.

[0067] In modern education, using positional cameras combined with artificial intelligence (AI) to capture students standing up to answer questions in class has become an effective means of enhancing interactive teaching and student engagement. However, when using optical zoom close-up cameras to capture specific students in greater detail and clarity, two major technical challenges arise: insufficient positioning accuracy—how to accurately identify and lock onto the target student's position within a panoramic image; and the limited relative mounting angles of multiple cameras, which restricts the system's ability to flexibly capture students in diverse positions. These issues not only impact the quality of the final video or image but also reduce the overall system efficiency and user experience. Traditional solutions often rely on manual calibration or rule-based methods to improve positioning accuracy, but these solutions often struggle to adapt to complex classroom environments and struggle with multiple moving objects. Furthermore, due to physical layout constraints, adjusting the relative angles of multiple cameras to optimize coverage is difficult.

[0068] This application provides a solution for use in a panoramic fixed-focus camera, which includes a panoramic camera and a close-up camera. The solution includes: utilizing an AI person detection algorithm to capture the motion of a standing student and determine the position of the standing student in the panoramic camera; calculating a first offset from the standing student to the close-up camera based on pre-acquired calibration parameters and position information; performing coarse positioning of the standing student based on the first offset, obtaining a panoramic image and a close-up image after coarse positioning; calculating a second offset from the standing student to the close-up camera based on the panoramic image and the close-up image after coarse positioning, and controlling the close-up camera to a corresponding position based on the second offset to complete fine positioning of the standing student. This method ensures the accuracy of the close-up camera's target positioning after coarse and fine positioning, and, combined with advanced image processing technology and intelligent algorithms, effectively overcomes the limitations caused by the fixed installation angles between multiple cameras, allowing the system to flexibly adapt even in complex and changing environments, achieving high-precision target tracking and positioning functions.

[0069] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, camera device, etc., or an electronic device capable of performing the above functions. The following uses a personal computer as an example to illustrate this embodiment and the following embodiments.

[0070] Based on this, the embodiment of the present application provides a target positioning method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the target positioning method of the present application.

[0071] In this embodiment, the target positioning method includes steps S10 to S40:

[0072] Step S10, using an AI person detection algorithm to capture the motion of the standing student and determine the position information of the standing student in the panoramic camera;

[0073] It should be noted that the method in this embodiment is applied to a panoramic fixed-focus camera, which includes both panoramic and close-up cameras. The panoramic camera captures a wide-angle image of the entire classroom or a specific area, ensuring that all students are captured. The close-up camera captures detailed images of specific objects (such as a student standing up). While its field of view is smaller, it provides high-resolution local images.

[0074] In addition, it should be noted that AI task detection algorithms refer to the use of artificial intelligence technology, especially deep learning-based methods, to identify the location of people in images or videos. Such algorithms can automatically separate the human body from the complex background and determine its precise location in the image.

[0075] Specifically, the panoramic camera captures images inside the classroom in real time and uses AI person detection algorithms to analyze the images, identify standing students, and determine their location information in the panoramic image.

[0076] Step S20, calculating a first offset from the standing student to the close-up camera based on pre-acquired calibration parameters and the position information;

[0077] It should be noted that the first offset is the motion adjustment information that needs to be performed when the close-up camera performs the first phase of coarse positioning in order to align with the position of the standing student.

[0078] In addition, it should be noted that offline calibration refers to a series of tests and adjustments performed on the camera in a controlled environment before actual use to determine the camera's internal and external parameters.

[0079] It is understandable that since the lenses of panoramic cameras and close-up cameras may have inherent distortion and the specific parameters of each camera are different, step S201 is also included before performing step S20. Executing step S201 can avoid target positioning errors caused by lens distortion or inaccurate parameters, and improve the positioning accuracy and reliability of panoramic fixed-focus cameras in practical applications.

[0080] In a feasible implementation, step S20 may include step S201 before step S20:

[0081] Step S201, performing offline calibration on the panoramic camera and the close-up camera to obtain calibration parameters; the offline calibration on the panoramic camera and the close-up camera to obtain calibration parameters includes: using a person detection algorithm to enable the panoramic camera and the close-up camera to synchronously detect standing students, and recording the corresponding position information of the standing students in a calibration file.

[0082] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the calibration method for the panoramic camera and close-up camera provided in this embodiment. Specifically, Figure 2 As shown, Figure 2 The panoramic camera is on the left, and the close-up camera is in the center. During calibration, a standing student serves as the calibration subject. As the calibration subject walks in a public area, the panoramic and close-up cameras simultaneously record or capture photos. The two cameras are synchronized to ensure they capture data at the same moment.

[0083] An AI person detection algorithm is used to process images captured by the panoramic camera to determine the specific location of people within the panoramic view. The AI ​​person detection algorithm is also used to process images captured by the close-up camera to record the person's position within the close-up view. The person's position information from the two cameras is compared. By comparing these position points, the relative pose relationship between the panoramic and close-up cameras can be calculated. Multiple sets of corresponding points are selected. For example, the eight-point method requires at least eight pairs of matching points. These points represent the locations of the same target found in both the panoramic and close-up images. The eight-point method or other methods (such as RANSAC) are used to calculate the homography matrix H1 between the panoramic and close-up cameras. The homography matrix H1 describes the projective relationship between the target in the panoramic image and the close-up image. Based on the above analysis results, a calibration file containing all necessary parameters is created. This calibration file includes, but is not limited to, the homography matrix H1 and other necessary parameters.

[0084] Finally, additional testing is performed to verify the validity of the calibration results. If large deviations are found, it may be necessary to redo part or all of the calibration process until satisfactory accuracy is achieved.

[0085] Through the above steps, the panoramic camera and close-up camera are calibrated offline, which not only lays the foundation for subsequent precise positioning, but also uses the AI ​​person detection algorithm to make the entire calibration process more automated and efficient.

[0086] After the calibration parameters are obtained, the first offset from the standing student to the close-up camera can be calculated based on the calibration parameters and the position information of the standing student in the panoramic camera.

[0087] In another feasible implementation, step S20 may include steps S21 and S22:

[0088] Step S21, calculating the mapping position of the standing student in the panoramic camera in the close-up camera using a homography transformation formula according to the calibration parameters and position information;

[0089] Specifically, when the system starts, it reads the pre-generated calibration file and obtains the homography matrix H1 and other necessary calibration parameters. Then, the position information of the standing student in the panoramic image is mapped to the position in the close-up image based on the homography matrix H1. The formula is as follows:

[0090]

[0091] Where, and are homogeneous coordinates, 、 、 Indicates the horizontal coordinate of the transformed standing student in the close-up picture, 、 、 Represents the original horizontal coordinate of the standing student in the close-up picture before transformation. Represents the linear transformation parameters from one plane to another. These parameters together determine how to map points on one plane to another plane, where Determines the scaling and rotation of the transformed point in the x-axis direction, Determines the shearing and rotation of the transformed point in the x-axis direction, Determines the translation of the transformed point in the x-axis direction, Determines the shearing and rotation of the transformed point in the y-axis direction, Determines the scaling and rotation of the transformed point in the y-axis direction, Determines the translation of the transformed point in the y-axis direction, Determines the perspective deformation of the transformed point in the x-axis direction, Determines the perspective deformation of the transformed point in the y-axis direction, Usually set to 1 to maintain homogeneous coordinate form.

[0092] Step S22: Calculate a first offset from the standing student to the close-up camera based on the mapped position.

[0093] Optionally, after obtaining the mapping position of the standing student in the panoramic camera in the close-up camera, the first offset of the first coarse positioning of the close-up camera is calculated according to the following formula:

[0094]

[0095] Where tanα and tanβ are the tangent function values ​​in the horizontal and vertical directions, respectively. x_shift1 is the horizontal displacement of the standing pupil relative to the center of the close-up camera. img_width is the width of the close-up camera image.

[0096]

[0097] Where pan_angle1 is the parameter used to adjust the close-up camera's horizontal orientation during the first coarse positioning. β is half the close-up camera's horizontal field of view, also known as the half-field of view. pan_now1 is the current horizontal rotation angle of the close-up camera, or the direction the close-up camera is currently facing. img_width is the width of the close-up camera image. x_shift1 is the horizontal displacement of the standing student relative to the close-up camera's center, within the close-up camera's field of view.

[0098]

[0099] Where tanα and tanβ are the tangent function values ​​in the horizontal and vertical directions, respectively. y_shift1 is the vertical displacement of the standing pupil relative to the center of the close-up camera at the current close-up camera angle of view. img_height is the height of the close-up camera image.

[0100]

[0101] Where tit_angle1 is the parameter used to adjust the close-up camera's vertical orientation during the first coarse positioning. β is half the close-up camera's vertical field of view, also known as the half-field of view angle. tit_now1 is the current vertical rotation angle of the close-up camera, or the direction the close-up camera is currently facing. img_height1 is the height of the current close-up camera image.

[0102] Step S30, performing rough positioning of the standing student according to the first offset, and acquiring a panoramic image and a close-up image after rough positioning;

[0103] Optionally, a first control instruction is generated based on the calculated pan_angle1 and tit_angle1. The control instruction is then sent to the pan / tilt head of the close-up camera via a control system such as serial communication, network protocol, etc. Figure 3 As stated, Figure 3 A schematic diagram shows the offset angles required for the close-up camera's initial coarse positioning. Based on the first control command, the gimbal adjusts the close-up camera's orientation to align with the standing student, completing the initial coarse positioning of the close-up camera. After the close-up camera is aligned, both the panoramic camera and the close-up camera simultaneously capture a panoramic image of the standing student and a coarsely positioned close-up image.

[0104] Through the above steps, the system can automatically calculate the angle that needs to be adjusted for the close-up camera and generate corresponding control instructions to adjust the camera direction, thereby completing the first rough positioning of the close-up camera.

[0105] Step S40 , calculating a second offset from the standing student to the close-up camera based on the panoramic image and the roughly positioned close-up image, and controlling the close-up camera to a corresponding position based on the second offset to complete the fine positioning of the standing student.

[0106] like Figure 4 As shown, Figure 4This figure shows the target position after coarse positioning of the close-up camera. It should be noted that the blue frame represents the panoramic camera's field of view, while the green frame represents the close-up camera's field of view. Due to the difference between the actual student height and the preset calibration personnel height, the close-up camera may have some deviation during the initial coarse positioning of the target, for example, the target person may be offset from the center of the panoramic close-up. If only coarse positioning is performed once for the close-up camera, this error will be magnified after zooming in on the close-up image.

[0107] Therefore, this embodiment proposes a method of performing a second fine positioning of the close-up camera based on the coarse positioning of the close-up camera, thereby achieving the purpose of completing high-precision positioning when depth information is unknown.

[0108] In a feasible implementation, step S40 may include steps S41 to S45:

[0109] Step S41, extracting feature points in the panoramic image and feature points in the close-up image after rough positioning;

[0110] The close-up image and the panoramic image obtained after the first coarse positioning in step S30 are used to extract feature points of the two images using an AI feature extraction algorithm.

[0111] Specifically, feature point detection algorithms such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), and ORB (OrientedFAST and Rotated BRIEF) are used. These algorithms can identify key points in an image and are invariant to changes in scale, rotation, and illumination. The selected feature extraction algorithms are applied to the panoramic image and the coarsely localized close-up image to identify salient feature points in both images.

[0112] For each detected feature point, a descriptor is calculated to represent the local information around that point. A descriptor is typically a vector that reflects the distribution of gradient directions or other patterns within the feature point's neighborhood. The location of each feature point and its corresponding descriptor are recorded to prepare for feature matching in the next step.

[0113] Step S42, performing feature point matching on the feature points in the panoramic image and the feature points in the roughly positioned close-up image to obtain matching point pairs;

[0114] Optionally, after obtaining the feature points of the panoramic image and the feature points of the close-up image, a feature matching algorithm, such as brute force matching or FLANN matcher, is used to find the most similar feature point pairs between the two images. Figure 5 As shown, Figure 5 This is a schematic diagram of feature point matching. It should be noted that there are many pairs of matching feature points actually obtained. Figure 5 Only 4 pairs are drawn.

[0115] Step S43, calculating the homography matrix of the panoramic image and the homography matrix of the close-up image based on the matching point pairs;

[0116] Optionally, based on the matching feature point pairs of the panoramic image and the close-up image, the homography matrix H2 from the panoramic image to the close-up image is estimated using the eight-point method. )and( ) to construct a system of linear equations. Given more than 8 pairs of matching points, an overdetermined system of equations can be formed, which can then be solved through SVD decomposition to obtain an estimate of the homography matrix H2.

[0117] Step S44, calculating a second offset from the standing student to the close-up camera based on the homography matrix of the panoramic image and the homography matrix of the close-up image;

[0118] Optionally, the target position of the standing student in the panoramic camera is transformed according to the homography matrix H2 obtained in step S43 to obtain the coordinate position mapped to the close-up image. The homography transformation formula is:

[0119]

[0120] Where, and are homogeneous coordinates, 、 、 Indicates the horizontal coordinate of the transformed standing student in the close-up picture, 、 、 Represents the original horizontal coordinate of the standing student in the close-up picture before transformation. Represents the linear transformation parameters from one plane to another. These parameters together determine how to map points on one plane to another plane, where Determines the scaling and rotation of the transformed point in the x-axis direction, Determines the shearing and rotation of the transformed point in the x-axis direction, Determines the translation of the transformed point in the x-axis direction, Determines the shearing and rotation of the transformed point in the y-axis direction, Determines the scaling and rotation of the transformed point in the y-axis direction, Determines the translation of the transformed point in the y-axis direction, Determines the perspective deformation of the transformed point in the x-axis direction, Determines the perspective deformation of the transformed point in the y-axis direction, Usually set to 1 to maintain homogeneous coordinate form.

[0121] Then, the second offset is calculated based on the coordinate position. Similarly, the calculation formula is as follows:

[0122]

[0123] Where tanα and tanβ are the tangent function values ​​in the horizontal and vertical directions, respectively. Refers to the horizontal displacement of the standing student relative to the center of the close-up camera from the close-up camera's perspective. img_width is the width of the close-up camera image.

[0124]

[0125] Where pan_angle2 is the parameter used to adjust the close-up camera's horizontal orientation during the second fine-tuning phase. β is half the close-up camera's horizontal field of view, also known as the half-field of view. pan_now is the current horizontal rotation angle of the close-up camera, or the direction the close-up camera is currently facing. img_width is the width of the close-up camera image. x_shift2 is the horizontal displacement of the standing student relative to the center of the close-up camera at the current close-up camera viewing angle.

[0126]

[0127] Where tanα and tanβ are the tangent function values ​​in the horizontal and vertical directions, respectively. Refers to the vertical displacement of the standing student relative to the center of the close-up camera from the close-up camera's perspective. img_height is the height of the close-up camera image.

[0128]

[0129] In the formula, tit_angle2 is the parameter used to adjust the close-up camera's vertical orientation during the second fine-tuning phase. β is half the close-up camera's vertical field of view, also known as the half-field of view angle. tit_now2 is the current vertical rotation angle of the close-up camera, or the direction the close-up camera is currently facing. img_height2 is the height of the current close-up camera image. y_shift2 is the vertical displacement of the standing student relative to the center of the close-up camera, based on the current close-up camera's field of view.

[0130] Through the above formula calculation, the second offset required for the close-up camera to perform a second fine positioning of the standing student is obtained.

[0131] Step S45: controlling the close-up camera to a corresponding position based on the second offset to complete the detailed positioning of the standing student.

[0132] Optionally, first, obtain the current close-up image from the current close-up camera, and use a multi-task target detection algorithm, such as YOLO, SSD or Faster R-CNN, to identify and frame the position of the standing student, and at the same time calculate the size of the standing student in the close-up image, including width and height.

[0133] Then, the size of the student standing up is compared with the preset target size. The preset target size can be set according to actual needs, for example, the student standing up occupies a certain proportion of the close-up picture. Based on the ratio of the size of the current student standing up to the preset size, the zoom factor that needs to be adjusted is calculated. Assume that the size of the current student standing up is and , the default size is and , the preset zoom_factor can be calculated according to the following formula:

[0134] zoom_facto = =

[0135] Where, is the width of the student currently standing up, is the height of the student currently standing, The width of the preset standing students, The preset height for students standing up.

[0136] Since zooming will change the field of view of the close-up camera, it is necessary to adjust the second offset before zooming according to the preset zooming to obtain the second offset after zooming. Assume that the second offset after zooming is and , the focal length after zooming is focal_length_zoomed, then:

[0137] = arctan

[0138] = arctan

[0139] Where, Indicates the parameters of the close-up camera after horizontal zoom adjustment during the second fine positioning process. Indicates the parameters of the close-up camera after vertical zoom adjustment during the second fine positioning process. It refers to the vertical displacement distance of the standing student relative to the center of the close-up camera screen under the current close-up camera viewing angle after magnification adjustment. It refers to the horizontal displacement distance of the standing student relative to the center of the close-up camera screen under the current close-up camera viewing angle after magnification adjustment.

[0140] based on and Second control instructions are generated to instruct the close-up camera's pan / tilt head on how to adjust its angle. The second control instructions are sent to the close-up camera, which then adjusts its horizontal and vertical angles and zoom focal length accordingly, ensuring that the standing student is centered in the close-up image and fits within a preset size.

[0141] Through the above-mentioned embodiment method, close-up positioning is divided into two steps. The first step is to perform coarse positioning based on the camera projection relationship. Since the target height and distance information are unknown under multiple camera positions, it is an ill-posed problem in projection positioning and cannot be accurately positioned. In the second step, after the close-up changes for the first time, the feature points of the close-up picture and the panoramic picture are extracted and matched, and the homography matrix is ​​estimated. The panoramic target position is mapped to the close-up picture according to the homography matrix for secondary fine positioning. This achieves the purpose of completing high-precision positioning in the case of unknown depth information.

[0142] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 6 After step S40, the target positioning method further includes steps S401 to S403:

[0143] Step S401, using a close-up camera to determine the student standing in the center of the close-up image;

[0144] Step S402, detecting the standing student using target detection technology to obtain scale information of the standing student;

[0145] Step S403 : compose the current close-up image based on the scale information to obtain a double-dashed line close-up image.

[0146] Please refer to Figure 7 , Figure 7 The diagram below shows the target position of the close-up camera after fine positioning. Figure 7As can be seen, after the close-up camera's secondary fine-grained positioning, the standing student is at the exact center of the close-up frame. The close-up camera uses AI object detection technology to detect the target and obtain scale information for the centrally located target. Based on this information, the close-up camera frames the current frame and outlines the close-up area with a green double-dashed line. This not only enables the panoramic camera to capture the action, but also provides a high-definition close-up of the standing student through optical zoom.

[0147] For example, in order to help understand the implementation process of the target positioning method obtained by combining this embodiment with the above embodiment 1, please refer to Figure 8 , Figure 8 A brief flowchart of a target positioning method is provided, specifically:

[0148] In the panoramic fixed-focus camera, the panoramic and close-up cameras are first calibrated to obtain calibration parameters. This calibration data is then read to calculate the homography matrix H1. Deep learning techniques are then used to capture the standing-up motion of students in a classroom scene, obtaining the position of the standing students in the panoramic camera. Based on the calibration parameters and position information, the homography matrix H1 is then used to calculate the required motion offset from the close-up camera to the standing student's position, i.e., the first offset. This first offset is then used to control the close-up camera to the corresponding position, completing coarse positioning.

[0149] Due to the lack of depth information and the varying heights and positions of students relative to the camera, the initial coarse positioning of the target without precise depth information will result in a certain degree of deviation. The second re-positioning utilizes deep learning methods to detect and match feature points in the close-up image and panoramic image after the initial coarse positioning. The homography matrix H2 is calculated based on the matched feature points, and the offset for the second fine positioning, or the second offset, is calculated based on the homography matrix H2. The zoom factor z is then determined based on the size of the target in the close-up. The second offset after zooming is then determined based on the zoom factor z. Finally, the close-up camera is controlled according to the second offset after zooming, completing the fine positioning. After coarse and fine positioning, the close-up camera accurately locates the target.

[0150] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the target positioning method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0151] This application also provides a target positioning device, please refer to Figure 9 , the target positioning device includes:

[0152] The capture module 10 is used to capture the motion of the standing student using an AI person detection algorithm and determine the position information of the standing student in the panoramic camera;

[0153] The calculation module 20 includes a first calculation unit, a second calculation unit, and a third calculation unit:

[0154] The first calculation unit is configured to calculate a first offset from the standing student to the close-up camera based on pre-acquired calibration parameters and the position information;

[0155] The second calculation unit is configured to perform a rough positioning of the standing student according to the first offset, and obtain a panoramic image and a close-up image after the rough positioning;

[0156] The third calculation unit is used to calculate a second offset from the standing student to the close-up camera based on the panoramic image and the close-up image after coarse positioning, and control the close-up camera to a corresponding position based on the second offset to complete the fine positioning of the standing student.

[0157] The target positioning device provided in this application, using the target positioning method described in the above-mentioned embodiments, can address the technical issues of poor positioning accuracy and limited relative mounting angles of multiple cameras in the process of locating standing student targets using a panoramic wide-angle fixed-focus camera using AI technology in educational scenarios. Compared with the prior art, the beneficial effects of the target positioning device provided in this application are the same as those of the target positioning method described in the above-mentioned embodiments, and the other technical features of the target positioning device are the same as those disclosed in the above-mentioned embodiments and are not further described here.

[0158] The present application provides a target positioning device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the target positioning method in the above-mentioned embodiment 1.

[0159] Reference below Figure 10 , which shows a schematic diagram of the structure of a target positioning device suitable for implementing the embodiments of the present application. The target positioning device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), etc., as well as fixed terminals such as digital TVs and desktop computers. Figure 10 The target positioning device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0160] like Figure 10 As shown, the target positioning device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the target positioning device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to I / O interface 1006: input devices 1007, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003, such as a magnetic tape or hard disk; and communication device 1009. The communication device 1009 can allow the target positioning device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a target positioning device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have instead.

[0161] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0162] The target positioning device provided in this application, using the target positioning method described in the above-mentioned embodiments, can address the technical issues of poor positioning accuracy and limited relative mounting angles of multiple cameras in the process of locating standing student targets using a panoramic wide-angle fixed-focus camera using AI technology in educational scenarios. Compared with the prior art, the beneficial effects of the target positioning device provided in this application are the same as those of the target positioning method described in the above-mentioned embodiments, and the other technical features of the target positioning device are the same as those disclosed in the method of the previous embodiment, and are not further described here.

[0163] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0164] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0165] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer program) stored thereon, wherein the computer-readable program instructions are used to execute the target positioning method in the above embodiment.

[0166] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0167] The computer-readable storage medium may be included in the target positioning device, or may exist independently without being assembled into the target positioning device.

[0168] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the target positioning device, the target positioning device: uses the AI ​​character detection algorithm to capture the motion of the standing student and determine the position information of the standing student in the panoramic camera; calculates the first offset of the standing student to the close-up camera based on the pre-acquired calibration parameters and position information; performs coarse positioning of the standing student based on the first offset, and obtains a panoramic image and a close-up image after coarse positioning; calculates the second offset of the standing student to the close-up camera based on the panoramic image and the close-up image after coarse positioning, and controls the close-up camera to the corresponding position based on the second offset to complete the fine positioning of the standing student.

[0169] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0170] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0171] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0172] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned target positioning method. This computer-readable storage medium can address the technical issues of poor positioning accuracy and limited relative mounting angles of multiple cameras when using a panoramic, wide-angle, fixed-focus camera to capture standing student targets in educational settings using AI technology. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the target positioning method provided in the aforementioned embodiments and are not further elaborated here.

[0173] The present application also provides a computer program product, comprising a computer program, which implements the steps of the target positioning method as described above when the computer program is executed by a processor.

[0174] The computer program product provided in this application can address the technical issues of poor positioning accuracy and limited relative mounting angles of multiple cameras when using a panoramic, wide-angle, fixed-focus camera to capture standing student targets in educational settings using AI technology. Compared to the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the target positioning method provided in the aforementioned embodiments, and are not further elaborated here.

[0175] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A target positioning method, characterized in that: The method is applied to a panoramic fixed-focus camera, which includes a panoramic camera and a close-up camera. The method includes: Using an AI person detection algorithm to capture the motion of the standing student, and determine the position information of the standing student in the panoramic camera; Calculating a first offset from the standing student to the close-up camera according to the pre-acquired calibration parameters and the position information, including: calculating a mapping position of the standing student in the panoramic camera in the close-up camera by a homography transformation formula according to the pre-acquired calibration parameters and the position information, and calculating a first offset from the standing student to the close-up camera based on the mapping position; The method comprises: performing a rough positioning of the standing student according to the first offset to obtain a panoramic image and a close-up image after the rough positioning, including: converting the first offset into a first control instruction, controlling the close-up camera to a corresponding position based on the first control instruction so that the standing student is located at the center of the picture of the close-up camera, and obtaining a panoramic image and a close-up image after the rough positioning of the standing student; Calculate a second offset from the standing student to the close-up camera according to the panoramic image and the close-up image after rough positioning, and control the close-up camera to a corresponding position based on the second offset to complete the fine positioning of the standing student, including: extracting feature points in the panoramic image and feature points in the close-up image after rough positioning, matching feature points in the panoramic image and feature points in the close-up image after rough positioning to obtain matching point pairs, and calculating the homography matrix of the panoramic image and the homography matrix of the close-up image based on the matching point pairs, and calculating the second offset from the standing student to the close-up camera according to the homography matrix of the panoramic image and the homography matrix of the close-up image; The step of calculating the second offset of the standing student to the close-up camera according to the homography matrix of the panoramic image and the homography matrix of the close-up image comprises: performing a homography transformation on the standing student in the panoramic camera according to the homography matrix of the panoramic image and the homography matrix of the close-up image to obtain a coordinate position mapped on the close-up picture, and calculating the second offset of the standing student to the close-up camera based on the coordinate position; and controlling the close-up camera to a corresponding position based on the second offset to complete the detailed positioning of the standing student. Among them, controlling the close-up camera to a corresponding position based on the second offset to complete the detailed positioning of the standing student includes: performing multi-task target detection through the close-up camera to obtain the size of the standing student in the close-up picture, determining the preset magnification of the close-up camera based on the size, obtaining the second offset after magnification according to the second offset and the preset magnification, converting the second offset after magnification into a second control instruction, and controlling the close-up camera to a corresponding position based on the second control instruction to complete the detailed positioning of the standing student.

2. The method according to claim 1, characterized in that Before the step of calculating the first offset from the standing student to the close-up camera according to the pre-acquired calibration parameters and the position information, the method further includes: Performing offline calibration on the panoramic camera and the close-up camera to obtain calibration parameters; The offline calibration of the panoramic camera and the close-up camera to obtain calibration parameters includes: Through the person detection algorithm, the panoramic camera and the close-up camera synchronously detect the standing students, and record the corresponding position information of the standing students into the calibration file.

3. The method according to claim 1, characterized in that After the step of controlling the close-up camera to a corresponding position based on the second offset to complete the detailed positioning of the standing students, the method further includes: Use the close-up camera to identify the standing student in the center of the close-up frame; Detecting the standing students by using target detection technology to obtain scale information of the standing students; Based on the scale information, the current close-up picture is composed to obtain a double-dashed line close-up picture.

4. A target positioning device, characterized in that: The device comprises: A capture module is used to capture the motion of the standing student using an AI person detection algorithm and determine the position information of the standing student in the panoramic camera; A calculation module, the calculation module comprising a first calculation unit, a second calculation unit and a third calculation unit: The first calculation unit is used to calculate the first offset from the standing student to the close-up camera according to the pre-acquired calibration parameters and the position information, including: calculating the mapping position of the standing student in the panoramic camera in the close-up camera by a homography transformation formula according to the pre-acquired calibration parameters and the position information, and calculating the first offset from the standing student to the close-up camera based on the mapping position; The second calculation unit is used to roughly locate the standing student according to the first offset to obtain a panoramic image and a close-up image after rough positioning, including: converting the first offset into a first control instruction, controlling the close-up camera to a corresponding position based on the first control instruction so that the standing student is located at the center of the picture of the close-up camera, and obtaining a panoramic image and a close-up image after rough positioning of the standing student; The third calculation unit is used to calculate the second offset from the standing student to the close-up camera according to the panoramic image and the close-up image after coarse positioning, and control the close-up camera to a corresponding position based on the second offset to complete the fine positioning of the standing student, including: extracting feature points in the panoramic image and feature points in the close-up image after coarse positioning, performing feature point matching on the feature points in the panoramic image and the feature points in the close-up image after coarse positioning to obtain matching point pairs, calculating the homography matrix of the panoramic image and the homography matrix of the close-up image based on the matching point pairs, and calculating the second offset from the standing student to the close-up camera according to the homography matrix of the panoramic image and the homography matrix of the close-up image; The step of calculating the second offset of the standing student to the close-up camera according to the homography matrix of the panoramic image and the homography matrix of the close-up image comprises: performing a homography transformation on the standing student in the panoramic camera according to the homography matrix of the panoramic image and the homography matrix of the close-up image to obtain a coordinate position mapped on the close-up picture, and calculating the second offset of the standing student to the close-up camera based on the coordinate position; and controlling the close-up camera to a corresponding position based on the second offset to complete the detailed positioning of the standing student. Among them, controlling the close-up camera to a corresponding position based on the second offset to complete the detailed positioning of the standing student includes: performing multi-task target detection through the close-up camera to obtain the size of the standing student in the close-up picture, determining the preset magnification of the close-up camera based on the size, obtaining the second offset after magnification according to the second offset and the preset magnification, converting the second offset after magnification into a second control instruction, and controlling the close-up camera to a corresponding position based on the second control instruction to complete the detailed positioning of the standing student.

5. A target positioning device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the target positioning method according to any one of claims 1 to 3.

6. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the target positioning method according to any one of claims 1 to 3 are implemented.

Images