Positioning method and apparatus, computer device, and storage medium

Abstract

The application relates to a positioning method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: obtaining a target video and a target video frame; obtaining a current video frame from the target video, and determining the current video frame as a target video frame when the feature change of the current video frame and a historical video frame meets a preset requirement; performing triangulation processing on feature points of the target video frame and corresponding feature points of the historical video frame to determine a target coordinate system; determining pose information of the target video frame in the target coordinate system and three-dimensional coordinates of a to-be-identified mark in the target coordinate system; and mapping the three-dimensional coordinates of the to-be-identified mark to a plane coordinate system in which the target video frame is located according to the pose information of the target video frame to determine a target position of the to-be-identified mark in the plane coordinate system. The method can realize accurate identification and positioning of the to-be-identified mark.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a positioning method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the development of computer vision technology, barcode technology has emerged. We can scan and identify barcodes to extract the data stored within them, thus providing convenience in our lives. However, in practical use, barcode identification is relatively inefficient. Therefore, barcode location technology has been developed. This technology primarily utilizes the ability to identify barcodes to locate the area where the barcode is located in real time and accurately. In traditional technologies, barcode identification is usually based on a two-dimensional plane, identifying and tracking the target plane where the barcode is located. However, existing technologies have some limitations. For example, barcodes may be obscured, damaged, or affected by reflections, limiting their visibility and thus hindering effective tracking and location. Summary of the Invention

[0003] Therefore, it is necessary to provide a positioning method, apparatus, computer equipment, computer-readable storage medium, and computer program product to address the aforementioned technical problems.

[0004] Firstly, this application provides a positioning method. The method includes:

[0005] Acquire the target video and target video frames; wherein the target video frames are marked with the areas where the identifier to be identified is located;

[0006] The current video frame is obtained from the target video. If the feature changes between the current video frame and the historical video frames meet the preset requirements, the current video frame is determined as the target video frame.

[0007] Triangulation is performed on the feature points of the target video frame and the feature points of the corresponding historical video frames to determine the target coordinate system;

[0008] Determine the pose information of the target video frame in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system;

[0009] Based on the pose information of the target video frame, the three-dimensional coordinates of the identifier to be identified are mapped to the plane coordinate system where the target video frame is located, and the target position of the identifier to be identified in the plane coordinate system is determined; the target position is used to identify the identifier to be identified.

[0010] In one embodiment, the step of triangulating the feature points of the target video frame and the feature points of the corresponding historical video frames to determine the target coordinate system; determining the pose information of the target video frame in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system; and mapping the three-dimensional coordinates of the identifier to be identified to the planar coordinate system where the target video frame is located, based on the pose information of the target video frame, to determine the target position of the identifier to be identified in the planar coordinate system, includes:

[0011] When the number of target video frames reaches a preset threshold, the feature points of each target video frame and the feature points of the corresponding historical video frames are triangulated to determine the candidate coordinate systems corresponding to different target video frames, and the pose information of each target video frame in the corresponding candidate coordinate system and the candidate three-dimensional coordinates of the identifier to be identified in each candidate coordinate system are determined.

[0012] Based on the pose information of each target video frame in the candidate coordinate system, the three-dimensional coordinates of each candidate are converted into two-dimensional coordinates of the target video frame.

[0013] Compare the feature points in each target video frame and select two feature video frames;

[0014] The feature points of the two feature video frames are triangulated to determine the target coordinate system, and the pose information of the identifier to be identified in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system are determined.

[0015] Using the pose information of the identifier to be identified in the target coordinate system, the three-dimensional coordinates of the identifier to be identified are mapped to the planar coordinate system where the feature video frame is located, thereby determining the target position of the identifier to be identified in the planar coordinate system.

[0016] In one embodiment, the method further includes:

[0017] If the feature change between the current video frame and the historical video frame does not meet the preset requirements, the motion state of each feature point to be identified is determined based on the position change of the feature points in the current video frame and the historical video frame; wherein, the motion state includes at least one of the following: motion direction and motion distance.

[0018] Based on the motion state of the feature points, the target region to be identified in the current video frame is determined.

[0019] In one embodiment, acquiring the target video and target video frames includes:

[0020] Obtain the initial video and the preset initial target video frame in the corresponding initial video frame;

[0021] The initial video is corrected using a preset distortion strategy, and the resolution data of the initial video is adjusted to obtain the target video;

[0022] Based on the differences between the initial video and the target video, and the region to be identified in the initial target video frame, the target video frame and the target region to be identified in the target video frame are identified from the target video frame.

[0023] In one embodiment, the step of triangulating the feature points of the target video frame and the feature points of the corresponding historical video frames to determine the target coordinate system includes:

[0024] Based on the target video frame and the corresponding historical video frames, determine the pose information of the target video frame, and based on the pose information, determine the three-dimensional coordinate coefficients of the target video frame.

[0025] The three-dimensional coordinates of each point in the target video frame are determined by triangulation based on the three-dimensional coordinate coefficients.

[0026] In one embodiment, the method further includes:

[0027] Based on the initial frame images of the target video, establish an instant localization and map building system;

[0028] According to the preset frequency strategy, feature points are extracted from the real-time positioning and mapping system, and the display frequency of feature points in each video frame of the target video frame is recorded.

[0029] Based on the display frequency of the feature points, the pose information of each video frame of the target video is adjusted.

[0030] Secondly, this application also provides a positioning device. The device includes:

[0031] The data acquisition module is used to acquire the target video and the target video frame; wherein the target video frame is marked with the area where the identifier to be identified is located;

[0032] The target determination module is used to obtain the current video frame from the target video, and determine the current video frame as the target video frame when the feature changes between the current video frame and the historical video frames meet the preset requirements.

[0033] The coordinate system establishment module is used to triangulate the feature points of the target video frame and the feature points of the corresponding historical video frames to determine the target coordinate system.

[0034] The coordinate determination module is used to determine the pose information of the target video frame in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system.

[0035] The coordinate mapping module is used to map the three-dimensional coordinates of the identifier to be identified to the plane coordinate system where the target video frame is located, based on the pose information of the target video frame, and to determine the target position of the identifier to be identified in the plane coordinate system; the target position is used to identify the identifier to be identified.

[0036] In one embodiment, the coordinate determination module is further configured to, when the number of target video frames reaches a preset threshold, perform triangulation processing on the feature points of each target video frame and the feature points of the corresponding historical video frames, determine the candidate coordinate systems corresponding to different target video frames, and determine the pose information of each target video frame in the corresponding candidate coordinate system and the candidate three-dimensional coordinates of the identifier to be identified in each candidate coordinate system.

[0037] Based on the pose information of each target video frame in the candidate coordinate system, the three-dimensional coordinates of each candidate are converted into two-dimensional coordinates of the target video frame.

[0038] Compare the feature points in each target video frame and select two feature video frames;

[0039] The feature points of the two feature video frames are triangulated to determine the target coordinate system, and the pose information of the identifier to be identified in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system are determined.

[0040] The coordinate mapping module is further configured to use the pose information of the identifier to be identified in the target coordinate system to map the three-dimensional coordinates of the identifier to be identified to the planar coordinate system where the feature video frame is located, thereby determining the target position of the identifier to be identified in the planar coordinate system.

[0041] In one embodiment, the device further includes:

[0042] The motion state acquisition module is used to determine the motion state of each feature point to be identified based on the positional changes of feature points in the current video frame and the historical video frame when the feature change amount between the current video frame and the historical video frame does not meet the preset requirements; wherein, the motion state includes at least one of the following: motion direction and motion distance;

[0043] The target region localization module is used to determine the target region to be identified in the current video frame based on the motion state of the feature points.

[0044] In one embodiment, the data acquisition module includes:

[0045] The initial data acquisition submodule is used to acquire the initial video and the preset initial target video frame in the corresponding initial video frame;

[0046] The video correction submodule is used to correct the initial video using a preset distortion strategy and adjust the resolution data of the initial video to obtain the target video.

[0047] The target video frame determination submodule is used to identify the target video frame and the target region to be identified in the target video frame based on the difference between the initial video and the target video, and the region to be identified in the initial target video frame.

[0048] In one embodiment, the coordinate system establishment module includes:

[0049] The coefficient determination submodule is used to determine the pose information of the target video frame based on the target video frame and the corresponding historical video frames, and to determine the three-dimensional coordinate coefficients of the target video frame based on the pose information.

[0050] The coordinate determination submodule is used to perform triangulation based on three-dimensional coordinate coefficients to determine the three-dimensional coordinates of each point in the target video frame.

[0051] In one embodiment, the device further includes:

[0052] The system establishment module is used to establish an instant localization and map building system based on the initial frame images of the target video;

[0053] The frequency acquisition module is used to extract feature points from the real-time positioning and mapping system according to a preset frequency strategy, and record the display frequency of feature points in each video frame of the target video frame.

[0054] The pose adjustment module is used to adjust the pose information of each video frame of the target video based on the display frequency of the feature points.

[0055] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the positioning method as described in any one of the embodiments of this disclosure.

[0056] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the positioning method as described in any one of the embodiments of this disclosure.

[0057] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the positioning method as described in any of the embodiments of this disclosure.

[0058] The aforementioned positioning method, apparatus, computer equipment, storage medium, and computer program products acquire target video and target video frames containing the identifier to be identified. They determine the target video frame by analyzing feature changes between the current video frame and historical video frames. Triangulation is performed on the target video frame and historical video frames to obtain the target coordinate system and the pose information and three-dimensional coordinates of the identifier to be identified. Finally, the three-dimensional coordinates are mapped to a planar coordinate system, achieving accurate identification of the identifier's location. This method effectively achieves precise and efficient positioning of the identifier. By accurately capturing the target video and target video frames, and intelligently selecting the current video frame with significant feature changes as the target video frame, the timeliness of positioning is ensured. Simultaneously, the triangulation process constructs the target coordinate system, accurately calculates the position information of the target video frame and the identifier to be identified in three-dimensional space, and maps the three-dimensional coordinates to a planar coordinate system, achieving accurate identification and positioning of the identifier. This method extends two-dimensional planar positioning to three-dimensional space, meeting more complex application needs, and can accurately locate the target area even when the identifier to be identified is occluded. Attached Figure Description

[0059] Figure 1 This is a flowchart illustrating the positioning method in one embodiment;

[0060] Figure 2 This is a flowchart illustrating the selection and positioning of target video frames in one embodiment;

[0061] Figure 3 This is a flowchart illustrating the target area localization process in one embodiment;

[0062] Figure 4 This is a schematic diagram illustrating the process of acquiring the target video and target video frames in one embodiment;

[0063] Figure 5 This is a schematic diagram illustrating the process of establishing the target coordinate system in one embodiment;

[0064] Figure 6 This is a flowchart illustrating the optimization process of pose information for each video frame in one embodiment.

[0065] Figure 7 This is a flowchart illustrating the implementation of the positioning method in one embodiment;

[0066] Figure 8 This is a structural block diagram of the positioning device in one embodiment;

[0067] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] In one embodiment, such as Figure 1 As shown, a positioning method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0070] Step S100: Obtain the target video and the target video frame; wherein the target video frame is marked with the area where the identifier to be identified is located.

[0071] In one exemplary embodiment, the target video may include pre-stored video files or real-time captured video. The target video frame may be selected from target video frames. Marking the area to be identified may include selecting one or a group of feature points in the target video frame, or determining a specific area in the target video frame. Specifically, edge points of the target area may be selected to determine a specific area in the target video frame, i.e., the area where the identifier to be identified is located.

[0072] Step S200: Obtain the current video frame from the target video. If the feature changes between the current video frame and the historical video frames meet preset requirements, determine the current video frame as the target video frame.

[0073] In one exemplary embodiment, the current video frame can be extracted frame by frame from the target video and compared with historical video frames to detect feature changes. Feature changes may include, but are not limited to, changes in color, texture, shape, or motion trajectory. When these changes reach a preset threshold or meet specific conditions, the current video frame is considered to be significantly different from historical video frames, requiring repositioning of the identifier to be identified using three-dimensional coordinates.

[0074] In an exemplary embodiment, the feature changes between the current video frame and historical video frames may further include extracting feature points from each frame and matching the extracted feature points. The matching rules may include whether feature points from historical video frames exist in the current video frame, and the distance between the feature points in the current video frame and historical video frames, thereby determining the feature changes between the current video frame and historical video frames, and further determining whether the feature changes meet preset requirements, etc.

[0075] In one exemplary embodiment, the preset requirement may include whether the current video frame contains a preset proportion of feature points from historical video frames. For example, if the current video frame is required to contain 90% of the feature points from historical video frames, then the preset requirement is considered not met. In another exemplary embodiment, the preset requirement may further include that if the distance between a preset proportion of feature points in the current video frame and historical video frames is greater than a preset threshold, then the feature change is considered to meet the preset requirement. For example, if the distance between 90% of the feature points in the current video frame and historical video frames is greater than a preset threshold, then the feature change is considered to meet the preset requirement.

[0076] In one exemplary embodiment, if the feature changes between the current video frame and historical video frames do not meet preset requirements, optical flow can be used to locate the position of the identifier to be identified in the current frame. In another exemplary embodiment, the historical video frames may include previous video frames, i.e., each current video frame is compared with its corresponding previous video frame to determine whether the feature changes meet preset requirements.

[0077] Step S300: Triangulate the feature points of the target video frame and the feature points of the corresponding historical video frames to determine the target coordinate system.

[0078] In one exemplary embodiment, obtaining the feature points of the target video frame and the feature points of the historical video frames may include obtaining them using a feature extraction algorithm, determining the projection matrix of the two video frames based on the feature points of the target video frame and the historical video frames, and performing triangulation processing on the projection matrix to obtain the three-dimensional coordinates corresponding to the two images, thereby determining the target coordinate system, wherein the target coordinate system may include a three-dimensional coordinate system, etc.

[0079] Step S400: Determine the pose information of the target video frame in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system.

[0080] In one exemplary embodiment, the pose information may include the position and angle of the imaging device corresponding to the target video frame; that is, the position of the imaging device and the angle at which the target video frame was captured. The pose information may include, but is not limited to, the rotation matrix and translation vector of the target video frame in the target coordinate system. These parameters describe the pose of the target video frame relative to the target coordinate system. The three-dimensional coordinates of the identifier to be identified in the target coordinate system can be calculated based on the two-dimensional coordinates of the identifier to be identified in the target video frame and the target coordinate system, using projection relationships and triangulation principles.

[0081] Step S500: Based on the pose information of the target video frame, the three-dimensional coordinates of the identifier to be identified are mapped to the plane coordinate system where the target video frame is located, and the target position of the identifier to be identified in the plane coordinate system is determined; the target position is used to identify the identifier to be identified.

[0082] In one exemplary embodiment, the three-dimensional coordinates of the target coordinate system can be mapped to the planar coordinate system where the target video frame is located, based on the pose information of the target video, thereby determining the precise position of the identifier to be identified in the planar coordinate system. Specifically, a projection transformation matrix can be used to convert the three-dimensional coordinates of the identifier to be identified into the two-dimensional coordinates of the target video frame, achieving precise positioning of the identifier. In this way, even if the identifier to be identified moves or rotates in three-dimensional space, its position in the planar video frame can be accurately tracked and located.

[0083] In the aforementioned positioning method, the target video and its corresponding video frames containing the identifier to be identified are acquired. The target video frame is determined by analyzing the feature changes between the current video frame and historical video frames. Triangulation is performed on the target video frame and historical video frames to obtain the target coordinate system and the pose information and three-dimensional coordinates of the identifier. Finally, the three-dimensional coordinates are mapped to a planar coordinate system, achieving accurate identification of the identifier's location. This method effectively achieves precise and efficient positioning of the identifier. By accurately capturing the target video and its frames, and intelligently selecting the current video frame with significant feature changes as the target video frame, the timeliness of positioning is ensured. Simultaneously, the triangulation process constructs the target coordinate system, accurately calculates the position information of the target video frame and the identifier in three-dimensional space, and maps the three-dimensional coordinates to a planar coordinate system, achieving accurate identification and positioning of the identifier. This method extends two-dimensional planar positioning to three-dimensional space, meeting more complex application requirements, and can even accurately locate the target area when the identifier is occluded.

[0084] In one embodiment, such as Figure 2As shown, the process of triangulating the feature points of the target video frame and the corresponding feature points of historical video frames to determine the target coordinate system; determining the pose information of the target video frame in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system; and mapping the three-dimensional coordinates of the identifier to be identified to the planar coordinate system where the target video frame is located, based on the pose information of the target video frame, to determine the target position of the identifier to be identified in the planar coordinate system, includes:

[0085] Step S301: When the number of target video frames reaches a preset threshold, triangulation is performed on the feature points of each target video frame and the feature points of the corresponding historical video frames to determine the candidate coordinate systems corresponding to different target video frames, and the pose information of each target video frame in the corresponding candidate coordinate system and the candidate three-dimensional coordinates of the identifier to be identified in each candidate coordinate system are determined.

[0086] Step S302: Based on the pose information of each target video frame in the candidate coordinate system, convert each candidate three-dimensional coordinate into a two-dimensional coordinate of the target video frame.

[0087] Step S303: Compare the feature points in each target video frame and select two feature video frames.

[0088] Step S304: Triangulate the feature points of the two feature video frames to determine the target coordinate system, and determine the pose information of the identifier to be identified in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system.

[0089] Step S305: Using the pose information of the identifier to be identified in the target coordinate system, the three-dimensional coordinates of the identifier to be identified are mapped to the planar coordinate system where the feature video frame is located, and the target position of the identifier to be identified in the planar coordinate system is determined.

[0090] In one exemplary embodiment, comparing feature points in each target video frame and selecting two feature video frames may include calculating the reprojection error between the original target video frame and the two-dimensional coordinates, and selecting the two frames with the smallest reprojection error as feature video frames; specifically, the two-dimensional coordinates of each target video can be compared with the two-dimensional coordinates of the source target video frame, and the two target video frames with smaller errors can be selected as the two feature video frames, etc.

[0091] In one exemplary embodiment, after obtaining two feature video frames, the two feature video frames can be triangulated to obtain a target coordinate system. Based on this target coordinate system, the pose information and three-dimensional coordinates of the identifier to be identified in three-dimensional space can be accurately calculated. Furthermore, using the known pose information, the calculated three-dimensional coordinates are mapped to the planar coordinate system where the feature video frames are located, thereby determining the precise target position of the identifier to be identified in the planar coordinate system.

[0092] In one exemplary embodiment, comparing the feature points of each target video frame and selecting two feature video frames may include comparing the two-dimensional coordinates of the feature points in each target video frame and selecting the two video frames with the smallest error as feature video frames. Specifically, by analyzing the reprojection relationship between each frame and other consecutive frames and performing cross-validation, the two frames with the smallest error during the reprojection process can be selected. Subsequently, these two frames are used for the final triangulation calculation. This method enables precise localization of specific points in three-dimensional space.

[0093] In this embodiment, video frames with significant feature changes are selected from multiple target video frames and triangulated to construct multiple candidate coordinate systems. Then, the candidate 3D coordinates in each candidate coordinate system are converted into 2D coordinates of the target video frames for further comparison and analysis. To determine the most accurate target coordinate system, this embodiment selects two video frames with the smallest feature projection errors by comparing feature points in each target video frame. These two feature video frames provide more accurate and stable information, ensuring the accuracy of the triangulation process. Subsequently, the feature points of these two feature video frames are triangulated to determine the final target coordinate system. In the target coordinate system, the pose information and 3D coordinates of the identifier to be identified can be accurately calculated. Finally, using the known pose information, the 3D coordinates are mapped to the planar coordinate system where the feature video frames are located, thereby determining the precise target position of the identifier to be identified in the planar coordinate system. This method not only improves the accuracy and efficiency of positioning but also enhances the stability and reliability of the system. Even in complex application environments, it can accurately track and locate the position of the identifier to be identified.

[0094] In one embodiment, such as Figure 3 As shown, the method further includes:

[0095] Step S601: If the feature change between the current video frame and the historical video frame does not meet the preset requirements, determine the motion state of each feature point of the identifier to be identified based on the position change of the feature points in the current video frame and the historical video frame; wherein, the motion state includes at least one of the following: motion direction and motion distance.

[0096] Step S602: Based on the motion state of the feature points, determine the target region to be identified in the current video frame.

[0097] In an exemplary embodiment, when the feature changes between the current video frame and historical video frames do not meet preset requirements, the location of the region of interest (ROI) in the current frame can be determined using optical flow. Specifically, optical flow positioning of the ROI can include analyzing the positional changes of feature points between the current and historical video frames to determine the motion state of each feature point of the identifier to be identified. The motion state can be further divided into motion direction and motion distance, which helps to understand the dynamic behavior of the identifier more deeply. The motion direction refers to the trajectory of the feature point from the historical video frame to the current video frame, reflecting the spatial movement trend of the identifier. The motion distance is the actual length of the feature point's movement, providing specific numerical values ​​regarding the magnitude of the identifier's movement. After determining the motion state of the feature points, the target region of the identifier in the current video frame can be further determined based on this information. The target region is the area where the identifier is most likely to appear in the current video frame, predicted and delineated based on the motion state of the feature points. This method enables more accurate location of the identifier, allowing for effective tracking and positioning even in situations where feature changes are not significant or the environment is complex.

[0098] In this embodiment, by analyzing the positional changes of feature points between the current video frame and historical video frames, the dynamic behavior of the target identifier can be captured. When the amount of feature change does not meet the preset requirements, i.e., when the changes between video frames can be determined without directly using triangulation to determine the target video frame, this embodiment employs a more flexible and adaptable method. The motion state of each feature point of the target identifier is determined, including the motion direction and distance. The motion direction provides information on the movement trend of the target identifier in space, while the motion distance quantifies the magnitude of the movement. This information together constitutes a comprehensive description of the dynamic behavior of the target identifier. Based on the motion state of these feature points, the target region of the target identifier in the current video frame is predicted and determined. The target region is the region where the target identifier is most likely to appear, predicted based on the motion state of the feature points. This method enables effective tracking and positioning of the target identifier even when feature changes are not significant or the environment is complex. The introduction of this method not only improves the flexibility and adaptability of positioning but also further enhances the stability and reliability of the system in complex environments.

[0099] In one embodiment, such as Figure 4 As shown, acquiring the target video and target video frames includes:

[0100] Step S101: Obtain the initial video and the preset initial target video frame in the corresponding initial video frame.

[0101] Step S102: Using a preset distortion strategy, the initial video is corrected, and the resolution data of the initial video is adjusted to obtain the target video.

[0102] Step S103: Based on the difference between the initial video and the target video, and the region to be identified in the initial target video frame, the target video frame and the target region to be identified in the target video frame are identified from the target video frame.

[0103] In one exemplary embodiment, the memory matrix and related parameters can be recalculated based on the camera's intrinsic parameter matrix and distortion coefficients, and based on the resolution of the initial video, to match the resolution change. Then, the new intrinsic parameter matrix and distortion coefficients can be combined to perform resolution adjustment and distortion correction operations on the input image data, thereby obtaining the target video, etc.

[0104] In one exemplary embodiment, the initial target video frame may exist in the initial video. The initial target video frame may be a frame in the initial video, or a video frame that is labeled with the identifier to be identified. After distortion correction and resolution adjustment of the initial video frame, the identifier to be identified can be identified from the target video frame corresponding to the target video based on the identifier labeled in the initial video frame, and then labeled, thereby obtaining the target video frame with the labeled identifier.

[0105] In this embodiment, more accurate and clearer target video frames are obtained by preprocessing the initial video frames. First, a preset initial target video frame is determined from the acquired initial video. This initial target video frame is marked with the identifier to be identified, providing a benchmark for subsequent processing. Then, the initial video is corrected using a preset distortion strategy. This step aims to eliminate image distortion caused by camera lens distortion and other factors, improving video accuracy. Simultaneously, the resolution data of the initial video is adjusted to adapt to different application scenarios and requirements. The corrected and resolution-adjusted video is used as the target video. Further, based on the differences between the initial video and the target video, and the regional information of the identifier to be identified in the initial target video frame, the target video frame and the target region of the identifier to be identified within the target video frame are identified from the target video. This process fully utilizes the annotation information of the initial target video frame, combined with the correction and resolution adjustment results of the target video, to achieve precise positioning of the identifier to be identified. This method not only improves the efficiency and accuracy of video processing but also provides a solid foundation for subsequent tracking and positioning of the identifier to be identified.

[0106] In one embodiment, such as Figure 5As shown, the step of triangulating the feature points of the target video frame and the corresponding feature points of historical video frames to determine the target coordinate system includes:

[0107] Step S311: Determine the pose information of the target video frame based on the target video frame and the corresponding historical video frames, and determine the three-dimensional coordinate coefficients of the target video frame based on the pose information.

[0108] Step S312: Triangulation is performed based on the three-dimensional coordinate coefficients to determine the three-dimensional coordinates of each point in the target video frame.

[0109] In one exemplary embodiment, the three-dimensional coordinate coefficients may include the projection matrix of two frames of images, and the projection matrix may be triangulated to obtain the equation of the three-dimensional position. The equation may be solved to obtain the target three-dimensional coordinates, that is, to obtain the three-dimensional coordinate system of the target video frame, etc.

[0110] In one exemplary embodiment, feature points in the target video frame are matched with feature points in corresponding historical video frames to ensure temporal continuity and spatial consistency. Subsequently, based on these matched feature points, the pose information of the target video frame is calculated using triangulation principles. The pose information describes the position and orientation of the target video frame relative to a reference coordinate system and is crucial for determining the three-dimensional coordinates. After determining the pose information of the target video frame, this pose information is further utilized, combined with the camera's intrinsic parameter matrix, to construct a three-dimensional coordinate system for the target video frame. In this step, the three-dimensional coordinate system is obtained by analyzing the spatial positional relationships of feature points, reflecting the actual distribution of points in the target video frame in three-dimensional space. By triangulating the three-dimensional coordinate coefficients, the three-dimensional coordinates of each point in the target video frame can be accurately calculated, providing crucial spatial information for subsequent positioning tasks. This process not only improves positioning accuracy but also enhances the system's robustness, enabling it to accurately capture the position information of the identifier to be identified in complex environments.

[0111] In this embodiment, by performing in-depth processing on the target video frame and its corresponding historical video frames, the pose information of the target video frame is first determined. Pose information, as key data describing the position and orientation of the video frame in three-dimensional space, is the foundation for subsequent calculations. Based on this information, a three-dimensional coordinate system for the target video frame is further constructed, which accurately reflects the actual position of each point in the video frame in three-dimensional space. Subsequently, triangulation technology is used to meticulously calculate the three-dimensional coordinate coefficients, thereby accurately obtaining the three-dimensional coordinates of each point in the target video frame. This process not only ensures the accuracy of positioning but also significantly improves the stability and adaptability of the system, enabling it to accurately capture and locate the position information of the identifier to be identified in various complex environments.

[0112] In one embodiment, such as Figure 6 As shown, the method further includes:

[0113] Step S611: Establish an instant localization and map building system based on the initial frame image of the target video.

[0114] Step S612: According to the preset frequency strategy, feature points are extracted from the real-time positioning and mapping system, and the display frequency of feature points in each video frame of the target video frame is recorded.

[0115] Step S613: Adjust the pose information of each video frame of the target video based on the display frequency of the feature points.

[0116] In one exemplary embodiment, a pre-trained bag-of-words model and target video frames can be loaded to initialize the localization and mapping modules of the SLAM (Simultaneous Localization and Mapping) system. Simultaneously, based on multi-threading design principles, memory resources can be dynamically allocated according to the input thread ID, ensuring efficient resource utilization in a multi-threaded environment. The bag-of-words model can convert each image into a word frequency vector, helping the system quickly identify the match between the current keyframe and historical keyframes, determining whether the region has been hit (loop closure). After detecting a loop closure, the SLAM system can use these matching relationships to perform global pose graph optimization (e.g., image optimization, backend BA, etc.). This corrects accumulated errors and adjusts the pose information of each video frame of the target video.

[0117] In one exemplary embodiment, feature points are periodically extracted from the real-time localization and mapping (RTL) system according to a preset frequency strategy. This strategy ensures that feature point extraction is neither too frequent (to avoid increasing computational burden) nor too sparse (to avoid losing critical information). Simultaneously, the display frequency of feature points in each target video frame is recorded; this data reflects the stability and importance of feature points within the video frame. Based on the display frequency of feature points, the pose information of each video frame of the target video is fine-tuned. Feature points with high display frequency and good stability are given higher weights, allowing them to play a greater role in pose calculation. Conversely, feature points with low display frequency and poor stability have their weights appropriately reduced to minimize their interference with pose calculation. This process not only improves positioning accuracy but also enhances the system's robustness, enabling it to better adapt to complex and changing application environments. By combining the RTL system and the display frequency of feature points, this embodiment provides a more accurate and reliable pose adjustment method, laying a solid foundation for subsequent tracking and localization tasks.

[0118] In this embodiment, the system's autonomy and intelligence are enhanced by incorporating a Simultaneous Localization and Mapping (SLAM) system into the localization process. First, the basic framework of the SLAM system is constructed based on the initial frame images of the target video. This step provides necessary initialization information for subsequent pose adjustment and map building. Then, feature points are extracted from the SLAM system according to a preset frequency strategy. This strategy ensures efficient and accurate feature point extraction, avoiding waste of computational resources. Simultaneously, the display frequency of feature points in each target video frame is recorded; this data provides crucial information for subsequent pose adjustment. Based on the display frequency of feature points, the pose information of each video frame of the target video is finely adjusted. This process not only improves the accuracy of localization but also enhances the system's adaptability and robustness. By combining the SLAM system and the display frequency of feature points, this embodiment provides a more accurate and reliable pose adjustment scheme, offering strong support for subsequent tracking and localization tasks.

[0119] In one exemplary embodiment, the positioning method may be as follows: Figure 7 The schematic diagram implementation specifically includes:

[0120] Step S901, Data Initialization and Loading: The system first loads the camera's intrinsic parameter matrix and distortion coefficients, and recalculates the memory matrix and related parameters based on the input video stream to match resolution changes. Simultaneously, it loads the pre-trained bag-of-words model and the starting frame image to initialize the SLAM system's localization and mapping modules. Following multi-threading design principles, the system dynamically allocates memory resources based on the input thread ID, ensuring efficient resource utilization and stability in a multi-threaded environment.

[0121] Step S902, data preprocessing: Using the new intrinsic parameter matrix and distortion coefficients, resolution adjustment and distortion correction operations are performed on the input image data. This preprocessing step provides distortion-free, standardized input data for subsequent feature extraction, matching, and localization algorithms, improving the overall accuracy and robustness of the system.

[0122] Step S903, Local Inter-Frame Tracking and Local Optimization: Within a short time, the system uses the data from each frame in the video stream and the user-specified region of interest (ROI) to locate the ROI in the current frame using optical flow. Simultaneously, the system evaluates the current frame based on keyframe criteria, and for frames that meet the criteria, their pose and ROI information are recorded in the system.

[0123] In one exemplary embodiment, the positional change of the region of interest (ROI) in the current frame can be calculated based on the motion vector of each positioning point obtained from optical flow, thereby locating the region. Keyframe determination criteria may include selecting frames with significant changes in viewpoint compared to the previous frame; the change in viewpoint can be roughly matched to the degree of change in keypoints for judgment. The correlation may include establishing the relationship between the positional change of the ROI and the keyframes primarily through pose estimation during local inter-frame tracking. The ROI of each frame is tracked to its new position in the current frame using optical flow, and its actual position in 3D space is estimated based on the keyframe position and camera pose. Each keyframe forms a pair with its corresponding ROI, providing a basis for localization of the ROI in the next keyframe.

[0124] Step S904, Global Keyframe Localization and Optimization: During long-term operation, the system continuously monitors whether the number of keyframes associated with the region of interest reaches a set threshold. When the condition is met, the system first updates the pose information of the keyframe in the SLAM system. Then, for each point of interest, triangulation is performed using the projection matrices of the two frames in the keyframe to estimate its 3D position. Subsequently, by cross-validating with the reprojection errors of other frames, the two frames with the smallest errors are selected for the final triangulation calculation. This process not only achieves high-precision 3D point localization but also further optimizes the stability and anti-interference capability of the localization.

[0125] In one exemplary embodiment, the DLT (Direct Linear Transform) algorithm can be used to construct a system of linear equations to solve for the coordinates of 3D points. The cross-validation process may include first mapping the calculated 3D points back to the 2D image plane using the projection matrices of two image frames, comparing them with the actual image points, and calculating the reprojection error. Then, if multiple image frames are available, the two frames with the smallest reprojection error are selected for final triangulation by calculating the reprojection error for each pair of images, thus ensuring higher accuracy in the final 3D position. Selecting the two frames with the smallest error may include first calculating the projection matrix and corresponding image points for each pair of keyframes, using a triangulation algorithm to obtain the 3D points, and calculating the reprojection error. Then, by comparing all possible frame pairs, the image with the lowest reprojection error is selected. This can be achieved by sorting all reprojection errors and selecting the two frames with the smallest errors, etc.

[0126] Step S905, data post-processing: The system can remap the image point location information output by the positioning module back to the resolution coordinate system of the original image. This step ensures that the output result is consistent with the resolution and spatial scale of the input image, facilitating subsequent visualization and analysis on the original image.

[0127] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0128] Based on the same inventive concept, this application also provides a positioning device for implementing the positioning method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more positioning device embodiments provided below can be found in the limitations of the positioning method described above, and will not be repeated here.

[0129] In one embodiment, such as Figure 8 As shown, a positioning device 100 is provided, including: a data acquisition module 101, a target determination module 102, a coordinate system establishment module 103, a coordinate determination module 104, and a coordinate mapping module 105, wherein:

[0130] The data acquisition module is used to acquire the target video and the target video frame; wherein the target video frame is marked with the area where the identifier to be identified is located;

[0131] The target determination module is used to obtain the current video frame from the target video, and determine the current video frame as the target video frame when the feature changes between the current video frame and the historical video frames meet the preset requirements.

[0132] The coordinate system establishment module is used to triangulate the feature points of the target video frame and the feature points of the corresponding historical video frames to determine the target coordinate system.

[0133] The coordinate determination module is used to determine the pose information of the target video frame in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system.

[0134] The coordinate mapping module is used to map the three-dimensional coordinates of the identifier to be identified to the plane coordinate system where the target video frame is located, based on the pose information of the target video frame, and to determine the target position of the identifier to be identified in the plane coordinate system; the target position is used to identify the identifier to be identified.

[0135] In one embodiment, the coordinate determination module is further configured to, when the number of target video frames reaches a preset threshold, perform triangulation processing on the feature points of each target video frame and the feature points of the corresponding historical video frames, determine the candidate coordinate systems corresponding to different target video frames, and determine the pose information of each target video frame in the corresponding candidate coordinate system and the candidate three-dimensional coordinates of the identifier to be identified in each candidate coordinate system.

[0136] Based on the pose information of each target video frame in the candidate coordinate system, the three-dimensional coordinates of each candidate are converted into two-dimensional coordinates of the target video frame.

[0137] Compare the feature points in each target video frame and select two feature video frames;

[0138] The feature points of the two feature video frames are triangulated to determine the target coordinate system, and the pose information of the identifier to be identified in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system are determined.

[0139] The coordinate mapping module is further configured to use the pose information of the identifier to be identified in the target coordinate system to map the three-dimensional coordinates of the identifier to be identified to the planar coordinate system where the feature video frame is located, thereby determining the target position of the identifier to be identified in the planar coordinate system.

[0140] In one embodiment, the device further includes:

[0141] The motion state acquisition module is used to determine the motion state of each feature point to be identified based on the positional changes of feature points in the current video frame and the historical video frame when the feature change amount between the current video frame and the historical video frame does not meet the preset requirements; wherein, the motion state includes at least one of the following: motion direction and motion distance;

[0142] The target region localization module is used to determine the target region to be identified in the current video frame based on the motion state of the feature points.

[0143] In one embodiment, the data acquisition module includes:

[0144] The initial data acquisition submodule is used to acquire the initial video and the preset initial target video frame in the corresponding initial video frame;

[0145] The video correction submodule is used to correct the initial video using a preset distortion strategy and adjust the resolution data of the initial video to obtain the target video.

[0146] The target video frame determination submodule is used to identify the target video frame and the target region to be identified in the target video frame based on the difference between the initial video and the target video, and the region to be identified in the initial target video frame.

[0147] In one embodiment, the coordinate system establishment module includes:

[0148] The coefficient determination submodule is used to determine the pose information of the target video frame based on the target video frame and the corresponding historical video frames, and to determine the three-dimensional coordinate coefficients of the target video frame based on the pose information.

[0149] The coordinate determination submodule is used to perform triangulation based on three-dimensional coordinate coefficients to determine the three-dimensional coordinates of each point in the target video frame.

[0150] In one embodiment, the device further includes:

[0151] The system establishment module is used to establish an instant localization and map building system based on the initial frame images of the target video;

[0152] The frequency acquisition module is used to extract feature points from the real-time positioning and mapping system according to a preset frequency strategy, and record the display frequency of feature points in each video frame of the target video frame.

[0153] The pose adjustment module is used to adjust the pose information of each video frame of the target video based on the display frequency of the feature points.

[0154] Each module in the aforementioned positioning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0155] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores feature point data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a positioning method.

[0156] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0157] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0158] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0159] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0160] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A positioning method, characterized in that, The method includes: Acquire the target video and target video frames; wherein the target video frames are marked with the areas where the identifier to be identified is located; The current video frame is obtained from the target video. If the feature changes between the current video frame and the historical video frames meet the preset requirements, the current video frame is determined as the target video frame. Triangulation is performed on the feature points of the target video frame and the feature points of the corresponding historical video frames to determine the target coordinate system; Determine the pose information of the target video frame in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system; Based on the pose information of the target video frame, the three-dimensional coordinates of the identifier to be identified are mapped to the plane coordinate system where the target video frame is located, and the target position of the identifier to be identified in the plane coordinate system is determined; the target position is used to identify the identifier to be identified.

2. The method according to claim 1, characterized in that, The process of triangulating the feature points of the target video frame and the corresponding feature points of historical video frames to determine the target coordinate system; determining the pose information of the target video frame in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system; and mapping the three-dimensional coordinates of the identifier to be identified to the planar coordinate system where the target video frame is located, based on the pose information of the target video frame, to determine the target position of the identifier to be identified in the planar coordinate system, includes: When the number of target video frames reaches a preset threshold, the feature points of each target video frame and the feature points of the corresponding historical video frames are triangulated to determine the candidate coordinate systems corresponding to different target video frames, and the pose information of each target video frame in the corresponding candidate coordinate system and the candidate three-dimensional coordinates of the identifier to be identified in each candidate coordinate system are determined. Based on the pose information of each target video frame in the candidate coordinate system, the three-dimensional coordinates of each candidate are converted into two-dimensional coordinates of the target video frame. Compare the feature points in each target video frame and select two feature video frames; The feature points of the two feature video frames are triangulated to determine the target coordinate system, and the pose information of the identifier to be identified in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system are determined. Using the pose information of the identifier to be identified in the target coordinate system, the three-dimensional coordinates of the identifier to be identified are mapped to the planar coordinate system where the feature video frame is located, thereby determining the target position of the identifier to be identified in the planar coordinate system.

3. The method according to claim 1, characterized in that, The method further includes: If the feature change between the current video frame and the historical video frame does not meet the preset requirements, the motion state of each feature point to be identified is determined based on the position change of the feature points in the current video frame and the historical video frame; wherein, the motion state includes at least one of the following: motion direction and motion distance. Based on the motion state of the feature points, the target region to be identified in the current video frame is determined.

4. The method according to claim 1, characterized in that, The acquisition of the target video and target video frames includes: Obtain the initial video and the preset initial target video frame in the corresponding initial video frame; The initial video is corrected using a preset distortion strategy, and the resolution data of the initial video is adjusted to obtain the target video; Based on the differences between the initial video and the target video, and the region to be identified in the initial target video frame, the target video frame and the target region to be identified in the target video frame are identified from the target video frame.

5. The method according to claim 1, characterized in that, The step of triangulating the feature points of the target video frame and the corresponding feature points of historical video frames to determine the target coordinate system includes: Based on the target video frame and the corresponding historical video frames, determine the pose information of the target video frame, and based on the pose information, determine the three-dimensional coordinate coefficients of the target video frame. The three-dimensional coordinates of each point in the target video frame are determined by triangulation based on the three-dimensional coordinate coefficients.

6. The method according to claim 1, characterized in that, The method further includes: Based on the initial frame images of the target video, establish an instant localization and map building system; According to the preset frequency strategy, feature points are extracted from the real-time positioning and mapping system, and the display frequency of feature points in each video frame of the target video frame is recorded. Based on the display frequency of the feature points, the pose information of each video frame of the target video is adjusted.

7. A positioning device, characterized in that, The device includes: The data acquisition module is used to acquire the target video and the target video frame; wherein the target video frame is marked with the area where the identifier to be identified is located; The target determination module is used to obtain the current video frame from the target video, and determine the current video frame as the target video frame when the feature changes between the current video frame and the historical video frames meet the preset requirements. The coordinate system establishment module is used to triangulate the feature points of the target video frame and the feature points of the corresponding historical video frames to determine the target coordinate system. The coordinate determination module is used to determine the pose information of the target video frame in the target coordinate system and the three-dimensional coordinates of the identifier to be identified in the target coordinate system. The coordinate mapping module is used to map the three-dimensional coordinates of the identifier to be identified to the plane coordinate system where the target video frame is located, based on the pose information of the target video frame, and to determine the target position of the identifier to be identified in the plane coordinate system; the target position is used to identify the identifier to be identified.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images