A method, apparatus, and device for determining video route information.

Abstract

This application discloses a method, apparatus, and device for determining video route information. The method includes: acquiring the line flow of two-dimensional line segments in a video image frame corresponding to the video to be processed, and the line flow of three-dimensional line segments in the video image frame; supplementing the missing parts of the line flow of the three-dimensional line segments based on the line flow of the two-dimensional line segments to obtain the line flow of the target three-dimensional line segments; and determining the route information corresponding to the video to be processed based on the line flow of the target three-dimensional line segments.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a method, apparatus and device for determining video route information. Background Technology

[0002] Determining the shooting route in a video is a crucial task in video processing technology. There are two main methods for this: one is based on sensor data acquired by LiDAR (Light Detection and Ranging) sensors. This method is significantly affected by lighting conditions, and its accuracy is improved through photometric calibration, exposure time control, and sensor data processing. The other method extracts feature points from video frames as input for a local or global map. This method is unaffected by lighting variations. However, when the video scene contains occlusion, low texture, or repetitive textures, both methods struggle to achieve highly accurate shooting routes. Summary of the Invention

[0003] In view of this, embodiments of this application provide a method for determining video route information, which can accurately determine the shooting route of the video.

[0004] The technical solution of this application is implemented as follows:

[0005] This application provides a method for determining video route information, the method comprising:

[0006] Obtain the line flow of two-dimensional line segments in the video image frame corresponding to the video to be processed, and the line flow of three-dimensional line segments in the video image frame, wherein the line flow is a time series of the line segments;

[0007] The missing parts of the line flow of the three-dimensional line segment are supplemented based on the line flow of the two-dimensional line segment to obtain the line flow of the target three-dimensional line segment.

[0008] Based on the line flow of the target three-dimensional line segment, the route information corresponding to the video to be processed is determined, and the route information is used to represent the shooting trajectory of the image acquisition device that captured the video to be processed.

[0009] This application provides a device for determining video route information, including:

[0010] The acquisition module is used to acquire the line flow of two-dimensional line segments in the video image frame corresponding to the video to be processed, and the line flow of three-dimensional line segments in the video image frame, wherein the line flow is a time series of the line segments;

[0011] The first supplementary module is used to supplement the missing part of the line flow of the three-dimensional line segment based on the line flow of the two-dimensional line segment, so as to obtain the line flow of the target three-dimensional line segment.

[0012] The determination module is used to determine the route information corresponding to the video to be processed based on the line flow of the target three-dimensional line segment. The route information is used to represent the shooting trajectory of the image acquisition device that captured the video to be processed.

[0013] This application provides a device for determining video route information, including:

[0014] Memory, used to store executable instructions for determining video route information;

[0015] The processor, when executing the instructions for determining executable video route information stored in the memory, implements the method provided in the embodiments of this application.

[0016] This application provides a computer-readable storage medium storing computer-executable instructions configured to perform the aforementioned video route information determination step.

[0017] This application provides a method, apparatus, and device for determining video route information. Using this technical solution, firstly, the line flow of two-dimensional line segments and the line flow of three-dimensional line segments in the corresponding video image frames of the video to be processed are obtained; then, the missing parts of the line flow of the three-dimensional line segments are supplemented based on the line flow of the two-dimensional line segments to obtain the line flow of the target three-dimensional line segments; finally, the route information corresponding to the video to be processed is determined based on the line flow of the target three-dimensional line segments. Attached Figure Description

[0018] Figure 1 A flowchart illustrating a method for determining video route information provided in an embodiment of this application;

[0019] Figure 2 A schematic diagram of the line flow of a two-dimensional line segment provided in an embodiment of this application;

[0020] Figure 3 A schematic flowchart illustrating a method for obtaining the line flow of a two-dimensional line segment, provided in an embodiment of this application;

[0021] Figure 4 A flowchart illustrating a video-based route extraction method provided in this application embodiment;

[0022] Figure 5 A schematic diagram illustrating line flow provided in an embodiment of this application;

[0023] Figure 6A flowchart illustrating another video-based route extraction method provided in this application embodiment;

[0024] Figure 7 This is a schematic diagram of the composition structure of a video route information determination device provided in an embodiment of this application;

[0025] Figure 8 This is a schematic diagram of the composition structure of a video route information determination device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] In the following description, references to “some embodiments” or “other embodiments” describe a subset of all possible embodiments. However, it is understood that “some embodiments” or “other embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0029] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0031] Based on the problems existing in related technologies, embodiments of this application provide a method for determining video route information, which may be the shooting route or shooting trajectory of the video. For example... Figure 1 The diagram shown is a flowchart illustrating a method for determining video route information according to an embodiment of this application. The method includes the following steps:

[0032] S101. Obtain the line flow of two-dimensional line segments in the corresponding video image frame of the video to be processed, and the line flow of three-dimensional line segments in the video image frame.

[0033] It should be noted that the video to be processed can be video captured by an image acquisition device (camera, camera, etc.). This video can be acquired in real time or pre-captured and stored on a server or other device. The video image frames corresponding to the video to be processed can be multiple frames obtained after segmenting the video.

[0034] In some embodiments, the line flow is a time series of line segments. For example, the line flow of a two-dimensional line segment may include the time information and position information of the changes in the two-dimensional line segment. A corresponding two-dimensional line segment can be obtained from each video image frame. By extracting two-dimensional line segments at the same position from multiple changing video image frames, the line flow of the two-dimensional line segment can be obtained. The two-dimensional line segment can be of any shape, such as a straight line in an image frame. As the video image frames change, the position of the straight line segment remains unchanged, but the size of the straight line segment may change.

[0035] In some embodiments, the two-dimensional line segment can be an observed line segment in a video image frame. A two-dimensional line segment can be determined at any position in the first video image frame for observation. By extracting the observed two-dimensional line segments from multiple video image frames, the line flow of the two-dimensional line segment can be obtained. For example, if the straight line segment corresponding to the vertical edge of the display in the video image frame is observed as a two-dimensional line segment, the line flow of the finally obtained two-dimensional line segment includes information about the change of the straight line segment over time. The position of the straight line segment relative to the display does not change, but its size may change. For example, the straight line segment may become shorter; it may become longer; it may first become longer and then shorter. Of course, the situation of the change in the size of the straight line segment is merely an illustrative example, and this application does not limit it.

[0036] In some embodiments, within the same video segment, three-dimensional line segments correspond to two-dimensional line segments, and the reference objects for both are the same. A three-dimensional line segment can be the outline of the object containing the two-dimensional line segment. For example, if a two-dimensional line segment is a line segment on a display, the corresponding three-dimensional line segment can be the entire outline of that display. Similar to the line flow of a two-dimensional line segment, the line flow of a three-dimensional line segment can include the positional and temporal information of the three-dimensional line segment. The positional information can be the location of the three-dimensional line segment within a video frame. As the video frames change, the size of the three-dimensional line segment may also change.

[0037] In some embodiments, after selecting the observed two-dimensional line segment, the corresponding three-dimensional line segment can be determined. For example, if the selected observed two-dimensional line segment is a straight line segment on the table surface, that is, the reference object corresponding to the two-dimensional line segment is the table, then the corresponding three-dimensional line segment can be the entire outline of the table. In other embodiments, the observed three-dimensional line segment can be selected first, the reference object corresponding to the three-dimensional line segment can be determined, and then a two-dimensional line segment can be selected from the reference object. For example, if the selected observed three-dimensional line segment is the entire outline of a chair, then the corresponding two-dimensional line segment can be a line segment on the chair back, a line segment on the chair leg, etc.

[0038] S102. Based on the line flow of the two-dimensional line segment, the missing part of the line flow of the three-dimensional line segment is supplemented to obtain the line flow of the target three-dimensional line segment.

[0039] In some embodiments, during the video recording process, as the image acquisition device moves, some objects in the image may be obscured, resulting in the obstruction of some three-dimensional line segments and causing missing parts of the line flow of the three-dimensional line segments. In this case, the three-dimensional line segments can be supplemented based on the two-dimensional line segments to obtain the line flow of the complete three-dimensional line segments, i.e., the line flow of the target three-dimensional line segments.

[0040] For example, if the observed 3D line segment is the outline of a display, as the camera moves, the outline of the display may be obscured by objects such as bookshelves or walls, making it impossible to obtain the complete line flow of the 3D line segment. In this case, the camera pose can be obtained based on the size change of the observed 2D line segment on the display, thereby determining the line flow of the missing 3D line segment.

[0041] S103. Based on the line flow of the target three-dimensional line segment, determine the route information corresponding to the video to be processed.

[0042] In some embodiments, the line flow of the target 3D line segment may include the line flow of the 3D line segment obtained by direct detection of video image frames, and the line flow of the 3D line segment supplemented by 2D line segments. Based on the line flow of the target 3D line segment, the size change information and position change information of the reference object corresponding to the 3D line segment (which can be any object in the video to be processed) can be obtained, thereby determining the route information corresponding to the video to be processed. The route information is used to represent the shooting trajectory of the image acquisition device capturing the video to be processed.

[0043] In this embodiment, firstly, the line flow of two-dimensional line segments and the line flow of three-dimensional line segments in the video image frames corresponding to the video to be processed are obtained; then, the missing parts of the line flow of the three-dimensional line segments are supplemented based on the line flow of the two-dimensional line segments to obtain the line flow of the target three-dimensional line segments; finally, the route information corresponding to the video to be processed is determined based on the line flow of the target three-dimensional line segments. Thus, by supplementing the line flow of the three-dimensional line segments with the line flow of the two-dimensional line segments, the complete line flow of the target three-dimensional line segments can be obtained. This allows the shooting trajectory of the video to be processed to be accurately determined even when there is occlusion, low texture, or repeated texture in the shooting scene, based on the line flow of the target three-dimensional line segments.

[0044] In some embodiments of this application, the missing parts of a three-dimensional line segment are supplemented based on the line flow of a two-dimensional line segment to obtain the line flow of the target three-dimensional line segment. That is, step S102 can also be implemented by the following steps S1021A to S1024A. Each step is described below.

[0045] S1021A. Based on the line flow of a two-dimensional line segment, determine the size change information of the two-dimensional line segment.

[0046] It should be noted that the dimensional change information of a two-dimensional line segment can include changes in length, thickness, and direction. In some embodiments, the dimensional change information of a two-dimensional line segment can be determined by analyzing the line flow of the two-dimensional line segment. For example, from... Figure 2 As can be seen from the line flow of the two-dimensional line segment shown, the two-dimensional straight line segment changes from short to long and from thin to thick.

[0047] S1022A: Based on size change information, determine the pose information of the image acquisition device that captures the video to be processed.

[0048] In some embodiments, the pose information of the image acquisition device may include the shooting angle and orientation of the image acquisition device. Based on the size change information of the two-dimensional line segment, the shooting angle and orientation of the corresponding image acquisition device can be determined. For example, if the change in length of the two-dimensional line segment is determined based on the line flow, it can be determined that the image acquisition device capturing the video to be processed is gradually moving away from the reference object corresponding to the two-dimensional line segment.

[0049] S1023A. Based on pose information, determine the line flow of the first three-dimensional line segment.

[0050] In some embodiments, the line flow of the first three-dimensional line segment is the line flow of the missing three-dimensional line segments in the line flow of the three-dimensional line segments. When there is occlusion in the shooting scene, or as the shooting angle changes, some objects in the scene may not be fully presented in the video. If the object is used as a reference, the line flow of its corresponding three-dimensional line segment will be missing. In this case, the complete outline corresponding to the reference object cannot be detected in one or several frames of the video to be processed, so the complete three-dimensional line segment cannot be obtained, resulting in the missing line flow of the three-dimensional line segment.

[0051] In some embodiments, for the line flow of missing 3D line segments, each missing 3D line segment can be determined based on the pose change information of the image acquisition device. Based on each 3D line segment and its time change information, the line flow of the first 3D line segment can be obtained. In practice, based on the shooting angle and shooting position information of the image acquisition device, combined with the actual scene where the reference object is located, the complete contour corresponding to the reference object can be determined, thereby inferring the 3D line segment corresponding to the reference object.

[0052] S1024A: Obtain the line flow of the target three-dimensional line segment based on the line flow of the first three-dimensional line segment.

[0053] In some embodiments, after obtaining the line flow of the missing three-dimensional line segment, the line flow of the directly detected three-dimensional line segment and the line flow of the first three-dimensional line segment can be combined in chronological order to obtain the line flow of the complete three-dimensional line segment.

[0054] In some embodiments, the line flow of the first three-dimensional line segment may contain only a three-dimensional line segment corresponding to a single video image frame. In this case, the three-dimensional line segment can be inserted into the line flow of the directly detected three-dimensional line segment according to the timing information of the video image frame corresponding to the three-dimensional line segment. In other embodiments, the line flow of the first three-dimensional line segment may include three-dimensional line segments corresponding to multiple consecutive or non-consecutive video image frames. Similarly, according to the timing information of the video image frames corresponding to each three-dimensional line segment, each three-dimensional line segment is inserted into the corresponding position in the line flow of the directly detected three-dimensional line segment.

[0055] In other embodiments, all three-dimensional line segments in the video image frame may be directly detectable, that is, the line flow of the three-dimensional line segment does not include the missing part, and it is not necessary to supplement the line flow of the three-dimensional line segment based on the line flow of the two-dimensional line segment. In this case, the line flow of the target three-dimensional line segment is the line flow of the complete three-dimensional line segment.

[0056] Understandably, by obtaining the size change information of two-dimensional line segments based on the line flow of two-dimensional line segments, the pose information of the image acquisition device is determined based on the size change information, and the missing part in the line flow of three-dimensional line segments is determined based on the pose information. This realizes the supplementation of the line flow of three-dimensional line segments by the line flow of two-dimensional line segments, thereby obtaining the line flow of complete three-dimensional line segments and avoiding the problem of not being able to obtain the complete shooting route due to occlusion and other issues in the scene.

[0057] In some embodiments of this application, the missing parts of a three-dimensional line segment are supplemented based on the line flow of a two-dimensional line segment to obtain the line flow of the target three-dimensional line segment. The above step S102 can also be implemented by the following steps S1021B to S1023B. Each step is described below.

[0058] S1021B, Determine the reference object for the line flow that includes two-dimensional line segments in a video image frame.

[0059] In some embodiments, reference objects corresponding to the two-dimensional line segments in each video image frame can be determined first, thereby obtaining each reference object corresponding to the line flow of the two-dimensional line segment. In practice, the reference objects for the line flow of the two-dimensional line segment may include one or more. If the reference objects of the two-dimensional line segment are always presented in each video image frame corresponding to the video to be processed, then the reference objects for the line flow of the two-dimensional line segment may include one. If the reference object of the two-dimensional line segment is occluded at the beginning of a certain frame in the video image frame, then other reference objects can be selected to observe the two-dimensional line segment object from the beginning of that frame, thereby obtaining multiple reference objects for the line flow of the two-dimensional line segment.

[0060] In some embodiments, the reference object for the line flow of the two-dimensional line segment can be any object in the video image frame, such as a table, chair, monitor, lamp, etc. In some embodiments, once the two-dimensional line segment for observation in the initial image frame is determined, the corresponding reference object is also determined. Then, subsequent video image frames are traversed to determine the reference objects corresponding to the two-dimensional line segments, thereby obtaining the reference object for the line flow of the entire two-dimensional line segment.

[0061] S1022B, Determine the line flow of the second three-dimensional line segment that includes the line flow of the two-dimensional line segment in the reference object.

[0062] In some embodiments, the line flow of the second three-dimensional line segment is the line flow of the missing three-dimensional line segment in the line flow of the three-dimensional line segment. Within the same image frame, the reference objects corresponding to the two-dimensional and three-dimensional line segments are the same. When a three-dimensional line segment in a video image frame is occluded, the missing three-dimensional line segment can be determined based on the two-dimensional line segment in that image frame and its corresponding reference object. In implementation, the reference object corresponding to the two-dimensional line segment can be determined first, and then the corresponding three-dimensional line segment can be determined based on that reference object. For example, if the reference object corresponding to the two-dimensional line segment is a display (e.g., the two-dimensional line segment is a line segment in the display), the outline of the display can be determined as a three-dimensional line segment, thereby supplementing the three-dimensional line segment.

[0063] In some embodiments, if there are missing three-dimensional line segments in multiple video image frames, the three-dimensional line segments can be supplemented sequentially based on the two-dimensional line segments in each image frame to obtain the line flow of the missing three-dimensional line segments, i.e., the line flow of the second three-dimensional line segment.

[0064] S1023B: Obtain the line flow of the target three-dimensional line segment based on the line flow of the second three-dimensional line segment.

[0065] In some embodiments, after obtaining the line flow of the second three-dimensional line segment by supplementing the line flow of the three-dimensional line segment with the line flow of the two-dimensional line segment, the line flow of the second three-dimensional line segment can be combined with the line flow of the directly detected three-dimensional line segment. During the combination, the line flow of the complete three-dimensional line segment corresponding to each video image frame is obtained according to the temporal relationship of each video image frame, that is, the line flow of the target three-dimensional line segment.

[0066] It is understandable that by determining a reference object for the line flow of a two-dimensional line segment, and based on the line flow of the two-dimensional line segment included in that reference object, the line flow of the corresponding three-dimensional line segment can be determined, thereby supplementing the line flow of the three-dimensional line segment and obtaining the line flow of the complete three-dimensional line segment. This allows for the determination of a more accurate shooting route based on the line flow of the complete three-dimensional line segment.

[0067] In some embodiments of this application, after obtaining the line flow of two-dimensional line segments in the video image frame corresponding to the video to be processed, and the line flow of three-dimensional line segments in the video image frame, i.e., step S102, the following step S201 can also be performed, which will be described below.

[0068] S201. The missing part of the line flow of two-dimensional line segments is supplemented by the line flow of three-dimensional line segments.

[0069] In some embodiments, the line flow of the two-dimensional line segment obtained from the video image frame may be missing. For example, if the two-dimensional line segment is occluded, it cannot be detected in the video image frame. In this case, the corresponding two-dimensional line segment can be determined based on the three-dimensional line segment in the video image frame, thereby supplementing the two-dimensional line segment with the three-dimensional line segment. After processing each video image frame with missing two-dimensional line segments in sequence, the line flow of the missing two-dimensional line segment can be obtained.

[0070] For example, if the three-dimensional line segment is the entire outline of the display, the corresponding two-dimensional line segment may be the straight line segment corresponding to the left vertical edge of the display. If the left edge of the display is occluded in a certain video image frame, the straight line segment corresponding to the left edge can be obtained according to the complete outline of the display, or the corresponding occluded two-dimensional line segment can be determined by combining the line flow of multiple consecutive three-dimensional line segments.

[0071] Understandably, when the line flow of a two-dimensional line segment is missing, it can be supplemented by the line flow of a three-dimensional line segment to obtain a continuous line flow of two-dimensional line segments. This allows the line flow of a three-dimensional line segment to be supplemented by a continuous line flow of two-dimensional line segments when the line flow of a three-dimensional line segment is missing, ensuring the stability of the line flow of the three-dimensional line segment and providing a prerequisite for obtaining an accurate video shooting route.

[0072] In some embodiments of the present application, Figure 3 The diagram shown is a flowchart of a method for obtaining the line flow of a two-dimensional line segment according to an embodiment of this application. The step S101, "obtaining the line flow of the two-dimensional line segment in the video image frame corresponding to the video to be processed", can be achieved by the following steps S1011 to S1014. Each step is described below.

[0073] S1011. Determine whether a two-dimensional line segment is detected in the video image frame.

[0074] In some embodiments, the two-dimensional line segments in the initial video image frame (the first image frame corresponding to the video to be processed) can be preset to determine the shape, position and other information of the two-dimensional line segments. Then, the two-dimensional line segments are observed. As the image acquisition device moves, the two-dimensional line segments may be occluded and become undetectable. For undetectable two-dimensional line segments, prediction can be made based on the two-dimensional line segments in the previous and next frames, or the undetectable two-dimensional line segments can be supplemented based on the three-dimensional line segments.

[0075] In some embodiments, if a two-dimensional line segment is detected in a video image frame, step S1012 is performed; otherwise, steps S1013 to S1014 are performed.

[0076] S1012. Extract two-dimensional line segments from video image frames to obtain the line flow of the two-dimensional line segments.

[0077] In some embodiments, if two-dimensional line segments can be detected in each video image frame, the two-dimensional line segments in each video image frame can be directly extracted. For example, feature extraction algorithms such as principal component analysis and linear discriminant analysis can be used to extract the two-dimensional line segments, thereby obtaining the line flow corresponding to each two-dimensional line segment.

[0078] S1013. Determine the predicted two-dimensional line segments in the video image frame.

[0079] It should be noted that the prediction of two-dimensional line segments can be obtained by using a prediction method to predict undetected two-dimensional line segments. In some embodiments, the prediction of undetected two-dimensional line segments can be based on the line flow of the two-dimensional line segments themselves, or the prediction of undetected two-dimensional line segments can be based on the line flow of three-dimensional line segments.

[0080] S1014. The predicted two-dimensional line segment is determined as a two-dimensional line segment in the video image frame, and the line flow of the two-dimensional line segment is obtained.

[0081] In some embodiments, the predicted two-dimensional line segments can be directly used as two-dimensional line segments in video image frames. The predicted two-dimensional line segments and the directly detected two-dimensional line segments can be combined according to the corresponding order of the video image frames to obtain the line flow of the complete two-dimensional line segments.

[0082] In other embodiments, the predicted two-dimensional line segments can be further subjected to feature analysis. If the similarity between the predicted two-dimensional line segment and an adjacent directly detected two-dimensional line segment is high, the predicted two-dimensional line segment is then used as a two-dimensional line segment in the video image frame. For example, the feature value corresponding to the predicted two-dimensional line segment can be compared with the feature value corresponding to the two-dimensional line segment detected in the previous frame. If the absolute value of the difference between the two feature values ​​is less than or equal to a preset feature value threshold, then the predicted two-dimensional line segment can be determined as a two-dimensional line segment in the video image frame. Otherwise, it indicates that the prediction error is large, and the two-dimensional line segment needs to be re-predicted until the absolute value of the difference between the feature value corresponding to the predicted two-dimensional line segment and the feature value corresponding to the two-dimensional line segment detected in the previous or next frame is less than or equal to the preset feature value threshold, at which point the predicted two-dimensional line segment is used as a two-dimensional line segment in the video image frame.

[0083] In some embodiments of this application, determining the predicted two-dimensional line segment in the video image frame, i.e., step S1013, can be achieved by the following steps S301 to S302, and each step is described below.

[0084] S301. Perform content analysis and speech analysis on the video to be processed to obtain auxiliary prediction information.

[0085] In some embodiments, the video content and audio in the video to be processed can be analyzed to obtain auxiliary prediction information. This auxiliary prediction information can be information related to the shooting route or information related to reference objects. For example, the text, symbols, audio and other information in the video to be processed can be analyzed to determine the shooting angle and orientation of the image acquisition device or the position of the reference object. This information can then be used as auxiliary prediction information to predict two-dimensional line segments.

[0086] S302. Based on auxiliary prediction information, determine the predicted two-dimensional line segments in the video image frame.

[0087] In some embodiments, the obtained auxiliary prediction information can be used to predict two-dimensional line segments. For example, by determining the shooting direction and shooting angle of the image acquisition device through the auxiliary prediction information, the position and shape of the two-dimensional line segments in the current video image frame can be predicted based on the two-dimensional line segments in the previous or next video image frame and the reference objects of the two-dimensional line segments.

[0088] In some embodiments of this application, after performing the "obtaining the line flow of the two-dimensional line segment in the video image frame corresponding to the video to be processed" step S101, the following step S401 can also be performed, which will be described below.

[0089] S401. Remove two-dimensional line segments from the line flow whose timing and position information do not meet the preset pose change trend.

[0090] In some embodiments, a preset pose change trend indicates that two temporally adjacent two-dimensional line segments move in the same direction. For example, the directions of two-dimensional line segments in two adjacent video image frames have the same change trend. The temporal information and position information of the two-dimensional line segments have a one-to-one correspondence. Based on the temporal information of the two-dimensional line segments and the change trends of the two-dimensional line segments in the preceding and following video image frames, the position information of the two-dimensional line segments in the current video image frame can be determined. This position information can be the location of the two-dimensional line segment and its direction of movement.

[0091] For example, if the two-dimensional line segment in the third video image frame turns left, and the detected (or predicted) line segment in the fourth image frame turns right, but in reality, the line segments in the obtained video image frames do not turn left and right at the same time, then the two-dimensional line segment extracted in the fourth image frame can be deleted, and the two-dimensional line segment in the video image frame can be detected or predicted again until the temporal information and position information corresponding to the obtained two-dimensional line segment meet the predicted pose transformation trend.

[0092] Understandably, by removing two-dimensional line segments whose temporal and positional information does not meet the preset pose change trend from the line flow of the two-dimensional line segments, the temporal and positional information of the obtained two-dimensional line segments are correctly matched, ensuring the accuracy of the obtained line flow of the two-dimensional line segments. When supplementing the line flow of the three-dimensional line segments based on the line flow of the two-dimensional line segments in the subsequent process, a line flow of the three-dimensional line segments with higher accuracy can be obtained, providing a foundation for accurately determining the route information of the video to be processed.

[0093] In some embodiments of this application, the route information corresponding to the video to be processed is determined based on the line flow of the target three-dimensional line segment. That is, step S104 can be implemented by the following steps S1041 to S1042. Each step is described below.

[0094] S1041. Obtain the size and position change information of the reference object corresponding to the line flow of the target three-dimensional line segment.

[0095] It should be noted that the line flow of the target 3D line segment refers to the line flow of the complete 3D line segment corresponding to the video to be processed. The reference object corresponding to the line flow of the target 3D line segment can be any object in the video image frame, such as a monitor or a table. In practice, because the image acquisition device may move during the recording of the video to be processed, the reference object may be obscured. In this case, the 2D and 3D line segments corresponding to other reference objects can be observed, thus making it possible for the video to be processed to correspond to multiple reference objects.

[0096] In some embodiments, depending on the reference object, the video to be processed may be divided into multiple video segments, each video segment corresponding to its own reference object. The size transformation information of each reference object may be the change of each reference object from small to large or from large to small, and the position change information may be the change of the position of each reference object in the video image frame.

[0097] S1042. Determine route information based on size change information and location change information.

[0098] In some embodiments, the shooting path information of the image acquisition device can be determined based on the size transformation information and position change information of the reference object corresponding to the line flow of the target three-dimensional line segment. For example, if it is determined that the reference object in the video image frame changes from small to large (gradually increases in size), and from all video image frames, the reference object gradually moves to the left from the middle position of the video image frame, then the shooting path of the image acquisition device is likely a forward and rightward corresponding movement trajectory.

[0099] In this embodiment, firstly, the line flow of two-dimensional line segments and the line flow of three-dimensional line segments in the video image frames corresponding to the video to be processed are obtained; then, the missing parts of the line flow of the three-dimensional line segments are supplemented based on the line flow of the two-dimensional line segments to obtain the line flow of the target three-dimensional line segments; finally, the route information corresponding to the video to be processed is determined based on the line flow of the target three-dimensional line segments. Thus, by supplementing the line flow of the three-dimensional line segments with the line flow of the two-dimensional line segments, the complete line flow of the target three-dimensional line segments can be obtained. This allows the shooting trajectory of the video to be processed to be accurately determined even when there is occlusion, low texture, or repeated texture in the shooting scene, based on the line flow of the target three-dimensional line segments.

[0100] The implementation process of the application embodiments in practical application scenarios is described below.

[0101] In some embodiments, such as Figure 4 The diagram shown is a flowchart of a video-based route extraction method provided in an embodiment of this application. This method can be implemented through the following steps S501 to S502, and each step is described below.

[0102] S501. Perform frame-slicing processing on the video to obtain a time-series image (equivalent to "video image frame" in other embodiments).

[0103] In some embodiments, frame-slicing processing may involve extracting image frames from a video at a preset sampling frequency to obtain multiple image frames corresponding to the video, i.e., time-series images, where the time-series images may contain image frames containing time information. For example, Figure 5 The time sequence images t0 to t shown on the left are the time sequence images after the video has been processed by frame segmentation.

[0104] S502. Extract line segments from the time series image to obtain the line flow of two-dimensional line segments and the line flow of three-dimensional line segments.

[0105] In some embodiments, line segments in a time-series image can be extracted using feature extraction. In the initial time-series image, a two-dimensional line segment can be observed. The corresponding three-dimensional line segment is determined using a reference object corresponding to this observed two-dimensional line segment. Subsequently, the corresponding two-dimensional and three-dimensional line segments can be extracted from each subsequent time-series image, thereby obtaining the line flow of the two-dimensional line segment and the line flow of the three-dimensional line segment (equivalent to "obtaining the line flow of the two-dimensional line segment in the corresponding video image frame of the video to be processed, and the line flow of the three-dimensional line segment in the video image frame" in other embodiments). For example, Figure 5 The upper right side of the image shows the line flow of two-dimensional line segments in each time series image, where the observed two-dimensional line segments are the straight line segments corresponding to the right edge of the display screen.

[0106] In some embodiments, during the extraction of two-dimensional and three-dimensional line segments, due to problems such as occlusion, low texture, and repetitive texture in the scene, it is impossible to completely extract the two-dimensional and three-dimensional line segments. The two-dimensional and three-dimensional line segments that cannot be extracted can be obtained through prediction.

[0107] In some implementations, the prediction of a 2D line segment can be obtained by adding the 2D line segment corresponding to the previous frame's temporal image and the 2D line segment corresponding to the current frame's temporal image. The prediction of a 3D line segment can be obtained from the endpoints and pose information of the 3D line segment. If there is no reliable prior for the current frame's temporal image, the 2D line segment corresponding to the previous frame's temporal image is used as the predicted line segment. Using the predicted line segment as a reference point, the KLT algorithm is used to find the corresponding position in the current frame's temporal image. Simultaneously, a retention period is set for the observed line segment. If no matching line segment is detected, the predicted line segment replaces the detected line segment (observed line segment). If no line segment is found after the retention period ends, the current line segment tracking ends.

[0108] In other embodiments, the title or audio information in the video can also be used as parameter information to assist in the prediction of two-dimensional line segments (equivalent to "determining the predicted two-dimensional line segments in the video image frame based on the auxiliary prediction information" in other embodiments). By obtaining information such as text and audio in the video, it is possible to predict two-dimensional line segments that cannot be detected.

[0109] In some embodiments, for the predicted or extracted two-dimensional line segments, two-dimensional line segments that do not conform to the motion trend can be eliminated based on temporal information (equivalent to "removing two-dimensional line segments whose temporal and positional information in the line flow does not meet the preset pose change trend" in other embodiments), thereby obtaining the accurate line flow of the two-dimensional line segments. For example, if the current time series image is the third time series image, and the predicted line flow is the two-dimensional line segment in the fourth time series image, after the line segment in the third time series image turns left, the predicted two-dimensional line segment in the fourth time series image turns right. The spatiotemporal constraints will correct the fourth time series image, preventing the extracted line segment from moving left and right alternately.

[0110] Understandably, updating the line flow through prediction and using video information to assist in prediction solves the problem of failing to extract observed line segments when there are occlusions, low textures, and repetitive textures in the scene, under the constraint of coherence. Furthermore, eliminating false positive observations under the constraint of temporal information increases the accuracy of route information extraction.

[0111] S503. Determine the size change information of the image based on the line flow of the two-dimensional line segment, and determine the pose information of the image acquisition device based on the size change information.

[0112] In some embodiments, the size change information of the image is the scale change of the image. Pose estimation can be performed from video images based on the image scale from small to large, where the image scale from small to large refers to the change of two-dimensional line segments from small to large (or from short to long). Based on the change in image scale, the pose information of the image acquisition device can be determined, which may be information such as the shooting orientation and shooting angle of the image acquisition device.

[0113] S504. Based on the line flow of two-dimensional line segments, predict and construct the missing part of the line flow of three-dimensional line segments (equivalent to "supplementing the missing part of the line flow of three-dimensional line segments based on the line flow of two-dimensional line segments" in other embodiments).

[0114] In some embodiments, there is a spatial-temporal correspondence between two-dimensional line segments and three-dimensional line segments, i.e., a spatial-temporal coherence relationship. Based on the line flow of a two-dimensional line segment, a reference object corresponding to the two-dimensional line segment can be determined. Based on the size change information of the reference object, the line flow of the corresponding three-dimensional line segment can be determined, thereby realizing the prediction of the missing part of the line flow of the three-dimensional line segment.

[0115] S505. Based on the temporal-spatial coherence relationship between the line flow of two-dimensional line segments and the line flow of three-dimensional line segments, as well as the pose information of the image acquisition device, a route extraction model is established, and route information is obtained based on the route extraction model.

[0116] In some embodiments, the temporal-spatial coherence relationship between the line flow of two-dimensional line segments and the line flow of three-dimensional line segments may include the correspondence between the temporal information of two-dimensional line segments and three-dimensional line segments in different time-series images, as well as the correspondence between the temporal information of two-dimensional line segments and three-dimensional line segments. By combining the pose information of the image acquisition device and adopting a coarse-to-fine strategy, composite modeling of lines (two-dimensional line segments) and planes (three-dimensional line segments) (e.g., modeling of reference objects) can be performed. Based on the size change information and position change information of the reference objects, the extraction of the video shooting route can be realized.

[0117] It is understood that in the embodiments of this application, pose estimation is performed based on the image scale, and a coherent relationship with temporal information of line segment motion is constructed through the relationship mapping from two-dimensional line segments to three-dimensional line segments, thereby realizing route extraction in the video. In addition, by constructing the spatial and temporal coherent relationship of two-dimensional and three-dimensional line segment motion, the feature (route) extraction error caused by different lighting in the video scene is reduced.

[0118] In some embodiments, the video-based route extraction method provided in this application can be used... Figure 6 The process shown below will be implemented. Figure 6 The following example illustrates the video-based route extraction process provided in this application.

[0119] Step S1: Slice the video to obtain a time sequence image. At the same time, the video title content and audio can be parsed.

[0120] In some embodiments, slicing the video can involve extracting the corresponding image frames to obtain multiple time-series images, facilitating subsequent video analysis and processing. Simultaneously, to achieve accurate prediction of two-dimensional line segments when they cannot be extracted, the title content and audio in the video can be parsed to obtain auxiliary prediction information, which can then be used to predict the two-dimensional line segments.

[0121] Step S2: Extract two-dimensional line segments based on time-series images, and predict and update the two-dimensional line segments.

[0122] In some embodiments, after extracting the two-dimensional line segment, if it is determined that there is a missing part in the line flow of the two-dimensional line segment, the two-dimensional line segment can be predicted. Based on the two-dimensional line segments in the time sequence images of the preceding and following frames, the predicted two-dimensional line segment is obtained, and the predicted two-dimensional line segment is used as the two-dimensional line segment in the current frame time sequence image to realize the update of the two-dimensional line segment. For example, as shown... Figure 5 As shown, if a portion of the two-dimensional line segment in the t-th time series image is occluded, then the line segment in the (t+1)-th time series image is used to update the two-dimensional line segment in the t-th time series image.

[0123] Step S3: Perform pose estimation based on image scale.

[0124] In some embodiments, the image scale can be the scale change of the reference object corresponding to the line flow of the two-dimensional line segment. Based on the size change of the reference object, the posture information of the image acquisition device can be determined, that is, the shooting angle and shooting direction of the image acquisition device.

[0125] Step S4: Predict the line flow of the constructed 3D line segment.

[0126] In some embodiments, the line flow of a three-dimensional line segment is directly detected. For the line flow of a missing three-dimensional line segment, it can be predicted based on the detected three-dimensional line segment or based on the two-dimensional line segment, thereby obtaining the line flow of the complete three-dimensional line segment.

[0127] Step S5: Construct the time-space coherence relationship of line segment motion.

[0128] In some embodiments, by predicting the missing three-dimensional line segments or supplementing the missing parts of the three-dimensional line segments based on two-dimensional line segments, the line flow of the complete three-dimensional line segments corresponding to the video can be obtained. Based on the line flow of the complete three-dimensional line segments, the position change information and size change information of the corresponding reference object can be determined.

[0129] Step S6: Extract route information.

[0130] In some embodiments, the positional and dimensional changes of reference objects or landmarks in the video can be determined based on the temporal-spatial coherence of line segment motion. Then, based on this positional and dimensional change information, the shooting route of the image acquisition device can be determined, thus extracting the video route information. For example, the extracted route information may be... Figure 5 The shooting trajectory shown in the lower right corner indicates that the image acquisition device first moves forward to take a picture, and then moves to the left to take a picture.

[0131] Understandably, in this embodiment, firstly, the line flow of two-dimensional line segments and the line flow of three-dimensional line segments in the corresponding video image frames of the video to be processed are obtained; then, the missing parts of the line flow of the three-dimensional line segments are supplemented based on the line flow of the two-dimensional line segments to obtain the line flow of the target three-dimensional line segments; finally, the route information corresponding to the video to be processed is determined based on the line flow of the target three-dimensional line segments. Thus, by supplementing the line flow of the three-dimensional line segments with the line flow of the two-dimensional line segments, the complete line flow of the target three-dimensional line segments can be obtained, enabling the accurate determination of the shooting trajectory of the video to be processed based on the line flow of the target three-dimensional line segments, even when there is occlusion, low texture, or repetitive texture in the shooting scene.

[0132] This application also provides a device for determining video route information. Figure 7 This is a schematic diagram of the composition structure of a video route information determination device provided in an embodiment of this application, as shown below. Figure 7 As shown, the video route information determining device 600 includes:

[0133] The acquisition module 601 is used to acquire the line flow of two-dimensional line segments in the video image frame corresponding to the video to be processed, and the line flow of three-dimensional line segments in the video image frame, wherein the line flow is a time series of the line segments;

[0134] The first supplementing module 602 is used to supplement the missing part of the line flow of the three-dimensional line segment based on the line flow of the two-dimensional line segment, so as to obtain the line flow of the target three-dimensional line segment.

[0135] The determination module 603 is used to determine the route information corresponding to the video to be processed based on the line flow of the target three-dimensional line segment. The route information is used to represent the shooting trajectory of the image acquisition device that captures the video to be processed.

[0136] In some embodiments, the first supplementary module 602 may include:

[0137] The first determining submodule is used to determine the size change information of the two-dimensional line segment based on the line flow of the two-dimensional line segment;

[0138] The second determining submodule is used to determine the pose information of the image acquisition device that captures the video to be processed based on the size change information;

[0139] The third determining submodule is used to determine the line flow of the first three-dimensional line segment based on the pose information, wherein the line flow of the first three-dimensional line segment is the line flow of the missing three-dimensional line segment in the line flow of the three-dimensional line segment.

[0140] The first acquisition submodule is used to obtain the line flow of the target three-dimensional line segment based on the line flow of the first three-dimensional line segment.

[0141] In some embodiments, the first supplementary module 602 may further include:

[0142] The fourth determining submodule is used to determine the reference object of the line flow including the two-dimensional line segment in the video image frame;

[0143] The fifth determining submodule is used to determine the line flow of the second three-dimensional line segment in the reference object, which includes the line flow of the two-dimensional line segment, and the line flow of the second three-dimensional line segment is the line flow of the missing three-dimensional line segment in the line flow of the three-dimensional line segment;

[0144] The second acquisition submodule is used to obtain the line flow of the target three-dimensional line segment based on the line flow of the second three-dimensional line segment.

[0145] In some embodiments, the video route information determining device 600 further includes:

[0146] The second supplementary module is used to supplement the missing parts of the line flow of the two-dimensional line segment based on the line flow of the three-dimensional line segment.

[0147] In some embodiments, the acquisition module 601 includes:

[0148] The sixth determining submodule is used to determine the predicted two-dimensional line segment in the video image frame if the two-dimensional line segment is not detected in the video image frame; determine the predicted two-dimensional line segment as the two-dimensional line segment in the video image frame, and obtain the line flow of the two-dimensional line segment;

[0149] The line segment extraction submodule is used to extract the two-dimensional line segment from the video image frame if the two-dimensional line segment is detected in the video image frame, and obtain the line flow of the two-dimensional line segment.

[0150] In some embodiments, the fifth determining submodule includes:

[0151] The information parsing unit is used to perform content parsing and speech parsing on the video to be processed to obtain auxiliary prediction information;

[0152] The first determining unit is used to determine the predicted two-dimensional line segment in the video image frame based on the auxiliary prediction information.

[0153] In some embodiments, the determining module 603 includes:

[0154] The second acquisition submodule is used to acquire the size change information and position change information of the reference object corresponding to the line flow of the target three-dimensional line segment;

[0155] The seventh determination submodule is used to determine the route information based on the size change information and the position change information.

[0156] In some embodiments, the video route information determining device 600 further includes:

[0157] The line segment removal unit is used to remove two-dimensional line segments whose timing and position information in the line flow do not meet the preset pose change trend, wherein the preset pose change trend indicates that the motion directions of two temporally adjacent two-dimensional line segments are consistent.

[0158] It should be noted that the description of the video route information determination device in this application embodiment is similar to the description of the method embodiment above, and has similar beneficial effects as the method embodiment, therefore it will not be repeated. For technical details not disclosed in this device embodiment, please refer to the description of the method embodiment of this application for understanding.

[0159] It should be noted that, in the embodiments of this application, if the above-described method for determining video route information is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to related solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0160] Accordingly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method for determining video route information provided in the above embodiments.

[0161] This application also provides a device for determining video route information. Figure 8This is a schematic diagram of the composition structure of a video route information determination device provided in an embodiment of this application, as shown below. Figure 8 As shown, the video route information determination device 700 includes: a memory 701, a processor 702, a communication interface 703, and a communication bus 704. The memory 701 stores executable instructions for determining video route information; the processor 702 executes the executable instructions stored in the memory to implement the video route information determination method provided in the above embodiment.

[0162] The description of the above embodiments of the video route information determination device and storage medium is similar to the description of the above method embodiments and has similar beneficial effects. For technical details not disclosed in the embodiments of the video route information determination device and storage medium of this application, please refer to the description of the method embodiments of this application for understanding.

[0163] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising at least one…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0164] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0165] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0166] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0167] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0168] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a product to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0169] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining video route information, the method comprising: Obtain the line flow of two-dimensional line segments in the video image frame corresponding to the video to be processed, and the line flow of three-dimensional line segments in the video image frame, wherein the line flow is a time series of the line segments; The missing parts of the line flow of the three-dimensional line segment are supplemented based on the line flow of the two-dimensional line segment to obtain the line flow of the target three-dimensional line segment. Based on the line flow of the target three-dimensional line segment, the route information corresponding to the video to be processed is determined, and the route information is used to represent the shooting trajectory of the image acquisition device that captured the video to be processed.

2. The method according to claim 1, wherein supplementing the missing portion of the three-dimensional line segment based on the line flow of the two-dimensional line segment to obtain the line flow of the target three-dimensional line segment includes: Based on the line flow of the two-dimensional line segment, determine the size change information of the two-dimensional line segment; Based on the size change information, the pose information of the image acquisition device that captured the video to be processed is determined; Based on the pose information, the line flow of the first three-dimensional line segment is determined, and the line flow of the first three-dimensional line segment is the line flow of the missing three-dimensional line segment in the line flow of the three-dimensional line segment. The line flow of the target three-dimensional line segment is obtained based on the line flow of the first three-dimensional line segment.

3. The method according to claim 1, wherein supplementing the missing portion of the three-dimensional line segment based on the line flow of the two-dimensional line segment to obtain the line flow of the target three-dimensional line segment includes: A reference object is determined for the line flow that includes the two-dimensional line segment in the video image frame; The line flow of the second three-dimensional line segment, which includes the line flow of the two-dimensional line segment in the reference object, is determined. The line flow of the second three-dimensional line segment is the line flow of the three-dimensional line segment that is missing in the line flow of the three-dimensional line segment. The line flow of the target three-dimensional line segment is obtained based on the line flow of the second three-dimensional line segment.

4. The method according to claim 1, further comprising: The missing parts of the line flow of the two-dimensional line segment are supplemented based on the line flow of the three-dimensional line segment.

5. The method according to claim 1, wherein obtaining the line flow of the two-dimensional line segment in the video image frame corresponding to the video to be processed includes at least one of the following: If the two-dimensional line segment is not detected in the video image frame, the predicted two-dimensional line segment in the video image frame is determined; the predicted two-dimensional line segment is determined as the two-dimensional line segment in the video image frame, and the line flow of the two-dimensional line segment is obtained. If the two-dimensional line segment is detected in the video image frame, the two-dimensional line segment is extracted from the video image frame to obtain the line flow of the two-dimensional line segment.

6. The method according to claim 5, wherein, Determining the predicted two-dimensional line segment in the video image frame includes: The video to be processed is subjected to content analysis and speech analysis to obtain auxiliary prediction information; Based on the auxiliary prediction information, the predicted two-dimensional line segments in the video image frame are determined.

7. The method according to claim 1, wherein determining the route information corresponding to the video to be processed based on the line flow of the target three-dimensional line segment includes: Obtain the size and position change information of the reference object corresponding to the line flow of the target three-dimensional line segment; The route information is determined based on the size change information and the position change information.

8. The method according to any one of claims 1 to 7, further comprising: Remove two-dimensional line segments from the line flow whose timing and position information do not satisfy a preset pose change trend. The preset pose change trend indicates that the motion directions of two temporally adjacent two-dimensional line segments are consistent.

9. A device for determining video route information, comprising: The acquisition module is used to acquire the line flow of two-dimensional line segments in the video image frame corresponding to the video to be processed, and the line flow of three-dimensional line segments in the video image frame, wherein the line flow is a time series of the line segments; The first supplementary module is used to supplement the missing part of the line flow of the three-dimensional line segment based on the line flow of the two-dimensional line segment, so as to obtain the line flow of the target three-dimensional line segment. The determination module is used to determine the route information corresponding to the video to be processed based on the line flow of the target three-dimensional line segment. The route information is used to represent the shooting trajectory of the image acquisition device that captured the video to be processed.

10. A device for determining video route information, comprising: Memory, used to store executable instructions for determining video route information; A processor, when executing instructions for determining executable video route information stored in the memory, implements the method according to any one of claims 1 to 8.

Images