Method for making virtual riding route based on live-action route

Abstract

The invention provides a method for making a virtual riding route based on a live-action route, and the method comprises the following steps: S1, video collection: collecting video materials through shooting equipment, recording accurate data, and generating a riding file; s2, video analysis and processing: importing and analyzing video materials frame by frame, and performing segmentation processing on the video according to route characteristics; s3, model construction and calibration: performing point location marking according to the data obtained in the step S2, drawing a 3D model in combination with the marked point location data, and performing data matching and calibration; s4, video road book generation: comparing data in the data file and the. Fit file to generate a detailed video road book; s5, user access and equipment interconnection: establishing a user database, connecting the riding equipment with App software, and establishing a data interaction protocol; s6, model dynamic adjustment and riding feedback: performing model action fine adjustment according to data in the video road book; and giving corresponding feedback to the user according to the operation of the user and the riding equipment data.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method for making a virtual riding route based on a real-scene route. Background Art

[0002] Cycling, as a healthy and environmentally friendly way of travel and exercise, has gradually become popular around the world in recent years. With the acceleration of urbanization and the improvement of people's health awareness, more and more people are beginning to choose cycling as a way of daily commuting or leisure exercise. Cycling not only helps reduce traffic congestion and air pollution, but also improves physical fitness and promotes physical and mental health.

[0003] With the rapid development of science and technology, especially the popularization of mobile Internet, big data, cloud computing and other technologies, cycling software has emerged. These software use advanced technology to provide cycling enthusiasts with a more convenient and intelligent cycling experience, so that they can experience the training effect of outdoor cycling indoors, without worrying about the weather and environment affecting their cycling plans.

[0004] However, most traditional cycling software can only provide simple maps or virtual route displays, and cannot provide users with accurate cycling routes and scenery along the way, lacking a sense of immersion. Moreover, traditional cycling software can generally only record basic cycling data, such as speed, distance, time, etc., and it is difficult for users to understand their cycling status. The connection and interaction between traditional cycling software and cycling equipment is relatively limited, and can only perform some simple data transmission, which cannot bring users a better experience. Summary of the invention

[0005] The purpose of the present invention is to solve the above problems and to propose a method for making a virtual cycling route based on a real-life route.

[0006] In order to achieve the above purpose, the following technical solutions are adopted: The method for making a virtual cycling route based on a real-life route comprises the following steps: S1: Video capture: Use high-precision professional shooting equipment to capture stable and smooth video footage, and use speedometers and GPS devices to record accurate data and generate cycling .fit files; The data include speed, distance and time; S2: Video analysis and processing: Import the video into PFTrack software, analyze the video material frame by frame, use the object tracking and geometric shape tracking functions to extract information, accurately locate and track key feature points, and segment the video according to the route characteristics; The information is the motion information and position information of the camera; The key feature points include road signs, building edges, natural scenery and special landmarks; The route features include straight sections, curves, and ramps; S3: Model construction and calibration: Based on the data obtained in step S2, key route points in each frame are marked, and a 3D model is drawn in combination with the marked point data; the 3D model is exported for data matching and calibration; The key route points are feature points in the video image; The point data is the displacement of the feature point in each frame change of the video; S4: Video road book generation: Based on the data file of each frame and the data of each second in the .fit file, the video content is accurately matched with the riding data to generate a detailed video road book; S5: User access and device interconnection: Based on the information input by the user, a user database is established, and when the user logs in, the user data is retrieved from the database and the software is projected on the display screen; the riding device is connected to the App software, a data interaction protocol is established, and data is transmitted between the software and the device; S6: Dynamic model adjustment and riding feedback: Import the calibrated 3D model and fine-tune the model's movements based on the data in the video road book; after the App software and the riding device are mutually established, give the user corresponding feedback based on the user's operation and the riding device data; The data in the video road book are key turning frame points; The cycling equipment data include cycling speed, cycling distance, cycling power, cadence and heart rate.

[0007] Preferably, the system arranges the speed and position data points in time series, converts the data into binary data in a composite fit file format through a specific encoding method, and generates a riding .fit file by verifying and compressing the binary data.

[0008] Preferably, in step S2, the segmentation process is to mark key frames and key sections according to the characteristics of the route in the video, and segment the video according to the key frames and key sections.

[0009] Preferably, in step S3, the 3D model is drawn based on the key frame data and key feature points obtained in step 2 and using an object modeling technology based on scene analysis, wherein the object modeling technology based on scene analysis includes the following sub-steps: S31: Feature point recognition and tracking: Extract and recognize multi-scale features for each frame of the video, track the feature points, and obtain their motion information in the video; S32: Spatial position calculation and model construction: Calculate the spatial position of feature points, then form point cloud data, pre-process the point cloud data and reconstruct the surface to generate a 3D model; S33: Texture mapping: To make the 3D model more realistic, the texture information in the video is mapped to the surface of the 3D model.

[0010] Preferably, in sub-step S31, feature points in the video are extracted and identified by using SIFT algorithm and SURF algorithm, and the displacement of the feature points is tracked by calculating the movement speed of pixels between adjacent frames by using optical flow algorithm.

[0011] Preferably, in sub-step S32, the 3D spatial position of the feature points is calculated using a multi-view diffusion model. As the 3D spatial position of the feature points is continuously calculated, a 3D point cloud data structure is gradually formed. Finally, based on the point cloud data, a continuous 3D model surface is generated using an instant consistent mesh reconstruction algorithm.

[0012] Preferably, in sub-step S33, a texture mapping algorithm is used to accurately map the color information in the texture image to the model surface.

[0013] Preferably, in step S3, the XML configuration file and the original data model are used for matching and calibration, and the original data model includes the video analysis data and the data in the cycling fit file.

[0014] During the calibration process, the landmark spacing and altitude data measured on site are used as key verification standards to verify the accuracy of the model's geographic information. At the same time, multi-dimensional verification is performed in combination with the time, location, and action sequence data in the video road book to ensure that the model is highly consistent with the actual riding scene in terms of time and space dimensions and dynamic characteristics. This allows for all-round calibration and optimization of the 3D model, and improves the accuracy and realism of virtual route production.

[0015] Among them, data from multiple sources such as XML configuration files, video analysis data and cycling fit files are matched and calibrated to ensure that data in different data formats and collection methods can be accurately integrated in the model, thereby improving the integrity and accuracy of the data.

[0016] Preferably, in step S4, in order to correspond the video content with the riding data in real time, the following sub-steps are included: S41: Extract frames from the video by timestamp using the video processing library; S42: Parsing timestamp data in .fit file; S43: Convert the timestamp of the video to obtain the real time; S44: making the time data in the .fit file correspond to the time in the video, and giving the position information in the .fit file to each frame in the video; S45: using a map service API interface to obtain map data and route information; S46: Find the corresponding geographic location and route information according to the video frame, and overlay the map, route and current location mark on the video frame.

[0017] The above method for generating a video road book ensures that each frame of video is closely synchronized with its corresponding location information by accurately matching the timestamp of the video frame with the .fit file. In addition, the data in the .fit file can be combined with the video frame and displayed to the user in an intuitive manner. The location information is presented in a visual manner, which improves the authenticity of the video road book and the user experience.

[0018] Preferably, in step S6, two-way data transmission is performed between the App software and the cycling device, the cycling device transmits the cycling device data via the Bluetooth protocol, and the App software sends instructions to the cycling device via the cycling motion protocol.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: the method adopts high-precision professional video shooting equipment to ensure the acquisition of clear video materials with high color reproduction, uses video analysis tools to analyze the collected videos frame by frame, and adopts object modeling technology based on scene analysis to mark the key route points in the video frames, and then draws the 3D route prototype in combination with the marked point data. After the 3D model is constructed, texture mapping is used to improve the rendering efficiency of the model while ensuring the realism of the visual effect.

[0020] This method mines these data through data analysis algorithms, and clarifies the data type and data format transmitted between the software and the riding device to ensure accurate analysis and effective use of the data. In addition, based on the user's riding data, the video content and riding data are matched in real time to generate a video road book.

[0021] The App and the cycling device use a two-way communication interface to achieve two-way data transmission. The software can send instructions to the cycling device, such as adjusting the handlebar position and pedal resistance of the cycling device according to the slope changes in the virtual scene. At the same time, the cycling device can also transmit real-time data to Server A. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Schematic diagram of a method for making a virtual cycling route based on a real-scene route according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of sub-steps of step S3 of the method for making a virtual cycling route based on a real-scene route in embodiment 1 of the present invention; Figure 3 This is a schematic diagram of sub-steps of step S4 of the method for making a virtual cycling route based on a real-view route in embodiment 1 of the present invention; DETAILED DESCRIPTION

[0023] Hereinafter, the method for making a virtual cycling route based on a real-scene route of the present invention will be described in detail with reference to the accompanying drawings.

[0024] like Figure 1 As shown, the method for making a virtual cycling route based on a real-life route includes the following steps: S1: Video capture: Use high-precision professional shooting equipment to capture stable and smooth video footage, and use speedometers and GPS devices to record accurate data and generate riding files; S2: Video analysis and processing: Import the video into PFTrack software, analyze the video material frame by frame, use the object tracking and geometric shape tracking functions to extract information, accurately locate and track key feature points, and segment the video according to the route characteristics; S3: Model construction and calibration: Based on the data obtained in step S2, key route points in each frame are marked, and a 3D model is drawn in combination with the marked point data; the 3D model is exported for data matching and calibration; S4: Video road book generation: Based on the data file of each frame and the data of each second in the .fit file, the video content is accurately matched with the riding data to generate a detailed video road book; S5: User access and device interconnection: Based on the information input by the user, a user database is established, and when the user logs in, the user data is retrieved from the database and the software is projected on the display screen; the riding device is connected to the App software, a data interaction protocol is established, and data is transmitted between the software and the device; S6: Dynamic model adjustment and riding feedback: Import the calibrated 3D model and fine-tune the model's movements based on the data in the video road book; after the App software and the riding device are mutually established, give the user corresponding feedback based on the user's operations and the riding device data.

[0025] Wherein, step S3 adopts the object modeling technology based on scene analysis, and includes the following sub-steps: S31: Feature point recognition and tracking: Extract and recognize multi-scale features for each frame of the video, track the feature points, and obtain their motion information in the video; S32: Spatial position calculation and model construction: Calculate the spatial position of feature points, then form point cloud data, pre-process the point cloud data and reconstruct the surface to generate a 3D model; S33: Texture mapping: To make the 3D model more realistic, the texture information in the video is mapped to the surface of the 3D model.

[0026] Step S4 includes the following sub-steps in order to match the video content with the riding data in real time: S41: Extract frames from the video by timestamp using the video processing library; S42: Parsing timestamp data in .fit file; S43: Convert the timestamp of the video to obtain the real time; S44: making the time data in the .fit file correspond to the time in the video, and giving the position information in the .fit file to each frame in the video; S45: using a map service API interface to obtain map data and route information; S46: Find the corresponding geographic location and route information according to the video frame, and overlay the map, route and current location mark on the video frame.

[0027] In this embodiment, Figure 1~Figure 3 As shown in the figure, during the video acquisition stage, the planned route is filmed with professional filming equipment, the speedometer accurately collects speed data every second, the GPS device records the location information in real time, and the entire data is collected and integrated by the system to generate a fit file containing rich geographic and speed information. The video material is then analyzed frame by frame through the PFTrack software, key feature points such as trees and rocks along the way are locked, and the displacement trajectory of the feature points is accurately tracked, and the route is marked in sections according to the complexity of the route.

[0028] Then, based on the obtained feature point data, it is matched with strong feature points extracted from the video frame, such as unique rock textures and tree shapes, to monitor their pixel displacement in real time. The 3D spatial coordinates of the feature points are calculated to generate a dense 3D point cloud. A continuous and smooth 3D model surface of the mountain terrain is constructed based on the 3D point cloud data. Then, by using the high-definition texture materials collected on site, the texture details such as rough rocks and dense vegetation are accurately mapped to the corresponding areas of the model.

[0029] After the model is built, the preliminary model is exported first, and then the XML configuration file is used to accurately match it with the original data recorded by the speedometer, GPS device and the riding data in the fit file. After matching and calibration, the 3D model is optimized to reduce the number of polygons in the model and optimize the material texture.

[0030] After that, the timestamp of the fit file is parsed and the OpenCV video processing library is used to extract video frames according to time. The corresponding riding position information is assigned to each frame through the time conversion and matching method, thereby generating a video road book that integrates video images and riding data.

[0031] When registering and logging in, users enter their own data and connect the cycling device to the App via Bluetooth. The calibrated 3D model is imported, and the turning and climbing movements are precisely adjusted according to the key frames in the video road book to simulate real cycling dynamics. At the same time, commands are issued through the cycling motion protocol, and the cycling platform increases resistance to simulate the real climbing force, and reduces resistance to simulate inertial gliding when going downhill.

[0032] The above description is only a preferred example of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have other optimization schemes and additional functions. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for making a virtual cycling route based on a real-life route, characterized in that: The following steps are involved: S1: Video capture: Use high-precision professional shooting equipment to capture stable and smooth video footage, and use speedometers and GPS devices to record accurate data and generate riding files; The data include speed, distance and time; S2: Video analysis and processing: Import the video into PFTrack software, analyze the video material frame by frame, use the object tracking and geometric shape tracking functions to extract information, accurately locate and track key feature points, and segment the video according to the route characteristics; The information is the motion information and position information of the camera; The key feature points include road signs, building edges, natural scenery and special landmarks; The route features include straight sections, curves, and ramps; S3: Model construction and calibration: Based on the data obtained in step S2, key route points in each frame are marked, and a 3D model is drawn in combination with the marked point data; the 3D model is exported for data matching and calibration; The key route points are feature points in the video image; The point data is the displacement of the feature point in each frame change of the video; S4: Video road book generation: Based on the data file of each frame and the data of each second in the .fit file, the video content is accurately matched with the riding data to generate a detailed video road book; S5: User access and device interconnection: Based on the information input by the user, a user database is established, and when the user logs in, the user data is retrieved from the database and the software is projected on the display screen; the riding device is connected to the App software, a data interaction protocol is established, and data is transmitted between the software and the device; S6: Dynamic model adjustment and riding feedback: Import the calibrated 3D model and fine-tune the model's movements based on the data in the video road book; after the App software and the riding device are mutually established, give the user corresponding feedback based on the user's operation and the riding device data; The data in the video road book are key turning frame points; The cycling equipment data include cycling speed, cycling distance, cycling power, cadence and heart rate.

2. The method for making a virtual cycling route based on a real-life route according to claim 1, characterized in that: In step S1, the system arranges the speed and position data points in time series, converts the data into binary data in a composite fit file format through a specific encoding method, and generates a cycling fit file by verifying and compressing the binary data.

3. The method for making a virtual cycling route based on a real-life route according to claim 1, characterized in that: In step S2, the segmentation process is to mark key frames and key sections according to the characteristics of the route in the video, and segment the video according to the key frames and key sections.

4. The method for making a virtual cycling route based on a real-life route according to claim 1, characterized in that: In step S3, the 3D model is drawn based on the key frame data and key feature points obtained in step 2, and adopts the object modeling technology based on scene analysis, and the object modeling technology based on scene analysis includes the following sub-steps: S31: Feature point recognition and tracking: Extract and recognize multi-scale features for each frame of the video, track the feature points, and obtain their motion information in the video; S32: Spatial position calculation and model construction: Calculate the spatial position of feature points, then form point cloud data, pre-process the point cloud data and reconstruct the surface to generate a 3D model; S33: Texture mapping: To make the 3D model more realistic, the texture information in the video is mapped to the surface of the 3D model.

5. The method for making a virtual cycling route based on a real-life route according to claim 4, characterized in that: In sub-step S31, feature points in the video are extracted and identified by using SIFT algorithm and SURF algorithm, and the displacement of the feature points is tracked by calculating the movement speed of pixels between adjacent frames by using optical flow algorithm.

6. The method for making a virtual cycling route based on a real-life route according to claim 4, characterized in that: In sub-step S32, the 3D spatial position of the feature points is calculated using the multi-view diffusion model. As the 3D spatial position of the feature points is continuously calculated, a 3D point cloud data structure is gradually formed. Finally, based on the point cloud data, a continuous 3D model surface is generated using an instant consistent mesh reconstruction algorithm.

7. The method for making a virtual cycling route based on a real-life route according to claim 4, characterized in that: In sub-step S33, a texture mapping algorithm is used to accurately map the color information in the texture image to the model surface.

8. The method for making a virtual cycling route based on a real-life route according to claim 4, characterized in that: In step S3, matching and calibration are performed using the XML configuration file and the original data model, wherein the original data model includes the video analysis data and the data in the cycling fit file.

9. The method for making a virtual cycling route based on a real-life route according to claim 1, characterized in that: In step S4, in order to correspond the video content with the riding data in real time, the following sub-steps are included: S41: Extract frames from the video by timestamp using the video processing library; S42: Parsing timestamp data in .fit file; S43: Convert the timestamp of the video to obtain the real time; S44: making the time data in the .fit file correspond to the time in the video, and giving the position information in the .fit file to each frame in the video; S45: using a map service API interface to obtain map data and route information; S46: Find the corresponding geographic location and route information according to the video frame, and overlay the map, route and current location mark on the video frame.

10. The method for making a virtual cycling route based on a real-life route according to claim 1, characterized in that: In step S6, there is two-way data transmission between the App software and the cycling device. The cycling device transmits the cycling device data via the Bluetooth protocol, and the App software sends instructions to the cycling device via the cycling motion protocol.

Images