A method for dynamic real-time perspective fusion of virtual characters and real scenes

Abstract

The present invention relates to the field of virtual scene development technology, and provides a method for dynamic real-time perspective fusion of virtual characters and real scenes, including the following steps: S1: real scene information collection; S2: processing the real scene information collected in step S1, and building a virtual scene in a 3D engine based on this; S3: building a virtual character and a virtual camera in the virtual scene built in step S2; S4: importing the information flow of user operation equipment, controlling the position of the virtual character and the virtual camera in the 3D engine according to the information flow of the user operation equipment, controlling the playback of the sequence frame of the real scene picture according to the position of the virtual character, and the virtual camera simultaneously rendering the virtual character and the real scene picture to achieve real-time fusion. The present invention integrates the virtual character with the real scene in dynamic real-time perspective, greatly reducing the cost of scene model production, and some small and medium-sized enterprises can also use this method to develop virtual reality riding games. Expansion and updating based on this method will greatly enhance the user experience of the virtual reality industry and reduce production costs.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual scene development, and in particular to a method for dynamic real-time perspective fusion of a virtual character and a real scene. Background Art

[0002] With the continuous development of society, people pay more and more attention to their physical health. People usually improve their physical fitness through fitness. The popularity of the fitness industry has also driven the rapid development of the fitness equipment industry. Most young people now choose to relax at home instead of going out to play, and the number of people riding out has become very small, so many people have no way to experience the feeling of going out for a ride. Therefore, among the many fitness projects, the intelligent fitness of virtual scenes is deeply loved by the majority of cycling and fitness enthusiasts.

[0003] The existing intelligent fitness of virtual scenes collects pictures of real roads and other information, and hands it over to 3D modelers. The 3D modelers use realistic 3D models to restore the road information through the collected geographic location information and real road scene pictures, and then hand it over to 3D engines such as Unity for further game development.

[0004] This method requires the construction of a large number of terrain models. These complex terrains often consume a lot of manpower and financial resources to produce. At the same time, the development cycle of such games is relatively long, which is not conducive to players experiencing a large number of geographical landscapes. At the same time, such development investment is too large, and it takes a lot of manpower and material resources to restore the real scene, which requires a large number of production staff and a very long development cycle. This makes it impossible for many small and medium-sized game development companies to develop such projects. At the same time, for users, they need to download a large amount of model data to experience the fun of riding in a nearly real scene, which affects the user experience. Summary of the invention

[0005] In order to solve the problems existing in the background technology, the present invention provides a method for dynamic real-time perspective fusion of a virtual character and a real scene, a method for dynamic real-time perspective fusion of a virtual character and a real scene, which comprises the following steps:

[0006] S1: Real scene information collection;

[0007] S2: Processing the real scene information collected in step S1, and building a virtual scene in a 3D engine based on it;

[0008] S3: construct a virtual character and a virtual camera in the virtual scene constructed in step S2;

[0009] S4: Import the information flow of the user's operating equipment, control the position of the virtual character and the virtual camera in the 3D engine according to the information flow of the user's operating equipment, control the playback of the sequence frame of the real scene according to the position of the virtual character, and the virtual camera simultaneously renders the virtual character and the real scene to achieve real-time fusion.

[0010] In a preferred solution, step S1 includes collecting video information and geographic location parameters of the real scene.

[0011] In a preferred solution, the process of collecting video information of real scenes includes using a vehicle-mounted camera mode, fixing the camera on a car that is traveling at a constant speed, and simulating a riding perspective to collect video of the car within a certain period of time.

[0012] Furthermore, the collection of geographic location parameters of the real scene includes the following contents:

[0013] A. Time: including Beijing time when each piece of data was collected;

[0014] B. Location data: including the current location longitude and latitude;

[0015] C. Altitude data: including the altitude of the current location;

[0016] D. Road information: including the slope, inclination, and road width of the current location;

[0017] E. Weather data: including the temperature, wind direction, wind speed, and weather conditions in the current area;

[0018] F. Mobile data: including the moving speed of the current acquisition device.

[0019] Furthermore, the specific process includes: obtaining the longitude and latitude position information of the shooting point through GPS; obtaining the altitude information of the shooting point through an altitude detector; measuring the altitude through a high-precision handheld GPS collector; measuring the current slope through a slope meter; locating the position of the vehicle in the real space through the slope meter; while collecting the geographic location information, it is necessary to record the time of collection of each piece of location information, and then package the geographic location data that matches the time information together; when collecting information, the spatial coordinate information reading interval is set to be constant, and can be matched with the frame rate during video shooting, so that the actual geographic location information that should correspond to every few frames of the image can be obtained through post-processing.

[0020] Further, the specific process in step S2 includes:

[0021] P1: Processing geographic location data, converting the longitude and latitude coordinate data and altitude data collected in the real world into the corresponding x, y, and z axis coordinate data in the virtual world;

[0022] P2: Construct a virtual road. Generate a set of points through the data of P1 coordinate points, connect these points to construct a road curve in the virtual world; subdivide the points between two points separated by a collection time of Δt into Δt×m points, and smooth the curve; after constructing and processing the virtual road curve, build a road model with the same width as the real road based on this curve and the road width information in the real world, and fit and display it in the 3D engine;

[0023] P3: Split sequence frames, process the collected video information, split it into sequence frames and assign the shooting time information of each frame for subsequent processing;

[0024] P4: Match sequence frames to build a model and organize the collected information:

[0025] Geographic location information collection interval: Δt;

[0026] Video capture frame rate: m frames / second;

[0027] Unified start collection time: t0;

[0028] Then time t0 corresponds to the 0th frame image, and the geographical location information collected for the nth time corresponds to the n×Δt×mth frame image; and so on, the images corresponding to all the location information are obtained.

[0029] Further, the specific process of step S3 includes:

[0030] Q1: Make a virtual character model, and import the virtual character model and the model and curves constructed in step P4 into the 3D engine;

[0031] Q2: Construct a virtual camera in the 3D engine, where the virtual camera has the same focal length and depth of field parameters as the camera used to capture the image;

[0032] Q3: Bind the imported virtual character model and virtual camera to the road curve;

[0033] Q4: Match sequence frames and match the points on the road curve imported into the 3D engine to their corresponding images.

[0034] Furthermore, when the virtual character passes a point on the road, the image sequence frame is skipped, and the sequence frame playback continues until the virtual character moves to the next point, so that the movement of the virtual character on the virtual road is smooth.

[0035] Furthermore, the specific process of step S4 includes: importing the information flow of the user operating the equipment, controlling the position of the virtual character and the virtual camera in the 3D engine according to the information flow of the user operating the equipment, controlling the playback of the sequence frame of the real scene picture according to the position of the virtual character, and the virtual camera simultaneously rendering the virtual character and the real scene picture to achieve real-time fusion.

[0036] The input information of the user operating the equipment is instantaneous speed information, and the distance information is determined in the following way:

[0037] Record the instantaneous speed of the user operating the equipment: V 车 and the time interval between two data inputs: Δt; and the conversion relationship between the distance on the actual road and the distance on the virtual road, that is,

[0038] The specific distance that the user has traveled by operating the equipment at the current time is calculated by the following algorithm:

[0039]

[0040] The specific distance traveled on the virtual road is:

[0041] Assuming the total length of the virtual road is S, and the total number of frames of scene information of a road scene is N, the sequence frame number n played at the user's current position is calculated as follows:

[0042] The beneficial effects achieved by the present invention are:

[0043] The present invention provides a method for dynamic real-time perspective fusion of virtual characters and real scenes, and uses an innovative way to develop such an interactive interface, which dynamically integrates virtual characters and real scenes in real time, greatly reducing the cost of model production. Some small and medium-sized enterprises can also use this method to develop virtual reality somatosensory games, and expand and update based on this method, which will greatly enhance the user experience of the virtual reality industry.

[0044] In the specific scenario provided by the present invention, the filmed road video is combined with the collected road information, the filmed road video clip is fitted with the road route model constructed by collecting the road altitude and slope information, and the display interface is adjusted according to the user's control information of the smart bicycle, so that the user can experience the feeling of riding on a real road. At the same time, it supports the interaction of multiple players chasing each other in the same server, which will bring more players a fitness or game experience based on real scenery. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of the method for dynamic real-time perspective fusion of virtual characters and real scenes of the present invention;

[0046] Figure 2 is a schematic diagram of the real scene collected in step S2;

[0047] Figure 3 is a schematic diagram of constructing a virtual character and a virtual camera in step S3;

[0048] Figure 4 is a schematic diagram of controlling the playback of a sequence of frames of a real scene according to the position of a virtual character in step S4;

[0049] Figure 5 It is a schematic diagram of the real-time fusion of the virtual character and the real scene picture in step S4. DETAILED DESCRIPTION

[0050] To facilitate those skilled in the art to understand the present invention, specific implementations of the present invention are described below with reference to the accompanying drawings.

[0051] Reference Figure 1-4 , a method for dynamic real-time perspective fusion of a virtual character and a real scene, comprising the following steps:

[0052] S1: Real scene information collection; including video information collection and geographic location parameter collection of real scenes. The video information collection process of real scenes includes using a car-mounted camera mode, fixing the camera on a car that is traveling at a constant speed, and simulating the riding perspective to collect video of the car within a certain period of time. The geographic location parameter collection of real scenes includes the following:

[0053] A. Time: including Beijing time when each piece of data was collected;

[0054] B. Location data: including the current location longitude and latitude;

[0055] C. Altitude data: including the altitude of the current location;

[0056] D. Road information: including the slope, inclination, and road width of the current location;

[0057] E. Weather data: including the temperature, wind direction, wind speed, and weather conditions in the current area;

[0058] F. Mobile data: including the moving speed of the current acquisition device.

[0059] The specific process includes: obtaining the longitude and latitude position information of the shooting point through GPS; obtaining the altitude information of the shooting point through an altitude detector; measuring the altitude through a high-precision handheld GPS collector; measuring the current slope through a slope meter; locating the position of the vehicle in the real space through a slope meter; while collecting the geographic location information, it is necessary to record the time of collection of each piece of location information, and then package the geographic location data that matches the time information together; when collecting information, the spatial coordinate information reading interval is set to a constant, and can be matched with the frame rate during video shooting, so that post-processing can be used to obtain the real geographic location information that should correspond to every few frames of the image.

[0060] S2: Processing the real scene information collected in step S1, and building a virtual scene in a 3D engine based on the real scene information; specifically including:

[0061] P1: Processing geographic location data, converting the longitude and latitude coordinate data and altitude data collected in the real world into the corresponding x, y, and z axis coordinate data in the virtual world;

[0062] P2: Construct a virtual road. Generate a set of points through the data of P1 coordinate points, connect these points to construct a road curve in the virtual world; subdivide the points between two points separated by a collection time of Δt into Δt×m points, and smooth the curve; after constructing and processing the virtual road curve, build a road model with the same width as the real road based on this curve and the road width information in the real world, and fit and display it in the 3D engine;

[0063] P3: Split sequence frames, process the collected video information, split it into sequence frames and assign the shooting time information of each frame for subsequent processing;

[0064] P4: Match sequence frames to build a model and organize the collected information:

[0065] Geographic location information collection interval: Δt;

[0066] Video capture frame rate: m frames / second;

[0067] Unified start collection time: t0;

[0068] Then time t0 corresponds to the 0th frame image, and the geographical location information collected for the nth time corresponds to the n×Δt×mth frame image; and so on, the images corresponding to all the location information are obtained.

[0069] S3: The virtual scene constructed in step S2 is imported into the 3D engine, and a virtual character and a virtual camera are constructed; the specific process includes:

[0070] Q1: Make a virtual character model, and import the virtual character model and the model and curves constructed in step P4 into the 3D engine;

[0071] Q2: Construct a virtual camera in the 3D engine, where the virtual camera has the same focal length and depth of field parameters as the camera used to capture the image;

[0072] Q3: Bind the imported virtual character model and virtual camera to the road curve;

[0073] Q4: Match sequence frames and match the points on the road curve imported into the 3D engine to their corresponding images.

[0074] When the virtual character passes a point on the road, the image sequence frame jumps, and the sequence frame playback continues until the virtual character moves to the next point, so that the movement of the virtual character on the virtual road is smooth.

[0075] S4: Import the information flow of the user's operating equipment, control the position of the virtual character and the virtual camera in the 3D engine according to the information flow of the user's operating equipment, control the playback of the sequence frame of the real scene according to the position of the virtual character, and the virtual camera simultaneously renders the virtual character and the real scene to achieve real-time fusion.

[0076] The specific process includes: the virtual character and the virtual camera are controlled by the user's operating equipment to determine their positions on the virtual road.

[0077] The input information of the user operating the equipment is instantaneous speed information, and the distance information is determined in the following way:

[0078] Record the instantaneous speed of the user operating the equipment: V 车 and the time interval between two data inputs: Δt; and the conversion relationship between the distance on the actual road and the distance on the virtual road, that is,

[0079] The specific distance that the user has traveled by operating the equipment at the current time is calculated by the following algorithm:

[0080]

[0081] The specific distance traveled on the virtual road is:

[0082]

[0083] Assuming the total length of the virtual road is S, and the total number of frames of scene information of a road scene is N, the sequence frame number n played at the user's current position is calculated as follows:

[0084]

[0085] Through the above method, the positions of the virtual characters and the virtual camera on the 3D engine are controlled according to the user's operation of the equipment, so as to display the corresponding real scenery on the screen.

[0086] In the present invention, the equipment operated by the user can be a smart bicycle, and the 3D engine can be a unity engine. The above embodiments of the present invention do not constitute a limitation on the protection scope of the present invention. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for dynamic real-time perspective fusion of virtual characters and real scenes, characterized in that: It includes the following steps: S1: Real scene information collection; S2: Processing the real scene information collected in step S1, and building a virtual scene in a 3D engine based on it; S3: construct a virtual character and a virtual camera in the virtual scene constructed in step S2; S4: Import the information flow of the user's operation equipment, control the position of the virtual character and the virtual camera in the 3D engine according to the information flow of the user's operation equipment, control the playback of the sequence frame of the real scene according to the position of the virtual character, and the virtual camera simultaneously renders the virtual character and the real scene to achieve real-time fusion; The specific process in step S2 includes: P1: Processing geographic location data, converting the longitude and latitude coordinate data and altitude data collected in the real world into the corresponding x, y, and z axis coordinate data in the virtual world; P2: Construct a virtual road. Generate a set of points through the data of P1 coordinate points, connect these points to construct a road curve in the virtual world; subdivide the points between two points separated by a collection time of Δt into Δt×m points, and smooth the curve; after constructing and processing the virtual road curve, build a road model with the same width as the real road based on this curve and the road width information in the real world, and fit and display it in the 3D engine; P3: Split sequence frames, process the collected video information, split it into sequence frames and assign the shooting time information of each frame for subsequent processing; P4: Match sequence frames to build a model and organize the collected information: Geographic location information collection interval: Δt; Video capture frame rate: m frames / second; Unified start collection time: t0; Then the time t0 corresponds to the 0th frame image, the geographical location information collected for the nth time corresponds to the n×Δt×mth frame image; and so on, the images corresponding to all the location information are obtained; The specific process of step S3 includes: Q1: Make a virtual character model, and import the virtual character model and the model and curves constructed in step P4 into the 3D engine; Q2: Construct a virtual camera in the 3D engine, where the virtual camera has the same focal length and depth of field parameters as the camera used to capture the image; Q3: Bind the imported virtual character model and virtual camera to the road curve; Q4: Match the sequence frames and match the points on the road curve imported into the 3D engine to their corresponding images; When the virtual character passes a point on the road, the image sequence frame is skipped and the sequence frame playback continues until the virtual character moves to the next point, so that the movement of the virtual character on the virtual road is smooth; The specific process of step S4 includes: determining the movement distance of the imported user-operated equipment, thereby determining its position on the virtual road, so as to achieve the purpose of controlling the movement of the virtual character and the virtual camera by the user-operated equipment; The input information of the user operating the equipment is instantaneous speed information, and the distance information is determined in the following way: Record the instantaneous speed of the user operating the equipment: V 车 and the time interval between two data inputs: Δt; and the conversion relationship between the distance on the actual road and the distance on the virtual road, that is, The specific distance that the user has traveled by operating the equipment at the current time is calculated by the following algorithm: The specific distance traveled on the virtual road is: Assuming the total length of the virtual road is S, and the total number of frames of scene information of a road scene is N, the sequence frame number n played at the user's current position is calculated as follows:

2. The method for dynamic real-time perspective fusion of a virtual character and a real scene according to claim 1, characterized in that: Step S1 includes collecting video information and geographic location parameters of the real scene.

3. The method for dynamic real-time perspective fusion of a virtual character and a real scene according to claim 2, characterized in that: The process of collecting video information of real scenes includes using a vehicle-mounted camera mode, fixing the camera on a car that is traveling at a constant speed, and collecting video of the car within a certain period of time to simulate a riding perspective.

4. The method for dynamic real-time perspective fusion of a virtual character and a real scene according to claim 2, characterized in that: The acquisition of geographic location parameters of the real scene includes the following contents: A. Time: including Beijing time when each piece of data was collected; B. Location data: including the current location longitude and latitude; C. Altitude data: including the altitude of the current location; D. Road information: including the slope, inclination, and road width of the current location; E. Weather data: including the temperature, wind direction, wind speed, and weather conditions in the current area; F. Mobile data: including the moving speed of the current acquisition device.

5. The method for dynamic real-time perspective fusion of virtual characters and real scenes according to claim 4, characterized in that: The specific process includes: obtaining the longitude and latitude position information of the shooting point through GPS; obtaining the altitude information of the shooting point through an altitude detector; measuring the altitude through a high-precision handheld GPS collector; measuring the current slope through a slope meter; locating the position of the vehicle in the real space through a slope meter; while collecting the geographic location information, it is necessary to record the time of collection of each piece of location information, and then package the geographic location data that matches the time information together; when collecting information, the spatial coordinate information reading interval is set to a constant, and can be matched with the frame rate during video shooting, so that post-processing can be used to obtain the real geographic location information that should correspond to every few frames of the image.

Images