Motion information processing method and motion watching method based on virtual reality

Abstract

The invention provides a virtual reality-based motion information processing method and a motion watching method, and relates to the technical field of artificial intelligence such as virtual reality, motion simulation, digital humans and meta universe. The method comprises the following steps: acquiring image information of each object in a sports field through a plurality of image acquisition devices which are arranged in the sports field in advance; according to the image information, a target site area with the multi-object overlapping phenomenon is determined, overlapping motion information corresponding to the target site area is determined, and the overlapping motion information is formed after skeleton point motion information of multiple objects appearing in the target site area is overlapped; using a preset overlapping motion prediction model to disassemble and correct the overlapping motion information to obtain corrected motion information corresponding to each object appearing in the target site area, the overlapped motion prediction model is obtained by training a training sample formed by historical overlapped motion information and respective real motion information of each corresponding object; and constructing a virtual object for restoring a real motion state based on the corrected motion information in the virtual scene. By applying the method, the action restoration accuracy of the multi-object overlapping phenomenon can be better processed.

Description

Technical Field

[0001] The present disclosure relates to the fields of image processing and data processing, and specifically to the fields of artificial intelligence technologies such as virtual reality, motion simulation, digital humans, and metaverse, and in particular to a motion information processing method based on virtual reality and a motion viewing method based on virtual reality, as well as corresponding devices, electronic devices, computer-readable storage media, and computer program products. Background Art

[0002] With the rapid development of metaverse technology, the combination of virtual reality and augmented reality is changing the way people entertain themselves, especially in the field of sports.

[0003] Traditional sports games are limited by time and space, and spectators can only watch games in specific venues or through live broadcasts. The birth of the Metaverse Game breaks through these limitations and can apply virtual reality technology to sports events, allowing spectators to participate in the game in an immersive way, no longer restricted by the limitations of physical space. Summary of the invention

[0004] The embodiments of the present disclosure provide a motion information processing based on virtual reality, a motion viewing method, a device, an electronic device, a computer-readable storage medium, and a computer program product based on virtual reality.

[0005] In a first aspect, an embodiment of the present disclosure proposes a motion information processing method based on virtual reality, comprising: acquiring image information of each object in the sports venue by means of a plurality of image acquisition devices pre-arranged in the sports venue; determining a target venue area where multiple objects overlap based on the image information, and determining overlapping motion information corresponding to the target venue area; wherein the overlapping motion information is formed by overlapping skeletal point motion information of a plurality of objects appearing in the target venue area; utilizing a preset overlapping motion prediction model to disassemble and correct the overlapping motion information to obtain corrected motion information corresponding to each object appearing in the target venue area; wherein the overlapping motion prediction model is trained based on training samples consisting of historical overlapping motion information and real motion information of each corresponding object; and constructing a virtual object that restores the real motion state based on the corrected motion information in a virtual scene.

[0006] In a second aspect, an embodiment of the present disclosure proposes a motion information processing device based on virtual reality, comprising: an image information acquisition unit, configured to acquire image information of each object in a sports venue through a plurality of image acquisition devices pre-set in the sports venue; an overlapping motion information determination unit, configured to determine a target venue area where multiple objects overlap based on the image information, and determine overlapping motion information corresponding to the target venue area; wherein the overlapping motion information is formed by overlapping skeletal point motion information of a plurality of objects appearing in the target venue area; a motion information disassembly and correction unit, configured to use a preset overlapping motion prediction model to disassemble and correct the overlapping motion information, and obtain corrected motion information corresponding to each object appearing in the target venue area; wherein the overlapping motion prediction model is trained based on training samples consisting of historical overlapping motion information and corresponding real motion information of each object; a virtual scene and virtual object reconstruction unit, configured to construct a virtual object that restores the real motion state based on the corrected motion information in a virtual scene.

[0007] In a third aspect, an embodiment of the present disclosure proposes a motion viewing method based on virtual reality, comprising: determining virtual motion data corresponding to a target motion according to viewing requirements; wherein the virtual motion data is motion data of a virtual object that restores a real motion state and is rendered in a virtual scene according to the first aspect; parsing the virtual motion data into a virtual object that restores a real motion state in a virtual scene, and presenting it in an image display area.

[0008] In a fourth aspect, an embodiment of the present disclosure proposes a motion viewing device based on virtual reality, comprising: a virtual motion data acquisition unit, configured to determine virtual motion data corresponding to a target motion according to viewing requirements; wherein the virtual motion data is motion data of a virtual object that restores a real motion state and is rendered in a virtual scene according to the description of the second aspect; and a parsing and presenting unit, configured to parse the virtual motion data into a virtual object that restores a real motion state in a virtual scene, and present it in an image display area.

[0009] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, it is possible to implement the motion information processing method based on virtual reality as described in the first aspect or the motion viewing method based on virtual reality as described in the third aspect.

[0010] In a sixth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement the motion information processing method based on virtual reality as described in the first aspect or the motion viewing method based on virtual reality as described in the third aspect when executed.

[0011] In a seventh aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, can implement the steps of the motion information processing method based on virtual reality as described in the first aspect or the steps of the motion viewing method based on virtual reality as described in the third aspect.

[0012] The motion information processing solution based on virtual reality provided by the embodiment of the present disclosure first obtains all-round image information by photographing with multiple image acquisition devices set up in the sports field, then determines the target area where multiple objects overlap based on the image information, and determines the overlapping motion information corresponding to the multiple objects overlapping in the target area, and then uses the pre-trained overlapping motion prediction model to disassemble and correct the overlapping motion information obtained by multiple objects overlapping and mixing in a small area according to different objects, so as to obtain corrected motion information corresponding to each overlapping object, so that the virtual motion state expressed by the virtual object reconstructed based on the corrected motion information is more consistent with the real motion state expressed by the corresponding object in the sports field, avoiding abnormal limb movement posture caused by erroneous rendering.

[0013] On the basis of the motion information processing solution based on virtual display provided in the above-mentioned embodiments, the motion viewing solution based on virtual reality further provided in the embodiments of the present disclosure can directly present virtual reality motion that is as consistent as possible with real motion to the user by parsing and presenting the corresponding virtual reality data, so that the user can get an immersive effect without having to be present in person.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0016] Figure 1 is an exemplary system architecture in which the present disclosure may be applied;

[0017] Figure 2A flowchart of a motion information processing method based on virtual reality provided by an embodiment of the present disclosure;

[0018] Figure 3 A flowchart of a method for determining overlapping motion information provided by an embodiment of the present disclosure;

[0019] Figure 4 A flowchart of a method for constructing a virtual object in a virtual scene provided by an embodiment of the present disclosure;

[0020] Figure 5 A flowchart of a method for processing virtual motion data provided by an embodiment of the present disclosure;

[0021] Figure 6 A flowchart of a method for watching sports based on virtual reality provided by an embodiment of the present disclosure;

[0022] Figure 7 An exemplary screen of a virtual game viewing provided by an embodiment of the present disclosure;

[0023] Figure 8 A structural block diagram of a motion information processing device based on virtual reality provided in an embodiment of the present disclosure;

[0024] Fig. 9 A structural block diagram of a sports viewing device based on virtual reality provided in an embodiment of the present disclosure;

[0025] Fig.10 A schematic structural diagram of an electronic device suitable for executing a motion information processing method based on virtual reality and / or a motion viewing method based on virtual reality provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments may be combined with each other without conflict.

[0027] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0028] Figure 1An exemplary system architecture 100 is shown to which embodiments of the virtual reality-based motion information processing method and motion viewing method, apparatus, electronic device, and computer-readable storage medium provided in the present application can be applied.

[0029] like Figure 1 As shown, the system architecture 100 may include a moving object 101 (specifically including moving objects 1011, 1012, and 1013), an image acquisition device 102 (specifically including image acquisition devices 1021, 1022, 1023, 1024, 1025, 1026, 1027, and 1028), a sports field 103, a network 104, a server 105, and a user terminal 106 (specifically including user terminals 1061, 1062, and 1063). The network 104 is used as a medium for providing a communication link between the image acquisition device 102 and the server 105, and between the server 105 and the user terminal 106. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0030] The user can use the user terminal 106 to interact with the server 105 through the network 104 to receive or send messages, etc. The user terminal 106, the server 105 and the image acquisition device 102 can be installed with various applications for realizing information communication between the two, such as sports image shooting and transmission applications, virtual sports data generation applications, virtual viewing applications, etc.

[0031] The user terminal 106 and the server 105 can be hardware or software. When the user terminal 106 is hardware, it can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, head-mounted displays, laptop portable computers and desktop computers, etc.; when the user terminal 106 is software, it can be installed in the electronic devices listed above, which can be implemented as multiple software or software modules, or as a single software or software module, which is not specifically limited here. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server; when the server is software, it can be implemented as multiple software or software modules, or as a single software or software module, which is not specifically limited here.

[0032] The server 105 can provide various services through various built-in applications. Taking a virtual viewing application that can provide users with virtual reality viewing services as an example, the server 105 can achieve the following effects when running the virtual viewing application: first, the server 105 receives image information obtained by taking pictures of various objects 101 in the sports field 103 by multiple image acquisition devices 102 pre-set in the sports field through the network 104; then, the target field area where multiple objects overlap is determined based on the image information, and the overlapping motion information corresponding to the target field area is determined, and the overlapping motion information is formed by overlapping the motion information of the skeleton points of multiple objects appearing in the target field area; then, the overlapping motion information is disassembled and corrected by using a preset overlapping motion prediction model to obtain corrected motion information corresponding to each object appearing in the target field area; finally, a virtual object that restores the real motion state is constructed based on the corrected motion information in the virtual scene to obtain virtual motion data.

[0033] Among them, the overlapping motion prediction model can be trained by a model training application built into the server 105 based on training samples composed of historical overlapping motion information and the corresponding real motion information of each object, or it can be trained by other servers different from the server 105 that are dedicated to processing model training tasks. It only needs to be accessible and callable by the server 105.

[0034] After the server 105 has generated the virtual motion data through the above steps, the user terminal 106 can further realize viewing through the following steps: first, determine the virtual motion data corresponding to the target motion through the viewing requirements input by the user; then, receive the virtual motion data from the server 105 through the network 104, and finally parse the virtual motion data into a virtual object that restores the real motion state in the virtual scene, and present it in the image display area of ​​the user terminal.

[0035] The motion information processing methods based on virtual reality provided in the subsequent embodiments of the present application are generally executed by a server 105 having stronger computing power and more computing resources. Accordingly, the motion information processing device based on virtual reality is generally also disposed in the server 105.

[0036] The virtual reality-based motion viewing method provided in the subsequent embodiments of the present application is generally executed by the user terminal 106 , and accordingly, the virtual reality-based motion viewing device is generally also disposed in the user terminal 106 .

[0037] The trained overlapping motion prediction model can be used to obtain a lightweight overlapping motion prediction model suitable for placement in the user terminal 106 through model distillation. That is, it is possible to flexibly choose to use the lightweight overlapping motion prediction model in the user terminal 106 or the more complex overlapping motion prediction model in the server 105 according to the recognition accuracy actually required.

[0038] It should be understood that Figure 1 The number of objects, sites, image acquisition devices, networks, servers, and user terminals in the embodiment is only illustrative. Any number of objects, sites, image acquisition devices, networks, servers, and user terminals may be provided according to implementation requirements.

[0039] Please refer to Figure 2 , Figure 2 This is a flowchart of a motion information processing method based on virtual reality provided by an embodiment of the present disclosure, wherein process 200 includes the following steps:

[0040] Step 201: acquiring image information of each object in the sports field through a plurality of image acquisition devices pre-installed in the sports field;

[0041] This step is intended to be performed by an execution subject (e.g. Figure 1 The server 105 shown in the figure is pre-set at the sports venue (e.g. Figure 1 A plurality of image acquisition devices (eg, Figure 1 The image acquisition device 102 shown in the figure acquires the images of various objects (such as Figure 1 That is, multiple image acquisition devices are arranged at appropriate positions and angles to capture the motion image information of each object in the sports field as comprehensively and without blind spots as possible.

[0042] Among them, the objects in the sports field may include not only athletes participating in the game, but also spectators in the sports field; the image acquisition device may be a camera, a camcorder or other similar equipment with a shooting function. In order to supplement more subtle sports information as much as possible, the image acquisition device may also use a high-speed camera that can capture fast-moving details in the game at an extremely high frame rate.

[0043] Multiple image acquisition devices can capture images from different angles, and ultimately generate in real time the spatial position information data of each frame object, the spatial position information data of the ball, and the real-time skeleton information data of different objects.

[0044] Step 202: determining a target field area where multiple objects overlap based on the image information, and determining overlapping motion information corresponding to the target field area;

[0045] Based on step 201, this step aims to determine the target site area where multiple objects overlap according to the image information, and further determine the overlapping motion information corresponding to the target site area according to the image information. The overlapping motion information can be the product of preprocessing the motion information initially extracted according to the image information, and the preprocessing method can include: filtering, removing noise, adjusting contrast and other processing methods that can highlight the motion information.

[0046] The overlapping motion information is actually formed by the overlapping motion information of the skeleton points of multiple objects appearing in the target site area, and because the overlapping motion information is determined by the image information obtained by the image acquisition device in the area where the multiple objects overlap, it is possible that the captured image information is not sufficient to completely and accurately decompose the overlapping motion information into independent motion information of each overlapping object due to the fact that it is set up far away from the overlapping objects. In simple terms, the overlapping motion information as a representation obtained by the overlapping of multiple objects is obtained, but it is not necessarily possible to decompose the independent motion information of each specific object, that is, the "answer" as the result is obtained, but in order to accurately restore the independent motion information of each object, it is also necessary to know the process of obtaining the "answer".

[0047] An implementation method including but not limited to may be:

[0048] First, an area where there are more than a preset number of objects is determined based on the image information, and the area is selected as a multi-object venue area; then, the distance between the objects in the multi-object venue area is determined based on the image information; finally, the multi-object venue area with a distance less than the preset distance is determined as the target venue area. That is, according to the physical characteristics of the mixing or overlapping of multiple objects, that is, the distance between the two is small or even negative, this embodiment provides a specific implementation method for determining the target venue area based on the distance between the objects.

[0049] Further, the distance may include a minimum distance and an average distance, and correspondingly, the preset distance may include a preset minimum distance corresponding to the minimum distance and a preset average distance corresponding to the average distance; then, the method of determining the target site area may be adaptively changed to:

[0050] A multi-object site area whose minimum distance is less than a preset minimum distance and / or whose average distance is less than a preset average distance is determined as a target site area. That is, in the case where only the minimum distance is less than the preset minimum distance, only the average distance is less than the preset average distance, and not only the minimum distance is less than the preset minimum distance but also the average distance is less than the preset average distance, the corresponding site area can be determined as the target site area. Of course, in the case where different scenarios may have different standards for determining the target site area, the corresponding distance judgment standard can also be used to determine the target site area that meets the scenario requirements, which will not be listed one by one here.

[0051] Step 203: using a preset overlapping motion prediction model, disassemble and correct the overlapping motion information to obtain corrected motion information corresponding to each object appearing in the target site area;

[0052] On the basis of step 202, this step aims to use the preset overlapping motion prediction model by the above-mentioned execution subject to disassemble and correct the overlapping motion information, and obtain the corrected motion information corresponding to each object appearing in the target site area. Among them, the overlapping motion prediction model is trained based on the training samples composed of historical overlapping motion information and the corresponding real motion information of each object. The training sample is usually expressed as a set containing multiple sample pairs, and each sample pair can be specifically expressed as: the sample input is served by the historical overlapping motion information, and the sample output is served by the corresponding real motion information of each object. The historical overlapping motion information is the real overlapping motion information extracted from the image information of the historical motion with the same standard, and the real motion information of each object is the independent motion information of each object that is determined to be accurate. Of course, due to the overlap and mixing of multiple objects, the independent motion information of each object will also be associated with the motion information of other overlapping objects.

[0053] In order to deepen the understanding of how the overlapping motion prediction model specifically achieves the above-mentioned effect, this embodiment further specifically provides an implementation method including but not limited to:

[0054] First, a disassembly and recognition module in the overlapping motion prediction model is used to identify overlapping objects corresponding to overlapping motion information, and to disassemble independent motion information corresponding to each overlapping object. The disassembly and recognition module is trained based on a first training sample consisting of historical overlapping motion information and corresponding real overlapping objects. The first training sample is usually represented by a set of multiple first sample pairs, and each first sample pair can be specifically represented by: a sample input represented by the historical overlapping motion information and a sample output represented by the real motion information of each corresponding real overlapping object;

[0055] Next, the abnormal motion correction module in the overlapping motion prediction model is used to correct the independent motion information of each overlapping object to obtain the corrected motion information corresponding to each overlapping object. The abnormal motion correction module is trained based on a second training sample consisting of independent motion information historically decomposed and corresponding real motion information. The second training sample is usually expressed as a set including multiple second sample pairs, and each second sample pair can be specifically expressed as: a sample input composed of independent motion information historically decomposed and a sample output composed of the real motion information of each corresponding real overlapping object.

[0056] That is, this embodiment specifically realizes disassembling each overlapping object generating overlapping motion information and performing abnormal correction on each independent motion information obtained by disassembling in sequence through two core functional modules constituting the overlapping motion prediction model, so as to finally obtain the corrected motion information of each overlapping object.

[0057] It should be noted that, in some other embodiments of the present disclosure, the overlapping motion prediction model can also achieve similar technical effects through a combination of other built-in functional modules and functional layers, and is not limited to the specific implementation methods provided above.

[0058] Furthermore, in order to improve the accuracy of disassembly and the accuracy of anomaly correction as much as possible, it is also possible to require that the identity information of each real overlapping object be additionally recorded in the first training sample and the second training sample, so that the overlapping motion prediction model can additionally combine the motion inertia exhibited by objects of different identities when learning the implicit association between the overlapping motion information and the corrected motion information of each overlapping object. Therefore, when it is found that the overlapping object corresponding to the actually input overlapping motion information contains the motion inertia knowledge of a certain object that has been learned, the motion inertia knowledge of the object is preferentially used to participate in the disassembly and correction of the independent motion information, so that the final result is closer to the actual motion state.

[0059] Step 204: construct a virtual object that restores the real motion state based on the corrected motion information in the virtual scene.

[0060] On the basis of step 203, this step aims to enable the above-mentioned execution subject to construct a virtual object that restores the real motion state based on the corrected motion information in the virtual scene.

[0061] That is, a virtual scene corresponding to the sports field is constructed using a virtual reality technology engine (such as Unity), and then a virtual object is constructed for each object at the corresponding position on the sports field in the virtual scene. Then, the action of the corresponding virtual object is reconstructed according to the corrected motion information of each object, so that the motion state of the virtual object is consistent with the real motion state of the corresponding object.

[0062] The motion information processing method based on virtual reality provided by the embodiment of the present disclosure first obtains all-round image information by photographing with multiple image acquisition devices set up in the sports field, then determines the target area where multiple objects overlap based on the image information, and determines the overlapping motion information corresponding to the multiple objects overlapping in the target area, and then uses the pre-trained overlapping motion prediction model to disassemble and correct the overlapping motion information obtained by multiple objects overlapping and mixing in a small area according to different objects, so as to obtain corrected motion information corresponding to each overlapping object, so that the virtual motion state expressed by the virtual object reconstructed based on the corrected motion information is more consistent with the real motion state expressed by the corresponding object in the sports field, thereby avoiding abnormal limb movement posture caused by erroneous rendering.

[0063] Please refer to Figure 3 , Figure 3 A flowchart of a method for determining overlapping motion information provided by an embodiment of the present disclosure, that is, for Figure 2 In the process 200 shown, the part of the overlapping motion information corresponding to the target field area is determined according to the image information in step 202, which provides a specific implementation method. The other steps in the process 200 are not adjusted. The specific implementation method provided by this embodiment is replaced by how to determine the overlapping motion information part in step 202 to obtain a new complete embodiment. The process 300 includes the following steps:

[0064] Step 301: Determine the regional image information corresponding to the target site area;

[0065] This step aims at firstly determining the regional image information corresponding to the target venue area according to the image information corresponding to the complete sports venue by the execution subject.

[0066] Step 302: determining overlapping skeleton points of overlapping objects contained in the target site area according to the regional image information;

[0067] On the basis of step 301, this step aims at determining the overlapping objects contained in the target site area according to the regional image information by the above-mentioned execution subject, and then further determining the overlapping skeleton points on the overlapping objects.

[0068] Step 303: Determine the motion information matching the overlapping skeleton points according to the regional image information to obtain the overlapping motion information.

[0069] On the basis of step 302, this step aims at allowing the execution subject to determine the motion information matching the overlapping skeleton points according to the regional image information, and obtain the overlapping motion information describing how the overlapping skeleton points move.

[0070] This embodiment specifically provides an implementation method of first segmenting to obtain regional image information, then determining the overlapping skeleton points of overlapping objects based on the regional image information, and finally obtaining overlapping motion information based on the motion information of the overlapping skeleton points through steps 301 to 303. In some other embodiments of the present disclosure, similar or identical overlapping motion information can also be obtained by using a similar implementation method different from this embodiment. This embodiment exists only as an example with a high degree of feasibility.

[0071] Please refer to Figure 4 , Figure 4 A flowchart of a method for constructing a virtual object in a virtual scene provided by an embodiment of the present disclosure, that is, for Figure 2 Step 204 in the process 200 shown provides a specific implementation method, and the other steps in the process 200 are not adjusted. The specific implementation method provided in this embodiment is replaced by step 204 to obtain a new complete embodiment. The process 400 includes the following steps:

[0072] Step 401: constructing an initial virtual object in a virtual scene;

[0073] This step aims to have the above-mentioned execution subject first construct an initial virtual object with an initial posture that fails to reflect the real motion information at the corresponding position on the virtual scene according to the position information of each real object on the real sports field in the virtual scene corresponding to the real sports field.

[0074] Step 402: Rendering the motion information based on the same skeleton points between the corrected motion information and the initial virtual object to obtain an initial rendering result;

[0075] This step aims to render the motion information based on the same skeleton points between the corrected motion information of the object and the initial virtual object by the execution subject to obtain the initial rendering result. That is, the action rendering of the initial virtual object is based on the same skeleton points, and this step only renders the motion information.

[0076] Step 403: The initial rendering result is adjusted based on the physiological characteristics of the corresponding object, the environmental interaction with the sports venue, and the motion interaction feedback between different objects to obtain a virtual object that restores the real motion state.

[0077] On the basis of step 402, this step aims to allow the above-mentioned execution subject to further combine the physiological characteristics of the corresponding object, the environmental interaction with the sports venue, and the motion interaction feedback between different objects, and make detailed adjustments to the initial rendering result, so as to obtain a virtual object that perfectly restores the real motion state as much as possible, so that the motion posture of the virtual object is consistent with the motion posture of the real object, and can reflect the physiological characteristics of different objects and the real interaction with the specific environment, thereby improving the degree of restoration of virtual reality to events occurring in the real world.

[0078] Based on the specific implementation methods given from different angles for the target field area where multiple objects overlap, in any of the above embodiments, this embodiment mainly processes motion information in other field areas where multiple objects do not overlap. An implementation method including but not limited to the following may be:

[0079] First, other field areas in the sports field that are different from the target field area are determined; then, single-object motion information of objects appearing in other field areas is determined based on the image information; then, the single-object motion information is processed using a preset motion prediction model to obtain processed motion information, and the motion prediction model is trained based on training samples consisting of historical single-object motion information and real motion information; finally, a virtual object that restores the real motion state is constructed in a virtual scene based on the processed motion information.

[0080] Different from the overlapping motion prediction model used to process overlapping motion information, the action prediction model used in this embodiment is mainly used to process independent object motion information without overlapping phenomenon, and its effect is similar to the abnormal motion correction module mentioned in the above embodiment that constitutes the overlapping motion prediction model.

[0081] Similarly, the specific implementation method of constructing a virtual object that restores the real motion state based on the processed motion information in the virtual scene can be:

[0082] First, an initial virtual object is constructed in a virtual scene; then, based on the same skeleton points between the corrected motion information and the initial virtual object, the motion information is rendered to obtain an initial rendering result; finally, the initial rendering result is adjusted in combination with the physiological characteristics of the corresponding object and the environmental interaction with the sports venue to obtain a virtual object that restores the real motion state. Different from the adjustment scheme provided in step 403, this embodiment does not need to combine the motion interaction feedback between different objects because it is aimed at independent motion information without overlapping.

[0083] Please refer to Figure 5 , Figure 5 A flowchart of a method for processing virtual motion data provided by an embodiment of the present disclosure, that is, for Figure 2 Step 204 in the process 200 shown provides a specific implementation method, and the other steps in the process 200 are not adjusted. The specific implementation method provided in this embodiment is replaced by step 204 to obtain a new complete embodiment. The process 500 includes the following steps:

[0084] Step 501: Slice the virtual motion data corresponding to the virtual scene and each virtual object according to the time sequence to obtain slice data;

[0085] Step 502: compress each slice data using a preset compression algorithm, and distribute the compressed slice redundancy to different nodes of the content distribution network.

[0086] This embodiment specifically provides an implementation method of first sharding and then compressing the shard data and then distributing them to different nodes of the content distribution network through steps 501 and 502, so as to reduce the single transmission amount through sharding and compression, and improve the effectiveness of obtaining all shard data through redundant distribution.

[0087] Furthermore, considering that the virtual motion data may be affected by the types and performance of different user terminals when used by users in the future, different sampling frequencies can be set according to the types and performance levels of different terminals during slicing to provide the highest possible quality picture data that matches the performance of each terminal. By taking into account different display devices, the user coverage is also improved.

[0088] A specific implementation method including but not limited to may be:

[0089] First, obtain the virtual motion data corresponding to the virtual scene and each virtual object; then, sample each data frame in the virtual motion data according to different sampling frequencies to obtain sub-virtual motion data of different qualities, and different sampling frequencies correspond to different performances of various types of devices that can display virtual motion data; finally, slice each sub-virtual motion data according to the segmentation standard corresponding to the sampling frequency to which it belongs, to obtain sliced ​​data corresponding to each sampling frequency. Then, the device that initiates the acquisition request only needs to send its device model or performance parameters along with the acquisition request, and the sliced ​​data corresponding to the sampling frequency that matches its performance can be sent to the user device, so that the user device can present appropriate virtual motion data by presenting image content that matches its performance.

[0090] A more practical implementation can be divided into the following steps:

[0091] 1) Data preparation stage: obtaining original virtual motion information;

[0092] 2) In the data preprocessing stage, use tools to cut and segment the data (overall description file + segment data frame files);

[0093] Specifically, the data constituting the original virtual motion information can be divided into a group of n frames, and divided into frame data of various qualities according to different sampling frequencies (corresponding to different performances of different user terminals). The size of one segment is 76KB*n before compression, and an overall description data is designed to describe the game information, the total number of segments, the total number of frames, the number of frames in each segment, etc., where the value range of n is 50-250.

[0094] 3) Data deployment phase: deploy pre-processed data to the content distribution network and set the data compression speed format to brotli (an efficient lossless compression algorithm);

[0095] 4) During the data restoration phase, the Unity client pulls data in a loop and restores the game content frame by frame.

[0096] On the basis of any of the above embodiments, considering the publicity, authority and post-event traceability of sports events, blockchain technology can also be used to authenticate and prevent tampering of the action data of each participant in the competition, so as to ensure the authenticity and integrity of the actions of the participants restored in the virtual environment, especially in application scenarios such as game replays and referee decisions.

[0097] The above embodiments mainly describe how to generate virtual motion data from the perspective of the server. In order to highlight how the user side uses the virtual motion data from the actual usage scenario as much as possible, the present disclosure also describes how to generate virtual motion data from the perspective of the server. Figure 6 A flowchart of a method for watching sports based on virtual reality is provided, wherein the flowchart 600 includes the following steps:

[0098] Step 601: determining virtual motion data corresponding to a target motion according to viewing requirements;

[0099] This step is intended to be performed by an execution subject of the motion viewing method based on virtual reality (e.g. Figure 1 The user terminal 106 shown in FIG. 1 determines virtual sports data corresponding to the target sport (eg, game name, match number, number, etc.) according to the viewing requirements input by the user. The segmented data of the virtual sports data can be as follows: Figure 5 The method provided by the illustrated embodiment is stored on each node of the intranet distribution network.

[0100] An implementation method including but not limited to may be:

[0101] First, a target sampling frequency that matches the performance of the current viewing device is determined according to viewing needs; then, each slice data of the sub-virtual motion data corresponding to the target sampling frequency is obtained from each node constituting the content distribution network; finally, sub-virtual motion data of the target motion corresponding to the target sampling frequency is generated according to each slice data.

[0102] Step 602: parse the virtual motion data into virtual objects that restore the real motion state in the virtual scene, and present them in the image display area.

[0103] On the basis of step 601, this step aims to enable the above-mentioned execution subject to parse the virtual motion data into a virtual object that restores the real motion state in the virtual scene and present it in the image display area.

[0104] It should be noted that since the image information is obtained by capturing with a multi-perspective image acquisition device covering the entire sports field, combined with the virtual scene and virtual objects constructed with virtual reality technology, the user who initiates the viewing request can watch from any perspective in the virtual scene. For example, the user can freely switch to any player's perspective, referee's perspective, audience perspective or even any designated position.

[0105] On the basis of the motion information processing solution based on virtual display provided in the above-mentioned embodiments, the motion viewing solution based on virtual reality further provided in the embodiments of the present disclosure can directly present virtual reality motion that is as consistent as possible with real motion to the user by parsing and presenting the corresponding virtual reality data, so that the user can get an immersive effect without having to be present in person.

[0106] Furthermore, in order to ensure the smoothness of loading and display of the user terminal, a buffer that can resist network jitter and performance jitter can be set for the above-mentioned execution entity when playing and displaying the pictures corresponding to the virtual motion data. That is, some picture frames to be displayed can be reserved in the buffer in advance to avoid the problem of unstable loading caused by jitter.

[0107] Further references Figure 8 and Fig. 9 As an implementation of the methods shown in the above figures, the present disclosure provides a motion information processing device embodiment based on virtual reality and a motion viewing device embodiment based on virtual reality. Figure 2 The embodiment of the motion information processing method based on virtual reality shown in FIG. 1 corresponds to the embodiment of the motion viewing device based on virtual reality. Figure 6 The above device can be applied to various electronic devices.

[0108] like Figure 8 As shown, the motion information processing device 800 based on virtual reality of this embodiment may include: an image information acquisition unit 801, an overlapping motion information determination unit 802, a motion information decomposition and correction unit 803, and a virtual scene and virtual object reconstruction unit 804. Among them, the image information acquisition unit 801 is configured to acquire image information of each object in the sports venue through multiple image acquisition devices pre-arranged in the sports venue; the overlapping motion information determination unit 802 is configured to determine the target venue area where multiple objects overlap based on the image information, and determine the overlapping motion information corresponding to the target venue area; wherein the overlapping motion information is formed after the skeletal point motion information of multiple objects appearing in the target venue area overlaps; the motion information disassembly and correction unit 803 is configured to use a preset overlapping motion prediction model to disassemble and correct the overlapping motion information, and obtain corrected motion information corresponding to each object appearing in the target venue area; wherein the overlapping motion prediction model is trained based on training samples composed of historical overlapping motion information and the corresponding real motion information of each object; the virtual scene and virtual object reconstruction unit 804 is configured to construct a virtual object that restores the real motion state based on the corrected motion information in the virtual scene.

[0109] In this embodiment, in the motion information processing device 800 based on virtual reality, the specific processing of the image information acquisition unit 801, the overlapping motion information determination unit 802, the motion information decomposition and correction unit 803, and the virtual scene and virtual object reconstruction unit 804 and the technical effects thereof can be referred to in detail. Figure 2 The relevant descriptions of steps 201 - 204 in the corresponding embodiment are not repeated here.

[0110] In some optional implementations of this embodiment, the overlapping motion information determining unit 802 may include a target site area determining subunit configured to determine a target site area where multiple objects overlap according to the image information. The target site area determining subunit may include:

[0111] a multi-object site area determination module, configured to determine, based on the image information, an area where objects exceeding a preset number exist, and obtain a multi-object site area;

[0112] a distance determination module configured to determine the distance between objects in the multi-object venue area based on the image information;

[0113] The target site area determination module is configured to determine a multi-object site area whose distance is less than a preset distance as a target site area.

[0114] In some optional implementations of this embodiment, the distance includes a minimum distance and an average distance, and the preset distance includes a preset minimum distance corresponding to the minimum distance and a preset average distance corresponding to the average distance;

[0115] The target site area determination module is further configured to:

[0116] A multi-object site area whose minimum distance is less than a preset minimum distance and / or whose average distance is less than a preset average distance is determined as a target site area.

[0117] In some optional implementations of this embodiment, the overlapping motion information determining unit 802 may include an overlapping motion information determining subunit configured to determine overlapping motion information corresponding to the target venue area, and the overlapping motion information determining subunit is further configured to:

[0118] Determining regional image information corresponding to the target site area;

[0119] Determine overlapping skeleton points of overlapping objects contained in the target site area according to the regional image information;

[0120] According to the regional image information, the motion information matching the overlapping skeleton points is determined to obtain the overlapping motion information.

[0121] In some optional implementations of this embodiment, the motion information decomposition and correction unit 803 may be further configured to:

[0122] Using the disassembly and recognition module in the overlapping motion prediction model, each overlapping object corresponding to the overlapping motion information is identified, and independent motion information corresponding to each overlapping object is disassembled; wherein the disassembly and recognition module is trained based on a first training sample consisting of historical overlapping motion information and corresponding real overlapping objects;

[0123] The abnormal motion correction module in the overlapping motion prediction model is used to correct the independent motion information of each overlapping object to obtain the corrected motion information corresponding to each overlapping object; wherein the abnormal motion correction module is trained based on a second training sample consisting of independent motion information disassembled from history and corresponding real motion information.

[0124] In some optional implementations of this embodiment, the first training sample and the second training sample also record identity information of each real overlapping object.

[0125] In some optional implementations of this embodiment, the virtual scene and virtual object reconstruction unit 804 may be further configured to:

[0126] Constructing initial virtual objects in a virtual scene;

[0127] Based on the same skeleton points between the corrected motion information and the initial virtual object, the motion information is rendered to obtain an initial rendering result;

[0128] The initial rendering result is adjusted based on the physiological characteristics of the corresponding object, the environmental interaction with the sports venue, and the motion interaction feedback between different objects to obtain a virtual object that restores the real motion state.

[0129] In some optional implementations of this embodiment, the motion information processing device 800 based on virtual reality may further include:

[0130] The other field area determination unit is configured to determine other field areas in the sports field that are different from the target field area;

[0131] a single object motion information determination unit configured to determine single object motion information of an object appearing in other venue areas based on the image information;

[0132] The single object motion information processing unit is configured to process the single object motion information using a preset motion prediction model to obtain processed motion information; wherein the motion prediction model is trained based on training samples consisting of historical single object motion information and real motion information;

[0133] A single virtual object reconstruction unit is configured to construct a virtual object that restores the real motion state based on the processed motion information in a virtual scene.

[0134] In some optional implementations of this embodiment, the single virtual object reconstruction unit is further configured to:

[0135] Constructing initial virtual objects in a virtual scene;

[0136] Based on the same skeleton points between the corrected motion information and the initial virtual object, the motion information is rendered to obtain an initial rendering result;

[0137] The initial rendering result is adjusted based on the physiological characteristics of the corresponding object and the environmental interaction with the sports venue to obtain a virtual object that restores the real motion state.

[0138] In some optional implementations of this embodiment, the motion information processing device 800 based on virtual reality may further include:

[0139] A data slicing unit is configured to slice the virtual motion data corresponding to the virtual scene and each virtual object according to a time sequence to obtain slice data;

[0140] The compression distribution unit is configured to compress each shard data using a preset compression algorithm, and distribute the compressed shard redundancy to different nodes of the content distribution network.

[0141] In some optional implementations of this embodiment, the data slicing unit is further configured to:

[0142] Acquire virtual motion data corresponding to the virtual scene and each virtual object;

[0143] Sampling each data frame constituting the virtual motion data at different sampling frequencies to obtain sub-virtual motion data of different qualities; wherein different sampling frequencies correspond to different performances of various types of devices capable of displaying the virtual motion data;

[0144] Each sub-virtual motion data is sliced ​​according to a segmentation standard corresponding to the corresponding sampling frequency to obtain slice data corresponding to each sampling frequency.

[0145] like Fig. 9 As shown, the virtual reality-based motion viewing device 900 of this embodiment may include: a virtual motion data acquisition unit 901, and a parsing and presenting unit 902. The virtual motion data acquisition unit 901 is configured to determine virtual motion data corresponding to the target motion according to the viewing requirements; the parsing and presenting unit 902 is configured to parse the virtual motion data into a virtual object that restores the real motion state in the virtual scene, and present it in the image display area.

[0146] In this embodiment, in the virtual reality-based sports viewing device 900, the specific processing of the virtual sports data acquisition unit 901 and the analysis and presentation unit 902 and the technical effects brought by them can correspond to Figure 6 The relevant description in the method embodiment is omitted here.

[0147] This embodiment exists as an apparatus embodiment corresponding to the above-mentioned method embodiment. The motion information processing apparatus based on virtual reality provided by this embodiment first obtains all-round image information by photographing with multiple image acquisition devices arranged in a sports field, and then determines the target area where multiple objects overlap based on the image information, and determines the overlapping motion information corresponding to the multiple objects overlapping in the target area, and then uses the pre-trained overlapping motion prediction model to disassemble and correct the overlapping motion information obtained by multiple objects overlapping and mixed in a small area according to different objects, so as to obtain corrected motion information corresponding to each overlapping object, so that the virtual motion state expressed by the virtual object reconstructed based on the corrected motion information is more consistent with the real motion state expressed by the corresponding object in the sports field, thereby avoiding abnormal limb movement posture caused by erroneous rendering.

[0148] On the basis of the motion information processing device based on virtual display provided in the above-mentioned embodiments, the motion viewing device based on virtual reality further provided in the embodiments of the present disclosure can directly present virtual reality motion that is as consistent as possible with real motion to the user by parsing and presenting the corresponding virtual reality data, when the corresponding virtual motion data is determined according to the viewing needs initiated by the user. This allows the user to get an immersive effect without having to be present in person.

[0149] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, the motion information processing method based on virtual reality and / or the motion viewing method based on virtual reality described in any of the above embodiments can be implemented.

[0150] According to an embodiment of the present disclosure, the present disclosure also provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the virtual reality-based motion information processing method and / or the virtual reality-based motion viewing method described in any of the above embodiments when executed.

[0151] The embodiments of the present disclosure provide a computer program product. When the computer program is executed by a processor, the computer program can implement the motion information processing method based on virtual reality and / or the motion viewing method based on virtual reality described in any of the above embodiments.

[0152] Fig.10 A schematic block diagram of an example electronic device 1000 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0153] like Fig.10As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0154] A number of components in the device 1000 are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0155] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as a motion information processing method based on virtual reality and / or a motion viewing method based on virtual reality. For example, in some embodiments, a motion information processing method based on virtual reality and / or a motion viewing method based on virtual reality may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the motion information processing method based on virtual reality and / or the motion viewing method based on virtual reality described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute a virtual reality-based motion information processing method and / or a virtual reality-based motion viewing method in any other appropriate manner (e.g., by means of firmware).

[0156] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0157] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0158] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0159] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0160] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0161] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0162] The technical solution of the disclosed embodiment is to first obtain all-round image information by shooting with multiple image acquisition devices set in the sports field, then determine the target area where multiple objects overlap according to the image information, and determine the overlapping motion information corresponding to the multiple objects overlapping in the target area, and then use the pre-trained overlapping motion prediction model to disassemble and correct the overlapping motion information obtained by multiple objects overlapping and mixing in a small area according to different objects, so as to obtain the corrected motion information corresponding to each overlapping object, so that the virtual motion state of the virtual object reconstructed based on the corrected motion information is more consistent with the real motion state of the corresponding object in the sports field, avoiding abnormal limb motion posture caused by erroneous rendering. And in the case of determining the corresponding virtual motion data according to the viewing needs initiated by the user, the virtual reality motion that is as consistent as possible with the real motion can be presented to the user directly by parsing and presenting the corresponding virtual reality data, so that the user can get an immersive effect without being present in person.

[0163] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0164] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A motion information processing method based on virtual reality, comprising: Acquiring image information of each object in the sports field through a plurality of image acquisition devices pre-arranged in the sports field; Determine a target site area where multiple objects overlap according to the image information, and determine overlapping motion information corresponding to the target site area; wherein the overlapping motion information is formed by overlapping motion information of skeleton points of multiple objects appearing in the target site area; Using a preset overlapping motion prediction model, the overlapping motion information is disassembled and corrected to obtain corrected motion information corresponding to each object appearing in the target site area; wherein the overlapping motion prediction model is trained based on training samples consisting of historical overlapping motion information and corresponding real motion information of each object; A virtual object restoring the real motion state is constructed in a virtual scene based on the corrected motion information.

2. The method according to claim 1, wherein: The step of determining the target site area where multiple objects overlap according to the image information includes: Determine, based on the image information, an area where there are objects exceeding a preset number, to obtain a multi-object site area; determining the distances between the objects in the multi-object site area according to the image information; The multi-object site area whose distance is less than the preset distance is determined as the target site area.

3. The method according to claim 2, wherein: The distance includes a minimum distance and an average distance, and the preset distance includes a preset minimum distance corresponding to the minimum distance and a preset average distance corresponding to the average distance; The step of determining the multi-object site area whose distance is less than the preset distance as the target site area includes: A multi-object site area in which the minimum distance is smaller than the preset minimum distance and / or the average distance is smaller than the preset average distance is determined as the target site area.

4. The method according to claim 1, wherein: The determining of the overlapping motion information corresponding to the target site area includes: Determining regional image information corresponding to the target site area; Determining overlapping skeleton points of overlapping objects contained in the target site area according to the regional image information; According to the regional image information, the motion information matching the overlapping skeleton points is determined to obtain the overlapping motion information.

5. The method according to claim 1, wherein: The method of using a preset overlapping motion prediction model to decompose and correct the overlapping motion information to obtain corrected motion information corresponding to each object appearing in the target site area includes: Using the disassembly and recognition module in the overlapping motion prediction model, each overlapping object corresponding to the overlapping motion information is identified, and independent motion information corresponding to each overlapping object is disassembled; wherein the disassembly and recognition module is trained based on a first training sample consisting of historical overlapping motion information and corresponding real overlapping objects; The abnormal motion correction module in the overlapping motion prediction model is used to correct the independent motion information of each overlapping object to obtain the corrected motion information corresponding to each overlapping object; wherein the abnormal motion correction module is trained based on a second training sample consisting of independent motion information disassembled from history and corresponding real motion information.

6. The method according to claim 5, wherein: The first training sample and the second training sample also record the identity information of each of the real overlapping objects.

7. The method according to claim 1, wherein: The step of constructing a virtual object restoring a real motion state based on the corrected motion information in a virtual scene includes: constructing an initial virtual object in the virtual scene; Rendering the motion information based on the same skeleton points between the corrected motion information and the initial virtual object to obtain an initial rendering result; The initial rendering result is adjusted in combination with the physiological characteristics of the corresponding object, the environmental interaction with the sports venue, and the motion interaction feedback between different objects to obtain a virtual object that restores the real motion state.

8. The method according to claim 1, further comprising: Determining other field areas in the sports field that are different from the target field area; determining single object motion information of objects appearing in the other venue area based on the image information; Processing the single object motion information using a preset motion prediction model to obtain processed motion information; wherein the motion prediction model is trained based on training samples consisting of historical single object motion information and real motion information; A virtual object restoring a real motion state is constructed in the virtual scene based on the processed motion information.

9. The method according to claim 8, wherein: The step of constructing a virtual object restoring a real motion state based on the processed motion information in the virtual scene includes: constructing an initial virtual object in the virtual scene; Rendering the motion information based on the same skeleton points between the corrected motion information and the initial virtual object to obtain an initial rendering result; The initial rendering result is adjusted in combination with the physiological characteristics of the corresponding object and the environmental interaction with the sports venue to obtain a virtual object that restores the real motion state.

10. The method according to claim 1, further comprising: Slicing the virtual motion data corresponding to the virtual scene and each of the virtual objects in time sequence to obtain slice data; Each of the slice data is compressed using a preset compression algorithm, and the compressed slice redundancy is distributed to different nodes of the content distribution network.

11. The method according to claim 10, wherein: The step of slicing the virtual motion data corresponding to the virtual scene and each virtual object in time sequence to obtain slicing data includes: Acquire virtual motion data corresponding to the virtual scene and each of the virtual objects; Sampling each data frame constituting the virtual motion data at different sampling frequencies to obtain sub-virtual motion data of different qualities; wherein the different sampling frequencies correspond to different performances of various types of devices capable of displaying the virtual motion data; Each of the sub-virtual motion data is sliced ​​according to a segmentation standard corresponding to the sampling frequency to obtain slice data corresponding to each of the sampling frequencies.

12. A method for watching sports based on virtual reality, comprising: Determine virtual motion data corresponding to the target motion according to viewing requirements; wherein the virtual motion data is motion data of a virtual object rendered in a virtual scene to restore a real motion state according to any one of claims 1 to 11; The virtual motion data is parsed into a virtual object that restores the real motion state in the virtual scene, and is presented in an image display area.

13. The method according to claim 12, wherein: The step of determining virtual motion data corresponding to the target motion according to the viewing requirement includes: Determining a target sampling frequency that matches the performance of a current viewing device according to the viewing demand; Acquire each slice data of the sub-virtual motion data corresponding to the target sampling frequency from each node constituting the content distribution network; Sub-virtual motion data of the target motion corresponding to the target sampling frequency is generated according to each of the slice data.

14. A motion information processing device based on virtual reality, comprising: An image information acquisition unit is configured to acquire image information of each object in the sports field through a plurality of image acquisition devices pre-set in the sports field; an overlapping motion information determining unit, configured to determine a target field area where multiple objects overlap according to the image information, and determine overlapping motion information corresponding to the target field area; wherein the overlapping motion information is formed by overlapping motion information of skeleton points of multiple objects appearing in the target field area; The motion information disassembly and correction unit is configured to use a preset overlapping motion prediction model to disassemble and correct the overlapping motion information to obtain corrected motion information corresponding to each object appearing in the target site area; wherein the overlapping motion prediction model is trained based on a training sample consisting of historical overlapping motion information and corresponding real motion information of each object; The virtual scene and virtual object reconstruction unit is configured to construct a virtual object that restores the real motion state based on the corrected motion information in the virtual scene.

15. The device according to claim 14, wherein: The overlapping motion information determining unit includes a target site area determining subunit configured to determine a target site area where multiple objects overlap according to the image information, and the target site area determining subunit includes: a multi-object site area determination module, configured to determine, based on the image information, an area where objects exceeding a preset number exist, to obtain a multi-object site area; a distance determination module, configured to determine the distance between the objects in the multi-object venue area according to the image information; The target site area determination module is configured to determine the multi-object site area whose distance is less than a preset distance as the target site area.

16. The device according to claim 15, wherein: The distance includes a minimum distance and an average distance, and the preset distance includes a preset minimum distance corresponding to the minimum distance and a preset average distance corresponding to the average distance; Wherein, the target site area determination module is further configured to: A multi-object site area in which the minimum distance is smaller than the preset minimum distance and / or the average distance is smaller than the preset average distance is determined as the target site area.

17. The device according to claim 14, wherein: The overlapping motion information determining unit comprises an overlapping motion information determining subunit configured to determine overlapping motion information corresponding to the target venue area, and the overlapping motion information determining subunit is further configured to: Determining regional image information corresponding to the target site area; Determining overlapping skeleton points of overlapping objects contained in the target site area according to the regional image information; According to the regional image information, the motion information matching the overlapping skeleton points is determined to obtain the overlapping motion information.

18. The device according to claim 14, wherein: The motion information disassembly and correction unit is further configured to: Using the disassembly and recognition module in the overlapping motion prediction model, each overlapping object corresponding to the overlapping motion information is identified, and independent motion information corresponding to each overlapping object is disassembled; wherein the disassembly and recognition module is trained based on a first training sample consisting of historical overlapping motion information and corresponding real overlapping objects; The abnormal motion correction module in the overlapping motion prediction model is used to correct the independent motion information of each overlapping object to obtain the corrected motion information corresponding to each overlapping object; wherein the abnormal motion correction module is trained based on a second training sample consisting of independent motion information disassembled from history and corresponding real motion information.

19. The device according to claim 18, wherein: The first training sample and the second training sample also record the identity information of each of the real overlapping objects.

20. The device according to claim 14, wherein: The virtual scene and virtual object reconstruction unit is further configured to: constructing an initial virtual object in the virtual scene; Rendering the motion information based on the same skeleton points between the corrected motion information and the initial virtual object to obtain an initial rendering result; The initial rendering result is adjusted in combination with the physiological characteristics of the corresponding object, the environmental interaction with the sports venue, and the motion interaction feedback between different objects to obtain a virtual object that restores the real motion state.

21. The apparatus of claim 14, further comprising: Another field area determination unit is configured to determine another field area in the sports field that is different from the target field area; a single object motion information determination unit configured to determine single object motion information of an object appearing in the other venue area based on the image information; A single object motion information processing unit is configured to process the single object motion information using a preset motion prediction model to obtain processed motion information; wherein the motion prediction model is trained based on training samples consisting of historical single object motion information and real motion information; A single virtual object reconstruction unit is configured to construct a virtual object that restores the real motion state based on the processed motion information in the virtual scene.

22. The device according to claim 21, wherein The single virtual object reconstruction unit is further configured to: constructing an initial virtual object in the virtual scene; Rendering the motion information based on the same skeleton points between the corrected motion information and the initial virtual object to obtain an initial rendering result; The initial rendering result is adjusted in combination with the physiological characteristics of the corresponding object and the environmental interaction with the sports venue to obtain a virtual object that restores the real motion state.

23. The apparatus of claim 14, further comprising: A data slicing unit is configured to slice the virtual motion data corresponding to the virtual scene and each of the virtual objects in time sequence to obtain slice data; The compression distribution unit is configured to compress each of the slice data using a preset compression algorithm, and distribute the compressed slice redundancy to different nodes of the content distribution network.

24. The device according to claim 23, wherein: The data slicing unit is further configured to: Acquire virtual motion data corresponding to the virtual scene and each of the virtual objects; Sampling each data frame constituting the virtual motion data at different sampling frequencies to obtain sub-virtual motion data of different qualities; wherein the different sampling frequencies correspond to different performances of various types of devices capable of displaying the virtual motion data; Each of the sub-virtual motion data is sliced ​​according to a segmentation standard corresponding to the sampling frequency to obtain slice data corresponding to each of the sampling frequencies.

25. A sports viewing device based on virtual reality, comprising: A virtual motion data acquisition unit is configured to determine virtual motion data corresponding to the target motion according to viewing requirements; wherein the virtual motion data is motion data of a virtual object rendered in a virtual scene to restore a real motion state according to any one of claims 14 to 24; The parsing and presenting unit is configured to parse the virtual motion data into a virtual object that restores the real motion state in the virtual scene, and present it in the image display area.

26. The device according to claim 25, wherein The virtual motion data acquisition unit is further configured to: Determining a target sampling frequency that matches the performance of a current viewing device according to the viewing demand; Acquire each fragment data of the sub-virtual motion data corresponding to the target sampling frequency from each node constituting the content distribution network; Sub-virtual motion data of the target motion corresponding to the target sampling frequency is generated according to each of the slice data.

27. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the virtual reality-based motion information processing method described in any one of claims 1-11 and / or the virtual reality-based motion viewing method described in claim 12 or 13.

28. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the motion information processing method based on virtual reality described in any one of claims 1 to 11 and / or the motion viewing method based on virtual reality described in claim 12 or 13.

29. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the motion information processing method based on virtual reality according to any one of claims 1 to 11 and / or the steps of the motion viewing method based on virtual reality according to claim 12 or 13.

Images