File formats with fluctuating data

Abstract

A method for storing data representing virtual objects in a computer storage system. The method includes storing unchanging data corresponding to physical properties of the virtual objects that are to remain unchanging when the data is read. The unchanging data includes one or more unchanging elements representing one or more physical properties of the virtual objects. The method further includes storing varying data corresponding to physical properties of the virtual objects that are uncertain at the time of storing the data. The varying data includes one or more varying elements representing one or more uncertain physical properties of the virtual objects, each of the varying elements including a range of values ​​and a probability function for the range of values.

Description

Technical Field

[0001] The present invention relates to the field of storing data representing virtual objects in a computer memory system. The present invention further relates to methods of modifying and displaying such data.

Background Art

[0002] A file format is a standard way in which information is encoded for storage in a computer file. A file format specifies how bits are used to encode information in a digital storage medium.

[0003] Currently, many file formats exist. Exemplary 3D file formats are OBJ, Standard Triangle Language (STL), Filmbox (FBX), Collaborative Design Activity (COLLADA), and Graphics Language Transmission Format (glTF). These file formats are typically used to export a 3D scene description from a computer program so that the 3D scene description can be read by another computer program and rendered. Some known authoring tools can read and describe many different 3D graphic formats. Graphic file formats typically support the representation of 3D shapes, physical effects (light sources), shading effects, morphing, and animations.

[0004] The field of 3D computer vision attempts to infer object shape, physical properties, and semantic information from one or more camera images. Examples are depth estimation results from stereo camera images via disparity estimation results, and face detection via deep neural networks using exemplary images.

Summary of the Invention

Problems to be Solved by the Invention

[0005] It is desirable that the receiving computer program interprets and renders the file format using the most efficient method possible.

[0006] WO2018 / 109499A1 discloses a method for detecting objects located in the environment surrounding a vehicle by using particle filtering.

[0007] U.S. Patent No. 9406131(B2) discloses a method for generating a 3D representation of a dynamically changing 3D scene by acquiring two synchronized video streams, tracking the movement of an object in the two video streams, and identifying the object's location in the video streams, wherein the step of tracking the object's movement uses the location information at an earlier point in time. [Means for solving the problem]

[0008] The present invention is defined by the claims.

[0009] According to one aspect of the present invention, a method for storing data representing a virtual object in a computer storage system, wherein the method is A step of storing immutable data corresponding to the physical properties of a virtual object that will remain unchanged when the data is read, wherein the immutable data includes one or more immutable elements representing one or more physical properties of the virtual object, A method is provided for storing fluctuating data corresponding to the physical characteristics of a virtual object that are uncertain at the time the data is stored, wherein the fluctuating data includes one or more fluctuating elements representing the uncertain physical characteristics of one or more virtual objects, and each of the fluctuating elements includes a range of values ​​and a probability function for that range of values.

[0010] The representation of virtual objects in computer memory systems typically depends on an object designer that provides the object's physical properties, such as geometric shape, color, and texture. These physical properties are represented in different forms depending on the file format in which the data is stored. For example, in a given format, the object's color may be defined by a number between 0 and 1 for each of the colors green, red, and blue. Therefore, in this example, the object's color is represented by three numbers.

[0011] If an object's color is unknown at the time the data is stored, the object is stored without a color. However, in many cases, the designer has an intention about what the color should be.

[0012] In a simplified example, the designer wants to design a hypothetical tree, including a brown tree body and leaves, representing the current time of year. The leaves are green in spring and summer, and brown in autumn. Therefore, the leaf color can be remembered as [green, brown], and the designer can change the leaf color based on the time of year.

[0013] In the example above, the color of the tree's body, its shape, and the shape of its leaves are invariant elements in invariant data, as the designer does not want these properties to change for different points in time throughout the year. However, the leaf color is a variable element in variable data, as color is uncertain without other information (e.g., time of year, sensor data, etc.). The range of values ​​for color is green or brown, and the conditional probability function can be set by associating the data with a calendar. Furthermore, the number of leaves can also be a variable element (e.g., fewer leaves in winter), and therefore, the leaf color and number of leaves are variable elements in variable data. The probability function can be based on more than one conditional probability density function (e.g., outside temperature and calendar).

[0014] The range of values ​​and probability functions provide the data reader with prior knowledge about the object, enabling faster decision-making. In the example above, a human was the data reader and could determine the actual color of the leaf based on their own observations. However, in many cases, the data reader is a computer program. Prior knowledge given to a computer program can significantly improve efficiency in determining the color of a leaf. For example, instead of measuring the actual color of the leaf, a computer program could use information from sensors to determine whether the color is green or brown.

[0015] While using so-called "prior information with no informational value" regarding leaf color, if a low outside temperature (which is unlikely in summer) is observed, the inductive estimator will determine that the leaf is actually brown (for example, using Bayes' theorem and a likelihood function that correlates temperature with leaf color).

[0016] Therefore, this method of storing data allows future readers of the data to quickly and efficiently adapt fluctuating data to actual values.

[0017] The range of values ​​is set by the user (e.g., an operator for broadcasting a sporting event setting possible uniform colors for players). Alternatively, the range of values ​​is the maximum possible range within a scene, or even a random variable in an infinite domain (e.g., [-infinity, infinity]). Thus, the range of values ​​is a defined, limited range, or the range of values ​​is an infinite range. Explicit memory of the range of values ​​(and probability functions) is of particular interest in making it simple and intuitive for future changes.

[0018] The data representing an object corresponds to the 3D scene description.

[0019] The probability function for a fluctuating element is: Gaussian distribution, Poisson distribution, Two or more delta functions, a discrete probability distribution, a continuous probability distribution, or a conditional probability distribution, is one or more of the foregoing.

[0020] The probability function used for a particular varying element depends on the physical property and / or the nature of the physical property represented by the varying element. For the following example, the varying element represents the position of an object.

[0021] Consider a finite plane that includes a virtual object located somewhere on the plane. The position of the object on the plane can be random. In this case, the range of values for the position is all of the plane, and the probability function gives an equal probability of landing anywhere on the plane. Alternatively, the position of the object can be more likely to be at a particular point on the plane. In this case, a Gaussian distribution with an average value at the particular point can be used as the probability function (thus, the standard deviation depends on how "preferred" the object is to be close to the average value, but is not important as an example).

[0022] In another example, the object is located only at two particular points on the plane. In this case, the probability function can be the sum of two delta functions for the two particular points.

[0023] Another example is the positions of two goalkeepers during a soccer game. These positions are more likely to be near the goal and less likely to be at the center of the field. Such information can enable the introduction of context into a file format and thus may be useful for computer vision tasks.

[0024] The range of values is based on a range defined by the user for each particular varying element, and the probability function is also based on a function defined by the user for each particular variable.

[0025] The range of values can be selected by the user. In some examples, when little is known about a particular physical characteristic (e.g., the number of people in a particular street), the range of values is wide. However, in other cases, the range of values is small (e.g., the aforementioned example involving the color of leaves). The range of values can further be a continuous range or a discrete range.

[0026] Similarly, the probability function can be selected by the user for a physical characteristic based on the nature of the physical characteristic.

[0027] The present invention is a method for controlling a sensor system based on data stored by a method of storing data representing virtual objects in a computer storage system, the method comprising: controlling the sensor system to search for an actual object corresponding to a virtual object represented in the varying data based on the range of values and / or probability function of the varying elements in the varying data; obtaining sensor data from the sensor system representing the physical characteristics of the actual object corresponding to the varying elements in the varying data; further providing a method having.

[0028] The sensor system can be configured to search for an actual object corresponding to the virtual object. For example, in a sports game, the sensor system searches for players on the field. The sports field can be invariant data, and thus the sensor system need not measure the physical characteristics of the field. The number of players and the color of the players (i.e., the team) can be varying elements of the varying data, and thus the sensor system can be configured to search for these varying elements. The range of values and probability function of the varying elements give the sensor system a "starting point" of where to search for players and / or what colors to search for in the players.

[0029] Therefore, the sensor system only needs to acquire sensor data for physical properties corresponding to fluctuating elements, and can ignore all data regarding constant data. Furthermore, the sensor system has some prior knowledge about what to "look for," and thus the acquisition of sensor data becomes faster and more efficient.

[0030] The present invention relates to a method for displaying data stored by a method for storing data representing virtual objects in a computer storage system, wherein the method is A step of controlling the sensor system to acquire sensor data, A step of converting one or more fluctuating elements into transient elements based on corresponding sensor data for each of the fluctuating elements, wherein the transient elements are physical properties known at a specific point in time; A step of displaying a virtual object based on immutable elements and transient elements of the data, The present invention further provides a method having the following characteristics.

[0031] After sensor data is acquired for a variable element, the variable element may be converted into a transient element. For example, if the sensor system detects that the color of a virtual object is red or blue [red, blue], and the actual object (corresponding to the virtual object) is red, the variable element representing the color is temporarily converted into a transient element with the value [red]. The transient element is then displayed next to the immutable element of the data.

[0032] The sensor system is configured to periodically update the sensor data, and the steps for displaying the data are repeated each time the sensor data is updated.

[0033] For example, the sensor system could be a video camera that captures video of the position of a real object. Thus, each frame of the video could be used to update the position of a virtual object. Alternatively, the position of the real object could be captured once per second (or at any other arbitrary interval), and the virtual object could be updated accordingly.

[0034] The range and / or probability distribution of the first fluctuating element is: The previous value of the first fluctuating element, Other fluctuating element values, or Previous values ​​of other fluctuating elements, Based at least partially on one or more of the following.

[0035] In some examples, any particular value of a variable element depends on a previous value. For instance, the position of an object at a particular point in time might depend on its position at a second previous point in time.

[0036] Alternatively, the position of an object depends on other physical properties of the same object (e.g., size, orientation, etc.) or on the physical properties of other objects. For example, if a group of objects is typically observed together, it is likely that any one of the objects in the group will be observed near other objects.

[0037] The sensor data is Visual sensor data, Infrared sensor data, Microwave sensor data, Ultrasonic sensor data, Audio sensor data, Location sensor data, Accelerometer sensor data, or Global Positioning System data, Includes one or more of the following.

[0038] Magnetic sensor data will also be used.

[0039] The present invention is a method for modifying data representing a virtual object stored by the method defined above (for storing data representing a virtual object in a computer storage system), wherein the method is A step of acquiring historical sensor data logs from a sensor system, including the physical characteristics of the actual object corresponding to the fluctuating elements of the data representing the virtual object, and their respective changes over time, A step of changing the range of values ​​and / or the probability function of a fluctuating element based on historical data of the range of values ​​and / or the probability function and the corresponding historical sensor data log, The present invention further provides a method having the following characteristics.

[0040] Historical data from a sensor system (historical sensor data log) can provide useful information about the range of values ​​and / or what the probability function should be. For example, the probability function of a fluctuating element indicates that any particular value is equally likely. However, the sensor system observes that the fluctuating element has certain values ​​that it "prefers". Therefore, the probability function can be adapted to have a Gaussian distribution, and the mean and standard deviation can be derived from the historical sensor data log.

[0041] Modifying the value range and / or probability function is based on inputting the historical data and the corresponding historical sensor data log into a machine learning algorithm trained to identify patterns between the historical data and the historical sensor data log, and to output the value range and / or probability function for the corresponding fluctuating element.

[0042] This method further involves modifying the range of values ​​and / or the probability function based on calculating the maximum inductive estimation result of the historical sensor data log.

[0043] The present invention further provides a computer program product that includes computer program code means, when executed on a computing device including a processing system, causes the processing system to perform all of the steps of the method defined above (storing data representing virtual objects in a computer storage system).

[0044] The present invention further provides a processor configured to perform all the steps of the method defined above (for storing data representing virtual objects in a computer memory system).

[0045] The present invention is a system for displaying virtual objects, wherein the system is Processor and A computer memory system configured to store data representing virtual objects, A sensor system for acquiring sensor data representing actual objects, A display for showing virtual objects, We will further provide a system that includes the following features.

[0046] These and other aspects of the present invention will become apparent from the embodiments described below and will be described with reference to the embodiments described below.

[0047] To better understand the present invention and to more clearly illustrate how it is realized, the accompanying drawings are referenced below merely as examples. [Brief explanation of the drawing]

[0048] [Figure 1] Figure 1 shows an exemplary 3D scene. [Figure 2] Figure 2 shows an example of a file format that includes uncertainty information. [Figure 3] Figure 3 shows the visualization of the probability function on the field for a virtual player. [Figure 4] Figure 4 shows two different camera views of the actual scene and the corresponding virtual scene. [Modes for carrying out the invention]

[0049] The present invention will be described with reference to the figures.

[0050] Detailed descriptions and specific examples illustrate exemplary embodiments of the apparatus, systems, and methods, but it should be understood that they are for illustrative purposes only and not intended to limit the scope of the invention. These and other features, aspects, and advantages of the apparatus, systems, and methods of the invention will be better understood from the following description, the appended claims, and the accompanying drawings. It should be understood that the figures are schematic diagrams only and are not drawn to a specific scale. It should also be understood that the same reference numerals are used across multiple figures to indicate identical or similar parts.

[0051] The present invention provides a method for storing data representing virtual objects in a computer storage system. The method includes the step of storing immutable data corresponding to the physical properties of the virtual objects that remain unchanged when the data is read. The immutable data includes one or more immutable elements that represent one or more physical properties of the virtual objects. The method further includes the step of storing variable data corresponding to the physical properties of the virtual objects that are uncertain at the time the data is stored. The variable data includes one or more variable elements that represent one or more uncertain physical properties of the virtual objects, each of which includes a range of values ​​and a probability function for that range of values.

[0052] Figure 1 shows an exemplary 3D scene. The scene corresponds to a virtual soccer field containing field 102, background 104, and five players 106. Current 3D file formats cannot represent uncertainty about the 3D scene being imaged by a given (multi) camera setup (e.g., the movement of players 106 near field 102). Current 3D file formats specify only one possible rendering of a static scene, consisting of optionally moving (animated) parts.

[0053] The inventors have realized that a 3D file format capable of representing uncertainty about a 3D scene is useful because it allows prior knowledge information to be linked to measured sensor data in order to adapt the 3D scene to real-time measurements performed by a camera or other sensor. Therefore, the present invention relates to a method for storing, modifying, and displaying such a file format.

[0054] Figure 2 shows an example of a file format that includes uncertainty information. The file format includes deterministic data 202 (i.e., immutable data) that contains 206 elements that are certain to the user storing the data. For example, the size of field 102 shown in Figure 1 is an immutable element 206 because it is unlikely that the area of ​​field 102 will be measured as different from the current area.

[0055] The file format also includes variable data 204, which contains uncertainty in the 3D scene. Each of the variable elements 208 in the variable data 204 includes a range of values ​​210 and a probability function 212 corresponding to the possibilities of values ​​within the range 210.

[0056] A file format containing immutable data 202 and variable data 204 for player 106 can be illustrated, for example, in the following pseudocode.

number

[0057] The above file uses a custom XML format to describe the position of virtual player 106 in the soccer scene in Figure 1. Since it is unlikely that player 106's z position will change on the field 102 (ignoring jumps), the z position of virtual player 106 remains constant at 0.4. However, since virtual player 106 moves around the field 102, the x and y positions of virtual player 106 change. Therefore, a range of possible movement 210 is defined for each coordinate axis, and a probability function 212 is given. In this example, the x and y axes are variable elements 208 containing the corresponding ranges 210 and probability functions 212, while the z position is a constant element 206.

[0058] In this case, the probability function 212 is given as "random" for both the x and y axes. This is, for example, a starting point for the fluctuating element 208. Some players on field 102 may take positions that are largely random across field 102 (for example, midfielders tend to move across field 102), while other players have a higher probability of being in specific locations (for example, the goalkeeper is much more likely to be observed near the goal). Thus, the probability function 212 can be adapted to different players throughout the game (or from historical data).

[0059] Furthermore, the number of virtual players 106 on field 102 may be further specified.

number

[0060] By specifying that the fluctuating number of virtual players 106 are within the range 210 [0,22] using the probability function 212 "function_1", a computer program reading the data is informed that this information is uncertain. Any probability function 212 "function_1" is continuously adjusted based on the actual number of players. For example, there is a high probability that there are 22 players on the field 102 at the start of the match. However, based on the fact that any one of the actual players receives a red card, function_1 changes to have a higher probability for 21 players. Thus, the probability function 212 "function_1" can be coupled to a sensor (e.g., a camera) that can detect the use of a red card and adapt accordingly.

[0061] Other deterministic (invariant) elements 206 and variable (e.g., statistical) elements 208 can also be defined for the soccer field. For example, the position of the field 102 is deterministic, but the plane is subject to statistical rotations between -95 and -85 degrees around the x-axis and between -5 and 5 degrees around the z-axis.

[0062] It should be noted that the pseudocode uses a custom-formulated XML format to describe the scene. However, the concept can be added to the specifications of many existing 3D file formats. For example, applications capable of encoding / decoding 3D file formats, such as Blender or Unity, can simply choose to ignore the variable data 204 of the 3D graphics file, as is normally done from immutable data 202, and simply render / animate the scene.

[0063] The proposed file format is relevant to a new application. For example, consider a scenario in a (real) live sports game where each player wears a position sensor (e.g., GPS). During the live sports game, viewers of the match may receive a graphics rendering of the sports game, including the graphics appearance (shape, texture, etc.) as specified by the 3D file format, but the position of the virtual player 106 is adapted to the measured position of the corresponding real player on the playing field 102. Thus, viewers of the match see a mixture of the artificial graphics rendering of the virtual players 106 and their actual live positions.

[0064] Just like humans, computers benefit from prior knowledge of a scene, even when that prior knowledge is highly uncertain. For example, the knowledge that a camera is pointing to the player along a horizontal field 102 is currently used to estimate the pixel depth in multiple camera images. Without such an assumption, the depth estimation results could contain many errors.

[0065] The installation of multiple machine vision cameras near the stadium could add live textures to the virtual players 106 on the player field 102. This is a computer vision problem that typically requires machine learning-based person detection and pose estimation. However, deductively variable data 204 in the 3D graphics file could predict the expected colors of the actual players' clothing (socks, shirts, etc.). This could help computer vision applications observe the actual players and use the camera image data to correctly texture the virtual players 106.

[0066] While sensor data can adjust the variable elements 208, deterministic elements 206 are rendered as they were initially stored. Therefore, it is possible to deterministically set the graphical appearance of the soccer field, stadium, and light source, but to allow the virtual player 106 the flexibility to adapt to sensor data using a single 3D file format description.

[0067] Therefore, a computer vision search algorithm knows, based on a file format containing uncertain data, that it only needs to search for the orientation of a plane, a specified rotation, within a specified interval, without needing to search for the plane's position. Furthermore, based on the probability function 212, the algorithm may begin searching within that interval using the highest probability. Then, the probability function 212 may be updated based on historical data and / or current data.

[0068] Figure 3 shows the visualization of the probability function 212 on field 102 for a virtual player 106. In this case, the virtual player 106 corresponds to the goalkeeper. Figure 3(a) shows the first area 302 with equal probabilities for the goalkeeper. Figure 3(b) shows the second area 302b with probability gradients. The sizes of areas 302a and 302b correspond to the range 210 of positions for the goalkeeper, the gray shading corresponds to probabilities, and darker gray shading at any given point corresponds to a higher probability of observing the goalkeeper at that point.

[0069] At the start of the match, the goalkeeper's position is unknown, and therefore the probability can be defined as random (within range 210), as shown in Figure 3(a). However, during the match, the goalkeeper is most frequently observed at the rear of field 102 near the goal, and the probability gradually decreases as the distance from the goal increases. This can be quantified as probability function 212 corresponding to area 302b. Thus, probability function 212 evolves throughout the match.

[0070] Alternatively, probability function 212 could use historical data of the same player (or player of the same type) to predict probability function 212 before the match. Since the actual player is likely to be observed near the actual player's position in the previous frame, probability function 212 could further be conditional on the previous position of the virtual player 106.

[0071] Figure 4 shows two different camera views 402 of the actual scene and the corresponding virtual scene 403. Current computer vision search algorithms may use variable elements 208 in a 3D graphics file to fit the horizontal plane (i.e., field) 102 and the vertical background 104 based on at least two camera views 402.

[0072] Over a fixed number of iterations, the search algorithm evaluates small updates to the position and rotation of a geometric object, and in addition, evaluates pseudo-random values ​​at specified intervals. The search can begin during the interval. In some examples using a non-uniform (i.e., non-random) probability function 212, the search can begin with the value with the highest probability. This approach is applicable to both geometric parameters (position, orientation) and color / texture parameters.

[0073] An error scale is calculated after each proposed parameter change. The scene is first rendered to each viewpoint. The result is a predicted image and depth map for each viewpoint based on the graphic model. Given a depth map for each view, the error scale is calculated via the absolute difference in different colors between views, thereby using the disparity vector corresponding to the depth map. Arrows 404 in Figure 4 are exemplary corresponding pixels determined by the model state between camera views 402. Error minimization can be performed in real time, for example on a GPU, for output video at least 20 FPS.

[0074] The total error for a given parameter setting (plane position and orientation) is calculated by summing the color differences of the corresponding pixels (from arrow 404) in both views. The plane position and orientation identify the depth model in each camera view, which in turn determines the disparity vector field between the two views. The disparity vector field is used to calculate the color difference between the views, and one error is obtained after summing the absolute color differences across all pixels. This single (summed) total error is minimized by iteratively changing the plane parameters from the initial starting settings, always accepting proposed changes that reduce the error. For example, Powell's orientation set method may be used.

[0075] The rotation of field 102 around the x-axis 408 and the y-axis 406 in the virtual scene 403 can be estimated from the camera view 402. The range 210 and probability function 212 defined in the file format for field 102 provide the computer vision algorithm with prior knowledge information about which rotations to "look for". Similarly, the position of background 104 on field 102 (indicated by arrow 410) can also change. For example, the angle of background 104 relative to field 102 is fixed (e.g., 90 degrees) and is therefore an invariant element 206, while the position of background 104 on field 102 is a variable element 208.

[0076] There are various probability functions that can be used, and it may be useful to specify a particular probability density function. For example, the following:

number

[0077] This pseudocode represents that the x-coordinate of a geometric primitive follows a Gaussian distribution with a mean of 100.0m and a standard deviation of 10.0m. A Poisson distribution may be specified for the number of virtual objects (e.g., virtual players 106). To make the formal specification consistent by using a data 204 format that varies for all data, a delta function may be used to define an essentially deterministic and invariant element 206.

[0078] The use of variable elements 208 extends further to textures, light maps, shadow maps, and other elements, which are typically part of 3D graphics file formats. Therefore, the parameters of the probability function 212 can be specified as maps instead of scalar numbers. Combinations are also possible, namely, the mean of a Gaussian distribution may be represented by a 2D map, while the standard deviation is represented by a scalar number.

[0079] The 3D file format may further specify a conditional probability density function. This relates, for example, to animations where the position of a virtual player's feet at a given time point may conditionally depend on the position of their feet at an earlier time point. The probability density function may further be specified as a discrete distribution via a histogram.

[0080] Machine learning algorithms can be used to fit the parameters of the distribution based on sensor data and to output a better-fitting range 210 and / or probability function 212 for the next live event. Bayes' theorem can further be used to compute a maximum inductive (MAP) estimate of the scene model given the sensor data. This estimation result can then be written to a file.

[0081] Monte Carlo simulations can be used to sample from a given distribution. For example, Markov chain Monte Carlo (MCMC) can be used as part of a computer vision algorithm to search for the maximum inductive estimate of a real player's position.

[0082] The present invention is not limited to 3D scenes and can also be used in 2D scenes (or other dimensions). For example, the aforementioned soccer field is simply a 2D scene including virtual players 106 as points on a 2D field.

[0083] Figure 5 illustrates a method for storing data representing virtual objects in a computer storage system. This method includes, in step 502, storing immutable data in non-volatile computer storage. The immutable data corresponds to the physical characteristics of the virtual object that remain unchanged when the data is read. The immutable data includes one or more immutable elements representing one or more physical characteristics of the virtual object.

[0084] The method shown in Figure 5 further comprises, in step 504, storing fluctuating data in non-volatile computer storage. The fluctuating data corresponds to the uncertain physical properties of a virtual object at the time the data is stored. The fluctuating data includes one or more fluctuating elements representing one or more uncertain physical properties of the virtual object. Each fluctuating element includes a range of values ​​and a probability function for that range of values.

[0085] Step 502, which involves storing immutable data, is shown to be performed before step 504 (i.e., storing variable data). However, it will be understood by those skilled in the art that step 504 may be performed before step 502, or that both steps may be performed simultaneously.

[0086] Data representing virtual objects is called computer graphics data or graphics data for computer-generated objects.

[0087] Figure 6 shows a system used to store data representing virtual objects. Immutable data 202 and variable data 204 are stored in non-volatile computer storage 604 by processor 602. Processor 602 is configured to store both immutable data 202 and variable data in non-volatile computer storage 604.

[0088] Non-volatile computer storage 604 is also called non-volatile computer memory or non-volatile storage. Non-volatile computer storage 604 is a type of computer storage solution that can retain stored data and information after power has been removed from the physical computer storage. In contrast, volatile memory (e.g., random-access memory (RAM)) requires a certain amount of power to retain data or information, and the data is permanently lost after power is removed from the volatile memory.

[0089] Volatile memory is typically much faster (read / write speed) than non-volatile memory, and therefore operations can be performed more quickly. Data or information required for an operation is typically stored in volatile memory. When the data or information is no longer needed after the operation, it is removed from non-volatile memory.

[0090] In contrast, it is understood that storing immutable and variable data for virtual objects in non-volatile memory allows future users of the data to construct a partial image of the virtual object and to have deductive information about what the values ​​for the variable data are. This allows the user to control a sensor construct to observe the actual values ​​of the variable data. After the actual values ​​are observed, they are stored as temporary elements in volatile memory.

[0091] For example, the data represents a soccer field. The size of the field, the color of the field, the position and size of the goalposts, and the color of the goals are immutable data because they can be assumed to remain constant regardless of which team is playing. The number of players and the colors of the players' uniforms are variable data because they change from week to week as different teams play. When a match is being played, the actual colors and number of players can be measured, acquired, and stored as temporary elements for that match. Similarly, when a match is being played, the number of players can be continuously monitored and adapted. Since the data representing the soccer field is stored as non-volatile data, the data representing the soccer field can be used again for different matches (e.g., at different points in time) without requiring any adaptation after a previous match. Adaptations to the range of values ​​or probability functions of variable data are made based on previous matches.

[0092] Immutable and variable data are clearly distinguished in the file format (e.g., by a data type indicator or label) so that the application or processor reading the data can quickly differentiate between the two types of data.

[0093] This method comprises the step of encoding data representing a virtual object according to an instruction set, the instruction set comprising encoding immutable data for storage in non-volatile computer storage and encoding variable data for storage in non-volatile computer storage.

[0094] Modifications of the disclosed embodiments can be understood and realized by those skilled in the art who practice the claimed inventions, based on the drawings, this disclosure, and the accompanying claims. In the claims, the expression “including, having, possessing” does not exclude other elements or steps, and the expression corresponding to a singular element does not exclude a plural.

[0095] One processor or other unit may implement some of the functions described in the claims.

[0096] The mere fact that specific means are described in different dependent claims does not indicate that combinations of these means cannot be used to obtain advantages.

[0097] Computer programs may be stored in or distributed on suitable media such as optical storage media or solid-state media supplied together with or as part of other hardware, but may also be distributed in other forms, for example, via the Internet or other wired or wireless telecommunication systems.

[0098] When the phrase "adapted to" is used in the claims or description, it should be noted that the phrase "adapted to" is intended to be equivalent to the term "configured to."

[0099] No reference numeral in the claims shall be construed to limit the scope of the claims.

Claims

1. A method for storing data representing a virtual object in a computer memory system, wherein the method is A step of storing immutable data corresponding to the physical characteristics of the virtual object in non-volatile computer storage, wherein the immutable data includes one or more deterministic elements representing the physical characteristics of one or more of the virtual objects; A step of storing fluctuating data in non-volatile computer storage that corresponds to uncertain physical characteristics of the virtual object at the time the data is stored, wherein the fluctuating data includes one or more fluctuating elements representing uncertain physical characteristics of one or more of the virtual objects, and each of the fluctuating elements includes a range of values ​​and a probability function for the range of values. A method having

2. The method according to claim 1, wherein the data representing the virtual object corresponds to a 3D scene description.

3. The probability function for the aforementioned fluctuating element is, Gaussian distribution, Poisson distribution, Dirac delta function, Discrete probability distribution, Continuous probability distribution, or Conditional probability distribution, One or more of the following: The method according to claim 1 or claim 2.

4. The method according to any one of claims 1 to 3, wherein the range of the value is based on a range defined by the user for each of the specific fluctuating elements, and the probability function is further based on a function defined by the user for each of the specific fluctuating elements.

5. A method for controlling a sensor system based on data representing a virtual object, wherein the data includes immutable data and variable data, A step of reading data representing a virtual object from non-volatile computer storage, wherein the data is stored using the method described in any one of claims 1 to 4; A step of controlling the sensor system in order to search for an actual object in the scene that corresponds to the virtual object represented in the fluctuating data, based on the range of values ​​and / or probability function of the fluctuating elements in the fluctuating data; A step of acquiring sensor data from the sensor system that represents the physical characteristics of the actual object corresponding to the fluctuating elements in the fluctuating data, A method having

6. A step of converting one or more of the fluctuating elements into temporary elements based on corresponding sensor data for one or more of the fluctuating elements, wherein the temporary elements represent the physical properties of one or more of the virtual objects; A step of displaying the virtual object based on the deterministic elements and the transient elements of the data, The method according to claim 5, further comprising:

7. The sensor system periodically updates the sensor data, and the steps of the method according to claim 6 are repeated each time the sensor data is updated. The method according to claim 6.

8. The range of values ​​and / or probability distribution of the first fluctuating element is The previous value of the first variable element, Other fluctuating element values, or The previous values ​​of the other fluctuating elements mentioned above, Based at least partially on one or more of the following: The method according to claim 7.

9. The aforementioned sensor data, Visual sensor data, Infrared sensor data, Microwave sensor data, Ultrasonic sensor data, Audio sensor data, Location sensor data, Accelerometer sensor data, or Global Positioning System data, Including one or more of the following: The method according to any one of claims 5 to 8.

10. A method for modifying data representing a virtual object stored by the method of any one of claims 1 to 4, wherein the method is A step of acquiring historical sensor data logs from a sensor system, including the physical characteristics of an actual object corresponding to a fluctuating element of the data representing the virtual object, and the changes over time of each. A step of changing the range of values ​​and / or the probability function of the fluctuating element based on the historical data of the range of values ​​and / or the probability function and the historical sensor data log corresponding thereto. A method having

11. The method according to claim 10, wherein the step of changing the range of values ​​and / or the probability function is based on inputting the historical data and the corresponding historical sensor data log to a machine learning algorithm trained to identify a pattern between the historical data and the historical sensor data log and to output a range of values ​​and / or a probability function for the corresponding fluctuating element.

12. The method according to claim 10 or 11, further comprising the step of modifying the range of the values ​​and / or the probability function based on calculating the maximum inductive estimation result of the historical sensor data log.

13. A computer program including computer program code means that, when executed on a computing device including a processing system, causes the processing system to perform all the steps of the method according to any one of claims 1 to 12.

14. A system for storing data representing virtual objects in a computer memory system, wherein the system The method involves storing immutable data corresponding to the physical characteristics of the virtual object in non-volatile computer storage, wherein the immutable data includes one or more deterministic elements representing the physical characteristics of one or more of the virtual objects. The storage of fluctuating data in non-volatile computer storage that corresponds to uncertain physical characteristics of the virtual object at the time the data is stored, wherein the fluctuating data includes one or more fluctuating elements representing one or more uncertain physical characteristics of the virtual object, and each of the fluctuating elements includes a range of values ​​and a probability function for the range of values. Equipped with a processor that does the following: system.

Images