METHOD FOR CALCULATING THE VISIBILITY OF OBJECTS IN A 3D SCENE

Description

DOMAINE TECHNIQUE DE L'INVENTION

[0001] The technical field of the invention is that of three-dimensional rendering or 3D rendering, or 2D for two dimensions, and of the detection of visibility of objects in applications using graphics rendering engines such as video games, virtual worlds, virtual reality, augmented reality, etc.

[0002] The present invention relates to a method for calculating the visibility of objects within a three-dimensional scene, also called a 3D scene. ARRIERE-PLAN TECHNOLOGIQUE DE L'INVENTION

[0003] In the field of technology, graphics rendering engines such as Unity, Unreal and others are commonly used to generate and display a video stream giving a virtual and real-time representation of a scene composed of 3D objects.

[0004] The images that make up this video stream are two-dimensional images (or raster images) also called "renderings".

[0005] Each rendering is generated and displayed (also called "drawn") on a rendering plane called a "viewport" and from a predefined viewing angle called a "camera." This is achieved using combined algorithms within a graphics pipeline, or "render pipeline," used by the graphics rendering engine and integrated into a graphics card.

[0006] The graphics pipeline uses as input data the representation in a coordinate space (x,y,z) of the 3D objects that make up the scene to be represented.

[0007] Based on this input data, and with reference to the figure 1 , each object 100 is projected onto a viewport 110 from the viewpoint of a camera 120, also defined in the coordinate space (x,y,z).

[0008] Each rendering reproduces both the 3D projection, the textures (surface appearance), and the lighting effects (shadows, reflection, etc.) of object 100.

[0009] Objects, as seen by the end user (or viewer), are for example everyday objects, buildings, vehicles whose texture and lighting are realistic, that is to say, whose appearance is close to a real appearance.

[0010] The 120 camera determines the viewing angle from which the scene is drawn. The position of this 120 camera can change during the video stream. Therefore, the viewpoint used to draw the scene can change from one frame to the next. As a result, some objects may be visible in some frames, partially visible, or completely invisible in others.

[0011] For example, an object can be seen (and thus be visible) by camera 120 when no other object is interposed between object 100 and camera 120, and when the object is sufficiently illuminated. It can be partially visible when a semi-transparent object is interposed between it and the camera, or when it is partially located in a shadow. Finally, it can be invisible when it is hidden behind another object, or located in an unlit area.

[0012] The visibility of an object in the scene is therefore not fixed during the video stream. It is also not easy to determine it a priori because the movements of the 120 camera are generally unpredictable. However, for certain applications, there is a need to control and monitor this visibility in real time. Knowing this visibility allows, for example, precise memory management. Indeed, if an object is not or is no longer visible, the texture associated with that object can be either not loaded into, or unloaded from, memory. Since textures are the elements that take up the most space in memory, the technical advantage of such a visibility calculation method is clear.

[0013] Several solutions exist for this, which consist of representing the visibility of an object by a so-called visibility score associated with the object.

[0014] One solution involves using the so-called zbuffer technique. This technique determines the visibility of an object at the moment it is drawn. During the generation of the final render, other objects may be drawn on top of it, which can obscure the object whose visibility was determined, leading to an inaccurate visibility calculation.

[0015] A second solution involves using ray tracing, or "raycasting." This technique uses a collider, or collision detector, typically used by the physics engine to detect collisions between objects. The object whose visibility we want to calculate must therefore be associated with such a collider. In addition to this constraint, the technique cannot be applied when the object is behind another object, even a semi-transparent one, or in a shadow.

[0016] A third solution is described in patent FR3112638B1. It consists of using a stencil buffer in a second rendering process intended solely for visibility calculations, and therefore not displayed. This second rendering is generated in parallel with the main rendering (the one that renders the scene from a primary camera and is displayed to the user) by a second camera that constantly tracks the position and characteristics of the primary camera. Specifically, the stencil buffer is used in the pipeline that generates the second rendering, after a step of drawing the opaque objects in the scene. The graphics pipeline is necessarily "forward," meaning that it draws the opaque and transparent objects of the scene in that order.

[0017] This technique is very precise, meaning it takes into account the lighting and shadows of the scene. For example, it can calculate the visibility of an object even if it is partially obscured by another object or located in a shadow. However, it requires access to a stencil buffer, and therefore appropriate computing resources, and is not compatible with graphics pipelines other than forward-looking ones. For example, it cannot be implemented in a deferred-looking pipeline using a different rendering strategy, in which the drawing order is different.

[0018] Therefore, there is a need for a method of calculating object visibility scores that is compatible with all graphics pipeline architectures, whether software or hardware-based. Such a solution must also be able to adapt to available computing and energy resources. RESUME DE L'INVENTION

[0019] The invention is defined by the attached independent claim and offers a solution to the problems mentioned above, by allowing the visibility score to be calculated not from a second intermediate rendering corresponding to the rendering of opaque objects, but from a second final rendering.

[0020] For this, we use a color comparison technique deployed before and after the generation of the second rendering, and allowing us to cover cases where the object whose visibility we want to check has been partially masked by a semi-transparent object, or is made barely visible due to surrounding light that is too weak or too intense, or due to reflections on its surface.

[0021] One aspect of the invention relates to a method for calculating a visibility score of at least one object in a three-dimensional scene, the scene being composed of a plurality of objects and displayed in a display area by a main camera associated with a main graphics pipeline, characterized in that it comprises the following steps implemented by a dedicated camera, a clone of the main camera and associated with a dedicated graphics pipeline: Selection of at least one object from the plurality of objects in the scene; For the at least one selected object: Association of the object with a texture image containing only a single predetermined color and replacing the texture image of the object associated with the main graphics pipeline; Rendering, by the dedicated camera, of the scene to produce at least one dedicated image; For the at least one selected object: Reading, in the dedicated image, the color at at least one point corresponding to the object; Comparison of the read color with the predetermined color to obtain the visibility score associated with the object. The main and dedicated cameras operate sequentially, so that the dedicated image is produced before or after the scene is displayed in the display area.

[0022] Thanks to the invention, a second camera (called a dedicated camera because it is used solely for calculating visibility scores), a clone of the main camera, makes it possible to reproduce the scene context in real time and thus calculate and communicate (for example, to a user) the visibility scores in real time. The term "real-time" means in synchronization with the rendering of the scene as displayed to the user on the display area. To achieve this, the graphics pipelines are implemented using technical means: processing units and a display screen.

[0023] The association of an assignment step, on the object to be tested, of a solid texture of predetermined color (step of associating the selected object with a texture image containing only a predetermined color), a rendering step, and a color comparison step performed from the rendered image, makes it possible to represent the visible, partially visible or not visible aspect of an object by a visibility score, in a precise way (i.e. as close as possible to what the user sees) and without recourse to a stencil buffer.

[0024] The accuracy of the calculation is achieved through several characteristics.

[0025] Replacing the texture—called the "real" texture because it's associated with the main graphics pipeline—with a solid texture of a predetermined color provides a robust color reference compared to the real texture (displayed by the main camera), which typically includes multiple colors, as well as variations in hue and lighting. The advantage is that even slight color variations, such as those caused by semi-transparent objects or changes in lighting or brightness within the scene, can be detected.

[0026] The rendering step, performed after the texture exchange step, produces (without displaying it) a dedicated image that is identical in every respect to the one displayed by the main camera, except for the texture of the selected object, which has been given a solid, predetermined color texture. All shadow effects (shaders), lighting, etc., are identical to the rendering displayed by the main camera. The color read at at least one point corresponding to the selected object in the dedicated image carries the information of these different conditions.

[0027] Measuring the color difference at one or more points in the dedicated image corresponding to points positioned on the object allows us to increase the signal-to-noise ratio of the final visibility score (by averaging the measurements), and / or to obtain a map of the object's visibility.

[0028] It should be noted that the fact that the two cameras, main and dedicated, operate sequentially allows the step of assigning the solid texture image to be carried out before the dedicated camera renders.

[0029] Thus, the method according to the invention makes it possible to accurately represent, in real time, the visibility state of certain objects in a 3D scene, across a wide range of rendering configurations and pipelines. It is simpler to implement than the prior art technique based on the stencil buffer.

[0030] According to a preferred embodiment, the visibility score is calculated by taking the lowest visibility score from a first visibility score calculated during a first iteration of the dedicated camera, and a second visibility score calculated during a second iteration of the dedicated camera, the first and second iterations being successive, the predetermined color chosen for the second iteration being opposite on the color wheel the predetermined color chosen for the first iteration.

[0031] This preferred embodiment avoids false positives (i.e., an object detected as visible when it is not) and therefore improves the accuracy of the process.

[0032] This preferred embodiment makes it possible in particular to calculate the visibility score precisely even when the scene has a color texture close to the predetermined color, and / or when an object with a color texture close to the predetermined color hides or partially covers the object whose visibility we want to measure.

[0033] In addition to the characteristics mentioned in the preceding paragraph, the process according to one aspect of the invention may have one or more complementary characteristics from among the following, considered individually or according to all technically possible combinations.

[0034] The predetermined color is chosen from a list consisting of at least: pink with red, green, blue components equal to 255, 0, and 255 respectively, and green with red, green, blue components equal to 0, 255, 0 respectively.

[0035] Pink is a good choice because it is a color that is generally not very present in the scene. This predetermined color choice therefore reduces the probability of finding an object of the same color as the predetermined color sandwiched between the selected object and the camera. Since this configuration can lead to visibility detection errors (the small color difference found at the end of the comparison step abnormally indicating the visibility of the selected object), choosing pink is beneficial for a more reliable measurement of the visibility score.

[0036] The reading, in the dedicated image, of the color of the selected object is carried out on a plurality of points corresponding to the selected object in at least one dedicated image, for example 16 points, the points of the plurality of points being obtained by cutting the selected object into areas to be tested, and taking the central points of the areas to be tested.

[0037] Thus, it is possible to determine a visibility score at different locations on the object. The advantage is that it is possible to take into account the fact that an object partially covers the object being tested, or that part of the object is in a shadow area.

[0038] Depending on the size of the selected object (which also depends on the resolution of the dedicated image), the points of the plurality of points may overlap. In this case, the visibility score is calculated several times for the same point, and the final visibility score is the average of the calculated visibility scores.

[0039] The comparison of the read color with the predetermined color is obtained by calculating a color difference using: calculation, for each red, green and blue component, of the absolute difference between the color read and the predetermined color, and calculation of the average of the calculated absolute differences.

[0040] The comparison of the read color with the predetermined color is further based on a hue difference calculation comprising the following steps: Determination, from the red, green and blue components, of the hue of the read color and the hue of the predetermined color, calculation of the difference between the read hue and the hue of the predetermined color, weighting of the result of the calculated difference between the read hue and the hue of the predetermined color, to obtain the difference in hue, comparison of the difference in hue with respect to a predetermined threshold value such as the value 0.25: If the difference in hue is less than or equal to the threshold value, the difference between the read color and the predetermined color is equal to the average of the difference in hue and the difference in color, If the difference in hue is greater than the threshold value, the difference between the read color and the predetermined color is equal to the difference in color.

[0041] Thus, the process is more resilient to changes in scene brightness. Indeed, the hue difference measurement makes it possible to determine if two colors are similar, even if their brightness differs (one is light, the other dark). The advantage is that the visibility score is determined accurately even if the selected object has been obscured by a semi-transparent object, or / and if the ambient light in the scene is too intense or too dim, making the selected object difficult to see, or / and if the selected object is subject to reflections that prevent its surface from being clearly visible.

[0042] The image production frequency of the dedicated camera is lower than the calculation frequency of the main camera.

[0043] The dedicated camera's image production frequency is less than or equal to one-quarter of the main camera's image production frequency.

[0044] This limits the computing resources needed and does not degrade the performance of the application using the rendering engine.

[0045] The dedicated image resolution is lower than the rendering resolution of the main camera.

[0046] As with reducing the frequency, reducing the resolution of the dedicated image does not degrade the application's performance while allowing real-time visibility tracking.

[0047] The smallest resolution of the dedicated image corresponds to an image width of 96 pixels.

[0048] This value, although low, has been identified as providing a visibility score that combines accuracy and efficiency, and does not impact the operation of the application.

[0049] The process further includes, after the color comparison step, an additional step, performed for at least one selected object, of associating the object with the texture image associated with the main graphics pipeline, called the actual texture image, and replacing the texture image containing only the predetermined color.

[0050] The process further includes a step of testing the size of the selected object, including determining an area occupied by the selected object on the dedicated image, and an area ratio between said area occupied by the object and the area of ​​the dedicated image; if this ratio is less than a threshold value, for example a threshold value equal to 1.5%, then the visibility score is equal to 0, the selected object being considered too small to be visible.

[0051] In addition to the characteristics mentioned in the preceding paragraphs, the process may have one or more complementary characteristics from the following, considered individually or in all technically possible combinations: The method may further include a step for determining whether a user meets the visibility requirements for at least one object, these visibility requirements being dependent on the visibility score of the object. The visibility requirements may then depend further on predetermined parameters including a minimum visibility duration for the object and / or a surface area ratio between the area occupied by the object and the area of ​​the dedicated image and / or an angle between the direction normal to the area occupied by the object and the normal direction of the display area. After the step for determining whether the visibility requirements are met, the method may include a step involving a modification of the scene and / or the triggering of an additional event within or outside the scene.As an alternative to the step of determining whether a user meets the visibility conditions of at least one object, the process may further include, when the visibility score of at least one object differs from a predetermined visibility score, a step of modifying the object and a reiteration of the steps of calculating the visibility score of the modified object.

[0052] The invention and its various applications will be better understood by reading the following description and examining the accompanying figures. BREVE DESCRIPTION DES FIGURES

[0053] The figures are presented for illustrative purposes only and are in no way limiting to the invention. [ Fig. 1 ] shows a simplified illustration of how a graphics rendering engine works, [ Fig. 2 ] is a synoptic diagram illustrating the sequence of steps in the calculation process according to the invention, [ Fig. 3 [ ] schematically shows one possible implementation of a computer, [ Fig. 4 ] shows a simplified illustration of how the graphics rendering engine used to implement the steps of the figure 2 , [ Fig. 5 ] is an illustration of a 3D scene that will be rendered, during its next iteration, by the main camera of the graphics rendering engine of the figure 4 , [ Fig. 6 ] is a simplified illustration of the 3D scene of the figure 5 obtained at the end of the third stage of the figure 2 , [ Fig. 7 ] is a simplified illustration of the rendering, by the dedicated camera of the graphics engine of the figure 4 , of the 3D scene of the figure 5 , [ Fig. 8 ] is an enlarged view of the figure 7 , [ Fig.9 ] illustrates a preferred embodiment of the process of the figure 2 . DESCRIPTION DETAILLEE

[0054] Unless otherwise specified, the same element appearing on different figures has a unique reference.

[0055] The invention relates to a method for calculating a visibility score associated with an object in a 3D virtual scene, in particular a calculation method that combines efficiency, accuracy and simplicity of implementation.

[0056] There figure 2 illustrates schematically the sequence of steps S205, S210, S220, S230, S240, S250, S260, and S270 of the calculation method 200 according to the invention.

[0057] In practice, and with reference to the figure 3 , process 200 is implemented by a system 3 comprising a computer 30, a game console, a smartphone and more generally any device capable of implementing a graphics rendering engine.

[0058] In this description, processing actions are performed by processing units, i.e., microprocessors 300 which use memory 301. These microprocessors are general-purpose microprocessors also called CPUs, and / or graphics microprocessors also called GPUs.

[0059] System 3 may include a screen 31 and / or a keyboard 32.

[0060] The steps of process 200 are described below with reference to the figure 2 , there figure 4 , there figure 5 , there figure 6 , there figure 7 and the figure 8 .

[0061] The process begins with an S205 step of declaring a dedicated 420 camera (cf. figure 4 ) which is added to a main 120 camera. Rendering engine programming interfaces are used to declare and instantiate this dedicated 420 camera.

[0062] In the following description, the term "camera" refers to the (virtual) viewpoint from which the scene is rendered. The expression "the camera is drawn" refers to the process of generating (or rendering) a 2D image from a 3D scene, from the camera's viewpoint. The camera is therefore linked to a graphics pipeline implemented by the processing units.

[0063] The graphics pipeline consists of a sequence of algorithmic building blocks designed to pixelate the geometric objects of the 3D scene in order to obtain the 2D image.

[0064] Finally, the term "camera frequency" refers to the number of 2D images per second generated by the camera, which also corresponds to the number of camera iterations per second.

[0065] There figure 5 This schematically represents the 3D scene 50 drawn and displayed in a display area by the main camera 120, in which we wish to check the visibility of one or more objects. This scene 50 is composed of a plurality of 3D objects 501, 502, 503, 504, each object having an identifier. The objects are defined by a list of geometric shapes and attributes (texture, etc.).

[0066] The main 120 camera is associated with a main graphics pipeline.

[0067] The main camera 120 generates and displays a series of successive images to the application user on a display area (for example, occupying all or part of the screen 32 illustrated). figure 3 The time between two images in the image series depends on the frame rate of the main 120 camera. This frame rate is optimized for human vision, providing the necessary fluidity for real-time viewing of the scene. The minimum frame rate of the main 120 camera is 30 frames per second. Preferably, the frame rate is 60 frames per second. It can also be set at 120 frames per second.

[0068] The dedicated 420 camera is a clone of the main 120 camera, meaning it constantly tracks the position and characteristics of the main 120 camera (position, scale, filter options, cropping planes, etc.). For example, it reproduces the shadow and lighting effects exactly like the main 120 camera.

[0069] The dedicated 420 camera differs from the 120 camera in several aspects.

[0070] Firstly, the dedicated 420 camera is associated with a dedicated graphics pipeline, modified from the main graphics pipeline to integrate the algorithms enabling the implementation of steps S210 to S270 of process 200.

[0071] Secondly, the dedicated 420 camera renders a so-called dedicated image, "offscreen" according to English terminology. This means that this dedicated image is not displayed on screen 31. The dedicated image is therefore not intended for the application user (unlike the main rendering displayed by the main 120 camera), but solely for calculating the visibility score.

[0072] Third, the image production rate of the dedicated 420 camera may be lower than the rate of the main 120 camera.

[0073] The image production frequency of the dedicated camera can therefore be less than or equal to one-quarter of the image production frequency of the main camera.

[0074] Fourth, the resolution of the dedicated image rendered by the 420 camera can be degraded, meaning it is lower than the resolution of the main rendering. The resolution is defined, for example, by the dimensions of the dedicated image. Preferably, the smallest dimension of the dedicated image is 96 pixels, with the largest dimension calculated relative to the dimensions of the screen 31.

[0075] Reducing the frequency and resolution of the dedicated 420 camera prevents disruption to the performance of the graphics engine.

[0076] Finally, it should be noted that the dedicated 420 camera and the main 120 camera operate sequentially. In other words, one of the two 120 / 420 cameras is drawn before the other. The rendering of the dedicated image is therefore triggered with a slight time delay, for example, a 33 ms delay, relative to the rendering and display of the scene image in the main window.

[0077] Once the dedicated 420 camera is declared in step S205, steps S210, S220, S230, S240, S250 and, optionally, steps S260 and S270, are successively carried out, this at each iteration of the dedicated 420 camera.

[0078] Step S210 of process 200 is performed from the 3D scene 50 described previously in relation to the figure 5 This scene is the one that will be displayed by the main camera 120 during its next iteration.

[0079] The S210 step is a selection step of one or more objects 600 from the plurality of objects 501, 502, 503, 504 of the 3D scene 50. In the given example, an object 600 is selected, corresponding to object 502 of scene 50.

[0080] The 600 selected objects are those whose visibility we want to measure in real time, via a visibility score calculation.

[0081] The selection step S210 produces, in memory area 301 of the system 3 implementing the process 200 according to the invention, a list of identifiers associated with each selected object 600. This list includes the object's geometry and its texture. The texture of the selected object 600 is typically an image, called a texture image. At this step, the texture 510 of the object 600 is the one displayed to the user. It will be called the "actual texture 510" or "actual texture image 510" of the selected object 600.

[0082] Step S210 is followed by step S220 which consists of exchanging the actual texture image 510 of the selected object 600 with a texture image containing only one predetermined color.

[0083] To achieve this, in a substep S2201 of step S220, a texture image 610 containing only one predetermined color 6101 is associated with the selected object 600 (or each selected object). Furthermore, in a substep S2202, which can occur before or after the association step S220-S2201, the actual texture image 510 of the selected object 600 (or each selected object) can be extracted and saved in memory. The term "save" refers to saving the texture image by reference, not to copying the texture image into memory.

[0084] Thus, at the end of step S220, and with reference to the figure 6 We obtain a scene 60, called the initial scene, identical to the 3D scene 50 that will be displayed by the main camera 120, except for the selected objects 600, which will have a solid texture 610 of a predetermined color 6101. All other attributes (or settings and effects) of these selected objects 600 are preserved, for example, lighting, shadows, shaders, etc.

[0085] By solid texture, we mean that a single predetermined color 6101 is used for the same object 600.

[0086] Preferably the predetermined color 6101 is a color generally not very present in a 3D scene, such as pink, with a red component equal to 255, a green component equal to 0 and a blue component equal to 255 (or RGB(255,0,255)).

[0087] When multiple objects are selected, the same predetermined color 6101 or different predetermined colors can be used.

[0088] The S220 step is extended by an S230 generation step, by the dedicated camera, of a dedicated image of the real scene 60 obtained at the end of the S220 step.

[0089] There figure 7 shows an example of a dedicated 70 rendering of the real 60 scene containing objects 501, 503, and 504 and the selected object 600.

[0090] The selected object 600 is in the background.

[0091] The first object 501 is at a distance from the selected object 600 and has no influence on it.

[0092] The second object 503 is in an intermediate plane and partially overlaps the selected object 600.

[0093] The third object 504 is in the foreground. Furthermore, this third object 504 is transparent, covers the second object 503, and partially covers the selected object 600.

[0094] The selected object 600 has taken on a new color 710, which depends on the scene and the predetermined color 610 assigned in the previous step S220.

[0095] The new color 710 results in a modification of the solid texture 610 due to the presence of objects in planes between the plane of the selected object and the camera 420, or due to the presence of shadow areas covering the selected object 600.

[0096] In the example of the dedicated image 70 illustrated in figure 7 , color 710: was not changed from the predetermined color 6101 in area 711 where the selected object 600 was not covered, was changed from the predetermined color 6101 with some amplitude in area 712 where the selected object 600 is covered by the third transparent object 504, was changed in area 713, with a greater amplitude than in area 712, where the selected object 600 is covered by the second object 502.

[0097] Thus, the more visible the selected object 600 is (not covered, as in area 711), the smaller the color difference between the color 710 in the dedicated image and the predetermined color 6101.

[0098] Conversely, the less visible the selected object 600 is (covered by several objects, as in area 713), the greater the color difference between the color 710 in the dedicated image and the predetermined color 6101.

[0099] The color difference between the color 710 in the dedicated image (70) and the predetermined color texture 6101 is therefore correlated to the state of visibility or invisibility of the selected object 600: the greater the difference, the less visible the object, and, conversely, the smaller the difference, the more visible the object.

[0100] The dedicated image 70 is therefore essentially identical to the main image that will be generated during the next iteration of the main camera 120. One difference is the selected object(s) 600, which will have taken on a new color 710. Another difference may be, depending on a parameter setting of the implementation of the invention, the resolution of the dedicated image 70.

[0101] In other words, it is the same scene that is rendered on the dedicated 70 image and on the image displayed by the main 120 camera, but for some objects corresponding to the selected 600 objects, the actual 510 textures are changed to plain 610 textures, of the predetermined 610 color, for rendering by the dedicated 420 camera, but remain used by the main 120 camera.

[0102] Steps S240 and S250, which follow step S230, aim to measure this difference in color and represent it by a visibility score.

[0103] Step S240 is performed from the dedicated image 70. It consists of reading the color 710 of the object 600.

[0104] This S240 step can be implemented by adding a "compute shader" in the dedicated pipeline, at the end of rendering, by the dedicated 420 camera.

[0105] With reference to the figure 8 , the color 710 of the selected object 600 is read at at least one point 810 belonging to the object 600.

[0106] Preferably, color 710 is read successively at each point of a plurality of points belonging to the object, these points being determined by a prior step (not shown in figure 2 ) of dividing the object into areas to be tested, and of determining a central point in each area to be tested.

[0107] The number of points is, for example, 16. One point corresponds to one pixel.

[0108] Depending on the resolution chosen for the dedicated image, 810 points may overlap. In this case, the color reading is performed multiple times at the same point.

[0109] This is the case, for example, when the resolution of the dedicated image 70 is minimal (96 pixels for the smallest dimension of the dedicated image). The size of the object selected on the image may then occupy only a few pixels.

[0110] Step S250 extends step S240. It consists of comparing the color 710 obtained on the dedicated image 70 and the predetermined color 6101 at each measurement point 810.

[0111] The result of this color comparison is advantageously between 0 (identical or very close colors) and 1 (opposite or very distant colors).

[0112] To compare the colors, two calculation techniques can be performed.

[0113] The first technique consists of measuring a color difference by calculating, for each red, green and blue component, the absolute difference between the color 710 read and the predetermined color 6101, and then averaging these calculated absolute differences.

[0114] The second technique involves combining hue comparison with color difference measurement. Hue comparison allows us to detect that one color is identical to another, even if the lighting is different. This makes the comparison more accurate and more robust to changes in scene brightness.

[0115] The second technique more specifically includes the following steps: Determination, from the red, green, and blue components, of the hue of the read color 710 and the hue of the predetermined color 6101. Calculation of the difference between the read hue and the predetermined color hue. Weighting of the calculated difference between the read hue and the predetermined color hue, with the weighting result being between 0 and 1 and determining the hue difference. Comparison of the hue difference with a predetermined threshold value, such as 0.25: If the hue difference is less than or equal to the threshold value, the difference between the read color and the predetermined color is equal to the average of the hue difference and the color difference. If the hue difference is greater than the threshold value, the difference between the read color and the predetermined color is equal to the color difference.

[0116] The final result is a value between 0 (similar or identical colors) and 1 (opposite or distant colors).

[0117] A partial visibility score S p is then determined from the final result, this partial visibility score being between 0 (test area 800 not visible) and 1 (test area 800 visible).

[0118] We therefore obtain a partial visibility score S p for each 810 test point (i.e. each 800 test area).

[0119] The "final" visibility score S of the selected object 600 can then be determined by calculating the average of all the partial visibility scores S p obtained at step S250, this final score S also being between 0 (object not visible) and 1 (object visible).

[0120] It should be noted that the partial visibility scores S p allow verification of the visibility of the selected object 600 in the following cases: Object 600 was partially obscured by a semi-transparent object. The ambient light in the scene is either too intense or too weak, making the object difficult to see. The object is undergoing a reflection that prevents its surface from being clearly seen.

[0121] The visible / invisible state can be determined from the final visibility score S of the object 600 and a threshold value.

[0122] The final visibility score S and / or partial visibility scores S p are then communicated, via means of communication (not shown) to the application user or to other users.

[0123] When the actual texture image 510 is saved during step S220, step S250 can be followed by a step S260 consisting of exchanging, in the initial scene 60, the plain texture image 610 of the predetermined color 6101 with the actual texture image 510.

[0124] To do this, we associate with each selected object 600 the extracted texture image 510 saved during step S220.

[0125] Since the main 120 camera and the dedicated 420 camera operate sequentially, the rendering of the main 120 camera takes place before or after that of the dedicated 420 camera, therefore before replacing (step S220) the 510 real texture image with the 610 solid texture image, or after restoring (step S260) the 510 real texture image while the 610 solid texture image was used for the rendering of the dedicated 420 camera.

[0126] In other words, the fact that the rendering of the two main and dedicated 120, 420 cameras is never done simultaneously allows the 510 real texture image to always be used for the main 120 camera, and the 610 solid texture image for the dedicated 420 camera.

[0127] An S270 step can also be performed following the S260 step, which consists of adding a visibility criterion related to the size of the selected object 600 in the scene 50. This S270 step aims to mark as invisible any object 600 whose size is too small relative to the size of the display area. This could be because the object is located at a great distance from the camera 120.

[0128] For this, step S270 consists of determining an area occupied by the selected object 600 on the dedicated image 70, and an area ratio between the area occupied by the object determined previously, and the area of ​​the dedicated image 70, if this ratio is less than a threshold value, for example 1.5%, then the visibility score is equal to 0, the selected object being considered too small to be visible.

[0129] A preferred embodiment of process 200 is described below in relation to the figure 9 .

[0130] In this preferred mode, the visibility score S is calculated by taking the lowest visibility score from a first visibility score S1 calculated during a first iteration I1 of the dedicated camera 240, and a second visibility score S2 calculated during a second iteration I2 of the dedicated camera 240.

[0131] The first and second iterations 11, I2 are successive iterations.

[0132] The predetermined color chosen for the second iteration I2 is opposite on the color wheel the predetermined color chosen for the first iteration. For example, if pink RGB(255,0,255) is chosen for the first iteration 11, then green RGB(0,255,0) will be chosen for the second iteration I2.

[0133] This preferential mode has the effect of eliminating false positives.

[0134] To explain this, let's take the example of a scene that is primarily pink. If we use a predetermined pink color, then the selected object 600 will be abnormally detected as visible during a single iteration, even if it is hidden behind another object in the scene. The colors are indeed similar.

[0135] Similarly, if the object 600 to be tested is hidden behind another pink object, it will obtain an abnormally high visibility score S1 with a first iteration I1 using a pink texture.

[0136] By using a second color opposite to pink in the next iteration, for example green (with red, green, and blue components of 0, 255, and 0 respectively), object 600 will be detected with a much lower S2 visibility score, due to the difference between the pink (of the scene or object obscuring the object being tested) and the green (of object 600 being tested). By retaining this second S2 visibility score, the object is correctly detected as not visible or barely visible, and a false positive is avoided.

[0137] Conversely, if the object to be tested is hidden by another green object, the first iteration will give a low score which corresponds to the actual visibility of the object to be tested in the scene.

[0138] It should also be noted that it is not possible to obtain false negatives.

[0139] Thus, the method 200 according to the invention makes it possible to track in real time the visibility of an object in a 3D virtual scene, via a visibility score associated with the object combining precision and efficiency on a wide range of configurations and rendering pipelines, while being simpler to implement than the stencil buffer technique described in patent FR3112638B1.

[0140] The 200 method, by allowing real-time tracking of the visibility of an object in a 3D virtual scene by one or more users, can be used in any application area using a 3D virtual scene, such as video games, virtual reality and augmented reality.

[0141] The visibility score, for example, allows us to assess whether an object, containing information of interest, has been seen by the user(s). This object of interest could, for example, display marketing information, educational content, or information related to safety rules.

[0142] Process 200 can thus include an optional step to determine whether a user has, at least partially, seen the object based on the visibility score calculated for that object. The verb "saw" in the expression "the user saw or did not see the object" is understood here in the sense of visual perception, i.e., the user actually perceived the object. In other words, the process can include an optional step to determine whether visibility conditions for this object are met (for a user), where the visibility conditions depend on the visibility score calculated for the object and at least one predetermined visibility score.

[0143] Visibility conditions may also depend on predetermined parameters, including a minimum visibility duration for at least one object and / or a surface area ratio between the area occupied by the object and the area of ​​the dedicated image and / or an angle between the direction normal to the surface occupied by the object and the normal direction of the display area. The minimum visibility duration typically corresponds to the minimum time required for the user to perceive the object's presence in the scene and to understand its informational content. For example, visibility conditions are said to be met when the visibility score, such as the final visibility score S, calculated for a given object is equal to 1 for a duration greater than 2 seconds.

[0144] Process 200 can then include, upon determination of whether the visibility conditions are met or not, a scene modification step.

[0145] Thus, when an object's visibility score falls below a predetermined threshold, an optional final step of modifying the virtual environment can be performed. For example, the same object can be duplicated and / or moved within the three-dimensional scene as long as its visibility score remains below the predetermined threshold. In this case, the movement can be performed at a specific location within the three-dimensional scene where the probability of the object being seen is high.

[0146] In addition, and / or as an alternative, when an object's visibility score exceeds a second predetermined visibility score, the object can be removed and / or moved within the three-dimensional scene. In this case, the object can be moved to a specific location within the three-dimensional scene where the probability of it being displayed is low. This process thus optimizes the three-dimensional scene to ensure that relevant information has been seen by the user.

[0147] Optimization also saves memory and processing resources because if an object has already been seen, it can, for example, be deleted and / or moved so that it is no longer displayed. Furthermore, the object's display can also be modified. Indeed, thanks to the calculated visibility score, the texture associated with an invisible object can be either not loaded into or unloaded from memory.

[0148] In one initial application, the process can be used to evaluate the effectiveness of vocational training when employed within programs that utilize virtual and / or augmented reality. Indeed, thanks to the visibility score, it is possible to objectively assess where and how—that is, on which objects—learners focus their attention. This assessment can, for example, be used to improve teaching methods and / or evaluate learners.

[0149] For example, the visibility score can enable the analysis of a learner's behavior and help understand how different types of learners, for example, of different levels, focus their attention on different aspects of the vocational training program.

[0150] In a second application example, the process can be used to identify the objects, such as products for sale, that have generated the most interest from the user(s). When implemented on multiple objects in the scene, the process can then include an optional step comprising the following operations: Ordering of the calculated visibility scores for each object in the scene, and determination of the object with the highest visibility score.

[0151] This optional step can be followed by a step involving modifying the layout of one or more of these objects based on their visibility score ranking. Thus, the visibility score allows for optimization, for example, by customizing the layout and appearance of one or more products represented in the 3D virtual scene by one or more objects. Finally, it is also possible to conduct market research and product testing by analyzing the visibility score of different objects placed in the 3D virtual scene.

[0152] In a third application example, the process can be used to measure the effectiveness of one or more features of an object in order to generate interest in that object from one or more users. This object could, for example, be used to advertise security information or an item for sale.

[0153] In other words, the visibility score can be used to test several variants of an object. The visibility scores of these variants can then be compared, and the variant with the highest visibility score can be selected, using a testing method called "A / B testing." In practice, process 200 can thus also include a step using the visibility score of at least one object, comprising: When the visibility score of at least one object is different from a predetermined visibility score, modify the object and recalculate the visibility score for the modified object.

[0154] By object modification, we mean a change to the information of interest associated with the object, to the object's display, or to its location within the scene. This step, which utilizes the visibility score, allows for the reassignment of advertising content, for example, on the fly, to multiple locations within the scene to maximize its visibility. It also allows the visibility score to be used to compare different variations (or versions) of the same object.

[0155] In a fourth application example, the process can trigger an additional event when the visibility score is lower or higher than a predetermined threshold. This event could, for example, be the creation of an additional object or visual element and / or haptic feedback and / or sound. This event could, for instance, draw attention to the object so that its visibility score increases. This event could also transmit other information. For example, the event could consist of displaying additional advertising content in the form of audio content and / or an immersive and / or interactive element only when the user maintains a viewing angle for a longer time than a predetermined duration.

[0156] In a fifth application example, the process can be used to optimize a user interface and user experience. For instance, the visibility score can be used to analyze which parts of the user interface attract the most attention from players, thereby improving the design of the user interface and user experience. For example, the visibility score can be used to determine which buttons or features are most and / or least noticed by the user.

[0157] In a sixth application example, the process can allow the difficulty level of a video game or training exercise to be adapted to the user's skill level. For example, using the visibility score of key elements in a level, it is possible to ensure that these key elements are neither too visible nor too difficult to find.

[0158] In a seventh application example, the process can be used to improve tutorials. Specifically, the process can be used to track the visibility of instructions or hints given during tutorials to determine whether users notice and easily understand them.

[0159] In an eighth application example, the process can be used to identify abnormal user behavior. For example, a user whose behavior disregards an object with a visibility score exceeding a threshold, or conversely, a user like a player who spots opponents through walls in a shooting game.

[0160] In a ninth application example, the process can improve the behavior of different characters in a 3D virtual scene controlled by artificial intelligence. For example, using a visibility score, it is possible to adapt the AI's behavior of a character of interest based on its visibility. This results in more realistic and unpredictable characters.

Claims

1. A method (200) for computing a visibility score (S, Sp) of at least one object (600) in a three-dimensional scene (50), the scene (50) being comprised of a plurality of objects (501, 502, 503, 504) and displayed in a display zone by a main camera (120) associated with a main render pipeline, characterised in that it comprises the following steps implemented by a dedicated camera (420) which is a clone of the main camera (120) and associated with a dedicated render pipeline: - selecting (S210) at least one object (600) from the plurality of objects of the scene, - for the at least one object (600) selected: ∘ associating (S220) the object (600) with a texture image (610) including only one predetermined colour (6101) and replacing the texture image (510) of the object associated with the main render pipeline, - rendering (S230), by the dedicated camera, the scene to produce at least one dedicated image (70), - for the at least one object (600) selected, ∘ reading (S240), in the dedicated image (70), the colour (710) at at least one point (810) corresponding to the object (600) in the at least one dedicated image (70), ∘ comparing (S250) the colour (710) read with the predetermined colour (6101) to obtain the visibility score (S) associated with the object (600), the main (120) and dedicated (420) cameras operating sequentially, so that the dedicated image (70) is produced before or after the scene is displayed in the display zone.

2. The method (200) according to claim 1 characterised in that the visibility score (S) is computed by taking the lowest visibility score from a first visibility score (S1) computed during a first iteration (I1) of the dedicated camera (420), and a second visibility score (S2) computed during a second iteration (I2) of the dedicated camera (420), the first and second iterations (I1, I2) being successive, the predetermined colour chosen for the second iteration (I2) being opposite to the predetermined colour chosen for the first iteration (I1) on the hue circle.

3. The method (200) according to one of claims 1 to 2, characterised in that the predetermined colour (710) is chosen from a list formed by at least: pink with red, green and blue components respectively equal to 255, 0 and 255, and green with red, green and blue components respectively equal to 0, 255 and 0.

4. The method (200) according to one of claims 1 to 3, characterised in that reading (S240), in the dedicated image (70), the colour (710) of the object (600) selected is performed on a plurality of points (810) corresponding to the object (600) selected in the at least one dedicated image, for example 16 points, the plurality of points being obtained by cutting the object (600) selected into zones (800) to be tested, and taking the central points (810) of the zones to be tested.

5. The method (200) according to one of claims 1 to 4, characterised in that comparing the colour (710) read with the predetermined colour is obtained by computing a colour difference by: - computing, for each red, green and blue component, the absolute difference between the colour (710) read and the predetermined colour (6101), and - computing the average of the absolute differences computed.

6. The method (200) according to claim 5, characterised in that the comparison of the colour (710) read with the predetermined colour is further based on a hue difference computation comprising the following steps of: - determining, from the red, green and blue components, the hue of the colour (710) read and the hue of the predetermined colour (6101), - computing the difference between the hue read and the hue of the predetermined colour, - weighting the result of the computed difference between the hue read and the hue of the predetermined colour to obtain the hue difference, - comparing the hue difference with a predetermined threshold value such as the value 0.25: ∘ if the hue difference is less than or equal to the threshold value, the difference between the colour (710) read and the predetermined colour (6101) is equal to the average of the hue difference and the colour difference, ∘ if the hue difference is greater than the threshold value, the difference between the colour (710) read and the predetermined colour (6101) is equal to the colour difference.

7. The method (200) according to one of claims 1 to 6, characterised in that the image production frequency of the dedicated camera (420) is lower than the computation frequency of the main camera (120).

8. The method according to claim 7, characterised in that the image production frequency of the dedicated camera is less than or equal to a quarter of the image production frequency of the main camera.

9. The method (200) according to one of claims 1 to 8, characterised in that the resolution of the dedicated image is less than the render resolution of the main camera (120).

10. The method (200) according to claim 9, characterised in that the smallest resolution of the dedicated image (70) corresponds to an image width of 96 pixels.

11. The method (200) according to one of claims 1 to 10, characterised in that it further comprises a step (S270) of testing the size of the object (600) selected, comprising determining a surface area occupied by the object (600) selected on the dedicated image (70), and a surface area ratio between said surface area occupied by the object and the surface area of the dedicated image, if this ratio is less than a threshold value, for example a threshold value equal to 1.5%, then the visibility score is equal to 0, the object selected being considered too small to be visible.

12. The method (200) according to one of claims 1 to 11, further comprising a step of determining compliance, for a user, with visibility conditions of the at least one object, said visibility conditions depending on the visibility score (S, Sp) of the at least one object (600).

13. The method (200) according to claim 12, wherein the visibility conditions further depend on predetermined parameters comprising a minimum duration of visibility of the at least one object (600) and / or a surface area ratio between a surface area occupied by the object (600) and the surface area of the dedicated image and / or an angle between the direction normal to the surface area occupied by the object (600) and the normal direction of the display zone.

14. The method (200) according to one of claims 12 or 13, comprising, after the step of determining compliance with the visibility conditions, a step comprising modifying the scene (50) and / or a triggering an additional event inside or outside the scene (50).

15. The method (200) according to one of claims 1 to 11, further comprising, when the visibility score (S, Sp) of the at least one object (600) is different from a predetermined visibility score, a step of modifying the object (600) and reiterating the steps of computing the visibility score of the object modified.