METHOD FOR CALCULATING THE VISIBILITY OF OBJECTS WITHIN A 3D SCENE

Abstract

METHOD FOR CALCULATING THE VISIBILITY OF OBJECTS WITHIN A 3D SCENE One aspect of the invention relates to a method for calculating a visibility score of at least one object in a three-dimensional scene, comprising the following steps implemented by a dedicated camera, a clone of a main camera and associated with a dedicated graphics pipeline: Selection of at least one object (600) from among the plurality of objects in the scene, Association of the selected object (600) with a texture image containing only one predetermined color, Rendering, by the dedicated camera, of the scene to produce at least one dedicated image (70), For the selected object (600), Reading, in the dedicated image (70), the color (710) at at least one point corresponding to the object (600) in the at least one dedicated image (70), Comparison of the color read (710) with the predetermined color to obtain the visibility score associated with the object (600). Figure to be published with the abbreviation: Figure 7

Description

Title of the invention: METHOD FOR CALCULATING THE VISIBILITY OF OBJECTS WITHIN A 3D SCENE TECHNICAL FIELD OF THE INVENTION

[0001] The technical field of the invention is that of three-dimensional rendering or 3D rendering, or 2D for two dimensions, and of the detection of visibility of objects in applications using graphics rendering engines such as video games, virtual worlds, virtual reality, augmented reality, etc.

[0002] The present invention relates to a method for calculating the visibility of objects within a three-dimensional scene also called a 3D scene. TECHNOLOGICAL BACKGROUND OF THE INVENTION

[0003] In the technical field, graphics rendering engines such as Unity, Unreal and others are commonly used to generate and display a video stream giving a virtual and real-time representation of a scene composed of 3D objects.

[0004] The images that make up this video stream are two-dimensional images (or matrix images) also called “renderings”.

[0005] Each rendering is generated and displayed (also called "drawn") on a rendering plane called a "viewport" and according to a predefined viewing angle called a "camera". To do this, we use algorithms combined within a graphics pipeline, or "render pipeline" in English, used by the graphics rendering engine and integrated on a graphics card.

[0006] The graphics pipeline uses as input data the representation in a coordinate space (x,y,z) of the 3D objects which make up the scene to be represented.

[0007] From this input data, and with reference to [Fig. 1], each object 100 is projected onto a viewport 110 from the viewpoint of a camera 120, also defined in the coordinate space (x,y,z).

[0008] Each rendering restores both the 3D projection, the textures (appearance of the surfaces), and the lighting effects (shadows, reflection, etc.) of the object 100.

[0009] The objects, as seen by the end user (or viewer) are for example everyday objects, buildings, vehicles whose texture and lighting are realistic, that is to say whose appearance is close to a real appearance.

[0010] The camera 120 determines the viewing angle from which the scene is drawn. The position of this camera 120 may change during the video stream. Thus, the viewpoint used to draw the scene may change from one image to the next. As a result, Some objects may be visible in some images, or partially visible, or completely invisible in other images.

[0011] For example, an object may be seen (and thus be visible) by the camera 120 when no other object is interposed between the object 100 and the camera 120 and when the object is sufficiently illuminated. It may be partially visible when a semi-transparent object is interposed between it and the camera, or when it is located partly in a shadow area. Finally, it may be invisible when it is hidden behind another object, or located in an unlit area.

[0012] The visibility or not of an object in the scene is therefore not fixed during the video stream. It is also not easy to determine it a priori because the movements of the camera 120 are generally not predictable. However, there is a need, for certain applications, to control and monitor this visibility in real time. Knowledge of this visibility allows, for example, fine management of the memory. Indeed, if an object is not or no longer visible, it is possible not to load into, or unload from, the memory the texture associated with this object. To the extent that textures are the elements that take up the most space in memory, the technical interest of such a visibility calculation method is understandable.

[0013] Several solutions exist for this which consist of representing the visibility of an object by a so-called visibility score associated with the object.

[0014] A first solution consists of using the so-called zbuffer technique. This technique makes it possible to determine the visibility of an object at the moment it is drawn. During the generation of the final rendering, other objects can therefore be drawn on top, which can mask the object whose visibility has been determined, making the visibility calculation erroneous.

[0015] A second solution consists of using the technique known as ray casting or "raycast" in English. This technique uses a "collider" or collision detector usually used by the physics engine to detect collisions between objects. The object whose visibility we want to calculate must therefore necessarily be associated with such a "collider". Added to this constraint is the fact that the technique cannot be applied when the object is located behind another object, even semi-transparent, or in a shadow area.

[0016] A third solution is described in patent FR3112638B1. It consists of using a stencil buffer in a second rendering intended solely for calculating visibility, and therefore not displayed, and generated in parallel with the main rendering (the one which renders the scene from a main camera and is displayed to the user), by a second camera which follows at all times the position and characteristics of the main camera. Precisely, the stencil buffer is used in the pipeline used to generate the second rendering, after a step of drawing the opaque objects of the scene. The graphics pipeline is necessarily of the “forward” type, that is to say that it allows you to draw, in this order, the opaque objects and the transparent objects of the scene.

[0017] This technique is very precise, that is to say that it allows to take into account the lighting and the shadows of the scene. It allows for example to calculate the visibility of an object even if it is partially masked by another object or located in a shadow zone. However, it requires access to a stencil buffer, and therefore to adapted computing resources, and is not compatible with graphics pipelines other than "forward" type. For example, it cannot be implemented in a "deferred" type pipeline using a different rendering strategy, in which the drawing order is different.

[0018] There is therefore a need for a method for calculating object visibility scores that is suitable for all graphics pipeline architectures, whether these architectures are software or hardware. Such a solution must also be able to adapt to the available computing and energy resources. Summary of the invention

[0019] The invention offers a solution to the problems mentioned above, by making it possible to calculate the visibility score not from a second intermediate rendering corresponding to the rendering of opaque objects, but from a second final rendering.

[0020] To do this, a color comparison technique is used which is deployed before and after the generation of the second rendering, and which makes it possible to cover cases where the object whose visibility we wish to check has been partially masked by a semi-transparent object, or is made barely visible due to surrounding light being too weak or too intense, or due to reflections on its surface.

[0021] One aspect of the invention thus relates to a method for calculating a visibility score of at least one object in a three-dimensional scene, the scene being composed of a plurality of objects and displayed in a display area by a main camera associated with a main graphics pipeline, characterized in that it comprises the following steps implemented by a dedicated camera clone of the main camera and associated with a dedicated graphics pipeline: • Selection of at least one object from among the plurality of objects in the scene, • For at least one selected object: • Association of the object with a texture image comprising only one predetermined color and replacing the texture image of the object associated with the main graphics pipeline, • Rendering, by the dedicated camera, of the scene to produce at least one dedicated image, • For at least one selected object, • Reading, in the dedicated image, of the color at at least one point corresponding to the object in the at least one dedicated image, • Comparison of the read color with the predetermined color to obtain the visibility score associated with the object, the main and dedicated cameras operating sequentially, so that the dedicated image is produced before or after the scene is displayed in the display area

[0022] Thanks to the invention, a second camera (called dedicated because it is used only to calculate visibility scores), a clone of the main camera, makes it possible to reproduce the context of the scene in real time and thus to calculate and communicate (for example to a user) the visibility scores in real time. The term "in real time" means in synchronization with the rendering of the scene as displayed to the user on the display area. For this, the graphics pipelines are implemented by technical means: processing units, display screen.

[0023] The association of a step of assigning, on the object to be tested, a solid texture of predetermined color (step of associating the selected object with a texture image comprising only one predetermined color), a rendering step, and a color comparison step carried out from the rendered image, makes it possible to represent the visible, partially visible or non-visible aspect of an object by a visibility score, in a precise manner (i.e. as close as possible to what the user sees) and without recourse to a stencil buffer.

[0024] The accuracy of the calculation is achieved through several characteristics.

[0025] Exchanging the texture, called real because it is associated with the main graphics pipeline, associated with the selected object by a solid texture of predetermined color makes it possible to provide a robust color standard compared to the real texture (displayed by the main camera) which generally includes multiple colors, as well as variations in hues and lighting. The advantage is that even small color variations, due for example to the presence of semi-transparent objects or to changes in lighting or brightness in the scene, can be detected.

[0026] The rendering step carried out successively to the texture exchange step makes it possible to produce (without displaying it) an image (called dedicated) in every point identical to that displayed by the main camera with the exception of the texture of the selected object, which has taken the solid texture of predetermined color. All the shadow effects, (shaders), lighting, etc. are identical to the rendering displayed by the main camera. The color read at at least one point corresponding to the selected object in the dedicated image carries the information of these different conditions.

[0027] Measuring the color difference at one or more points of the dedicated image corresponding to points positioned on the object makes it possible to increase the signal-to-noise ratio of the final visibility score (by averaging the measurements), or / and to obtain a visibility map of the object.

[0028] It should be noted that the fact that both the main and dedicated cameras operate sequentially allows the step of assigning the solid texture image to be performed before the rendering of the dedicated camera.

[0029] Thus, the method according to the invention makes it possible to represent precisely and in real time the visibility state of certain objects in a 3D scene, over a wide range of configurations and rendering pipelines. It is simpler to implement than the prior art technique based on the stencil buffer.

[0030] According to a preferred embodiment, the visibility score is calculated by taking the lowest visibility score from a first visibility score calculated during a first iteration of the dedicated camera, and a second visibility score calculated during a second iteration of the dedicated camera, the first and second iterations being successive, the predetermined color chosen for the second iteration being opposite on the color wheel to the predetermined color chosen for the first iteration.

[0031] This preferred embodiment makes it possible to avoid false positives (i.e., a detected object visible when it is not) and therefore to improve the precision of the method.

[0032] This preferred embodiment makes it possible in particular to precisely calculate the visibility score even when the scene has a color texture close to the predetermined color, and / or when an object with a color texture close to the predetermined color hides or partially covers the object whose visibility is to be measured.

[0033] In addition to the characteristics which have just been mentioned in the preceding paragraph, the method according to one aspect of the invention may have one or more complementary characteristics among the following, considered individually or according to all technically possible combinations.

[0034] The predetermined color is chosen from a list formed of at least: pink with red, green, blue components respectively equal to 255, 0, and 255, and green with red, green, blue components respectively equal to 0, 255, 0.

[0035] Pink is advantageously chosen because it is a color that is generally not very present in the scene. This choice of predetermined color therefore makes it possible to reduce the probability of finding an object of the same color as the predetermined color inserted between the selected object and the camera. As this configuration can lead to visibility detection errors (the small color difference then found at the end of the comparison step abnormally indicating visibility of the selected object), the choice of pink is favorable to a more reliable measurement of the visibility score.

[0036] The reading, in the dedicated image, of the color of the selected object is carried out on a plurality of points corresponding to the object selected in the at least one dedicated image, for example 16 points, the points of the plurality of points being obtained by cutting the selected object into zones to be tested, and by taking the central points of the zones to be tested.

[0037] Thus, it is possible to determine a visibility score at different locations on the object. The advantage is that it is thus possible to take into account the fact that an object partially covers the object to be tested, or that part of the object is in a shadow area.

[0038] Depending on the size of the selected object (which also depends on the resolution of the dedicated image), the points of the plurality of points may overlap. In this case, the visibility score is calculated several times for the same point, and the final visibility score is the average of the calculated visibility scores.

[0039] The comparison of the read color with the predetermined color is obtained by calculating a color difference by: • calculation, for each red, green and blue component, of the absolute difference between the read color and the predetermined color, and • calculation of the average of the calculated absolute differences.

[0040] The comparison of the read color with the predetermined color is further based on a hue difference calculation comprising the following steps: • Determination, from the red, green and blue components, of the hue of the read color and the hue of the predetermined color, • calculation of the difference between the read shade and the shade of the predetermined color, • weighting of the result of the difference calculated between the read shade and the shade of the predetermined color, to obtain the shade difference, • comparison of the difference in shade with a predetermined threshold value such as the value 0.25: • If the hue difference is less than or equal to the threshold value, the difference between the read color and the predetermined color is equal to the average of the hue difference and the color difference, • If the hue difference is greater than the threshold value, the difference between the read color and the predetermined color is equal to the color difference.

[0041] Thus, the method is more resilient to changes in scene brightness. Indeed, the hue difference measurement makes it possible to determine whether two colors are similar, even if their brightness is different (one is light, the other dark). The advantage is that the visibility score is determined accurately even in the case where the selected object has been masked by a semi-transparent object, or / and if the surrounding light of the scene is too intense or too weak and makes the selected object barely visible, or / and again if the selected object undergoes a reflection which prevents its surface from being clearly seen.

[0042] The image production frequency of the dedicated camera is lower than the calculation frequency of the main camera.

[0043] The image production frequency of the dedicated camera is less than or equal to a quarter of the image production frequency of the main camera.

[0044] This limits the necessary computing resources and does not degrade the performance of the application that uses the rendering engine.

[0045] The resolution of the dedicated image is lower than the resolution of the main camera rendering.

[0046] As with reducing the frequency, reducing the resolution of the dedicated image allows the application's performance to be maintained while still allowing real-time tracking of visibility.

[0047] The smallest resolution of the dedicated image corresponds to an image width of 96 pixels.

[0048] This value, although low, has been identified as allowing a visibility score to be given that combines precision and efficiency, and does not impact the operation of the application.

[0049] The method further comprises, after the color comparison step, an additional step, carried out for the at least one selected object, of associating the object with the texture image associated with the main graphics pipeline, called the real texture image, and replacing the texture image comprising only the predetermined color.

[0050] The method further comprises a step of testing the size of the selected object, comprising determining a surface area occupied by the selected object on the dedicated image, and a surface area ratio between said surface area occupied by the object and the surface area of ​​the dedicated image, if this ratio is less than a threshold value, for example a threshold value equal to 1.5%, then the visibility score is equal to 0, the selected object being considered too small to be visible.

[0051] The invention and its various applications will be better understood upon reading the following description and examining the accompanying figures. BRIEF DESCRIPTION OF THE FIGURES

[0052] The figures are presented for information purposes only and in no way limit the invention. • [Fig. 1] shows a simplified illustration of how a graphics rendering engine works, • [Fig.2] is a block diagram illustrating the sequence of steps of the calculation method according to the invention, • [Fig.3] shows schematically an embodiment of a computer, • [Fig.4] shows a simplified illustration of the operation of the graphics rendering engine used to implement the steps in [Fig.2], • [Fig.5] is an illustration of a 3D scene that will be rendered, during its next iteration, by the main camera of the graphics rendering engine of [Fig.4], • [Fig.6] is a simplified illustration of the 3D scene of [Fig.5] obtained at the end of the third step of [Fig.2], • [Fig.7] is a simplified illustration of the rendering, by the dedicated camera of the graphics engine of [Fig.4], of the 3D scene of [Fig.5], • [Fig.8] is an enlarged view of [Fig.7], • [Fig.9] illustrates a preferred embodiment of the method of [Fig.2]. DETAILED DESCRIPTION

[0053] Unless otherwise specified, the same element appearing in different figures has a single reference.

[0054] The invention relates to a method for calculating a visibility score associated with an object of a 3D virtual scene, in particular a calculation method which combines efficiency, precision and simplicity of implementation.

[0055] [Fig.2] schematically illustrates the sequence of steps S205, S210, S220, S230, S240, S250, S260, and S270 of the calculation method 200 according to the invention.

[0056] In practice, and with reference to [Fig. 3], the method 200 is implemented by a system 3 comprising a computer 30, a game console, a smartphone and more generally any device capable of implementing a graphics rendering engine.

[0057] In this description, the processing actions are carried out by processing units, i.e. microprocessors 300 which use memory 301. These microprocessors are general microprocessors also called CPUs, and / or graphics microprocessors also called GPUs.

[0058] The system 3 may comprise a screen 31 and / or a keyboard 32.

[0059] The steps of the method 200 are described below with reference to [Fig.2], [Fig.4], [Fig.5], [Fig.6], [Fig.7] and [Fig.8].

[0060] The method begins with a step S205 of declaring a dedicated camera 420 (see [Fig.4]) which is added to a main camera 120. Programming interfaces of the rendering engine are used to declare and instantiate this dedicated camera 420.

[0061] In the remainder of the description, the term “camera” designates the (virtual) point of view from which the scene is rendered. The expression “the camera is drawn” designates the fact of generating (or rendering), from a 3D scene, a 2D image from the point of view of the camera. The camera is therefore associated with a graphics pipeline implemented by the processing units.

[0062] The graphics pipeline consists of a sequence of algorithmic bricks aimed at pixelating the geometric objects of the 3D scene in order to obtain the 2D image.

[0063] Finally, the term “camera frequency” refers to the number of 2D images per second generated by the camera, which also corresponds to the number of iterations per second of the camera.

[0064] [Fig. 5] schematically represents the 3D scene 50 drawn and displayed in a display area by the main camera 120 and in which it is desired to check the visibility of one or more objects. This scene 50 is composed of a plurality of 3D objects 501, 502, 503, 504, each object having an identifier. The objects are defined by a list of geometric shapes and attributes (texture, etc.).

[0065] The main camera 120 is associated with a main graphics pipeline.

[0066] The main camera 120 generates and displays to the user applying a series of successive images to a display area (for example occupying all or part of the screen 32 illustrated [Fig. 3]). The time between two images in the series of images depends on the frequency of the main camera 120. This is adapted to human vision, to provide the fluidity necessary for real-time viewing of the scene. At a minimum, the image production frequency of the main camera 120 is 30 images per second. Preferably, the frequency of the main camera 120 is 60 images per second. It may also be 120 images per second.

[0067] The dedicated camera 420 is a clone of the main camera 120, that is to say that it follows at all times the position and the characteristics of this main camera 120 (position, scale, filter options, cutting planes, etc.). For example, it reproduces the effects of shadows, lighting, etc. identically to the main camera 120.

[0068] The dedicated 420 camera differs from the 120 camera in several respects.

[0069] First, the dedicated 420 camera is associated with a dedicated graphics pipeline, modified with respect to the main graphics pipeline to integrate the algorithms for implementing steps S210 to S270 of the method 200.

[0070] Secondly, the dedicated camera 420 renders a so-called dedicated image, in an “offscreen” manner according to English terminology. This means that this dedicated image is not displayed on the screen 31. The dedicated image is therefore not intended for the user of the application (unlike the main rendering displayed by the main camera 120), but only for calculating the visibility score.

[0071] Third, the image production frequency of the dedicated camera 420 may be lower than the frequency of the main camera 120.

[0072] The image production frequency of the dedicated camera can thus be less than or equal to a quarter of the image production frequency of the main camera.

[0073] Fourthly, the resolution of the dedicated image rendered by the camera 420 may be degraded, i.e. lower than the resolution of the main rendering. The resolution is for example defined by the dimension of the dedicated image. Preferably, the smallest dimension of the dedicated image is 96 pixels, the largest dimension being calculated relative to the dimensions of the screen 31.

[0074] Reducing the frequency and resolution of the dedicated 420 camera allows the performance of the graphics engine not to be disrupted.

[0075] Finally, it should be noted that the dedicated camera 420 and the main camera 120 operate sequentially. In other words, one of the two cameras 120, 420 is drawn before the other. The rendering of the dedicated image is thus triggered with a slight time lag, for example a time lag of 33 ms, compared to the rendering and display of the scene image in the main window.

[0076] Once the dedicated camera 420 has been declared in step S205, steps S210, S220, S230, S240, S250 and, optionally, steps S260 and S270, are successively performed, at each iteration of the dedicated camera 420.

[0077] Step S210 of the method 200 is carried out from the 3D scene 50 described previously in relation to [Fig.5]. This scene is the one that will be displayed by the main camera 120 during its next iteration.

[0078] Step S210 is a step of selecting one or more objects 600 from among the plurality of objects 501, 502, 503, 504 of the 3D scene 50. In the example given, an object 600 is selected, corresponding to the object 502 of the scene 50.

[0079] The objects 600 selected are those whose visibility we want to measure in real time, via a visibility score calculation.

[0080] The selection step S210 produces, in the memory area 301 of the system 3 implementing the method 200 according to the invention, a list of identifiers associated with each selected object 600, this list comprising the geometry of the object and its texture. The texture of the selected object 600 is typically an image, called a texture image. At this step, the texture 510 of the object 600 is the one that is displayed to the user. It will be called “real texture 510”, or “real texture image 510” of the selected object 600.

[0081] Step S210 is followed by a step S220 which consists of exchanging the real texture image 510 of the selected object 600 with a texture image comprising only one predetermined color.

[0082] For this, during a sub-step S2201 of step S220, a texture image 610 is associated with the selected object 600 (or with each selected object) which comprises only one predetermined color 6101. It is also possible, during a sub-step S2202 which can take place before or after the association step S220-S2201, to extract and save in memory the real texture image 510 of the selected object 600 (or of each selected object). The term “save” designates a saving by reference of the texture image, and not a copy, in memory, of the texture image.

[0083] Thus, at the end of step S220, and with reference to [Fig.6], a scene 60 is obtained, called initial, identical to the 3D scene 50 which will be displayed by the main camera 120 with the exception of the selected objects 600 which will have taken a plain texture 610, of predetermined color 6101. All the other attributes (or settings and effects) of these selected objects 600 are preserved, for example the light, the shadows, the “shaders”, etc.

[0084] By solid texture, we therefore mean that a single predetermined color 6101 is used for the same object 600.

[0085] Preferably the predetermined color 6101 is a color generally not very present in a 3D scene, such as pink, with red components equal to 255, green components equal to 0 and blue components equal to 255 (or RGB(255,0,255)).

[0086] When multiple objects are selected, the same predetermined color 6101 or different predetermined colors may be used.

[0087] Step S220 is extended by a step S230 of generation, by the dedicated camera, of a dedicated image of the real scene 60 obtained at the end of step S220.

[0088] [Fig.7] shows an example of a dedicated rendering 70 of the actual scene 60 comprising the objects 501, 503, and 504 and the selected object 600.

[0089] The selected object 600 is in the background.

[0090] The first object 501 is at a distance from the selected object 600 and has no influence on it.

[0091] The second object 503 is in an intermediate plane and partially covers the selected object 600.

[0092] The third object 504 is in the foreground. Furthermore, this third object 504 is transparent, covers the second object 503, and partially covers the selected object 600.

[0093] The selected object 600 has taken on a new color 710, which depends on the scene and the predetermined color 610 assigned in the previous step S220.

[0094] The new color 710 in fact results from a modification of the solid texture 610 due to the presence of objects in planes between the plane of the selected object and the camera 420, or due to the presence of shadow zones covering the selected object 600.

[0095] In the example of the dedicated image 70 illustrated in [Fig.7], the color 710: • has not been changed from the predetermined color 6101 in the area 711 where the selected object 600 has not been covered, • has been modified from the predetermined color 6101 with a certain amplitude in the area 712 where the selected object 600 is covered by the third transparent object 504, • has been modified in area 713, with a greater amplitude than in area 712, where the selected object 600 is covered by the second object 502.

[0096] Thus, the more the selected object 600 is visible (not covered, as in area 711), the smaller the color difference between color 710 in the dedicated image and the predetermined color 6101.

[0097] Conversely, the less visible the selected object 600 is (covered by several objects, as in the zone 713), the greater the color difference between the color 710 in the dedicated image and the predetermined color 6101.

[0098] The color difference between the color 710 in the dedicated image (70) and the predetermined color texture 6101 is therefore correlated to the state of visibility or invisibility of the selected object 600: the greater the difference, the less visible the object is, and, conversely, the smaller the difference, the more visible the object is.

[0099] The dedicated image 70 is therefore a rendering essentially identical to the main rendering which will be generated during the following iteration of the main camera 120. A difference is the selected object or objects 600 which will have taken on a new color 710. Another difference may be, depending on a parameter setting of the implementation of the invention, the resolution of the dedicated image 70.

[0100] In other words, it is the same scene which is rendered on the dedicated image 70 and on the image displayed by the main camera 120, but for certain objects corresponding to the selected objects 600, the real textures 510 are changed by plain textures 610, of the predetermined color 6101, for the rendering of the dedicated camera 420, but remain used by the main camera 120.

[0101] Steps S240 and S250 which follow step S230 aim to measure this color difference and to represent it by a visibility score.

[0102] Step S240 is performed from the dedicated image 70. It consists of reading the color 710 of the object 600.

[0103] This step S240 can be implemented by adding a “compute shader” in the dedicated pipeline, at the end of rendering, by the dedicated camera 420.

[0104] With reference to [Fig.8], the color 710 of the selected object 600 is read at least one point 810 belonging to the object 600.

[0105] Preferably, the color 710 is read successively at each point of a plurality of points belonging to the object, these points being determined by a prior step (not represented in [Fig.2]) of dividing the object into areas to be tested, and determining a central point in each area to be tested.

[0106] The number of points is for example 16. One point corresponds to one pixel.

[0107] Depending on the resolution chosen for the dedicated image, 810 points may overlap. In this case, the color reading is carried out several times at the same point.

[0108] This is for example the case when the resolution of the dedicated image 70 is minimal (96 pixels for the smallest dimension of the dedicated image). The size of the object selected on the image can then occupy only a few pixels.

[0109] Step S250 extends step S240. It consists of comparing the color 710 obtained on the dedicated image 70 and the color 6101 predetermined at each measurement point 810.

[0110] The result of this color comparison is advantageously between 0 (identical or very close colors) and 1 (opposite or very distant colors).

[0111] To compare colors, two calculation techniques can be performed.

[0112] The first technique consists of measuring a color difference by calculating, for each red, green and blue component, the absolute difference between the color 710 read and the predetermined color 6101, then averaging these calculated absolute differences.

[0113] The second technique is to combine a hue comparison with the color difference measurement. Hue comparison makes it possible to detect that one color is identical to another, even if the lighting is different. Thus, the comparison is more accurate and more robust to changes in scene brightness.

[0114] The second technique more precisely comprises the following steps: • Determination, from the red, green and blue components, of the shade of the color 710 read and the shade of the predetermined color 6101, • calculation of the difference between the read shade and the shade of the predetermined color, • weighting of the result of the difference calculated between the read shade and the shade of the predetermined color, the result of the weighting being between 0 and 1 and determining the difference in shade. • comparison of the difference in shade with a predetermined threshold value, such as the value 0.25: • If the hue difference is less than or equal to the threshold value, the difference between the read color and the predetermined color is equal to the average of the hue difference and the color difference, • If the hue difference is greater than the threshold value, the difference between the read color and the predetermined color is equal to the color difference.

[0115] The final result is a value between 0 (close or identical colors) and 1 (opposite or distant colors).

[0116] A partial visibility score Sp is then determined from the final result, this partial visibility score being between 0 (test zone 800 not visible) and 1 (test zone 800 visible).

[0117] We therefore obtain a partial visibility score Sp for each test point 810 (i.e. each test zone 800).

[0118] The “final” visibility score S of the selected object 600 can then be determined by calculating the average of all the partial visibility scores Sp obtained in step S250, this final score S also being between 0 (object not visible) and 1 (object visible).

[0119] It should be noted that the partial visibility scores Sp make it possible to check the visibility of the selected object 600 in the following cases: • Object 600 was partially obscured by a semi-transparent object, • The surrounding light in the scene is too intense or too weak and makes the object difficult to see, • The object undergoes a reflection which prevents its surface from being clearly seen.

[0120] The visible / non-visible state can be determined from the final visibility score S of the object 600 and a threshold value.

[0121] The final visibility score S and / or the partial visibility scores Sp are then communicated, via communication means (not shown) to the user of the application or to other users.

[0122] When the actual texture image 510 is saved in step S220, step S250 may be followed by a step S260 of exchanging, in the initial scene 60, the solid texture image 610 of the predetermined color 6101 with the actual texture image 510.

[0123] To do this, each selected object 600 is associated with the texture image 510 extracted and saved during step S220.

[0124] Since the main camera 120 and the dedicated camera 420 operate sequentially, the rendering of the main camera 120 takes place before or after that of the dedicated camera 420, therefore before having replaced (step S220) the real texture image 510 with the plain texture image 610, or after having restored (step S260) the real texture image 510 while the plain texture image 610 was used for the rendering 70 of the dedicated camera 420.

[0125] In other words, the fact that the rendering of the two main and dedicated cameras 120, 420 is never carried out simultaneously makes it possible to always use the image 510 of actual texture for the main 120 camera, and the solid texture 610 image for the dedicated 420 camera.

[0126] A step S270 can also be performed following step S260, which consists of adding a visibility criterion linked to the size of the selected object 600 in the scene 50. This step S270 aims to indicate as not visible any object 600 whose size is too small compared to the size of the display area. This may be because the object is located at a great distance from the camera 120.

[0127] For this, step S270 consists of determining a surface occupied by the selected object 600 on the dedicated image 70, and a surface ratio between the surface occupied by the object determined previously, and the surface of the dedicated image 70, if this ratio is less than a threshold value, for example 1.5%, then the visibility score is equal to 0, the selected object being considered too small to be visible.

[0128] A preferred embodiment of the method 200 is described below in relation to [Fig.9].

[0129] In this preferred mode, the visibility score S is calculated by taking the lowest visibility score from a first visibility score SI calculated during a first iteration II of the dedicated camera 240, and a second visibility score S2 calculated during a second iteration 12 of the dedicated camera 240.

[0130] The first and second iterations II, 12 are successive iterations.

[0131] The predetermined color chosen for the second iteration 12 is opposite on the color wheel to the predetermined color chosen for the first iteration. For example, if pink RGB(255,0,255) is chosen for the first iteration II, then green RGB(0,255,0) will be chosen for the second iteration 12.

[0132] This preferential mode has the effect of eliminating false positives.

[0133] To explain this, let's take the example of a scene that is primarily pink in color. If we use a predetermined color that is also pink, then the selected object 600 will be abnormally detected as visible during a single iteration, even if it is hidden behind another object in the scene. The colors are indeed close.

[0134] Similarly, if the object 600 to be tested is hidden behind another pink-colored object, it will obtain an abnormally high visibility SI score with a first iteration II using a pink-colored texture.

[0135] By using a second color opposite to pink during the following iteration, for example green (with red, green, blue components respectively equal to 0, 255 and 0), the object 600 will be detected with a much lower visibility score S2, due to the difference between pink (of the scene or the object which hides the object to be tested) and green (of the object 600 to be tested). By retaining this second visibility score S2, the object is precisely detected as not or barely visible, and a false positive is avoided.

[0136] Conversely, if the object to be tested is hidden by another green object, the first iteration will give a low score which corresponds to the real visibility of the object to be tested in the scene.

[0137] It should also be noted that it is not possible to obtain false negatives.

[0138] Thus, the method 200 according to the invention makes it possible to monitor visibility in real time of an object in a 3D virtual scene, this via a visibility score associated with the object combining precision and efficiency on a wide range of configurations and rendering pipelines, while being simpler to implement than the stencil buffer technique described in patent FR3112638B1.

[0139] Such a method makes it possible, for example, to address the problem of managing memory and computing resources as closely as possible to requirements. Indeed, thanks to the visibility score thus calculated, it is possible not to load into, or unload from, the memory the texture associated with an object that is not visible.

Claims

1.

2. Claims Method (200) for calculating a visibility score (S, Sp) of at least one object (600) in a three-dimensional scene (50), the scene (50) being composed of a plurality of objects (501, 502, 503, 504) and displayed in a display area by a main camera (120) associated with a main graphics pipeline, characterized in that it comprises the following steps implemented by a dedicated camera (420) clone of the main camera (120) and associated with a dedicated graphics pipeline: - Selection (S210) of at least one object (600) from among the plurality of objects in the scene, - For at least one object (600) selected: • Association (S220) of the object (600) with a texture image (610) comprising only one predetermined color (6101) and replacing the texture image (510) of the object associated with the main graphics pipeline, - Rendering (S230), by the dedicated camera, of the scene to produce at least one dedicated image (70), - For at least one object (600) selected, • Reading (S240), in the dedicated image (70), of the color (710) at at least one point (810) corresponding to the object (600) in the at least one dedicated image (70), • Comparison (S250) of the read color (710) with the predetermined color (6101) to obtain the visibility score (S) associated with the object (600), the main (120) and dedicated (420) cameras operating sequentially, such that the dedicated image (70) is produced before or after the display of the scene in the display area. Method (200) according to claim 1 characterized in that the visibility score (S) is calculated by taking the lowest visibility score from a first visibility score (SI) calculated during a first iteration (II) of the dedicated camera (420), and a second visibility score (S2) calculated during a second iteration (12) of the dedicated camera (420), the first and second iterations (II, 12) being successive, the predetermined color chosen for the second iteration (12) being opposite on the color wheel to the predetermined color chosen for the first iteration (II).

3. Method (200) according to one of claims 1 to 2 characterized in that the predetermined color (710) is chosen from a list formed of at least: pink with red, green, blue components respectively equal to 255, 0, and 255, and green with red, green, blue components respectively equal to 0, 255, 0.

4. Method (200) according to one of claims 1 to 3 characterized in that the reading (S240), in the dedicated image (70), of the color (710) of the selected object (600) is carried out on a plurality of points (810) corresponding to the selected object (600) in the at least one dedicated image, for example 16 points, the points of the plurality of points being obtained by cutting the selected object (600) into zones (800) to be tested, and by taking the central points (810) of the zones to be tested.

5. Method (200) according to one of claims 1 to 4 characterized in that the comparison of the color (710) read with the predetermined color is obtained by calculating a color difference by: - calculating, for each red, green and blue component, the absolute difference between the color (710) read and the predetermined color (6101), and - calculating the average of the calculated absolute differences.

6. Method (200) according to claim 5 characterized in that the comparison of the color (710) read with the predetermined color is further based on a calculation of the difference in hue comprising the following steps: - Determination, from the red, green and blue components, of the hue of the color (710) read and the hue of the predetermined color (6101), - calculation of the difference between the hue read and the hue of the predetermined color, - weighting of the result of the difference calculated between the hue read and the hue of the predetermined color to obtain the difference in hue, - comparison of the shade difference with respect to a predetermined threshold value such as the value 0.25: • If the shade difference is less than or equal to the threshold value, the difference between the color (710) read and the predetermined color (6101) is equal to the average of the shade difference and the color difference, • If the shade difference is greater than the threshold value, the difference between the color (710) read and the predetermined color (6101) is equal to the color difference.

7. Method (200) according to one of claims 1 to 6 characterized in that the image production frequency of the dedicated camera (420) is lower than the calculation frequency of the main camera (120).

8. Method according to claim 7 characterized in that the image production frequency of the dedicated camera is less than or equal to a quarter of the image production frequency of the main camera.

9. Method (200) according to one of claims 1 to 8 characterized in that the resolution of the dedicated image is lower than the resolution of the rendering of the main camera (120).

10. Method (200) according to claim 9 characterized in that the smallest resolution of the dedicated image (70) corresponds to an image width of 96 pixels.

11. Method (200) according to one of claims 1 to 10 characterized in that it further comprises a step (S270) of testing the size of the selected object (600), comprising the determination of a surface occupied by the selected object (600) on the dedicated image (70), and of a surface ratio between said surface occupied by the object and the surface of the dedicated image, if this ratio is less than a threshold value, for example a threshold value equal to 1.5%, then the visibility score is equal to 0, the selected object being considered too small to be visible.

Images