METHOD AND MODULE FOR GENERATION OF REAL-TIME VISUAL OR AUDIOVISUAL DATA

Abstract

Method (P) of generating an image stream by a computing unit, each image being formed of a plurality of planes, each of the planes presenting a plurality of objects (O1, O2) arranged on said plane, each of the objects of a plane presenting a plurality of parameters (P1, P2, P3), the method comprising: • an assignment step (A1), for each of the parameters of the objects: of so-called primitive data (E2), comprising motion data, rhythm data and intensity data; • an assignment step (A2), for each of the planes: of object data (E1) comprising a plurality of objects, each of the objects having coordinates, called associated with the object, in the plane, of so-called primitive data (E2), comprising motion data, rhythm data and intensity data;• a generation step (G) of the image stream (E3) in which, for each of the planes, each of the objects is initially placed at its associated coordinates, the parameters of each object initially placed on the plane are determined and evolve from the primitive data assigned to them, a sequence of displacements is applied to each object initially placed on the plane, the sequence of displacements being determined from the primitive data of the plane. Figure 1;

Description

Title of the invention: METHOD AND MODULE FOR GENERATING VISUAL OR AUDIOVISUAL DATA IN TIME REAL Field of invention

[0001] The present invention relates to the audiovisual field in the broad sense, both as visual representation (design, manipulation and diffusion of digital imagery, design and manipulation of visuals of colors and lights via projection and lighting devices or display screens), as well as as sound representation and expression (manipulation, capture, creation and diffusion of sound flows, whether they are of the order of sound effects, Sound Design, or musical creation in the studio or in public, soloist or concert, DJ.

[0002] The invention also concerns and touches the field of digital or analog event installation and broadcasting (sound and light); it is intended to be a direct bridge between the manufacturing, manipulation, broadcasting of audiovisual multimedia content, and the impact on the public linked to an event organization. It thus particularly touches the field known as VJing, which is a broad term that designates real-time visual performance, which is characterized by the broadcasting of visual content during a musical event.

[0003] Finally, the invention may also relate to the creation, manipulation, and diffusion of multimedia for the fields of interior or exterior space design, home automation or point-of-sale advertising, as well as the fields of chromotherapy, art therapy, or any form of therapy which would involve visual and sound elements.

[0004] Sound and image have completely invaded our modern lifestyle, and all the corresponding modes of expression and communication which, today, are almost entirely based on two of our senses: sight and hearing.

[0005] The desire to find a correlation between light, color and sounds related to musical creation is far from new. Isaac Newton, upon his discovery of the decomposition of white light into seven different primitive colors, instinctively associated these seven colors with the seven notes of a musical octave, thus attempting to demonstrate a mental structure underlying these two fields located in human perception. Then it was the German philosopher, physicist, musician and inventor Athanasius Kircher who would more firmly theorize a link between the eye and the ear in his Musurgia Universalis written in 1650. The first plans for a device combining sound and light were made by Louis Bertrand Castel in 1725 through his invention: the ocular harpsichord, which, according to him, allowed a concert to be played to a deaf person with colors and light. However, his invention remained at the planning stage and was not built.

[0006] The first real device would not appear until the American inventor Bainbridge Bishop, who filed a patent for a color organ in 1877. It was a set of lights that were attached to a tube organ and projected colors onto a screen depending on the note played. It was then the Englishman Alexander Wallace Rimington who also manufactured and filed a patent for his light keyboard in 1893.

[0007] In the 20th century, with the development of cinema, audiovisual and electronics, new, more advanced systems appeared: video synthesizers. Designed by the most ambitious artists and electronics engineers such as Nam Jun Paik or Steve Rutt and Bill Etra, these devices made it possible to transform source videos according to spatial (width, height, position), temporal (speed, display frequency), or visual (color) parameters, according to the wishes of the artists to obtain interesting abstract visual objects.

[0008] These visual systems accompanied, throughout the 70s and 80s, the emergence of club culture and Disk Jockeys (DJs), until they became an indispensable asset when organizing parties, giving rise to what the videographer Merrill Aldighieri called Video Jockeys (VJs), a visual counterpoint to DJs, who were responsible for the visual and multimedia atmosphere of the scenes.

[0009] The 1980s also saw a boom in electronic technologies with the arrival of microprocessors. The first video synthesizers were then marketed, allowing numerous computer-generated effects to be applied to video sources. Among these devices, we can remember the name of the Fairlight Computer Video Instrument (CVI), which revolutionized the world of television and real-time video editing by allowing high-quality abstract images to be obtained quickly from a source.

[0010] From the 90s, micro-computer technology allowed an explosion of visual design systems, with software like Photoshop, After Effects, or even Flame and Infemo, driven by the ever more creative demand for Visual Effects in cinema, the generation of artificial figurative or abstract images quickly became an essential communication tool.

[0011] The audiovisual sector is thus seeing the development of disciplines based entirely on the production, animation and movement of visual content for communication, advertising or cinema. We will then speak of "Motion Design". Motion Design (or Motion Graphics) is a particular branch of graphic design, the aim of which is to set graphic elements in motion. Over time, it has become an art form in its own right.

[0012] By adapting these technologies to the world of real-time performance, artists can get a little closer to instant visual creation, combining real-life shots and computer-generated generative art. Among the communities of computer-based visual content creators are DemoScene participants, also known as DemoMakers. These are programmers capable of creating visual and audio elements based on the fewest possible lines of code. Every year, DemoMaking competitions are held, allowing each team to showcase their visual and audio programming skills and compete to determine the best team.

[0013] The latest major development in terms of visual event communication was the arrival in the 2000s of the phenomenon of Projection Mapping (or video mapping), which consists of projecting videos onto surfaces such as walls, monuments, or objects in volume and thus giving a dynamic visual dimension to a particular place. Several software programs have appeared such as MadMapper, HeavyM, Resolume, or Notch. They allow visual content to be broadcast live and deformed to adapt it to surfaces.

[0014] The present invention falls directly within the aforementioned line of all the research and practice of an instant audiovisual art form. As a technical audiovisual generative system, it allows for creative expression in the moment via this system, with the possibility of improvisation. State of the art

[0015] Currently, there are three types of audiovisual systems that allow the design of animated multimedia content that can be used directly or subsequently for real-time broadcasting in an event, exhibition or stage context: • Digital content creation systems, or Digital Content Creation (DCC), such as: After Effects, Cinema4D, Blender, Maya or Smode. • Visual state programming systems, or Visual Coding, which are most often manifested by a nodal graphical representation (node ​​trees or graphs), such as: PureData, QuartzComposer, Touch-Designer, VVVV, Notch or PraxisLive. • Pure programming interfaces: Processing or GLSL with Kodelife.

[0016] Some software mixes several aspects and allows, for example, the inclusion of lines of code while remaining content creation or stateful visual programming software.

[0017] Digital content creation tools were originally invented for the creation of digital images in film and television, and their evolution has been depending on the evolution of these fields, namely Visual Effects and Motion Design. They inherit directly from the cinematographic design of images and are therefore based on a model derived from film. The creation of a visual by a digital content creation system thus first involves the creation of a sequence that will contain the evolution of the visual. Within this sequence, the evolution of the animation will be carried out by "key frames", that is to say by marking in time the different states of the animated objects and by interpolating these states.

[0018] Once the sequence is created, it must be rendered as a video file with an immutable timecode and cannot be modified thereafter. This is therefore a linear production model. Videos can be broadcast, and effects or filters can be applied to them in the broadcast systems, but their content is fixed when the sequence is rendered and therefore remains immutable.

[0019] Stateful visual programming systems are lightweight programming tools, where each function is most often represented by a block, and the entire execution of the script is represented by a tree connecting each of the blocks: the graph. As a programming system, they often allow the user great flexibility in the choice of functions used. The creation of a programmed procedural system takes a lot of time, requires knowledge of computers, programming, and electronic equipment if the content is to be broadcast. These systems require designing the scene upstream and then deploying it on the ground. The prior knowledge necessary for their use hinders their democratization within the general public and the artistic community.

[0020] Finally, creative programming languages ​​are reserved for programmers and computer scientists who wish to create visuals or generative art directly from lines of code. These languages ​​are capable of interpreting commands in near real time. This is pure programming, even if it is simplified by the language used.

[0021] All these contents once created can be sent to a diffusion or projection mapping system in order to adapt to the environment. Among the common diffusion software, we find: Resolume, MadMapper or HeavyM. These software programs most of the time have configurable predefined effects and filters, allow to mix pre-existing elements (created for example in DCC tools) and to apply these filters. These tools also most of the time allow to react to external control hardware such as MIDI or OSC type instruments, in order to control the diffusion and the behavior of the filters.

[0022] If a user wants to share content between all these tools directly (without having to first calculate the video renderings), he needs the latest software allowing you to share graphics memory between different applications. For this purpose, on PC we use Spout, while on Mac we prefer to use Siphon.

[0023] Creating a complete environment for broadcasting and creating multimedia content in a real-time context is therefore a task that requires a great deal of technical expertise, several software programs that must be used and linked together and which, ultimately, will not really allow for improvisation, because they are limited most of the time to content produced according to a linear logic and rendered in the form of an immutable file.

[0024] The current set of tools does not allow improvisation or the instantaneous expression of an audiovisual artistic intention because of this linearity.

[0025] The current technical problem of the audiovisual is that its current forms of manifestation are subject to a linearity resulting from a top-down manufacturing process: it is necessary to construct a first time line upstream (called track, sequence or in English timelinef to which the rest of the process will be subjected. Whether it is editing, shooting, or animation, all the current processes allowing the creation of an audiovisual object with a view to its diffusion in a real-time context, are prisoners of this temporal linearity which cannot be modified once established.

[0026] This linear model is directly inherited from a cinematographic paradigm which consists of seeing the film, and by extension the video, as a sequence of images; and current systems do not take into account the latest research carried out on non-linear processes.

[0027] The main symptom of this technical limitation to a linear representation is the almost total absence of the possibility of creating directly on the ground or of modifying an audiovisual element once it has been produced, which closes the door to any form of improvisation.

[0028] There is currently no tool that allows for instantaneous visual creative expression in motion. There are audiovisual systems that allow for the development of non-linear visual and sound objects, however these systems are extremely complex and require knowledge of programming and therefore of computers and are not accessible at first glance. In addition, once the elements are programmed, they are no longer really modifiable for improvisation.

[0029] The invention is situated as a bridge between the codes of visual creation and the creative structure naturally open to improvisation that is music. The invention can therefore be seen as an instrument and aims to overcome at least one of the aforementioned drawbacks. Statement of the invention

[0030] To this end, according to a first aspect of the invention, there is proposed a method for generating a stream of images by a calculation unit, each of the images being formed from a plurality of planes, each of the planes presenting a plurality of objects arranged on said plane, each of the objects arranged on the plane presenting a plurality of visual and behavioral parameters, the method comprising: • an allocation step, for each of the object parameters: • data, called primitives, comprising movement data, rhythm data and intensity data, • an attribution step, for each of the plans: - object data comprising a plurality of objects, each of the objects comprising coordinates, called associated, in the plane, - data, called primitives, comprising movement data, rhythm data and intensity data, • a step of generating the image stream in which, for each of the planes: - each of the objects is initially arranged at its associated coordinates, - the visual and behavioral parameters of each object initially placed on the plane are determined from the assigned primitive data and evolve according to said primitive data, - a sequence of movements is applied to each object initially placed on the plane, the sequence of movements being determined from the primitive data of the plane.

[0031] The method may include other steps of allocation or reallocation subsequent to the step of generating the image stream. The allocation of primitive data to the parameters of an object makes it possible to act on its representation and its behavior, namely for example: its color, its opacity, its size, its rotation angle, its lifetime.

[0032] Preferably, the movement data is generated by the user in real time using a pointing tool.

[0033] Preferably, the intensity data is generated by the user in real time using a pointing tool.

[0034] According to a second aspect of the invention, a module is proposed for generating a stream of images, each of the images being formed from a plurality of planes, each of the planes presenting a plurality of objects arranged on said plane, each of the objects of a plane presenting a plurality of parameters, the module comprising a calculation unit configured to: • assign, for each of the object parameters: • so-called primitive data, including movement data, rhythm data and intensity data, • assign, for each of the plans: • object data comprising a plurality of objects, each of the objects comprising coordinates, said to be associated with the object, in the plane, • so-called primitive data, including movement data, rhythm data and intensity data, • generate the image stream in which, for each of the shots, • each of the objects is initially placed at its associated coordinates, • the visual and behavioral parameters of each object initially placed on the plan are determined from the data of the primitives which have been assigned to them and evolve according to this primitive data. • a sequence of movements is applied to each object initially placed on the plane, the sequence of movements being determined from the primitive data of the plane. Brief description of the figures

[0035] Other characteristics and advantages of the invention will appear during the reading of the detailed description which follows for the understanding of which reference will be made to the appended drawings in which: • [Fig.l] is a block diagram of an embodiment of a system according to the invention, and • [Fig.2] illustrates different predetermined intensity profiles recorded in a storage unit. Detailed description of the invention

[0036] The embodiments described below being in no way limiting, it will be possible in particular to consider variants of the invention comprising only a selection of the characteristics described, subsequently isolated from the other characteristics described, if this selection of characteristics is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art. This selection comprises at least one characteristic, preferably functional without structural details, or with only a part of the structural details if this part alone is sufficient to confer a technical advantage or to differentiate the invention compared to the state of the prior art.

[0037] An embodiment of a method P for generating a image stream by a calculation unit 2, at the same time as a system 100 according to the invention, integrating a module 1 for generating an image stream comprising said calculation unit to implement said method.

[0038] Each of the images in the image stream is formed from one or more planes.

[0039] The computing unit 2 is configured to receive A2, for each of the plans: • object data El comprising a plurality of objects, each of the objects comprising a plurality of parameters (PI, P2, P3) and coordinates, said to be associated with the object, in the plane, • data, called E2 primitives, comprising movement data, rhythm data and intensity data.

[0040] The calculation unit 2 is configured to receive Al, for each of the parameters of the objects: • data, called E2 primitives, comprising movement data, rhythm data and intensity data.

[0041] The calculation unit 2 is further configured to generate G an image stream E3 in which, for each of the planes: • each of the objects is initially placed at its associated coordinates, • the visual and behavioral parameters (P1, P2, P3) of each object initially placed on the plane (for example its color, its opacity, its size, its rotation angle, its lifetime) are determined from the data of the primitives which have been attributed to them and evolve according to this primitive data, • a sequence of movements is applied to each object initially placed on the plane, the sequence of movements being determined from the primitive data of the plane.

[0042] In the example shown, the module 1 is produced in the form of a touchscreen tablet comprising the calculation unit 2, the control of the module 1 being able to be hardware or manual. Intensity data: forces

[0043] In the invention the intensity data are called forces.

[0044] In the artistic sense of the term, we are not considering here a vector form of force, but rather an intensity I which will be impregnated with one or more parameters of objects or plane. The notion of object is defined further on.

[0045] A force is a numerical sequence defining an evolution profile as a function of time. This time is discretized according to the classic expression t = nTe, where Te is the sampling period. The smaller the sampling period, the finer the granularity of the profile. In return, the profile has a greater number of points to calculate, implying an increase in calculation times. According to one possibility:

[0046] Te = 0.15 => f = 4- = 10 Hz

[0047] The force is by nature periodic, but one can imagine cases where it would vanish after a single application.

[0048] The force profile is therefore defined over exactly one period T = NTe. Without loss of generality, it is possible to arbitrarily set the duration of the period T to 1 minute (this choice is linked to the unit of rhythm which will be discussed in the next paragraph). Thus, by default, all force profiles have:

[0049] N = f- = 600 points

[0050] that is to say in memory an array of 600 single precision floating numbers.

[0051] The computing unit may be configured to implement a predetermined force profile stored in a storage unit.

[0052] The predetermined force profile is for example a sinusoidal profile, referenced fl in [Fig.2]. This is an oscillating profile. The mathematical expression is given by:

[0053] = cos(2t7^ )

[0054] The sinusoidal profile could have been a sin instead of a cos, but the advantage of the latter is that it starts at 1 (whereas sin starts at 0), which in practice has some advantages in terms of synchronization with the rhythm of the music.

[0055] The force profile is for example a logarithmic profile, referenced f2 in [Fig.2]. It is a “powerful” profile, with fairly strong amplitudes at the beginning of the period, but for a short duration, followed by a phase of quasi-linear decrease to a zero value. The mathematical expression is given by:

[0056] / 2(f) = -log(J±9

[0057] The predetermined force profile is for example an exponential profile, referenced f3 in [Fig.2]. It is an increasing profile, starting from a unit amplitude to peak fairly quickly at the largest of the proposed amplitudes. The mathematical expression is given by:

[0058] f3(t)=e£

[0059] It is possible to play on the “acceleration” of the curve by adjusting the coefficient 3 / 2.

[0060] The force profile is for example a stepped profile, referenced f4 in [Fig.2]. This is an “all or nothing” type profile, with an abrupt transition zone in the middle of the period. The mathematical expression is given by: [OObi] = 2 sgn {f-1}

[0062] The invention also makes it possible to create personalized forces, not derived from mathematical expressions, but directly determined by the user by means of of moving a tool on a touch screen, the tool being the user's finger. Rhythm data

[0063] In the previous paragraph, we introduced the time variable t. This time does not necessarily correspond to that which elapses in reality.

[0064] If one wishes to modify the frequency of the oscillations, the natural way to do it would be to reduce it to cos ( j T ) — COS ( 2tf ftj and to want to change the frequency / 0 by applying a coefficient to it. This view does not apply here. Indeed, the time t associated with the force always covers the interval [0; T]. Modifying the formula would lead to constructing either a truncated sinusoid or a repeated pattern, but not necessarily a whole number of times, which is an undesirable result in both cases.

[0065] The invention proposes to introduce rhythm data B. The rhythm, whose unit is in BPM (for the English beats per minute), determines in the model the number of times the force profile is covered in one minute. Thus, for a rhythm of 1 BPM, the profile is covered 1 time, and 1 minute elapses. This is the only case where the apparent time coincides with the absolute time. For a rhythm of 100 BPM, the force profile is covered 100 times per minute. The apparent time, also called frequency time (it is no longer homogeneous to seconds), therefore elapses 100 times faster than the absolute time. If we note the value of the rhythm, we therefore have:

[0066] = V t

[0067] To change the frequency of the oscillations, simply change the value of the rhythm (for example, double the rhythm to double the frequency of the oscillations).

[0068] It is possible to establish a correspondence between the time being measured and the corresponding index of the cell in the force table, for the force being evaluated, subject to the chosen rhythm. The origin of the times is set at the start of the application. We call T the current absolute instant modulo T:

[0069] tfreg = r[T]

[0070] The index sought is none other than:

[0071] n =

[0072] Suppose that the real elapsed time t is in milliseconds, then the expression becomes:

[0073] „ _ mod 60000 11 “ 1000 / 10

[0074] Considering an offset on time, the final expression of the index n is therefore:

[0075] „ _ ^,)^60000 n~ 1000 / U)

[0076] The computing unit may be configured to implement a predetermined rate

[0079] rX1 [y * recorded in a storage unit.

[0077] The invention also makes it possible to create personalized rhythms, directly determined by the user using a tool on a touch screen, the tool being able to be the user's finger. Motion Data: Vector Fields

[0078] The calculation unit is configured to apply a displacement to the center of each object, identified by its coordinates (x, y). Thus, at any point in space, a displacement vector V forming a field is made to correspond: X(x, y)ly).

[0080] If, at time t, the coordinates of the center of an object are (x(t), >Û)), then at time t + A / ; the new coordinates of the center become:

[0081] %(t + At) =x(t) +X(x(f), y(t)) y(t+At) = y(t)+K{x(t), y(t))

[0082] At is called the refresh period. It is the time that elapses between the display of 2 consecutive images. It is therefore the inverse of the refresh frequency, called FPS (for the English freintes per second). Without loss of generality, it is proposed to arbitrarily set a reference duration for the refresh period At at 40 milliseconds: this choice corresponds to a value of 25 FPS, a usual value in the audiovisual field. However, the generation module 1 can be set according to a preferred range of 10 FPS to 100 FPS.

[0083] A constant is introduced in the previous formula, rewritten in a simplified way:

[0084] x —*==--------x + aX(x. v) fps , with a = y ——--------y + ax(x, y) FPSuscr

[0085] When a = 0.5, images are refreshed at a FPSuser frequency twice as high as the reference FPSref frequency.

[0086] Another constant is introduced to constrain the movements:

[0087] x—~------x + aaX(x,y) with <7 = 0.1 by default y •—==--------y + aaX(x. y)

[0088] Other values ​​of the constant o are possible, such as 0.01 or 0.02.

[0089] By incorporating the force (displacement intensity) applied to the object, the displacement expression is written:

[0090] x “----------x + af{t)(yX{x, y) y ------------y + af(t)(jX(x, y)

[0091] The space is discretized according to a number of points Lx L, for example according to a value L of 21 x 21 points.

[0092] The computing unit may be configured to implement a predetermined vector field stored in a storage unit.

[0093] The predetermined vector field is for example a constant field, the mathematical expression of which is given by:

[0094] X(x,y)=l K(x, y) = 1

[0095] The objects are then moved in a straight line. It is also possible to play on the angle 0 — arctan(-^ ) 'a trajectory, with for example in the case above 3 = 45 °.

[0096] The predetermined vector field is for example a wave field, the mathematical expression of which can be given by:

[0097] X(xy)=3 K(x,y) = cosx

[0098] The objects are then moved horizontally to the right (the value 3 is to be adjusted according to the desired speed and its sign according to the desired direction), while oscillating slightly vertically.

[0099] The predetermined vector field is for example a circular field, the mathematical expression of which can be given by: [01001 r(*y)

[0101] Objects move concentrically, counterclockwise. The radius of the circular path is larger the further the object is placed from the center.

[0102] The predetermined vector field is for example a spiral field, the mathematical expression of which can be given by: ""i:l «(«)=$ Y ( x, y ) = -4~L

[0104] Objects move concentrically, counterclockwise. They move further and further away from the center as they move.

[0105] The predetermined vector field is for example a trigonometric field, the mathematical expression of which can be given by:

[0106] X ( x, y ) = xcosy Y (x, y) = y cosx

[0107] The invention also makes it possible to create personalized fields, not derived from mathematical expressions, but directly determined by the user by means of moving a tool on a touch screen, the tool being able to be the user's finger. Object data

[0108] There are four categories of object: • multimedia content, for English asset, • launchers, for the English launcher, • behaviors, for the English behavior, • plans.

[0109] Assets are objects placed on a plane, having internal parameters, these parameters can be visual or behavioral, such as position, rotation angle, opacity, color, lifetime, or any other parameter programmed inside the object to define its representation, behavior or appearance. The values ​​of the asset parameters are determined: • either by the data of the primitives which have been attributed to them; • either statically.

[0110] Assets move according to the movement determined by the primitive data of the plane on which they are placed. These can be either elements: • of a visual nature such as bitmaps (.bmp, .jpg, .png images or .gif animated images or .mp4, .avi videos) or procedural entities (drawn by code instructions); • of a sound nature such as audio streams (.wav, .mp3, .ogg sounds or streaming or microphone sounds) or procedural entities (played by code instructions).

[0111] Launchers are virtual objects placed on the plane. Launchers have their own primitives, independent of those of the plane on which they are placed. Thus, a launcher can move without connection to the plane, and place itself on the plane the desired type of object at regular time intervals. Thus, a launcher can, for example, periodically place either multimedia content or another launcher. In order to know how to move, the launcher receives 3 primitives: force, rhythm, movement (in case of omission, the launcher will remain immobile). In order to know at what interval to place its object, the launcher receives 1 primitive: rhythm (mandatory).

[0112] Behaviors are virtual objects that can be applied to one (or more) planes or one (or more) objects. Behavior is understood to mean any change in size, color, rotation, scale, opacity, or any other scripted behavior (described by code instructions). In order to know which transformation to apply, the behavior can expose inputs to receive primitives: force, rhythm or movement (in case of constant force, omission of force or rhythm, the com- behavior will adopt the default values).

[0113] The visual composition results from a multitude of objects whose parameters evolve according to the data of the primitives which have been assigned to them. A plane is a layer, that is to say a display surface having a transparent background, capable of receiving objects moving according to the primitives assigned to the plane. The final composition is constituted by the superposition of any number of planes, from the most distant (background), to the closest. The final composition can also be distributed over several rendering windows. Each rendering window then broadcasts all or part of the worked planes. Sequencer

[0114] A sequencer allows you to emulate primitives in a composite way. The type of the sequencer is determined by the type of the first primitive inserted into it. Once the type is fixed, only primitives of the same type can be inserted into it. The sequencer becomes in turn (in a loop or in a single application) each of the primitives constituting it, either successively, randomly, or according to a predetermined pattern.

[0115] Note: in order to know at what rate it must mutate, the sequencer receives 1 primitive: rhythm (mandatory).

[0116] For example, if we want to create a composite force having a rising action (increasing), a plateau action (constant) and a falling action (decreasing), all in a periodic manner and where each action lasts 2 seconds, we just need to create a sequencer in loop mode and of type force (in which we will insert the 3 increasing / constant / decreasing forces) receiving as input a rhythm of 30 BPM. Real-time audiovisual instrument

[0117] An embodiment of the invention may be compiled software, capable of running on a computer or a touchscreen tablet. This software embeds and implements the method described in the invention, this software also being called an instrument whose user is called artist A.

[0118] The instrument has in particular: • a system for managing forces, rhythms and movements; • a plan management system; • a system for managing objects and their parameters; • a composition space; • a library of assets allowing you to manage, manipulate, sort and order them

[0119] The instrument allows the artist to create his work on a working window and to distribute it on a plurality of rendering windows (whose content can possibly

[0120]

[0121]

[0122]

[0123]

[0124]

[0125] vary from window to window). Custom motion fields by “painting” In an alternative embodiment of the invention, the motion fields are no longer necessarily derived from mathematical equations, but can be directly traced by the user using a pointing tool, such as their finger on a touch screen, or otherwise such as a mouse. The notations used in this paragraph are as follows: • Coordinates of an anchor point, point initially pointed by the pointing tool: ( • Coordinates of a current point, pointed by the pointing tool: -y ) Gradient tools By implementing this tool, the artist can indirectly define a dividing line, around which vectors orthogonal to it and of attenuated standards according to its distance will be organized. The anchor point and the current point are sufficient to define a straight line. We want to determine the equation of the line that is orthogonal to it, and that passes through the anchor point. Knowing that the product of the slope coefficients of two orthogonal lines is -1, that of the desired dividing line is therefore:

[0126]

[0127] The coordinates of the anchor point must verify the equation of the dividing line, we can therefore deduce that the ordinate at the origin is:

[0128] b = y0-a.r0

[0129] The field that we wish to paint will contain vectors orthogonal to this line (therefore collinear with that drawn by the artist) and whose norm will be unitary when they are in contact, then all the weaker as their point of application is distant and the length of the segment drawn by the artist is short.

[0130] The distance between the line and the point of application is given by:

[0131] j _ |y-ax-b|

[0132] The length of the segment drawn by the artist is:

[0133] , r ~r~

[0154] Condition to respect collinearity: Y=— X / a because K = aX and aa — -1

[0155] Condition to respect an exponential decrease as a function of the distance: ^X2 + Y2 = eT

[0156] We can therefore solve the system:

[0137]

[0138]

[0139]

[0140]

[0141]

[0142]

[0143]

[0144]

[0145]

[0146]

[0147]

[0148]

[0149]

[0150]

[0151]

[0152]

[0153]

[0154]

[0155]

[0156] Finally the expression of the vector field sought is: X = -^= e-Ki Y = -7=== To obtain vectors only on the correct side of the dividing line, simply make sure that Sgn fy _ ax -b] - sgn y {- yo | • An empirical constant is introduced solely for the purpose of improving the artistic experience. In the source code, the value retained is K — 3. Brush tool Using this tool, the artist can directly imbue the vector field with the movement of the pointing tool. The speed of movement of the brush (i.e. the mouse) will reflect the intensity of the field at the point in question. There is no need for an anchor point for this tool. In one example, the artist works on a 21x21 grid while the number of pixels on the screen can reach 1920x1080. Since the number of points in the vector field is small compared to the number of pixels over which the brush can actually pass, the idea is not to simply paint the points in the field closest to those hovered over, but to paint them all, with an intensity that is lower the further they are from the brush stroke. The distance between the current point and the application point is given by: F d^^x-xj + (yy J If x। = X] ( t ) and y = y ( t ) characterize the position of the brush at time r, then their respective derivatives characterize its speed at the same instant. Condition to respect the link between intensity and speed: Xe* y dy dt The notion of a continuous-time derivative translates to a simple difference (between 2 consecutive values) in discrete time. Condition to respect an exponential decrease with respect to the distance: [X-^ l 7 « g-Kd. Finally the expression of the vector field sought is: This time again, the value of the constant is determined empirically, in the

[0157]

[0158]

[0159]

[0160]

[0161]

[0162]

[0163]

[0164]

[0165]

[0166]

[0167]

[0168]

[0169]

[0170]

[0171]

[0172]

[0173]

[0174]

[0175] sole objective of improving the artistic experience. In practice, a value that can be retained is K = 1 / 20. Radial tool Using this tool, the artist can draw a circle inside which a Newtonian field will be established, consisting of centrifugal (or centripetal) vectors of norms attenuated according to their distance, following the so-called "inverse square" law. Thus, each time we move away from the center of the circle by a factor of 2, the intensity of the field is divided by 4. We also want to involve the radius of the circle in the field intensity: the larger the radius, the more the norm of the vectors will be increased. The anchor point is the center of the circle. The distance between the current point and the anchor point is the radius of the circle: a = ^(xrxü)2+(yey^ The distance between the application point and the anchor point is: d=^(x-x0) +(y-y0) Condition to respect the link between intensity and radius: (X « a There is Condition for respecting the centrifugal (or centripetal) direction relative to the anchor point: (X = h(x-x0) [ Y = b^yy^ Condition to respect a quadratic decrease with respect to the distance: Jx 2 +y 2 - 4 * d- We can therefore solve: Jx2 + F2 -bJ(x-x0)"+ (v-vn I ^b--Y y V v -o / d" dj Finally the expression of the vector field sought is: The value of the constant that can be used is K = 1 / 25. Magnus Tool Using this tool, the artist can define a zone of turbulence inspired by the "Magnus effect" in fluid mechanics. The latter highlights the tangential force experienced by a rotating object moving in a fluid. It is this force that explains the change in trajectory, which then takes on a curved shape.

[0176] This tool is therefore intended to be used over a constant field, for example, in order to asymmetrically modify the velocity field around the object.

[0177] A NaN field, for Not a Number, is imposed inside the turbulence zone, so that no object can exist there.

[0178] The anchor point is the center of the turbulence zone. The distance between the current point and the anchor point is the radius of the turbulence zone:

[0179] ? a = ^xï-x0) +

[0180] We will subsequently use the angle between the current point and the anchor point relative to the horizontal to play on the direction of the depression: S^arctan^

[0182] We decide to place ourselves in a polar coordinate system whose origin is the center of the turbulence zone: 101831 r =

[0184] The literature in fluid mechanics teaches that, for a constant horizontal resting velocity field of amplitude V, the perturbed field is written in the coordinate system defined above as:

[0185] yr=Vcose^.^^V()=-Vsin0(ïY^)+^

[0186] The sign and magnitude of the constant G are related to the pressure difference in the fluid on either side of the turbulence zone.

[0187] According to the invention, the constant and horizontal field is replaced by the average field inside the turbulence zone, before the step where it is fixed to NaN.

[0188] Noting respectively A) and A) the amplitude and direction of this average field, the modified equations for the disturbed field are:

[0189] Vr=

[0190] The calculations to obtain these expressions are relatively classic and they have therefore not been detailed in this description. Returning to Cartesian coordinates:

[0191] X = VyosO - V ^inO Y = V^inQ + Vy:os3

[0192] The value retained can be G = (fis in ( - 0 ( ) / 20-

[0193] Displace tool

[0194] Using this tool, the artist can superimpose regular undulations over an existing field, stretching with varying intensity in a direction that would typically be orthogonal to the main field.

[0195] The anchor point is a wave antinode, but it is actually of little importance because the generated "waves" will extend over the entire plane. The distance between the current point and the anchor point corresponds to the stretched segment: a =

[0196]

[0197]

[0198]

[0199]

[0200]

[0201]

[0202]

[0203]

[0204]

[0205]

[0206]

[0207]

[0208]

[0209]

[0210]

[0211]

[0212]

[0213] To simplify, we initially consider the case of a horizontal main field and a vertical stretch. Condition to respect the link between the intensity and the size of the segment; Y has Condition for generating regular vertical waves: from where Y cosx Y= acosx We now need to generalize to any orientation. To do this, we will consider that we will always have a vertical undulation, but in a frame rotated by an angle: The expression of the required vector field is: X = - Kasind^cos (xcosB^ + ysinB}) The constant can be equal to K = 50. Twirl tool, for English swirl Using this tool, the artist can create circles, spirals and swirls, either internal or external. A NaN (Not a Number) field will be imposed in the center of the twirl, so that no objects can exist there. The anchor point is the center of the twirl. The distance between the current point and the anchor point corresponds to the power of the twirl: We will then use the angle between the current point and the anchor point relative to the vertical to play on the inclination of the twirl: B, = % + arctan^: Changing the tilt allows the artist to create very different visual effects: • Zero inclination: radial effect • Angular increase: spiral effect (external) • Critical point: circular effect • Angular increase: spiral effect (internal) Two special cases are therefore easily recognizable: • When = 0, the field is radial X = A • When ^ = -, the field is spiral / circular [ X = - y l Y = X We intuit the following trigonometric expressions (weighted by intensity):

[0214]

[0215]

[0216]

[0217]

[0218]

[0219]

[0220]

[0221]

[0222]

[0223]

[0224]

[0225]

[0226]

[0227]

[0228] X - Ka(xcos^-ysin&i) Y = KaixsinOi + ycosdx) The value of the constant can be K = 0.0075. Field combination Only the equations of individual fields have been discussed so far. But the artist has the possibility of drawing several movements in succession. For example, a radial field X^ Yb then two brush strokes X2, Y2 and Xy Yen ending with a gradient X4, Y4. The resulting field is simply the sum of the individual fields. For N movements drawn by the artist, this is therefore written: x=E fcl x, £n Custom motion fields by “engraving” In an alternative embodiment of the invention, the forces are no longer necessarily derived from equations but can be directly traced by the artist. Some examples of profiles have been given previously. But the artist may wish to use more complex and / or irregular profiles. In all cases, the idea is to feel the evolution of a "tension" according to the time and rhythm of the music, then to "engrave" what results from the artistic will that will have given birth to it through this gesture. In the analog world, what best translates this phenomenon is a wire or a rope that is stretched more or less according to the intensity that one wants to apply. In the instrument, a circle is proposed with a reference radius R(0), its radius R can be modified at any time t by a pointing tool moved by the user (for example one of his fingers): the larger the radius of the current circle R(t), the stronger the intensity and conversely, the smaller the radius of the current circle R(t), the weaker the intensity. At each instant, the intensity of the force (engraved value) will be calculated as the ratio between the current radius R(t) and the reference radius R(0): JV 1 MB) Of course, we can have force values ​​less than 1 and conversely, if we want very large force values, it is enough to have a small reference circle. A dotted dividing line is automatically drawn as soon as the artist places their finger on the line. Crossing this line instantly reverses the force polarity for values ​​engraved after that point. This allows for the generation of a negative force. Finally, it should be noted that this engraving method de facto requires that the first point of the force is always equal to 1, which does not necessarily correspond to the artist's wishes. The instrument is configured to offer a cropping system which allows, after engraving, to adjust the start and end limits of the profile actually rendered. Decentralized display system

[0229] In an alternative embodiment of the invention, the calculations for the broadcasting of the rendering windows can be transferred to boxes distributed at different points in the space (room(s) or exterior). Each box can then be connected to at least one broadcasting system associated with a rendering window. The broadcasting systems 102 can be diverse: a computer screen, a television, a video projector, a screen wall. The instrument as a hardware console

[0230] In an alternative embodiment of the invention, the software gives way to a partially or entirely hardware control console.

[0231] The elements constituting it can be (non-exhaustive list) touch screens, sliders, potentiometers, touchpads or more generally any accessory traditionally used by DJs / VJs. Remote control

[0232] In an alternative embodiment of the invention, the system can be controlled remotely via the Internet, in order to ensure its use off-site as part of an event service, or its maintenance as part of a dedicated installation. Projection mapping

[0233] In an alternative embodiment of the invention, the rendering windows transmitted to the broadcasting systems can undergo a transformation called “4-corner deformation” in order to be able to be projected onto volumes or relief structures. Video mapping thus makes it possible to adhere as closely as possible to the chosen locations. Transition to 3D

[0234] In an alternative embodiment of the invention, the motion fields and the drawn objects have an additional dimension z in addition to x and y. This makes it possible to create a sensation of relief on the visual compositions.

[0235] Of course, the invention is not limited to the examples which have just been described and numerous adjustments can be made to these examples without departing from the scope of the invention. In addition, the various characteristics, forms, variants and embodiments of the invention can be associated with each other in various combinations to the extent that they are not incompatible or exclusive of each other.

Claims

Claims

1. Method (P) for generating an image stream by a calculation unit, each of the images being formed from a plurality of planes, each of the planes having a plurality of objects (01, 02) arranged on said plane, each of the objects of a plane having a plurality of parameters (PI, P2, P3), the method comprising: • a step of attribution (A1), for each of the parameters of the objects: i. of so-called primitive data (E2), comprising movement data, rhythm data and intensity data, • a step of attribution (A2), for each of the planes: i. of object data (El) comprising a plurality of objects, each of the objects having coordinates, said to be associated with the object, in the plane, ii. of so-called primitive data (E2), comprising movement data, rhythm data and intensity data, • a step of generation (G) of the image stream (E3) in which, for each of the planes, i.each of the objects is initially placed at its associated coordinates, ii. the visual and behavioral parameters (P1, P2, P3) of each object initially placed on the plane are determined from said assigned primitive data and evolve according to this primitive data. iii. a sequence of movements is applied to each object initially placed on the plane, the sequence of movements being determined from the primitive data of the plane.

2. The method of claim 1, wherein the motion data is generated by the user in real time using a pointing tool.

3. A method according to claim 1 or 2, wherein the forces are generated by the user in real time using a pointing tool.

4. Module (1) for generating a stream of images, each of the images being formed from a plurality of planes, each of the planes having a plurality of objects (01, 02) arranged on said plane, each of the objects of a plane having a plurality of parameters (PI, P2, P3), the module comprising a calculation unit (2) configured to: • assign (Al), for each of the object parameters: i. so-called primitive data (E2), comprising movement data, rhythm data and intensity data, • assign (A2), for each of the plans: i. object data (El) comprising a plurality of objects, each of the objects comprising coordinates, said to be associated with the object, in the plane, ii. so-called primitive data (E2), comprising movement data, rhythm data and intensity data, • generate the image stream (E3) in which, for each of the shots, i. each of the objects is initially arranged at its associated coordinates, ii. the visual and behavioral parameters (PI, P2, P3) of each object initially placed on the plane are determined from the said primitive data which have been assigned to them and evolve according to these primitive data, iii. a sequence of displacements is applied to each object initially placed on the plane, the sequence of displacements being determined from the primitive data of the plane.

Images