Method and device for composing a video and method for analysing the video

EP4460979A1Pending Publication Date: 2024-11-13ROKNI TOMAJ
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
EP2023703513
Authority / Receiving Office
EP · EP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2022-01-05
Filing Date
2023-01-03
Publication Date
2024-11-13

AI Technical Summary

Technical Problem

The complexity of capturing and recomposing videos from multiple cameras positioned around an event, such as concerts or sporting events, is exacerbated by the need for complex mobility management and human intervention to maintain video quality, especially due to camera movement.

Method used

A method that synchronizes videos from multiple capture devices using metadata to determine fixed points of interest, automatically recalculates these points when camera movement is detected, and composes a single high-quality video without requiring human intervention, ensuring constant video quality and efficient recomposition.

Benefits of technology

This method enables the automatic recomposition of high-quality videos from multiple sources with minimal latency, allowing for live broadcasting or on-demand recording without the need for complex camera repositioning or manual recalculations, providing enriched content with additional data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 1.1
    Figure 1.1
Patent Text Reader

Abstract

The invention relates to a video capture method and device, to a video composition method and device and to a video analysis method. More specifically, the video composition method in a server comprises: - receiving, from at least one first video capture device, a first video representative of a first portion of a scene and items of metadata that are associated with at least one fixed image of the first portion of the scene, the item of metadata being representative of a geographical position of the first capture device at the time of capturing the image; - receiving, from at least one second video capture device, a second video representative of at least one second portion of the scene and items of metadata that are associated with at least one fixed image of the second portion of the scene, the item of metadata being representative of a geographical position of the second capture device at the time of capturing the image, the first and second videos being synchronised; - determining fixed points of interest in each of the videos; - composing a composite video representative of the scene from the first and at least one second video by matching up the points of interest; - delivering the video representative of the scene, detecting the movement of at least one of the two capture devices on the basis of the received items of metadata triggering a new step of determining fixed points of interest in the video captured by the moving device and subsequently composing a new composite video.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] METHOD AND DEVICE FOR COMPOSITION OF A VIDEO AND

[0002] VIDEO ANALYSIS METHOD

[0003] Technical Field

[0004] The invention relates to the processing of a video of a scene from several videos representative of a part of the scene.

[0005] Prior art

[0006] The broadcasting and recording of events, such as, but not limited to, concerts and sporting events, often requires the deployment of expensive and complex means to implement. Indeed, it is often necessary to position numerous cameras to capture the event, the cameras generally being positioned all around the scene to be filmed. Not only is the capture of the scene made complex by the positioning of the numerous cameras, but also the composition and restitution of the scene by the broadcasters.

[0007] The complexity of such systems may lead spectators to use simpler devices to handle, such as cell phones for recording. However, recomposing a video from phones positioned around the scene remains complex, particularly due to the mobility of such phones and therefore the respective positions of the phones in relation to the scene.

[0008] Statement of the invention

[0009] The present invention aims to remedy all or part of the drawbacks of the prior art, in particular those set out above, by proposing a solution which makes it possible, in comparison with the solutions of the state of the art, to recompose a single video from several videos of the same scene while avoiding the complex solutions of managing the mobility of the capture terminals and the intervention of a user.

[0010] To this end, the invention relates to a method for composing a video comprising:

[0011] - receiving from at least one first video capture device a first video representative of a first part of a scene and metadata associated with at least one still image of the first part of said scene, the metadata being representative of a geographical position of the first capture device at the time of capturing the image,

[0012] - receiving from at least one second video capture device a second video representative of at least a second part of said scene and metadata associated with at least one still image of the second part of the scene, the metadata being representative of the geographical position of the second capture device at the time of capturing the image, the first and second videos being synchronized,

[0013] - the determination of fixed points of interest in each of the videos,

[0014] - the composition of a composite video representative of said scene from the first and at least one second videos by matching said points of interest,

[0015] - providing the video representative of the scene, detecting a movement of at least one of the two capture devices on the basis of the metadata received triggering a new determination of the fixed points of interest in the video captured by the moving device and then composing a new composite video.

[0016] Thus, the method can advantageously detect that the capture devices have moved, or at least one of them, and automatically recalculate the points of interest. In this way, the composition of the two videos generates a video of constant quality from the two videos. No human intervention is necessary either to reposition the capture devices or to request a recalculation of the points of interest as long as the method can calculate or recalculate the points of interest or as long as the variation of the metadata remains below a threshold.

[0017] In particular implementations, determining fixed points of interest in each of the videos includes:

[0018] - transmitting to at least one video display device, the first and at least second videos,

[0019] - receiving from at least one video display device, points of interest relating to each of the parts of the scene. In particular modes of implementation, the provision comprises one or more of:

[0020] - recording of composite video,

[0021] - display of composite video,

[0022] - transmission of the composite video to at least one analysis device.,

[0023] Advantageously, the recomposed video is intended for broadcast, for example to a broadcast group. It can be broadcast live, the process being fast and causing only a very slight delay with the event. The video can also be recorded and broadcast on demand.

[0024] In particular embodiments, when said video is transmitted to at least one analysis device, it further comprises

[0025] - reception, following the transmission of scene enrichment data,

[0026] - synchronization of enrichment data with composite video;

[0027] - the provision of composite video enriched with enrichment data.

[0028] Thus, in addition to the automatic recomposition of the video from the two videos, the method can advantageously offer an end user content enriched with additional data relating to the captured content.

[0029] According to a third aspect, the invention relates to a method of video analysis comprising

[0030] - Receiving a composite video representative of a scene obtained by a method of composing a video according to the invention,

[0031] - Adding enrichment data to said composite video to annotate said composite video,

[0032] - Provision of annotated composite video and enrichment data.

[0033] Preferably, the enrichment data is obtained by positioning the enrichment data by a user through a graphical interface.

[0034] According to another embodiment, the composition method analyzes the composite video (for example using artificial intelligence) and compares the received enrichment data with the analyzed data and in the event of inconsistency, submits said inconsistencies to the user through the graphical interface for final validation. The invention also relates to a video capture device comprising video capture means, a communication interface with at least one server and at least one processor capable of implementing a video capture method according to the invention.

[0035] The invention also relates to a server comprising

[0036] - at least one first communication interface with a video capture device,

[0037] - at least one second communication interface with a video display device,

[0038] - at least one processor capable of implementing a method of composing a video according to the invention.

[0039] The invention also relates to a system comprising at least one first and one second video capture device according to the invention, at least one server according to the invention and at least one analysis device capable of implementing an analysis method according to the invention.

[0040] The invention also relates to a computer program comprising instructions for implementing one of the methods according to the invention when the program is executed by a computer.

[0041] The invention also relates to a computer-readable recording medium on which a computer program according to the invention is recorded.

[0042] Brief description of the drawings

[0043] [Fig. 1] Figure 1 represents a video capture method according to a particular embodiment of the invention.

[0044] [Fig. 2] Figure 2 represents a method of composing a video according to a particular embodiment of the invention.

[0045] [Fig. 3] Figure 3 represents the position of the points of interest according to a particular embodiment of the invention.

[0046] [Fig. 4] Figure 4 represents a video analysis method according to a particular embodiment of the invention.

[0047] [Fig. 5] Figure 5 represents a video capture device according to a particular embodiment of the invention. [Fig. 6] Figure 6 represents a video composition server according to a particular embodiment of the invention.

[0048] [Fig. 7] Figure 7 represents a video analysis device according to a particular embodiment of the invention.

[0049] [Fig. 8] Figure 8 represents a system according to a first particular embodiment of the invention.

[0050] [Fig. 9] Figure 9 represents the position of two cameras around a scene to be captured according to a particular embodiment of the invention.

[0051] [Fig. 10] Figure 10 shows the position of three cameras around a scene to be captured according to a particular embodiment of the invention.

[0052] [Fig. 11] Figure 11 represents a system according to a second particular embodiment of the invention.

[0053] Description of the embodiments

[0054] In the embodiments described below, the captured scene is a scene representative of a football match and therefore more precisely of the events taking place on a football field during a match. Thus, when reference is made to a first part of the scene and to a second part of the scene, reference is particularly made to a first part of the football field, for example a little more than half a field for a first part and a little more than half a field for a second part. Of course, this is an example of a captured scene and should in no way constitute a limitation of the invention to the capture of a football match. Similarly, other sports can be envisaged but also other events such as concerts or other scenes from life.

[0055] Furthermore, the description of Figures 1 to 9 refers mainly to embodiments based on the capture of two parts of the same scene. It should be noted that speaking of two parts of the scene is not limited to two and that the invention is not limited to two captures but to at least two. Figure 10 illustrates for example the possible positioning of three capture means.

[0056] Figure 1 represents in the form of a flowchart, the main steps of a video capture method according to a preferred embodiment of the invention. Said method comprises steps Ela and Elb during which two videos are captured. More precisely, during step Ela, a video of a part of said scene is captured and during step Elb, a video of at least one other part of said scene is captured. The two steps Ela and Elb are carried out simultaneously. The two captured parts contain at least one common portion of the scene, this common portion comprising at least two common fixed points of interest.

[0057] The two videos are captured synchronously or can be synchronized using time labels. To do this, prior to starting the capture process, a synchronization process (not shown in the figures) is launched so as to associate a time label ("timecode" in English) with each captured video. This is for example advantageously achieved by using the NTP protocol ("network Time Protocol" in English). Since each of the captured images has its time label, it is possible to synchronize the two videos when they are received by a remote device, regardless of the transmission delays of each of the videos which could be caused by network disturbances or different transmission speeds depending on the network used by each of the capture devices.

[0058] In other embodiments, the synchronization of the two streams can be done using the soundtracks of the streams. The sound sampling frequency being much higher than the frame rate, and the soundtracks being practically identical on all streams because the microphones are very close, synchronizing the soundtracks also allows the time labels, and therefore the streams, to be synchronised. This system is used for synchronizing and editing multi-camera recordings while avoiding echo effects.

[0059] Then, during a step E2a, E2b, metadata are obtained, preferably from data from the GPS sensor (for "Global Positioning System" in English or global positioning system in French) and / or data from an accelerometer and / or a gyroscope and / or a magnetometer. Each video of a part of the scene is captured by capture means and these means can for example be associated or collaborate with a GPS, an accelerometer, a gyroscope or a magnetometer.

[0060] According to another embodiment, a processor associated with the capture means can also, using a software program, obtain said metadata.

[0061] Thus, when capturing each part of the scene, it is possible to associate metadata with each captured image. Each image is therefore associated with a time tag and metadata. Of course, some metadata varies little, such as latitude and longitude, while in the event of bad weather, some metadata will vary more, such as attitude, compass, or tilt. Thus, longitude and latitude can be associated with the video and transmitted less frequently than others that could vary more during capture.

[0062] The metadata mainly relates to the geographical position, which may also include the orientation, of a capture device implementing the method and for example described in Figure 4. Thus, a change in the metadata makes it possible to identify a displacement or movement of the camera. Such a displacement / movement is not necessarily, but may be, a displacement made, voluntarily or involuntarily, by a person. Such a displacement / movement may also be linked to the weather, for example. If there is a lot of wind, it is possible that the capture device implementing the method deviates from its initial position. This is all the more true if the capture device is attached at a height (to have a better view of the scene to be captured). Preferably, to avoid significant modifications to the metadata, the capture means are rigidly fixed and anchored to the ground.For example, they can be fixed on one or more tripods, removable or fixed to the ground.

[0063] During a step E3a, E3b, each video stream (including the time labels) and the metadata associated with the capture are transmitted to a server, represented in figure 5.

[0064] According to a preferred embodiment, the metadata are transmitted periodically to the server. Periodically means at regularly spaced times or at irregularly spaced times. For example, it may be envisaged that the metadata are transmitted every 60s. The frequency of sending the metadata may also be programmed and vary according to weather conditions, for example if the capture devices are located outdoors. When there is a lot of wind, for example, the capture means may possibly move, depending on their location or their attachment to the ground, and thus it may be chosen to transmit the metadata with a greater frequency than when the weather is calm.Each of the capture devices may also transmit metadata at a different frequency since each of the capture devices may be subject to different variations depending on its location or the conditions specific to its location.

[0065] The transmission can be done using the wireless communication network, such as the 2G, 3G, 4G, 5G and more cellular network or using a WIFI connection. Each of the capture devices can also use a different network to transmit the video and metadata. The images of the video streams are preferably transmitted at a rate of 30 frames per second. In other embodiments, they can be transmitted at a rate of 60, 120 or 240 frames per second. The evolution of communication networks and video standards can allow the transmission of larger videos, for example Full HD, 4K, 8K using transmission networks such as 5G and later versions, or WIFI networks of future standards.

[0066] Of course, the video can be transmitted encoded (compressed) before transmission using coding algorithms such as MPEG-4, H.264 or X.264 or other formats...

[0067] Figure 2 represents a method of composing a composite video according to a particular embodiment of the invention. The method is advantageously implemented in a server as represented in Figure 5.

[0068] In a step T1, the video (including the time labels) and the metadata representative of a first part of the scene, and at least a second part of the scene, are received. The videos were captured with the same time label, so they can be synchronized using these time labels. Similarly, since the metadata is associated with an image, each image being associated with a time label, the metadata is therefore associated with the same label as the image with which it is associated.

[0069] In a step T2, fixed points of interest in the scene are determined in each of the two videos. The composition of the composite video representative of said scene is produced from the first and at least one second video by matching the points of interest.

[0070] In this step, the determination of points of interest is linked to the nature of the captured scene. For example, when the captured scene is a football match, 4 points of interest are determined on each part of the scene.

[0071] According to a first embodiment, as illustrated in figure 3, the 4 points of interest are defined as being the points of intersection between

[0072] - The center line and the two touchlines:

[0073] - the points Rlu and RJd for the first part of the video representing the right part of the scene in figure 3 right and the points Lru and Lrd for the second part of the video in figure 3 left representing the left part of the scene - The goal line and the two touch lines: the points Rru and Rrd for the first part of the video representing the right part of the scene in figure 3 right and the points Llu and Lld for the second part of the video in figure 3 left representing the left part of the scene

[0074] The composition process, and particularly the detection of points of interest, are linked to the type of scene being filmed. Thus, the composition process can be configured in advance with the type of event being captured.

[0075] According to a preferred embodiment, the points of interest are determined using different functions and algorithms from the open-cv library. First, the color masks (green for the field and white for the lines). Edge detection can be performed by the Canny filter (canny algorithm), by the gradient method or by the Laplacian. In order to eliminate the remaining false positives, the method implements the open-cv, numpy, and skimage.morphologie libraries in python (or equivalent in C++ language) or the watershed algorithm (or watershed line). The morpho function is calibrated to keep only the longest straight line elements (thus eliminating the goals including the posts and the crossbar, the center circle or the penalty area for the example of football).

[0076] According to a particular embodiment, the determination, in step T2, of fixed points of interest in each of said videos may comprise:

[0077] - transmission to at least one video analysis device of at least two videos received, each representing at least one part of the scene,

[0078] - Reception from at least one video analysis device, of points of interest relating to each of the parts of said scene.

[0079] The video analysis device is shown in Figure 7

[0080] This may possibly allow the determination of points of interest by another device and possibly allow, if necessary, the adjustment of points of interest in the event that automatic detection cannot take place, for example by user intervention.

[0081] For example, if only three points of interest can be determined during step T2 by the method, then the user may be led to position a fourth point of interest to improve the composition process, which is more reliable with four points of interest than with three points of interest.

[0082] According to a particular embodiment, the determination, in step T2, of fixed points of interest in each of said videos may comprise:

[0083] - transmission to at least one video analysis device of at least two videos received, each representing at least one part of the scene, and points of interest,

[0084] - Reception from at least one video analysis device, of new points of interest relating to each of the parts of said scene.

[0085] Thus, this can advantageously adjust one or more points of interest when automatic detection does not work correctly or accurately enough.

[0086] According to another particular embodiment, the determination, in step T2, of fixed points of interest in each of said videos may comprise:

[0087] - transmission to at least one video analysis device of at least two videos received, each representing at least one part of the scene, and points of interest,

[0088] - Reception from at least one video analysis device of a message validating the position of at least one of the points of interest.

[0089] This can advantageously allow a user or operator to confirm or not the automatic detection of the position of points of interest.

[0090] According to another embodiment, during transmission to at least one video analysis device, of the at least two received videos each representing at least one part of the scene, the two videos are transmitted at the same resolution (definition) and more particularly at the lowest resolution of that of the two videos when the two videos are not captured according to the same definition.

[0091] According to another preferred embodiment, in addition to the position of the points of interest, data such as the detection of the boundaries of the terrain can be transmitted and validated or corrected by the video analysis device. More generally, geometry data relating to said scene can be transmitted to the video analysis device and repositioned by the latter or validated by the latter. Thus, according to this embodiment, step T2 can comprise:

[0092] - transmission to at least one video analysis device of at least two videos received, each representing at least one part of the scene, points of interest or data relating to the geometry of the part of the scene,

[0093] - Reception from at least one video analysis device of a message validating the position of at least one of the points of interest or the geometry of the scene.

[0094] In a step T3, the metadata received with each video is analyzed to study its variation. As described in Figure 1, the metadata is representative of a geographical position of the capture means, the attitude, the inclination or the compass (the inclination of the capture means with respect to North). The metadata is transmitted periodically, at regular or irregular time intervals, and analyzed upon receipt by comparing it to reference metadata. The reference metadata includes metadata transmitted by the capture means during their initialization or implementation (during locking). If this comparison detects that the geographical position (for example latitude or longitude), or the inclination, or the attitude or the compass have varied and that the variation is greater than a given threshold, then the points of interest are recalculated.As mentioned, these variations are due to a shift in the capture means. Advantageously, when the metadata does not vary, for example because the capture means are not easily accessible and therefore cannot be inadvertently moved or when the weather conditions are good (no wind for example), then the comparison can be made more spaced out over time and not systematically each time new metadata is received.

[0095] If the result of the comparison, step T4, is greater than a threshold, each threshold being defined for each type of metadata, then the points of interest are redefined. Otherwise the points of interest are not redefined. Of course, since the points of interest are used during step T5, they are always defined at the start of the process when the first metadata is received, with the analysis and comparison steps T3 and T4 then occurring during the capture to verify the geographical stability of the capture devices.

[0096] According to another embodiment, the metadata are not compared to a threshold but must be included in a tolerance range centered around the value of the first corresponding received metadata called reference metadata. It is possible that only one of the capture devices has moved and that the variation of the associated metadata is sufficiently significant to require a new calculation of the points of interest. In this case, the points of interest are recalculated only for this part of the video during step T2.

[0097] According to another embodiment, several thresholds can be defined.

[0098] The first SI threshold is defined as a threshold below which the variation of the metadata does not imply a recalculation of the position of the points of interest.

[0099] The second threshold S2 is defined as a threshold above which the at least two capture devices no longer capture a common area allowing the determination of a sufficient number of fixed points of interest for the composition of the composite video.

[0100] If the variation of the metadata is below the first SI threshold, then there is no recalculation of the points of interest because this low variation does not imply deformations of the symmetries or parallelisms of the composite image to be constructed.

[0101] If the variation of the metadata is between this first threshold SI and the second threshold S2, then a recalculation of the points of interest is necessary as indicated above.

[0102] If the variation of the metadata is greater than this second threshold S2, then the repositioning of the means or at least one of the means of capture is necessary before the recalculation of the points of interest.

[0103] In step T5, a composite video representing the scene is composed from the two videos, each representing at least part of the scene and points of interest.

[0104] In this step, referring to Figure 3, it is a matter of creating a single video, called a composite video, in which a single scene is created. The scene recomposed from the two half-scenes of the left and right videos is shown below.

[0105] So, in this step, the two videos are matched, by matching the Lru and Rlu points and the Lrd and Rld points. This is illustrated in Figure 3 bottom, the Lru and Rlu interest points are merged as well as the Lrd and Rld interest points.

[0106] The composition is achieved using stitching and cropping algorithms. In the case of a scene depicting a sports field, the following constraints are taken into account for the composition of the composite video: - The composite image must have parallel top and bottom key lines,

[0107] - The composite image must have the center line perpendicular to the key lines,

[0108] - The lines along the goals in the composite image are chiral,

[0109] - The low keyline length must be 2 times the definition of the lowest resolution stream, in number of pixels,

[0110] - the height of the center line is chosen by comparing the heights of the two points of common interest and keeping the largest, the second will have to stretch to match it.

[0111] - In the case of 3 capture devices, the height of the center line on the central capture device will be the reference,

[0112] - The length of each half of the high key lines is the same

[0113] - The width of the composite image is 2 times the width of the stream (if 2 streams), its height is reduced to eliminate the empty areas (holes) created by the stitching.

[0114] According to a particular embodiment, the cutting and pasting algorithms use python libraries such as open-cv and cv2.getPerspectiveTransform and cv2.getWarpPerspective functions. An encoding is then carried out in X.264 format with the cv2.videoWriter function by specifying the size of the image (size) and the number of frames per second (fps). Other equivalent functions can be used when the coding language used is C++.

[0115] According to a particular embodiment, this step may comprise a validation sub-step. This validation sub-step comprises the transmission to a video analysis device of a version, for example low definition, of the composite video.

[0116] The subscriber terminal may receive instructions from a user to adjust certain geometric points of the recomposed scene obtained during the composition. For example, in the context of a sports field, this may involve adjusting the 4 corners of the sports field. More specifically, according to a preferred mode, only the two points of interest defined by the intersection of the upper touchline (top of the image) and the goal lines are modifiable. The instructions may also include an adjustment, such as reducing the height of the image, for example by eliminating unnecessary elements. The instructions may also include replacing these unnecessary areas with advertising banners.

[0117] Unnecessary areas can be understood, for example, as areas outside the boundaries of the field.

[0118] During this T5 composition step, according to a variant, the differences between the color and brightness parameters, such as for example contrast, brightness, saturation, exposure between the two videos are also taken into account in order to generate the composite video. Thus, these differences can be smoothed, filters can also be applied to homogenize, or even transform, one or the other stream to have a homogeneous composite video on the two parts thus assembled.

[0119] Once the video has been recomposed from the points of interest and the two videos, it is made available, step T6.

[0120] According to a first variant, the provision consists of recording the composite video in order, for example, to broadcast it later.

[0121] According to another variant, the provision consists of the display.

[0122] Alternatively, provision involves transmitting the composite video to one or more devices for display or recording for later viewing or use. The video may be recorded at different resolutions depending on the capability of the communications networks and terminals for which it is intended.

[0123] All three variants can of course be combined. Similarly, different definitions of composite video can be made available depending on the bandwidth, the end use of the video, and the capabilities of the end device using the video.

[0124] When the provision consists of the display of the composite video (which may also be preceded by recording), according to a particular embodiment, the composite video can be enriched by a user, or automatically by a program.

[0125] In one embodiment, when it is a program that enriches the composite video, the enrichment can be carried out using image analysis software and detection of preferential actions linked to the broadcast event. For example, during a match, it is possible to detect goals, corners, and to compile statistics, it is also possible to add for example the names of the players, the number of ball possessions, the number of balls touched, lost, recovered, the time of ball possession per player or per team, the number of shots on target or not, the distance covered by each player, per match, per position and thus to see the evolution of the statistics during a complete season.

[0126] When a user enriches the composite video, it is transmitted to at least one analysis device, for example a device 3 as illustrated in Figure 7 and used by the user who will enrich the composite video. In this case, the method of composing the video comprises

[0127] - the reception, following transmission, of data enriching said scene,

[0128] - synchronization of enrichment data with composite video;

[0129] - the provision of composite video enriched with enrichment data.

[0130] When video is transmitted for enrichment, it can advantageously be transmitted with a high compression ratio so as to make the latency very low. The compression ratio is adapted to the available bitrate. Thus, the enrichment is done almost live.

[0131] One embodiment, when the user enriches the video, is described later in Figure 4.

[0132] Figure 3, briefly described above, illustrates the two parts of the scene and their composition by the composition process.

[0133] The video on the left represents the video capture of a first capture means which captures the right part of a sports field, more specifically a football field. On this video are positioned 4 points of interest Llu, Lru, Lrd and Lld. These four points of interest are fixed and represent as described above for example intersections of lines on a field.

[0134] The hatched area on both the right and left videos represents a common or overlapping area of ​​the scene, i.e. a part of the scene captured by both video capture devices. Thus, points Lru, Rlu represent identical points in the scene and points Lrd and RJd also represent the same point in the scene.

[0135] The video compositing process will generate the composite video, seen from below in Figure 3, as previously described with respect to Figure 2.

[0136] In Figure 3, the points Lld and Rrd do not coincide with the edges of the image. This represents a non-limiting version of the composite image. In other embodiments, it is envisaged that the points of interest Lld and Lrd coincide with the edges of the composite image, the latter then representing in length the length of the pitch. Thus, Figure 3 represents one of several composite images, this depending on the desired zoom. It is thus possible to envisage having a composite image comprising only the pitch or also the stands or part of them, or for example a buffer zone around the pitch or around certain edges of the pitch.

[0137] Figure 4 represents an analysis method according to a particular embodiment of the invention in which the composite video made available during step T5 with reference to Figure 2 is transmitted to a user device. This user device may be, in a non-limiting manner, a mobile phone, a computer, a tablet, a television, etc. The composite video is received on the user device during step U1. It is also decoded if it was transmitted in a coded manner to then be displayed on a screen of the user's device. During a step U2, the user positions information, called enrichment information, on the composite video or associates enrichment information with the composite video. The enrichment information, in the example of a football match, may for example correspond to information making it possible to annotate the match in the form of words such as "goal", "corner", "throw-in".They can also identify players, add personal user comments, add statistics on the number of successful passes, the score, etc. This information, along with the composite video, can then be displayed on the user's terminal, transmitted to the server for broadcasting to other display terminals, or recorded, or both.

[0138] To do this, the user advantageously but not exclusively uses a user interface. This graphical interface can, for example, allow the user to select predefined events corresponding to the type of scene captured. For example, when the scene is a football match, the predefined events can be "goal", "corner", "touch". The user can also associate names with the players, request the compilation of statistics, etc.

[0139] If the event is a concert then it can add the title of the song, it can also position the name of the musicians, the year of the song, the title of the album...

[0140] Thus, the recording information is obtained or positioned based on an analysis of the scene content.

[0141] Once the enrichment data is positioned, in step U3, the composite video and the enrichment data are synchronized if necessary. Finally, in step U4, the composite video enriched with the enrichment information is made available.

[0142] According to a first variant, the provision consists of recording the enriched composite video in order to broadcast it later, for example.

[0143] According to another variant, the provision consists of the display of the enriched composite video on the user's terminal.

[0144] Alternatively, making available involves transmitting the enhanced composite video to a server or one or more devices for display or for recording for later viewing or use.

[0145] When the server receives the enriched video, it can also compile or have analyzed the video and compare its analysis to the enrichment data to verify the latter. In case of conflict during verification, a validation request can be transmitted to the video analysis device and therefore to the user via the graphical interface for example.

[0146] All three variants can, of course, be combined.

[0147] The method is implemented in an analysis device and for example an analysis device as described in figure 6.

[0148] Figure 5 shows a capture device configured to implement a method as described in Figure 1.

[0149] The device 1 comprises capture means 11. The capture means 11 are preferably a camera such as can be found in a mobile phone. Thus, preferably, the device 1 is a mobile phone but can also be a computer or more generally a device equipped with capture means. The capture means capture at least part of a video scene according to a preferred embodiment.

[0150] The device 1 also comprises display means 16. These display means are optional but can advantageously allow viewing of the scene captured by the capture means 11. The device 1 also comprises a communication interface 14 adapted to communicate with remote devices, such as for example but not limited to a server 4. This communication interface can be of the cellular type (3G, 4G, 5G, etc.) or WIFI or both, or even other protocols. The device 1 also comprises a memory 15 containing data and program instructions and a memory 12 allowing recording of data, such as for example video data. The memory 12 can for example record temporarily or longer term the data captured by the capture means 11, before transmission to the communication interface 14.The device 1 comprises a processor 13 capable of cooperating with the aforementioned means of the device 1 and mainly configured to implement the method described in FIG. 1. The device 1 also comprises an accelerometer 18, a GPS sensor 17, a magnetometer 19 and an (electronic) gyroscope 20. The GPS sensor 17 makes it possible to determine the geographical position of the capture device and the accelerometer 18 makes it possible to determine the movements of the device (the speed of movement). The gyroscope 20 makes it possible to measure the rotation of the capture device and the magnetometer 19 functions as a digital compass. Thus, the inclination, the compass, and the attitude of the capture device are measured. In addition, a log of the positions of the capture device can be kept up to date.

[0151] Before any launch of the capture according to a preferred embodiment of the invention, the capture device is locked, that is to say it is positioned in a position considered as a reference position. This is therefore a physical lock which can be followed by a digital lock, making it possible to have a constant rendering, namely for example to obtain a video having no overexposed area. During the physical lock, the capture device transmits metadata obtained by the GPS sensor 17 and the accelerometer 18 which constitute the reference metadata used by the method according to FIG. 2 for comparison with the metadata subsequently transmitted periodically.

[0152] Figure 6 describes a video composition device 4 and preferably a server-type device. Thus, this server can, for example, be located remotely or in the “cloud”. Furthermore, according to certain embodiments, the functions of this device can be distributed across several servers. For example, the analysis device 3 can receive the data (composite video and points of interest) from a first server and transmit the enriched video and / or the enrichment data to another server, physically different from the server that transmitted the composite video and possibly the points of interest to it. Similarly, a third server can communicate / cooperate with one or other of the servers to transmit the video at one or more additional resolutions.

[0153] The video communication device 4 comprises a processor 47 configured to implement a video composition method as described in FIG. 2. The processor 47 collaborates with a recording memory 43 and a program memory 44. The recording memory 43 can for example temporarily or longer term record the data received or transmitted by the communication interfaces 41a, 41b, 41c. The communication interfaces 41a, 41b, 41c are preferably adapted to communicate respectively with several capture devices 1, one or more analysis devices 3. The number of communication interfaces is given for illustrative purposes. Such communication interfaces are preferably cellular type interfaces (3G, 4G, 5G and future...) or WIFI standard 5, 6 and future or both.

[0154] Figure 7 represents a video analysis device 3 according to a preferred embodiment.

[0155] The device 3 comprises a processor 33 configured to implement a video analysis method as described in FIG. 4. The memory 35 comprises data and program instructions of the processor 33 and the memory 32 is used to record in particular temporarily or permanently the video received through the communication interface 34 and enrichment data entered by the user via the graphical interface 37. The communication interface 34 is preferably a cellular type interface (3G, 4G, 5G, etc.) or WIFI or both.

[0156] Figure 8 represents a system according to a preferred embodiment of the invention. Two capture devices 1a and 1b are arranged around a scene 2, here a football match. The two devices are arranged so as to each film at least part of the football pitch, with an overlapping area, that is to say that each of the devices can capture more than half a pitch. The two devices 1a and 1b are configured to implement a capture method according to the invention and described with reference to Figure 1. The two devices 1a and 1b each transmit, through their respective communication interfaces, the captured video (including in a preferred embodiment the time labels) and the metadata to the server 4. The server 4 may be composed of one or more separate devices. The server 4 is configured to implement a video composition method according to the invention and more precisely described in Figure 2.The server 4 transmits the video obtained according to the video composition method to a video analysis device 3 which enriches the video according to the analysis method described in FIG. 4 and retransmits the enriched video and / or the enrichment data and / or the validation of points of interest or the validation of enrichment data to the server 4. The server 4 then records the recording data and / or the annotated video or transmits them to the device 5 for display or recording or to the device 3. The device 5 is an electronic device such as for example a mobile phone, a computer, a television. According to a preferred embodiment, such a system can be used according to the following scenario.

[0157] The analysis device 3 receives instructions from a user to launch an application. Preferably, the analysis device 3 is a mobile phone. The application receives from the user a setting of the application. Such a setting may be, for example, an indication of the type of event captured, for example, a sports match, for example, football. The application also receives information relating to the number of capture devices 1a, 1b used for capturing the event. Likewise, these capture devices are preferably mobile phones. The application creates a QR code for each capture device 1a, 1b.

[0158] Each of the capture devices 1a, 1b scans one of the QR codes present on the analysis device 3. This makes it possible to link each capture device to one of the streams. This makes it possible to launch an application on the capture devices or to download the application if it was not present and then to launch the clock synchronization according to the NTP protocol. According to advantageous embodiments, the capture devices can also perform throughput tests to check the appropriate bandwidth according to the network parameters, the type of network available (2G, 4G, 5G, WIFI standard 5, 6, etc.), the environment.

[0159] The capture devices also identify themselves to the server 4 so as to allow the server 4 to detect which stream is associated with which capture device and possibly modify this.

[0160] An operator then locks the position of the capture devices around the scene by verifying that each part of the scene captured by each of the capture devices includes at least one overlapping area in which two fixed points of interest can be determined. This determination can be visual by the operator but this determination can also be done at the server level. For example, the server can transmit notifications to the capture devices to orient / move the capture devices, as long as it has not succeeded in determining common points of interest for the first time. Once the common points of interest are detected, the operator locks the physical position of the capture devices. Then, as described previously, the points of interest are automatically recalculated according to the variation of the metadata.Only a significant variation in the metadata that no longer allows the detection of points of interest will make it necessary to physically reposition the capture devices. During locking, the metadata is also recorded and thus constitutes reference metadata for the position of the capture devices 1a, 1b. This reference metadata is used when comparing with the metadata received periodically (regularly or irregularly) to determine the need for recalculation of the position of the points of interest as indicated with reference to Figure 2.

[0161] Figure 9 illustrates the positioning of two video capture devices 1a and 1b around a football scene. Devices 1a and 1b are similar to device 1 described in Figure 5. They are positioned geographically close to the center line of the field. Device 1a is oriented towards the left part of the field and device 1b is oriented towards the right part of the field, so that their respective capture means capture more than half of the scene, i.e. more than half a field. Capture device 1a has a capture angle a and capture device 1b has a capture angle p. The hatched area in Figure 8 shows the overlapping area of ​​the two video captures made by device 1a and device 1b.

[0162] Figure 10 illustrates the positioning of three video capture devices 1c, 1d and 1e around a football scene. Devices 1c, 1d and 1e are similar to devices 1a and 1b described with reference to Figure 9 but the capture means have different capture angles. The third capture device 1c may be positioned such that its capture means are parallel to the touchline.

[0163] It may be necessary to use a larger number of capture devices. This is particularly true when two capture devices are not sufficient to fully capture the entire scene, for example because their capture angle is insufficient or because they cannot be positioned to cover the entire scene. Certain technical or geographical limitations may also lead to the addition of new capture means, for example. Here, since the respective capture angles y and e of devices le and ld are lower than the capture angles of devices la and lb, a third device le is added whose capture angle is p. Thus, the positioning of the three capture devices and their angles make it possible to cover the entire terrain.

[0164] Figure 11 represents a system according to a second embodiment of the invention. Such a system is particularly advantageous when the capture devices 1a and 1b as well as 3 are not connected to a network such as a cellular network (3G, 4G, 5G, 6G) or WIFI. In this case, in fact, they cannot communicate with a server as illustrated in Figure 8 and therefore transmit the captured videos for recomposition to a server. Such a configuration is also advantageous when the bandwidth between the capture devices 1a and 1b and the server 4 is low and does not allow the transmission of the video streams and metadata to the server 3 at sufficient quality.

[0165] In this embodiment, the capture devices 1a, 1b and the analysis device 3 communicate via a local network established between them. This is notably possible via new generation WIFI networks or 5G, 6G and 6E cellular networks.

[0166] In this embodiment, the analysis device 3 synchronizes the two capture devices 1a and 1b by creating a temporary time clock which it transmits to them. The video streams of the two devices 1a and 1b can then be synchronized by the device 3.

[0167] Device 3 receives the two video streams and constructs the composite video from the two received videos and the received metadata. To do this, device 3 implements the method according to Figure 2. Once recomposed, the video is then enriched by device 3 according to steps U2 to U4 of the method described in Figure 4.

[0168] The video thus enriched is transmitted to the server 4 when the device 3 is again connected to a cellular or WIFI network or allowing it to benefit from sufficient bandwidth for the transmission. The device 3 and the server 4 can thus enrich, analyze the composite video. In the same way as previously, the server 4 can request validations of the enrichment information transmitted by the device 3 and compile statistics.

[0169] Of course, by video stream or by video, we also mean throughout the previous description, the audio associated with this video.

Claims

Demands

1. A method for composing a video comprising: - the reception (Tl) of at least one first video capture device (la) of a first video representative of a first part of a scene (2) and of metadata associated with at least one still image of said first part of said scene, said metadata being representative of a geographical position of the first capture device (la) at the time of image capture, - the reception (Tl) by at least one second video capture device (lb) of a second video representative of at least a second part of said scene (2) and of metadata associated with at least one still image of said second part of said scene, said metadata being representative of the geographical position of the second capture device (lb) at the time of image capture, said first and second videos being synchronized, - the determination (T2) of fixed points of interest in each of said videos, - said composition (T5) of a composite video representative of said scene from said first and at least one second video by matching said points of interest, - the provision (T6) of said video representative of said scene, the detection (T3, T4) of a movement of at least one of the two capture devices on the basis of the metadata received triggering a new determination (T2) of the fixed points of interest in said video captured by the moving device then said composition (T5) of a new composite video.

2. The method according to claim 1 characterized in that the determination (T2) of fixed points of interest in each of said videos comprises: - the transmission to at least one video analysis device (3), of said first and at least second videos, - the reception of said at least one video analysis device (3), of said points of interest relating to each of said parts of said scene (2).

3. A method according to any one of claims 1 to 2 characterized in that the provision (T6) comprises one or more of the following: - the recording of said composite video, - the display of said composite video, - the transmission of said composite video to at least one analysis device (3).

4. A method according to claim 3 characterized in that when said composite video is transmitted to at least one analysis device (3), it further comprises - the receipt, following said transmission, of data enriching said scene, - the synchronization of the enrichment data with said composite video; - making available said composite video enriched with enrichment data.

5. A method for analyzing video comprising - The reception (Ul) of a composite video representative of a scene (2) obtained by a method according to any one of claims 1 to 4, - The addition (U2) of enrichment data to said composite video to annotate said composite video, - The provision (U4) of the annotated composite video and enrichment data.

6. Analysis method according to claim 5 characterized in that the enrichment data is obtained by a user positioning the enrichment data through a graphical interface (37).

7. Server including: - at least one first communication interface (41a) with a video capture device, - at least one second communication interface (41b) with a video display device, - at least one processor (47) capable of implementing a method according to one of claims 1 to 4.

8. System comprising at least a first (a) and a second (b) video capture device, at least one server (4) according to claim 7 and at least one analysis device (3, 5) capable of implementing a method according to claim 5 or 6.

9. A computer program comprising instructions for carrying out steps 6 of the video composition method according to any one of claims 1 to 4 or of the video analysis method according to any one of claims 5 or 6 when said program is executed by a computer.

10. Computer-readable recording medium on which is recorded a computer program comprising instructions for carrying out the steps of the video composition method according to any one of claims 1 to 4 or of the video analysis method according to any one of claims 5 or 6.