Method for segmenting plurality of data and corresponding encoding method, decoding method, apparatus, system and computer program

Abstract

The invention relates to a method for segmenting a plurality of input data, comprising:-determining (61) a weight value to be applied to the plurality of input data before the plurality of input data is processed by at least one processing device, depending on a criterion for optimizing the quality of the result of the processing and a further criterion for optimizing the amount of input data to be processed; the at least one processing device is configured to generate a processing result according to the criteria; -determining (62) segment information of the plurality of input data, which is assigned a first value or a second value according to a weight; and-obtaining (63) a subset of data to be processed by applying the determined segment information to the plurality of input data, the subset of data to be processed comprising data of the plurality of input data associated with an item of segment information equal to the first value.

Description

Technical Field

[0001] The present invention generally relates to processing multiple data (such as multiple views of a scene acquired by multiple cameras) Figure 3 3D images or physiological data of a patient obtained from multiple sensors).

[0002] In particular, the present invention relates to segmenting the plurality of data before encoding them and then transmitting them to a processing device via a communication network. Background Art

[0003] In the field of virtual reality and immersive video, free navigation allows a viewer to view a scene from any viewpoint using a device such as a virtual reality headset, whether that viewpoint corresponds to a viewpoint captured by a camera or a viewpoint that has not yet been captured by the camera. This view that has not yet been captured by the camera is called a virtual view or an intermediate view, because it lies between the views captured by the camera and must be synthesized for rendering the scene to the viewer from the captured views.

[0004] In the context of immersive video, i.e., in situations where the viewer has the feeling of being immersed in a scene, the scene SC is typically captured by a set of cameras, such as Figure 1 These cameras can be of 2D type (cameras C1, C2, ..., C N , where N is Figure 1 is a non-zero integer in ), i.e., each of them captures the view from one viewpoint, or is of type 360, i.e., they capture the entire scene 360 ​​degrees around the camera ( Figure 1 Camera C 360 ), thus capturing from several different viewpoints. The cameras can be arranged in an arc, a rectangle, or any other configuration that provides good coverage of the scene.

[0005] about Figure 2 , at a given time, a set of images representing the scene from different views is obtained. Since this involves video, the captured images are temporally sampled (e.g., 30 images per second) to produce a raw multi-view video VMV, as Figure 3 shown.

[0006] The MIV (MPEG Immersive Video) standard enables the transmission of videos suitable for immersive navigation. The encoder selects the parts (patches) of each view that it wants to transmit in order to maximize the quality of view synthesis from these patches while reducing the amount of data to be transmitted. Patches are extracted from the views and collected in one or more atlases, so that an atlas is an assembled image comprising patches from different views. Patches are usually arranged in atlases so as to fill the atlas as completely as possible. An occupancy map is sent with each atlas, an occupancy map being an image in which each pixel can take a first value or a second value different from the first value (e.g. corresponding to the color "white" or "black") to indicate whether the pixel in the atlas belongs to an image patch. Thus, the segmentation of the patches in the atlas can be obtained from this occupancy map. In addition, other information is transmitted for each patch, including its coordinates in the atlas and the view associated with it.

[0007] From the transmitted data, the MIV decoder can find the image blocks and arrange them in the view to which they belong. This view is then called "partial" because it does not contain all the pixel values ​​of the original view acquired by one of the cameras. However, if the encoder has effectively selected the parts of the views to be transmitted, they are sufficient to generate or synthesize any viewpoint of the scene. In this regard, view synthesis from decoded views is not specified by the MIV standard. It relies on occupancy maps to determine whether the pixels of a given view contain relevant information items.

[0008] A method for texture-based view synthesis using neural networks is also known from the paper entitled "IBRNet: Learning Multi-View Image-Based Rendering" by Wang et al., published by arXiv:2102.13090 v2 in April 2021. In particular, the paper describes a neural network called IBRN, which takes as input the parameters of a camera, texture images captured from various viewpoints, and the coordinates of the viewpoint of the scene to be synthesized, and produces as output a synthetic view corresponding to the view captured by the camera from the viewpoint corresponding to the provided coordinates. One advantage of this method is that it provides very good qualitative results.

[0009] One disadvantage of this approach, and more generally of current view synthesis methods, is that they require all views acquired by the camera. This means a lot of data to be transmitted and then decoded, which creates complexity issues at the decoder, especially when it is embedded in a mobile terminal such as a smartphone or augmented reality headset.

[0010] Therefore, a solution is needed to reduce the amount of data, in particular to reduce the number of pixels to be transmitted, while maintaining the quality of view synthesis as much as possible.

[0011] The present invention improves this situation. Summary of the invention

[0012] The present invention responds to this need by proposing a method for segmenting a plurality of data acquired by a sensor, called input data, comprising:

[0013] - determining weight values ​​to be applied to the plurality of input data before the plurality of input data are processed by at least one processing device, the at least one processing device being configured to generate a processing result according to a criterion for optimizing the quality of the processing result of the input data, the weight values ​​being determined according to the criterion and a criterion for optimizing the amount of input data to be processed,

[0014] - determining segment information of the plurality of input data, wherein one of the segment information items of one of the data items is assigned a first value or a second value different from the first value according to the weight, and

[0015] - obtaining a data subset to be processed by applying the determined segment information to the plurality of input data, the data subset to be processed comprising data in the plurality of input data associated with the segment information item equal to the first value.

[0016] The present invention proposes a completely new and inventive method for segmenting a plurality of input data before they are processed by a given processing device, the method comprising configuring weight values ​​to be applied to the input data so that both the amount of data resulting from the segmentation and presented at the input of the processing device and the quality of the processing are optimized. The values ​​of the weights are determined according to the plurality of input data themselves and are therefore specifically selected for them.

[0017] For example, the criteria for optimizing the amount of input data to be processed include the minimization of the cumulative value of weight values ​​or of a number of weight values ​​below a given threshold. According to a variant, it includes the minimization of the cumulative value of input data or of the number of these items of input data that remain in the subset of data to be processed. For example, the criteria for optimizing the quality of processing include the minimization of the squared error between the result obtained from the plurality of input data and the result obtained from the subset of data to be processed, or the maximization of the peak signal-to-noise ratio or PSNR.

[0018] Then, the obtained configured weight values ​​are used to determine one or more items of segmentation information of the plurality of input data. The segmentation information indicates:

[0019] - which input data should be provided at the input of the processing device, insofar as they are useful for the processing device to produce processing results that comply with the criteria for optimizing the quality of the processing results of the input data, and

[0020] - Which input data should not be presented to the processing device, as they are useless, as it does not contribute to improving the quality of the obtained processing results.

[0021] The segmentation information thus determined is used to obtain a subset of data to be processed. In this way, only the subset of data to be processed can be encoded and transmitted to a receiver incorporating similar processing equipment in terms of structure and configuration.

[0022] Therefore, the present invention not only proposes to perform effective segmentation of multiple input data, but also considers the influence of the value of the weight of the segmentation module performing the segmentation on the output of the processing device when determining the value of the weight of performing the segmentation. Therefore, it does not consider a simple segmentation module independent of the processing device, but considers the combination of the two, and more precisely, considers their continuous action on the input data.

[0023] The present invention is applicable to any type of data acquired by any type of sensor. For example, multiple sensors are arranged around a scene, an object or an object... It involves, for example, multiple cameras, each camera having a different viewpoint of the scene and configured to acquire a sequence of images or views of the scene. In this case, the processing device can be a device for synthesizing additional views from the original views acquired by the multiple cameras, and the segmentation of the original view according to the present invention means that only data useful for synthesizing the additional views is retained and redundant data is eliminated.

[0024] According to another example, the plurality of data includes a time series of physiological measurements of a patient captured by a plurality of sensors of various types (electrocardiogram (ECG), electroencephalogram (EEG), scanner, magnetic resonance imaging (MRI), X-ray images, blood composition indicators, etc.) In this other example, the processing device may be a diagnostic aid.

[0025] According to another aspect of the present invention, the determining comprises learning the weight values ​​from the plurality of input data, the learning being performed by back propagation of gradients of a loss function combining two standards.

[0026] Advantageously, the determination of the weight values ​​of the segments of the input data is performed using the plurality of input data themselves to implement a learning technique of the artificial intelligence module. More specifically, the artificial intelligence module (hereinafter referred to as the segmentation module) learns the best possible internal configuration of its weights, thereby optimizing the data volume of the subset of the input data to be presented at the input of the processing device and optimizing the data processing quality.

[0027] The present invention is different from the usual practice in machine learning because it implements learning specific to a plurality of given input data. Therefore, it does not require the prior acquisition of a large amount of labeled learning data, but only requires the processing of a plurality of current data.

[0028] According to a further aspect of the invention, the processing device comprises weights referred to as processing weights, the values ​​of which have been predetermined according to criteria for optimizing the quality of the input data processing results, and the method further comprises determining a modified value of the processing weights.

[0029] Advantageously, the processing device also implements artificial intelligence techniques based on applying weights to input data. For example, the processing device has been previously trained in a conventional manner from a basis of labeled data to process a plurality of input data in order to provide as output a result that complies with a criterion for optimizing processing quality.

[0030] Advantageously, the learning according to the invention is a combined learning of the segmentation module and the processing device. It comprises updating the configuration of the processing device so that it can optimally process a data subset resulting from the segmentation of the plurality of input data.

[0031] For example, the segmentation modules and processing devices are each organized in layers including weights, and the succession of their respective layers forms an artificial intelligence module, such as a neural network, which is trained to segment input data and then process them optimally.

[0032] According to yet another aspect of the present invention, the plurality of input data comprises a plurality of views acquired by a plurality of cameras, one of the views comprising pixels, the weights being included in a plurality of layers, one of the layers being associated with one of the views and comprising one of the weights for each pixel, and the segmentation information comprising a plurality of segmentation maps, one of the maps being associated with one of the views.

[0033] The present invention is particularly applicable to input data of the type of multi-view video sequences. Advantageously, a hierarchically structured segmentation module is considered, wherein each view Figure 1 layers and one weight per pixel, the weight for each layer making it possible to derive a segmentation information item for the pixel of the associated view.

[0034] For example, for an image captured by a camera, the segmentation information may take the form of a segmentation map having the same size as the input image and whose useful pixels are assigned a first value (e.g., corresponding to “white”) and whose non-useful pixels are assigned a second value (e.g., corresponding to “black”).

[0035] Advantageously, the processing device is configured to synthesize the additional view associated with a given viewpoint from at least one original view received as input therefrom.

[0036] It will be appreciated that in this case, the learning according to the invention may comprise presenting at the input of a segmentation module combined with a processing device a plurality of acquired views in addition to the view to be synthesized. In this way, the loss function may be calculated by comparing the synthesized view with the original view. Advantageously, this process will be repeated for each of the acquired views.

[0037] According to another aspect of the present invention, the multiple input data include multiple measurement data sequences acquired by multiple sensors, the weights are included in multiple layers, one of the layers is associated with one of the measurement data sequences and includes one of the weights for each measurement data item, and the segmentation information includes multiple segmentation sequences, one of the segmentation sequences is associated with one of the measurement data sequences.

[0038] According to this further example, the plurality of data includes a time series of physiological measurements of the patient captured by a plurality of sensors of various types.In this further example, the processing device may include one or more diagnostic aids.

[0039] Relatedly, the present invention also relates to a device for segmenting a plurality of data acquired by a sensor, referred to as input data, the device being configured to implement:

[0040] - determining weight values ​​to be applied to the plurality of input data before the plurality of input data are processed by at least one processing device, the at least one processing device being configured to generate processing results according to a criterion for optimizing the quality of the processing results of the input data, the weight values ​​being determined according to the criterion and a criterion for optimizing the amount of input data to be processed,

[0041] - determining segment information of the plurality of input data, wherein one of the segment information items of one of the data items is assigned a first value or a second value different from the first value according to the weight, and

[0042] - obtaining a data subset to be processed by applying the determined segment information to the plurality of input data, the data subset to be processed comprising data in the plurality of input data associated with the segment information item equal to the first value.

[0043] The above-described device implements the segmentation method according to the present invention in its various embodiments.

[0044] Advantageously, said segmentation device is integrated into an item of server equipment, said item of server equipment being configured to receive a plurality of input data and further comprising the above-mentioned device for processing a plurality of input data.

[0045] The invention also relates to a method for encoding a plurality of data acquired by a sensor, referred to as input data, comprising:

[0046] - obtaining the segmentation information and the data subset to be processed by applying segmentation information of the plurality of input data to the plurality of input data, the segmentation comprising determining a weight value to be applied to the plurality of input data before the plurality of input data are processed by at least one processing device, the at least one processing device being previously configured to produce a processing result according to a criterion for optimizing processing quality, the weight value being determined according to the criterion and another criterion for optimizing the amount of input data to be processed, one of the segmentation information items of one of the data items being assigned a first value or a second value different from the first value according to the weight value, the data subset to be processed (USS) comprising data of the plurality of input data associated with the segmentation information item equal to the first value, and

[0047] - Encoding the segment information and the data subset to be processed.

[0048] By virtue of the invention, the encoded data contain all the information to reconstruct the plurality of segmented input data at a receiver comprising processing equipment similar in structure and configuration to that implemented by the transmitter.

[0049] According to another aspect of the present invention, the obtaining further comprises:

[0050] - obtaining modified weight values, said weights being intended to be applied by said processing device to said input data, said weight values ​​being previously determined according to said criteria for optimizing the quality of the result of processing the input data and for processing said plurality of input data, said weight values ​​having been modified according to said criteria for processing said subset of data to be processed,

[0051] - encoding the modified weight value.

[0052] Advantageously, the invention proposes to send the modified weight values ​​of the processing device in the coded data at the transmitter side to update the configuration of the processing device at the receiver side. This ensures that the processing device produces optimal results in the sense of an optimization criterion.

[0053] Relatedly, the present invention also relates to a device for encoding a plurality of data acquired by a sensor, referred to as input data, the device being configured to implement:

[0054] - obtaining the segmentation information and the data subset to be processed by applying segmentation information of the plurality of input data to the plurality of input data, the segmentation comprising determining a weight value to be applied to the plurality of input data before the plurality of input data are processed by at least one processing device, the at least one processing device being previously configured to produce a processing result according to a criterion for optimizing processing quality, the weight value being determined according to the criterion and another criterion for optimizing the amount of input data to be processed, one of the segmentation information items of one of the data items being assigned a first value or a second value different from the first value according to the weight value, the data subset to be processed (USS) comprising data of the plurality of input data associated with the segmentation information item equal to the first value, and

[0055] - Encoding the segment information and the data subset to be processed.

[0056] The above-mentioned device implements the encoding method according to the present invention in its various embodiments.

[0057] Advantageously, said encoding device is integrated into the above-mentioned item of server equipment.

[0058] The invention also relates to a method for decoding encoded data, comprising:

[0059] - decoding encoded data, the encoded data comprising segmentation information of a plurality of data, referred to as input data, acquired by a sensor, a data subset to be processed by a processing device, the processing device being configured to apply weights to a plurality of decoded input data and to generate a processing result according to a criterion for optimizing the quality of the processing, one of the segmentation information items of one of the input data items being assigned a first value or a second value different from the first value, the data subset to be processed having been obtained by applying the segmentation information to the plurality of input data, the data subset to be processed comprising data of the plurality of input data associated with one of the segmentation information items equal to the first value, the encoded data further comprising modified values ​​of the weights, the modified values ​​having been determined for processing the plurality of segmented input data according to a criterion for optimizing the quality of the processing and a criterion for optimizing the amount of data of the data subset to be processed,

[0060] - constructing a plurality of decoded segmented input data from the decoded data subset to be processed and the decoded segment information;

[0061] - providing a plurality of decoded segment input data and the modified values ​​of the weights to the processing device.

[0062] Advantageously, the invention proposes to encode the modified weight values ​​of the processing device at the transmitter side to transmit them to the receiver and update the configuration of the processing device at the receiver side. This ensures that the processing device produces optimal results in the sense of an optimization criterion.

[0063] Relatedly, the present invention also relates to a decoding device, comprising:

[0064] - decoding encoded data, the encoded data comprising segmentation information of a plurality of data, referred to as input data, acquired by a sensor, a data subset to be processed by a processing device, the processing device being configured to apply weights to a plurality of decoded input data and to generate a processing result according to a criterion for optimizing the quality of the processing, one of the segmentation information items of one of the input data items being assigned a first value or a second value different from the first value, the data subset to be processed having been obtained by applying the segmentation information to the plurality of input data, the data subset to be processed comprising data of the plurality of input data associated with one of the segmentation information items equal to the first value, the encoded data further comprising modified values ​​of the weights, the modified values ​​having been determined for processing the plurality of segmented input data according to a criterion for optimizing the quality of the processing and a criterion for optimizing the amount of data of the data subset to be processed,

[0065] - constructing a plurality of decoded segmented input data from the decoded data subset to be processed and the decoded segment information;

[0066] - providing a plurality of decoded segment input data and the modified values ​​of the weights to the processing device.

[0067] The above-mentioned device implements the decoding method according to the present invention in its various embodiments.

[0068] Advantageously, said decoding device is integrated into an item of terminal equipment configured to receive the encoded data and also comprising the above-mentioned processing device.

[0069] The invention also relates to a signal carrying coded data. The coded data comprises segmentation information of a plurality of data, referred to as input data, acquired by a sensor, a data subset to be processed obtained by applying the segmentation information to the plurality of input data, one of the segmentation information items of one of the input data items being assigned a first value or a second value different from the first value, the data subset to be processed comprising data associated with the segmentation information item equal to the first value in the plurality of input data, the data subset to be processed being intended to be decoded and then used to reconstruct a plurality of decoded segmented input data according to the decoded segmentation information, so as to be processed by a processing device, the processing device being configured to generate a processing result according to a criterion for optimizing processing quality by applying weights to the plurality of input data. The coded data also comprises modified values ​​of the weights, which have been determined for processing the plurality of segmented input data according to a criterion for optimizing processing quality and a criterion for optimizing the data volume of the data subset to be processed, and are intended to be used by the processing device to update the weights before processing the plurality of reconstructed decoded segmented input data.

[0070] The invention also relates to a system comprising a segmentation device, a coding device, a decoding device and the above processing device. The system and the segmentation, coding and decoding devices according to the invention have at least the same advantages as those conferred by the above method.

[0071] The invention finally relates to a computer program product comprising program code instructions for implementing the above-mentioned segmentation, encoding and decoding methods, respectively, when they are executed by a processor.

[0072] Such a program may use any programming language, and may be in the form of source code, object code, or a code intermediate between source code and object code, such as a partially compiled form, or in any other desired form.

[0073] The present invention also relates to a computer-readable storage medium on which a computer program is stored, the computer program comprising program code instructions for implementing the steps of the method according to the present invention as described above.

[0074] Such a storage medium may be any entity or device capable of storing a program. For example, the medium may include a storage device such as a ROM, for example a CD-ROM or a microelectronic circuit ROM, or a magnetic recording device, for example a removable medium (memory card) or a hard disk or SSD.

[0075] On the other hand, such a storage medium may be a transmissible medium such as an electrical or optical signal, which may be carried via an electrical or optical cable, by radio or by other means, so that the computer program contained therein may be remotely executed. The program according to the invention may in particular be downloaded on a network, such as the Internet.

[0076] Alternatively, the one or more storage media may be one or more integrated circuits in which the program is embedded, the circuit(s) being suitable for executing or for executing the above-described methods.

[0077] According to one embodiment, the present technology is implemented using software and / or hardware components. In this context, the term "module" may be used in this document to refer to a software component, a hardware component, or a combination of hardware and software components.

[0078] A software component is one or more computer programs, one or more subroutines of a program, or more generally any element of a program or software capable of implementing a function or set of functions, as described below for the relevant modules. Such a software component is executed by a data processor of a physical entity (terminal, server, gateway, set-top box, router, etc.) and has access to the hardware resources of that physical entity (memory, recording medium, communication bus, electronic input / output card, user interface, etc.). Hereinafter, a resource is understood to be any collection of hardware and / or software elements that support a function or service, either individually or in combination.

[0079] In the same way, a hardware component is any element of a hardware assembly capable of implementing a function or a set of functions, as described below for the relevant modules. It can be a programmable hardware component or a component with an integrated processor for executing software, such as an integrated circuit, a smart card, a memory card, an electronic card for executing firmware, etc.

[0080] Each component of the previously described system naturally implements its own software module.

[0081] The various embodiments mentioned above can be combined with each other to implement the present technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Other features and advantages of the invention will emerge from the following description, with reference to the accompanying drawings which illustrate non-limiting embodiments of the invention. In the drawings:

[0083] Figure 1 shows an example of an arrangement of multiple cameras forming a system for acquiring multi-view video of a scene according to the prior art;

[0084] Figure 2 schematically illustrates multiple images of a scene captured by multiple cameras at a given time according to the prior art;

[0085] Figure 3 Schematically illustrates a sequence of multiple images captured by multiple cameras at several consecutive times and forming an original multi-view video according to the prior art;

[0086] Figure 4 A first example of the architecture of a system according to an embodiment of the present invention is schematically shown, the system comprising: when the plurality of data are multi-view videos of a scene acquired by a plurality of cameras, a device for segmenting a plurality of input data, a device for encoding segmentation information and a data subset to be processed resulting from the segmentation, a device for decoding the encoded data representing the data subset to be processed and the associated segmentation information, and a device for processing the decoded data;

[0087] Figure 5 A second example of the architecture of a system according to an embodiment of the present invention is schematically shown, the system comprising a device for segmenting a plurality of input data, when the plurality of data comprises physiological measurements of a patient acquired by a plurality of sensors, a device for encoding segmentation information and a subset of data to be processed resulting from the segmentation, a device for decoding the encoded data representing the subset of data to be processed and the associated segmentation information, and a device for processing the decoded data;

[0088] Figure 6 The steps of a method for segmenting a plurality of input data according to one embodiment of the present invention are described in the form of a flowchart;

[0089] Figure 7 The steps of a method for encoding data generated from a plurality of segments of encoded data according to one embodiment of the present invention are described in the form of a flowchart;

[0090] Figure 8 The steps of a method for decoding data generated by segments of a plurality of data according to one embodiment of the present invention are described in the form of a flowchart;

[0091] Fig. 9 A first embodiment of the segmentation and encoding method described above is described in detail when the plurality of data includes multi-view video;

[0092] Fig.10 A first embodiment of the above decoding method is described in detail when the plurality of input data include multi-view videos;

[0093] Fig.11 A second embodiment of the above segmentation and encoding method is described in detail when the plurality of input data includes a plurality of physiological measurement sequences of a patient;

[0094] Fig.12A second embodiment of the above decoding method is described in detail when the plurality of input data includes a plurality of physiological measurement sequences of a patient;

[0095] Fig.13 Describe an example of the hardware structure of the device for segmenting a plurality of data according to the present invention;

[0096] Fig.14 Describe an example of the hardware structure of the apparatus for encoding a plurality of data according to the present invention; and

[0097] Fig.15 An example of the hardware structure of an apparatus for decoding data generated by segmentation and encoding according to the present invention is described. DETAILED DESCRIPTION

[0098] The principle of the present invention is based on segmenting a plurality of input data acquired by sensors arranged around a scene, object or subject, so as to be processed by a given processing device. Segmentation here means identifying those input data to be retained for a given subsequent processing among a plurality of input data and collecting them into a subset of data to be processed.

[0099] In other words, it consists in pruning data from one or more of these sensors when the data from one or more of these sensors are not useful or relevant for the performance of the processing provided by the processing device, for example because they are redundant with respect to data acquired by one or more other of these sensors. According to the invention, the decision to prune or retain input data is made based on two criteria, one for optimizing the amount of segmented data retained in the subset of data to be processed and the other for optimizing the quality of a given processing from the subset of data to be processed.

[0100] According to one embodiment of the invention, segmentation is implemented by an artificial intelligence module that is placed upstream of the processing device and is able to configure itself by machine learning. The structure of the automatic segmentation module includes a plurality of weights intended to be applied to a plurality of input data, and the learning of the configuration of the automatic segmentation module includes the module learning the best possible internal configuration of its weights, both minimizing the amount of data of a subset of the input data to be presented at the input of the processing device and maximizing the quality of data processing.

[0101] According to the invention, the learning is performed on the one hand specific to the plurality of input data and on the other hand taking into account the result of the combination of the actions of the segmentation module and of a processing device placed downstream of the segmentation module, thus recovering as input the plurality of data weighted by the configured weights of the automatic segmentation module according to at least two criteria. A first performance criterion associated with the segmentation module is to minimize the amount of data to be supplied to the processing device, which means to prune the plurality of input data as much as possible. A second performance criterion is linked to the processing device and requires that a quality measure of the processing performed by the processing device be maximized.

[0102] Then, segmentation information of the plurality of data is determined using the configuration value of the automatic segmentation module, and a data subset to be processed is finally obtained by applying the segmentation information to the plurality of input data.

[0103] The invention thus reduces the amount of data to be encoded and transmitted to a remote receiver item of equipment in which processing means similar to those implemented by the sender item of equipment are embedded, while maintaining the quality of this data processing.

[0104] The present invention is applicable to any type of input data and is independent of the artificial intelligence technology implemented by the automatic segmentation module, given that it can be configured using multiple configuration weights associated with multiple input data.

[0105] The invention has particular application in a system for free navigation within a multi-view video, for example embedded in an item of terminal equipment, such as a mobile phone or a virtual reality headset. In this case, the processing comprises synthesizing additional views desired by the user of the item of terminal equipment from the segmented data.

[0106] Of course, the invention is not limited to this use case but can be applied to any other plurality of data acquired by sensors, such as sensors for physiological measurements of a patient.

[0107] In the following, embodiments of the present invention are described for these two specific use cases.

[0108] In the following, "scene" refers in the broadest sense to any object, subject, or multiple objects or objects in its environment.

[0109] about Figure 4 , shows an example of the architecture of a system S according to the invention, comprising a system configured to receive input data from a plurality of sensors C1, C2, ..., CN. In this case, it comprises a plurality of cameras, and the received input data are a plurality of video sequences taken from different viewpoints of a scene SC, also called multi-view videos VMV.

[0110] In this example, a server device ES comprises a sending-receiving module E / R, a segmentation device 100 according to the invention, devices PROC1, SYNT1 for processing a plurality of input data and a device 200 for encoding segmented data according to the invention. Such a server device item ES is configured to send the encoded data using its E / R module, for example in a data stream STR or a file F D It also includes memory M ES , for example it stores encoded data.

[0111] According to the present invention, the segmentation device 100 is configured to: determine weight values ​​to be applied to a plurality of input data before the plurality of input data are processed by at least one processing device (PROC1, SYNT1), the processing device being configured to generate a processing result according to a criterion for maximizing a quality measurement of the input data processing result, the weight values ​​being determined according to the criterion and another criterion for minimizing the amount of input data to be processed; determine segmentation information of the plurality of input data, a segmentation information item of one of the data items being assigned a first value or a second value different from the first value and a second threshold value equal to the first value according to the weight.

[0112] In this example, the system S comprises an artificial intelligence module or segmentation module SEG1 comprising said weights and situated upstream of the processing device PROC1, SYNT1. Advantageously, the determination comprises learning said weight values ​​from said plurality of input data by back propagation of the gradients of a loss function combining two criteria.

[0113] As a variant, the module SEG1 is integrated into the device 100 .

[0114] Therefore, the device 100 implements the method for segmenting a plurality of data representing a scene according to the present invention, which will be described below with respect to Figure 6 Detailed description.

[0115] According to the present invention, the encoding device 200 is configured to obtain the plurality of data generated by the segmentation device 100 and the segmentation information of the data subset to be processed, and encode the data subset to be processed and the obtained segmentation information. Figure 4 In the embodiment, device 100 is independent of device 200 , but according to a variant (not shown), it is integrated into device 200 .

[0116] Thus, the device 200 implements a method for encoding data representing a scene according to the present invention, which will be described below with respect to Figure 7 Detailed description.

[0117] In this example, the processing device PROC1, SYNT1 is configured to synthesize a view selected by a user from the decoded input views when this view does not correspond to any view of the multi-view video already sent in the coded data stream or file STR, FD. It should be understood that according to the invention, the processing device PROC1, SYNT1 implemented in the server equipment item ES is only used for learning purposes of the automatic segmentation module SEG1.

[0118] In this example, the system S also comprises an item of terminal equipment UE, such as a smartphone or an augmented reality headset, for example of the head mounted device (HMD) type worn by a user UT, which is at a distance from the server equipment item ES and connected to it, for example via a communication network RC.

[0119] According to the invention, the terminal device UE comprises a device capable of receiving a coded data stream or file STR, F via a communication network RC. D , a sending / receiving module E / R, a device 300 for decoding received coded data, a device PROC2, SYNT2 for processing segmented data (which is similar in structure and configuration to the equipment item ES), and a memory MUE.

[0120] In this example, the device 300 is configured to decode the received coded data and make them available to the processing devices PROC2, SYNT2. According to the invention, the coded data comprises segmentation information of a plurality of input data (here a multi-view video VMV) and a subset of data to be processed, which is generated by the device 100 of the equipment item ES as described above as a result of applying the determined segmentation information to the plurality of input data.

[0121] Therefore, the device 300 implements the decoding method according to the present invention, which will be described below with respect to Figure 8 Detailed description.

[0122] exist Figure 4 In the example of FIG. 1 , the processing device PROC2, SYNT2 of the terminal device item UE is therefore configured to, when the view selected by the user from the other decoded views of the multi-view video does not correspond to a view in the coded data stream or file STR, F DWhen any view of the multi-view video sent in the ES is synthesized, the view is synthesized. It is similar to the device PROC1, SYNT1 of the server device item ES in that it has the same structure and the same configuration. In a specific embodiment, the processing device PROC1, PROC2 also includes an artificial intelligence module, which includes a weight structure, the value of which is configured using a machine learning technique. According to this embodiment, the previous configuration of the two processing devices PROC1, PROC2 (ES side and UE side) is automatically learned from the same basis of the input data (e.g. labeled) and under the constraints of a criterion for maximizing the quality measure of the processing result.

[0123] Memory M UE For example, it is used to store additional views synthesized by the processing devices PROC2, SYNT2.

[0124] about Figure 5 , shows another example of the architecture of a system S' according to the invention, comprising a server device item ES' configured to receive input data acquired by a plurality of sensors S1, S2, ..., SN of various types, for example presenting ECG, EEG, scanner, MRI, blood component analysis, radio, etc. In this case, it comprises a plurality of physiological sensors placed on or near a patient SB, and the received input data are a plurality of sequences of physiological measurements acquired by the plurality of sensors at different consecutive moments. According to this example, the server device item ES' has the same features as previously described with regard to Figure 4 The server device item ES has a similar structure to that described. It comprises a device 100' for segmenting a plurality of physiological measurements according to the invention, a module SEG1' for segmenting input data configured by the device 100' to segment the input data according to the invention, a device PROC1' for processing the data generated by the segmentation of the module SEG1', DIAG1, a device 200' for encoding the segmented data according to the invention, a sending / receiving module E / R and a memory M. ES’ The devices 100' and 200' are similar to the devices 100 and 200 described above. Therefore, the device 100' implements a method according to the invention for segmenting input data (here physiological measurements of the patient SB), which will be described below with respect to Figure 6 Therefore, the device 200' implements the method for encoding input data according to the present invention, which will be described below with respect to Figure 7 Detailed description.

[0125] In this example, the processing device PROC1 ', DIAG1 is configured to process the physiological measurements provided by the sensor in order to perform a diagnosis-aiding process. It should be understood that, according to the invention, on the server device item ES side, the processing device PROC1 ', DIAG1 is here only used to configure the segmentation module SEG1 ' of the device 100 ' according to the invention.

[0126] The system S' also comprises a terminal equipment item UE' which is at a distance from the server equipment item ES' and is connected to it via a communication network RC. The server equipment item UE' has the same Figure 4 It includes the same structure as described above regarding the terminal equipment item UE. Figure 4 At least one decoding device 300' similar to the mentioned decoding device 300, a processing device PROC2' previously configured to process a plurality of data generated from the sensor once decoded by the decoding device 300', DIAG2, a transmission / reception module E / R and a memory M UE’ The device 300' is similar to the device 300 described above and implements a method according to the invention for decoding data representing a scene, which will be described below with respect to Figure 8 Advantageously, it also comprises a display device DISP, for example of the screen type, used by the user DOC to view the physiological measurements received and decoded by 300 ′.

[0127] about Figure 6 , an example of implementing a method for segmenting a plurality of input data according to an embodiment of the present invention is now presented in the form of a flow chart. Advantageously, the method is implemented by the above-mentioned device 100 or 100'.

[0128] SQ1(t), SQ2(t), …, SQN(t), where N is a non-zero integer, refers to N time series data acquired at Nt consecutive moments by N sensors placed around the object, subject, or scene of interest, where Nt is a non-zero integer, for example Figure 4 and Figure 5SC or patient SB of the scene. It is assumed that N sequences are obtained in 60. In 61, the weight values ​​of the segmentation module SEG1 (respectively, SEG1') to be applied to the input data are determined. For example, the module SEG1 (respectively, SEG1') is an artificial intelligence module that is able to learn from the input data SQ1(t), SQ2(t), ..., SQN(t) those that are useful to the processing device PROC1 (respectively, PROC1') located downstream (the output of the module SEG1 (respectively, SEG1') is connected to the input of the processing device PROC1 (respectively, PROC1')) and those that are preconfigured to apply a predetermined processing to the plurality of input data, and those that are not. According to the present invention, the segmentation module SEG1 (respectively, SEG1') is configured to apply weights to the plurality of input data, whose values ​​are determined during this learning. Advantageously, the segmentation module SEG1 (respectively, SEG1') includes as many weights as input data. These weights have floating values ​​between, for example, 0 and 1, and their application (multiplication) to the value of an input data item has the effect of keeping that data item "on" or "off" before it is considered by processing device PROC1 (respectively PROC1').

[0129] For example, in a particular configuration of the automatic segmentation module SEG1 (respectively, SEG1'), these weights are distributed in consecutive layers associated with each of the N sequences of input data at each of the Nt time instants of each sequence. For example, when the input sequence is a sequence of images, the module SEG1 comprises N × Nt layers, one layer per image or view and per time instant t, whose dimensions correspond to the dimensions of the images of the N sequences, respectively.

[0130] According to the invention, learning is performed based on a first performance criterion of the segmentation module SEG1 (respectively SEG1'), which here is a criterion for optimizing the amount of data to be presented to the processing device PROC1 (respectively PROC1'), but also based on a second performance criterion specific to the processing device PROC1 (respectively PROC1') itself, which is a criterion for optimizing the processing quality. This consideration of the combination of two consecutive processing operations and the satisfaction of these two performance criteria ensure that the pruning of the input data by the segmentation module SEG1 (respectively SEG1') does not penalize the performance of the processing device PROC1 (respectively PROC1').

[0131] For example, the first criterion consists in minimizing the amplitude of the cumulative values ​​of the weights of the layer of segmentation modules SEG1, SEG1'. According to a variant, it consists in minimizing a number of weight values ​​above a given threshold in the layer or maximizing a number of weight values ​​below the threshold. According to yet another variant, it consists in minimizing the entropy of the input data at the output of the segmentation modules SEG1, SEG1', i.e. once the values ​​of the weights of the layer have been applied to them. According to another variant, it consists in minimizing the amplitude of the cumulative values ​​of the input data maintained in the subset of data to be processed. According to yet another variant, it consists in minimizing the number of input data maintained in the subset of input data to be processed. According to yet another variant, it consists in minimizing the entropy of the input data values ​​maintained in the subset of input data to be processed. Of course, the first criterion may also consist in a combination of several of these various criteria.

[0132] In 62, the segmentation information SGI of the input data is determined according to the weight value determined for the module SEG1 (respectively, SEG1'). For the input data item, according to the value of the corresponding weight, the corresponding item of the segmentation information SGI is set to a first value or a second value different from the first value. For example, the first value is higher than the second value. The value of the weight is compared with a given threshold. When the value of the weight is higher than the threshold, the segmentation information item is assigned a first value (which means "useful", therefore "on"), otherwise it is assigned a second value (which means "useless", therefore "off"). For example, for input data of an image type of size width W × height H, the segmentation information associated with the image includes a segmentation map of the same size W × H, so when the pixels correspond to data judged to be "useful", they are assigned a gray level corresponding to "white", otherwise they are assigned a gray level corresponding to "black". As a variant, the first value is equal to 1 and the second value is zero. The segmentation information SGI is stored in the memory M1.

[0133] In 63, a subset of data to be processed USS is obtained by applying segment information to a plurality of input data SQ1, SQ2, ..., SQN and retaining only those data associated with segment information items assigned a first value. For example, the data subset USS to be processed is stored in a memory M1 and may be provided to an encoder.

[0134] Optionally, according to a particular embodiment of the invention, the processing device PROC1 (respectively PROC1 ') further comprises an artificial intelligence module having a layer structure comprising processing weights. It is assumed that a learning base has been previously used and trained in a manner known per se to process input data of the type N time series SQ1(t), SQ2(t), ..., SQN(t) according to a criterion for maximizing a measure of processing quality. It is therefore assumed that the values ​​of its processing weights have been determined during this previous learning.

[0135] According to a first embodiment of the invention, the processing weights of the processing device PROC1 (respectively PROC1 ′) are fixed, ie their values ​​are not modified by the determination 61 .

[0136] According to a second embodiment of the invention, the determination 61 also applies to the processing device PROC1 (respectively PROC1') and has the effect of modifying its processing weight values. Modified processing quantity values ​​MW are obtained in 64. They are provided to the encoder or stored in the memory M1.

[0137] The following will be about Figures 9 to 11 Embodiments of the present invention are described in detail in specific use cases.

[0138] about Figure 7 , an example of implementing a method for encoding a plurality of input data according to an embodiment of the present invention is now presented in the form of a flow chart. Advantageously, the method is implemented by the above-mentioned device 200 or 200'.

[0139] In 70, the device 200, 200' obtains N time series of input data SQ1(t), SQ2(t), ..., SQN(t) acquired by N sensors placed around the object, subject or scene of interest, where N is a non-zero integer. In 71, it obtains the N time series of input data SQ1(t), SQ2(t), ..., SQN(t) acquired by N sensors placed around the object, subject or scene of interest, where N is a non-zero integer. Figure 6 The described segmentation of the data is performed by the segmentation method according to the invention. It is assumed that the device 200, 200' obtains at least the segmentation information SGI and the data subset USS to be processed provided by the segmentation method according to the invention. Advantageously, it also obtains the modified weight values ​​MW intended for the processing device PROC1, PROC1'.

[0140] It should be noted that the device 100, 100' may or may not be integrated into the encoding device 200, 200'. In the first case, the device 200, 200' directly implements the segmentation method according to the invention. In the second case, it instructs the device 100, 100' to segment the input data and receives the segmented data in response.

[0141] In 72, the device 200, 200' encodes the obtained data SGI, USS in a conventional manner known per se. In 73, the encoded data are stored in a data file F in the memory M2. D or sent to a remote device item in a data stream STR, for example Figure 4 and Figure 5 Terminal equipment items UE, UE'.

[0142] about Figure 8, an example of a method for decoding data acquired by a plurality of sensors according to an embodiment of the present invention is now presented in the form of a flow chart. Advantageously, the method is implemented by the above-mentioned device 300 or 300'.

[0143] In 80, the encoded data is obtained, for example, from the memory M3, or via a communication network (such as Figure 4 and Figure 5 network RC) to receive them.

[0144] According to the present invention, the encoded data at least includes segmentation information SGI and a subset of data to be processed USS, and the segmentation information SGI identifies the data to be processed of the subset USS in N input data sequences expected by the decoder.

[0145] In 81, the encoded data is decoded in a manner known per se. A decoded subset of data to be processed USSD and decoded segment information SGID are obtained. Optionally, in a particular embodiment, a decoded modified processing weight value MWD intended for the processing device PROC2 (respectively PROC2') is also obtained.

[0146] In 82, the decoded segment information SGID and the decoded data subset USSD to be processed are used to construct a sequence of segmented decoded data SQ1DS(t), SQ2DS(t), ..., SQNDS(t) to be presented to the processing device PROC2, PROC2'. If the decoded modified weight values ​​MWD have been obtained, they are pre-applied to the processing device PROC2, PROC2', whose structure and initial configuration are similar to those of the processing device PROC1, PROC1' on the encoder side. Once the configuration is completed, the segmented decoded data sequence SQ1DS(t), SQ2DS(t), ..., SQNDS(t) is provided to the device PROC2, PROC2' for processing.

[0147] Now about Fig. 9 and Fig.10 A first embodiment of the present invention is described.

[0148] As in Figure 1 and Figure 4 In , consider a scene SC captured by a set of N cameras C1, C2, ..., CN. Each camera observes the scene from a given viewpoint and generates texture images or views T1, T2, ..., TN corresponding to that viewpoint at Nt consecutive times t.

[0149] The processing to be applied to the input data corresponds here to view synthesis. It specifically includes synthesizing additional views associated with additional viewpoints PVS different from N viewpoints of N cameras. The synthesis device in question is configured to take all or some of the N images T1(t), T2(t), ..., TN(t) and the intrinsic (focal length, sensor resolution, number of color components, etc.) and non-intrinsic (position in space, orientation, etc.) parameters of the N cameras and the coordinates of the additional viewpoints PVS of the scene as input, and produce additional images or views TS as output. In this example, it is assumed that the synthesis devices SYNT1, SYNT2 include IBRNet neural networks. Such a network has a structure of M consecutive layers L'1, L'2, ..., L'M, where M is a non-zero integer, which are connected together so that the output of the upstream layer is presented at the input of the next downstream layer. Each layer has a configuration weight WjL'i, i is an integer between 1 and M, j is an integer between 1 and a plurality of processing weights associated with layer i, whose values ​​are determined during previous learning.

[0150] about Fig. 9 , first describe the operations implemented on the encoder side, i.e. Figure 4 According to the invention, the automatic segmentation module SEG1 also has the structure of a layered neural network, also called a relevance layer, because the values ​​of its weights are intended to finely indicate the level of relevance or usefulness of the input data item for its subsequent processing by the synthesis device SYNT1.

[0151] According to this embodiment of the invention, the automatic segmentation module comprises as many layers L1, L2, ..., LN as there are input images T1(t), T2(t), ..., TN(t) at a given moment t. For simplicity, in the following, this given moment t is considered, and the corresponding sequence of input images will be referred to as T1, T2, ..., TN.

[0152] Advantageously, each layer Li with i between 1 and N has the same size as the corresponding input image or view. For example, if the input image has a size of W × H pixels, the layer Li also has a size of W × H, i.e., it comprises W × H weights, each weight being a unique multiplication coefficient of a scalar (grayscale) or vector (RGB or YUV triplet) value associated with a corresponding pixel of the input image. Advantageously, the multiplication coefficients are floating real numbers.

[0153] As a variant, the weight of each layer Li takes a binary value. If the weight bit has the value 1, the associated pixel is sent unchanged to the input of the processing device SYNT1, otherwise the associated pixel is not sent to the processing device SYNT1 or it is sent with a predefined value (for example, -1) to indicate that it is useless.

[0154] The neural network of the segmentation module SEG1 is then learned to assign appropriate values ​​to the weights of these various layers so that it performs the intended task (here, synthesis of additional views) satisfactorily.

[0155] According to this embodiment of the invention, in practice, it is the neural network CRN1 combining the N layers L1, L2, ..., LN of the segmentation module SEG1 and the M layers L'1, L'2, ..., L'M of the processing device PROC1 that is subjected to this learning. For this purpose, two different performance criteria are used, referred to in this document as loss functions. The first criterion relates to the loss associated with the task of the neural network of the synthesis device SYNT1 and measures the quality of the synthesized image, and the second criterion relates to the loss associated with the segmentation of the input data by the neural network of the module SEG1 and measures the cumulative amplitude of the weight values ​​of its layers L1, ..., LN. A global loss function is defined that takes into account these two criteria. An example is described in detail below.

[0156] In particular, the learning is similar to that described in the paper by Wang et al. already cited. In short, the learning works as follows: for each value j in 1, ..., N, the combination network CRN1 is asked to generate a view Tj from the other views. Since the original view Tj associated with the viewpoint PVj is available, the quality of the synthesis performed by the device PROC1 can be evaluated, for example, by calculating the difference between the synthesized view TSj and the original view Tj (for example, based on the mean squared error). This difference is then used to calculate the gradient, which is back-propagated to the layer Lj of the module SEG1 in a manner known per se. It should be noted that in the case of a synthesis device that is not based on artificial intelligence methods, the calculated gradient is simply propagated in the layer Lj.

[0157] For example, the global loss function to be minimized is as follows (where ||∙|| is the L2 norm, based on the square root of the sum of the squares of the coordinates of the vector):

[0158] | | TSj-Tj | | + . | | PLj | |

[0159] in, is a parameter set by the user to determine the size of the segmented region to minimize (high ) and view synthesis performance (low ). It is a positive real number, which can be greater or less than 1. PLj is the vector concatenation of W×H weights of N layers Li.

[0160] According to a first option, the weights of the neural network of the synthesis device SYNT1 are fixed and are not modified by the learning of the module SEG1. In other words, only the weights of the layers L1, L2, ..., LN of the module SEG1 are optimized in the sense of a loss function.

[0161] As a variant, the weights of the network layer of the processing device SYNT1 can be varied. Advantageously, they are optimized during the learning of the module SEG1, which becomes the combined learning of the module SEG1 and the device SYNT1. In this case, the modified values ​​MWL'1, MWL'2, ..., MWL'M of the processing weights of the device SYNT1 are obtained and stored for transmission, for example to Figure 4 Terminal equipment item UE.

[0162] Once the learning ( 61 ) is completed, the segment information SGI of the input data T1 , T2 , . . . , TN is determined in 62 .

[0163] According to this embodiment of the invention, each layer L1, L2, ..., LN of the module SEG1 is transformed into a segmentation map SGI1, SGI2, ..., SGIN of an associated view Tl, T2, ..., TN. It should be noted that in this example, the segmentation map of a given image or view is a map having the same size as the image and whose elements or pixels are assigned a first value, for example a value of 1, meaning that the associated pixel in the image Tj is useful for a subsequent task of the processing device PROC1, and a value of 0 means that the associated pixel in the image Tj is useless for the processing device.

[0164] Advantageously, when the value of the weights of the layers L1, L2, ..., LN is a floating real number, for example between 0 and 1, the corresponding value of the segmentation map is determined by comparing each weight with a given threshold value, for example equal to 0.01. If the weight value is below the threshold value, the corresponding pixel in the segmentation map is set to 0, otherwise the pixel is set to 1.

[0165] As a variant, when the values ​​of the weights are binary, they are copied directly into the corresponding segment map SGIi (the SGIi map is equal to the weight map of layer Li).

[0166] The views T1, T2, ..., TN are then encoded (72) by the encoding device 200 using the segment maps SG1, SG2, ..., SGN. Advantageously, a method for encoding multi-view video is used, preferably the MIV standard. As a variant, other standards may be used, such as MV-HEVC or 3D HEVC.

[0167] According to the MIV standard, at time t, parts of views T1(t), T2(t), ..., TN(t), called image tiles, are arranged in a large common image called an atlas. Typically, these parts correspond to the results of an a priori segmentation of views acquired, for example, by N cameras, which is designed to keep only non-redundant information between views. For example, parts of an object of a scene that will be visible on a given view and transmitted in an image tile of that view no longer need to be transmitted in an image tile of another view.

[0168] The atlases corresponding to each moment of the video are then encoded as a conventional 2D video using, for example, the HEVC video coding standard. The MIV standard does provide a syntax that enables the decoder to identify the image blocks within the atlas and rearrange them in the initial view. Therefore, the decoded and then reconstructed view on the decoder side is partial.

[0169] According to the invention, segmentation into image patches is performed for each view Tj based on a segmentation map SGIj generated from the values ​​of the weights assigned to the automatic segmentation module after learning (61).

[0170] In this regard, it should be noted that there are several ways of creating image blocks from the same segmentation map. For example, as many image blocks are formed as there are groups of connected pixels with "useful" values ​​in the segmentation map under consideration. As a variant, a minimum limit can be imposed for forming image blocks from such a group of connected pixels. For example, if a group of segmentation pixels contains a number of pixels below a first threshold (e.g. equal to 100 pixels), the formation of a new image block will not be considered, but it will be sought to attach it to a group of adjacent segmentation pixels until this limit is exceeded. According to another variant, it is also possible to simply omit consideration of groups of segmentation pixels whose number of pixels is below a second threshold, below the first threshold and, for example, equal to 5.

[0171] The image blocks thus formed for a given view Ti(t) are merged into an atlas, where other image blocks may originate from other views Tj(t) associated with the same time instant t. In order to reconstruct the partial views at the decoder side, the following information must be sent to it and is therefore encoded:

[0172] - the view to which the image tile is associated,

[0173] - coordinates of the image patch in the atlas,

[0174] - Occupancy maps for image patches in the atlas.

[0175] It should be noted that the MIV standard already allows for the view segmentation map to be taken into account in the coding scheme for multi-view video, so that only the input data of the data subset USS to be processed is encoded and sent to the receiver (UE).

[0176] Once the atlas has been populated with all image blocks identified in all views, the atlas is encoded by a conventional encoder, e.g. compliant with the HEVC standard.

[0177] In one embodiment, the configuration of the neural network of the synthesis device SYNT1 is fixed.

[0178] In another embodiment, the configuration of the neural network of the synthesis device SYNT1 is modified by learning 61 and modified values ​​MWL'1, MWL'2, ..., MWL'N of the processing weights of the various layers L'1, L'2, ..., L'M of the neural network of the synthesis device SYNT1, these modified values ​​MWL'1, MWL'2, ..., MWL'N are also coded (for example using the MPEG NNR (Neural Network Representation) coding standard) as specified in part 17 of the MPEG-7 standard for their transmission to the corresponding synthesis device SYNT2 on the receiver side (UE). It should be noted here that several modified processing weight values ​​MWL'i can be sent for a layer L'i and that this set of values ​​MWL'i can take the form of a map or image whose dimensions are the same as the dimensions of the layer L'i concerned.

[0179] Assume that the encoded data is, for example, in the form of an encoded data stream STR or an encoded data file F D is sent to a receiver device item via a communication network RC in the form of, for example Figure 4 Terminal equipment item UE.

[0180] They are received by a decoding device 300 which, according to this embodiment, decodes the coded data relating to complete or partial views T1 , T2 , . . . , TN according to the MIV standard.

[0181] This decoding produces the following decoded data:

[0182] - a series or sequence of atlases,

[0183] - For each atlas, data representing the image tiles it contains, and for each of these image tiles:

[0184] - the view to which the image tile is associated,

[0185] - the coordinates of the image patch in the atlas, and

[0186] - Occupancy maps for image patches in the atlas.

[0187] From the decoded data, the decoded views T1D, T2DS, ..., TNDS of the segment are reconstructed (82) completely or partially (depending on the information contained in the block).

[0188] In the embodiment according to which the modified weight values ​​MWL'1, MWL'2, ..., MWL'M are encoded, they are also decoded and then used to update the configuration of the layers of the neural network of the processing device SYNT2.

[0189] The N decoded views T1D, T2D, ..., TNDS are then presented to a neural network corresponding to device PROC2.

[0190] Assume that the user UT wants to synthesize an additional view according to an additional viewpoint PVS. In this case, according to the invention, according to the embodiment considered, at the input of the synthesis device SYNT2, the segmented decoded views T1DS, T2DS, ..., TNDS (some of whose layers L'i may have been updated with modified decoding weight values ​​MWL'1, MWL'2, ..., MWL'M) are presented. The coordinates of the viewpoint PVS that the user UT wants are also input. The device SYNT2 finally generates an additional synthesized view TSj'.

[0191] Now about Fig.11 and Fig.12 A second embodiment of the present invention is described.

[0192] about Figure 5 and Fig.11 , consider a patient SB surrounded by a plurality of sensors S1, S2, ..., Si, ..., SN of various types (ECG, EEG, scanner, MRI, chemical analysis, radio, ...). Each sensor acquires at least one of the data T1(Y), T2(Y), ..., Ti(Y), ..., TN(Y) of the type physiological measurement, image, etc.

[0193] Furthermore, assume that the data is then processed by several neural networks R1, R2, ..., Rk, ..., RK, where K is a non-zero integer less than or equal to N, each neural network being dedicated to a diagnosis or diagnosis-aiding task. For example, all these networks are considered to form Figure 5Each of these networks is configured to take as input a subset of the N input data T1(Y), ..., Ti(y), ..., TN(Y) and to produce as output an item of diagnostic information Di(Y) (e.g. presence of a tumor, suspected epilepsy, ...) or an item of diagnostic auxiliary information (e.g. segmentation, i.e. geographical delimitation of an area that may contain a tumor, etc.). The subset of input data to be presented to each neural network Rk is selected by an algorithm external to the neural network under consideration and may include some or all of the data Ti(Y). In the paper titled “Multi-channel deep convolutional neuralnetworks for multi-classifying thyroid disease” by Zhang et al. (published in March 2022 by https: / / arxiv.org / abs / 2203.03627), the paper titled “Joint brain tumorsegmentation from multi MR sequences through a deep convolutional neuralnetwork” by Dehghani et al. (published in March 2022 by https: / / arxiv.org / abs / 2203.03338), and the paper titled “CNN-Aided Factor Graphs with Estimated Mutual Information Features for Seizure Detection” by Salafian et al. (published in March 2022 by https: / / arxiv.org / abs / 2203.05950), the paper titled “CNN-Aided Factor Graphs with Estimated Mutual Information Features for Seizure An example of a neural network Rk suitable for diagnosing a specific pathology is given in the paper "Detection in MRI Scanning" (published in March 2022 by the website https: / / arxiv.org / abs / 2203.05950).

[0194] Therefore, the processing of the input data to be applied to each of the networks Rk corresponds here to a diagnosis-assisting process.

[0195] In this example, it is assumed that each processing device DIAG11, DIAG12, ..., DIAG1K comprises a neural network in layers R1, R2, ..., RK. Each of these networks Rk has a structure in consecutive layers connected together so that the output of the upstream layer is present at the input of the next downstream layer. Each layer has configuration weights whose values ​​are adjusted during previous learning.

[0196] about Fig.11 , first describe the operations implemented on the encoder side, i.e. Figure 5 In this example, K segmentation modules SEG11', SEG1K', ..., SEG1K' are considered, each of which has the structure of a neural network in a layer or associated layers, because the value of its weight is intended to finely indicate the level of relevance or usefulness of an input data item, the subsequent processing of which is performed by the neural network R1, R2, ..., RK corresponding to the associated processing device DIAG11, DIAG1K, ..., DIAG1K.

[0197] According to this embodiment of the invention, each segmentation module SEG1k' comprises as many layers L1, L2, ..., LN as there are input data intended for the network Rk. According to a variant not shown, the segmentation layers can be pooled so that the segmentation module SEG1 has one layer Li for each type of input data item Ti and the corresponding segmented data are sent to each network Rk that needs them. For example, in Fig.11 , the network Rk receiving data T3, T8 and T9 is connected to layers L3(y), L8(y) and L9(y) at the input.

[0198] Advantageously, each layer Lik(y) of the module SEG1k' has the same size as the associated input data item. For example, if the input data has size H parameters, then the layer Li(y) also has size H weights, each weight being a unique multiplication coefficient associated with a data item (the data item having, for example, a scalar or vector value).

[0199] In a particular embodiment, the weight of each layer Lik(y) of the segmentation module SEG1k' takes a binary value. If the value of the weight bit is 1, the associated data or parameter item is sent to the network Rk, otherwise it is not sent. It should be noted that in the latter case, a predefined value (e.g. equal to -1) is sent to the processing device DIAG1k.

[0200] According to the invention, the learning of a network combining a certain number of layers Li(Y) and layers of a network Rk (not shown) is performed using a loss function that combines a loss associated with the task of the network Rk (i.e., the diagnostic quality) and a loss representing the cumulative magnitude of the weights of the layers Ci(Y). CRk refers to the network obtained by combining (multiple) input layers Li(Y) with the network Rk. Prior to this learning of the network of the segmentation module SEG1k', an input data sequence T1(y), T2(y), ..., TN(y) is presented to each network Rk in order to obtain an original diagnosis Dk(y).

[0201] Then, the learning, known per se, works as follows. For each value k in 1, ..., K, an attempt will be made to maximize the diagnostic power of the network CRk based on the data. Since the original diagnosis Dk(y) is available, the difference between this original diagnosis and the output of the network CRk can be calculated (e.g., the diagnostic digital error). This difference is used to calculate the gradient according to the known formula, which is back-propagated to the weights of the combined network CRk to be optimized.

[0202] In the following, Destk refers to the value of the diagnosis generated by CRk(Y) and Dk(y) to the original diagnosis. The loss function to be minimized (where is the L2 norm) can be expressed as follows:

[0203] | | Destk-Dk(y) | | + . | | Pli | |

[0204] in is a user-set parameter that determines the size of the segmented region to minimize (high ) and diagnostic performance (low ) and PLi is the vector concatenation of the weights of the K layers Li present in the neural network of the segmentation module SEG1k' at the input of the processing device DIAG1k. These are positive real numbers that can be greater or less than 1.

[0205] In a first embodiment, the weights of the network Rk contained in the neural network CRk are fixed during learning, only the weights of the layers Li of the network of the segmentation module SEG1k′ are optimized in the sense of the loss function explained above.

[0206] In a second embodiment, the weights of the layers of the network Rk are also modified by learning. Their modified values ​​(not shown) are stored in a memory.

[0207] Then, each layer Lik of the network SEG1k' generated by learning is transformed into a segmentation map SGIi of the associated data. A segmentation map is a map of the same size as the data, containing a value of 0 or 1 for each parameter in the data item, 0 means that the relevant parameter in the data item is not used for the subsequent diagnosis task, and 1 means the opposite.

[0208] Advantageously, when the value of the weights of the layer Li is a floating real number, for example between 0 and 1, the corresponding value of the segmentation map is determined by comparing each weight with a given threshold, for example equal to 0.01. If the value of the weight is below this threshold, the corresponding data item of the segmentation map is set to 0, otherwise it is set to 1.

[0209] As a variant, when the values ​​of the weights are binary, they are copied directly into the corresponding segment map SGIi (the SGIi map is equal to the weight map of layer Ci).

[0210] The segmented parts of the input data T1(y), T2(y), ..., TN(y) are then encoded, i.e. the processed data set USS obtained by applying the segmentation graph SGIi obtained at each of the N input data sequences, for example using a standard such as MPEG NNR or gzip.

[0211] It is also necessary to encode the segmentation map, for example in the form of a series of binary values ​​indicating for each parameter or measurement of the considered sequence Ti whether it is encoded or not.

[0212] The encoded data is stored in memory and / or transmitted in a data file or data stream.

[0213] about Fig.12 Now consider the encoded data stream STR or the encoded data file F D Receivers, such as Figure 5 The terminal equipment UE', the receiver decodes the stream STR or file F D The coded data received in the . Segment diagrams SGD1, SGD2, ..., SGDN indicating which parameters of the N sequences T1, T2, ..., TN have been sent and are therefore contained in the subset of data to be processed are decoded and then the values ​​of the parameters in question (contained in the coded subset of data to be processed) are decoded using a suitable decoding method opposite to the one used during encoding (MPEG NNR decoding or gzip decoding). The decoded subset of data to be processed USSD is obtained.

[0214] The segmented decoded data sequences T1DS, T2DS, ..., TNDS are then reconstructed for each of the diagnostic devices DIAG21, DIAG22, ..., DIAG2K, respectively (by means of CONST1, CONST2, ..., CONSTK in 82), whether they are complete (all parameters of the data sequence can be reconstructed) or partial (the parameters sent allow only a part of the sequence to be reconstructed) from the decoded subset of data to be processed USSD and the decoded segment graphs SGD1, SGD2, ..., SGDK. Optionally, when the decoded data include modified weight values ​​for the network Rk, these modified weight values ​​are injected into the relevant network Rk.

[0215] Now use Figure 5 In this example, the user (here the practitioner) wants to use Fig.11 The diagnostics are performed on a network Rk (e.g. network R2) and some reconstructed data sequences (e.g. T3D, T8D and T9D). The segmented data T1DS, T2DS, ..., TNDS (possibly partial) reconstructed during decoding are input into layer Ci, which itself is connected to the input of the network CRk of the device DIAG2k. The network Rk can then generate its diagnostics from the segmented data alone.

[0216] about Fig.13 An example of a hardware structure of a device 100 for segmenting multiple input data acquired by a sensor includes: a module for determining weight values ​​to be applied to multiple input data before the multiple input data are processed by at least one processing device, the at least one processing device being configured to generate a processing result according to a criterion for maximizing a quality measurement of the input data processing result, the weight value being determined according to the criterion and another criterion for minimizing the amount of input data to be processed; a module for determining segmentation information of the multiple input data, a segmentation information item of one of the data items being assigned a first value or a second value different from the first value according to the weight; and a module for obtaining a data subset to be processed by applying the determined segmentation information to the multiple input data, the data subset to be processed including data in the multiple input data associated with a segmentation information item equal to the first value.

[0217] The term "module" may correspond to a software component as well as a hardware component or a set of hardware and software components, the software component itself corresponding to one or more computer programs or subroutines, or more generally, to any element of a program capable of realizing a function or a set of functions.

[0218] More generally, such a device 100 comprises a random access memory 103 (e.g. a RAM memory), a processing unit 102, which is equipped with, for example, a processor and controlled by a computer program Pg1 stored in a read-only memory 101 (e.g. a ROM memory or a hard disk), which represents the modules of the above-mentioned device 100. At initialization, the code instructions of the computer program are loaded into the random access memory 103, for example, before being executed by the processor of the processing item 102. The random access memory 103 may also contain segment information and a subset of useful data.

[0219] Fig.13 Only a particular one of several possible ways of implementing the apparatus 100, 100' is shown so that it performs in its various embodiments as described above with respect to Figure 6 , 9 The steps of the segmentation method are described in detail in 1 and 11. In practice, these steps can be implemented indifferently on a reprogrammable computing machine (PC computer, DSP processor or microcontroller) executing a program comprising a sequence of instructions, or on a dedicated computing machine (e.g. a set of logic gates such as an FPGA or an ASIC, or any other hardware module).

[0220] In the case where the device 100 is implemented by a reprogrammable computing machine, the corresponding program (i.e., sequence of instructions) may be stored in a removable (such as, for example, an SD card, a USB flash drive, a CD-ROM, or a DVD-ROM) or non-removable storage medium that can be partially or fully read by a computer or processor.

[0221] about Fig.14 , an example of the hardware structure of the device 200, 200' for encoding multiple input data according to the present invention includes at least a module for obtaining segmentation information of the multiple input data and obtaining a data subset to be processed by applying the segmentation information to the multiple input data, the segmentation includes determining a weight value to be applied to the multiple input data before the multiple input data are processed by at least one processing device, the at least one processing device is previously configured to generate a processing result according to a criterion for maximizing a quality measurement of processing, the weight value is determined according to the criterion and another criterion for minimizing the amount of input data to be processed, one of the information items of one of the data items is assigned a first value or a second value different from the first value according to the weight value, the data subset to be processed includes data associated with the segmentation information item equal to the first value in the multiple input data; and a module for encoding the segmentation information and the data subset to be processed.

[0222] The term "module" may correspond to a software component as well as a hardware component or a set of hardware and software components, the software component itself corresponding to one or more computer programs or subroutines, or more generally, to any element of a program capable of realizing a function or a set of functions.

[0223] More generally, such a device 200, 200' comprises a random access memory 203 (e.g. a RAM memory), a processing unit 202 equipped with, for example, a processor and controlled by a computer program Pg2, which represents segmentation and encoding modules stored in a read-only memory 201 (e.g. a ROM memory or a hard disk). At initialization, the code instructions of the computer program are loaded into the random access memory 203, for example, before being executed by the processor of the processing item 202. The random access memory 203 may also contain encoding information.

[0224] Fig.14 Only one of several possible ways to implement the device 200, 200' is shown, so that it performs the above-mentioned Figure 7 , 9 The steps of the encoding method are described in detail in 1 and 11. In practice, these steps can be implemented on a reprogrammable computing machine (PC computer, DSP processor or microcontroller) that executes a program comprising a sequence of instructions, or on a dedicated computing machine (e.g. a set of logic gates such as an FPGA or an ASIC, or any other hardware module).

[0225] In the case where the device 200, 200' is implemented by a reprogrammable computing machine, the corresponding program (i.e., the instruction sequence) can be stored in a removable (such as, for example, an SD card, a USB flash drive, a CD-ROM or a DVD-ROM) or non-removable storage medium, which can be partially or completely read by a computer or a processor.

[0226] Finally, about Fig.15An example of a hardware structure of a device 300, 300' for decoding encoded data includes: a module for decoding encoded data including segmentation information of multiple data called input data acquired by a sensor and a data subset to be processed, one of the segmentation information items of the input data items being assigned a first value or a second value different from the first value, the data subset to be processed having been obtained by applying the segmentation information to multiple input data, the data subset to be processed including data associated with one segmentation information item equal to the first value in the multiple input data; a module for constructing multiple decoded segmented input data from the decoded data subset to be processed and the decoded segmentation information; and a module for providing the multiple decoded segmented input data to a processing device, the processing device being configured to generate a processing result according to a criterion for maximizing a quality measurement of processing.

[0227] The term "module" may correspond to a software component as well as a hardware component or a set of hardware and software components, the software component itself corresponding to one or more computer programs or subroutines, or more generally, to any element of a program capable of realizing a function or a set of functions.

[0228] More generally, such a device 300, 300' comprises a random access memory 303 (for example a RAM memory), a processing unit 302 equipped with, for example, a processor and controlled by a computer program Pg3 representing the above modules stored in a read-only memory 301 (for example a ROM memory or a hard disk). At initialization, the code instructions of the computer program are loaded into the random access memory 303, for example, before being executed by the processor of the processing item 302.

[0229] Fig.15 Only one particular one of several possible ways to implement the apparatus 300, 300' is shown so that it performs the functions described above with respect to the apparatus 300, 300' in its various embodiments. Figure 8 , Fig.10 and Fig.12 The steps of the decoding method are described in detail. In practice, these steps can be implemented on a reprogrammable computing machine (PC computer, DSP processor or microcontroller) executing a program comprising a sequence of instructions, or on a dedicated computing machine (e.g. a set of logic gates such as an FPGA or ASIC, or any other hardware module).

[0230] In the case where the device 300, 300' is implemented by a reprogrammable computing machine, the corresponding program (i.e., the instruction sequence) can be stored in a removable (such as an SD card, USB flash drive, CD-ROM or DVD-ROM) or non-removable storage medium, which can be partially or fully read by a computer or processor.

[0231] The invention just described in its different embodiments has many advantages. This makes it possible to select from a plurality of data representing scenes, objects or subjects those data which will actually be useful for processing the plurality of data by a processing device of a receiver item of equipment before the plurality of data is sent in a communication network. To this end, the invention uses machine learning techniques, which are implemented in a clever way to train an automatic segmentation module specifically at the sender item of equipment to segment the plurality of input data in order to minimize the amount of data to be sent while maximizing the quality of processing.

Claims

1. A method for segmenting a plurality of data acquired by a sensor, referred to as input data, the method comprising: - determining (61) weight values ​​to be applied to a plurality of input data before the plurality of input data are processed by at least one processing device (PROC1, SYNT1, DIAG1), the at least one processing device (PROC1, SYNT1, DIAG1) being configured to generate processing results according to a criterion for optimizing the quality of the processing results of the input data, the weight values ​​being determined according to the criterion and another criterion for optimizing the amount of input data to be processed, - determining (62) segment information (SGI) of the plurality of input data, one of the segment information items of one of the data items being assigned a first value or a second value different from the first value according to the weight, and - obtaining (63) a data subset to be processed by applying the determined segment information to the plurality of input data, the data subset to be processed comprising data of the plurality of input data associated with the segment information item equal to the first value.

2. Segmentation method according to the preceding claim, characterized in that The determining (61) comprises learning the weight values ​​from the plurality of input data, the learning being performed by back propagation of gradients combining two standard loss functions.

3. The segmentation method according to any one of claims 1 and 2, characterized in that: The processing device (PROC1, SYNT1, DIAG1) comprises weights, called processing weights, whose values ​​have been predetermined according to criteria for optimizing the quality of the input data processing result, and the method further comprises determining (64) modified values ​​of the processing weights.

4. The method for segmenting a plurality of data according to any one of claims 2 and 3, characterized in that: The plurality of input data comprises a plurality of views acquired by a plurality of cameras, one of the views comprising pixels, the weights being comprised in a plurality of layers, one of the layers being associated with one of the views and comprising one of the weights for each pixel, and the segmentation information comprising a plurality of segmentation maps, one of the maps being associated with one of the views.

5. The method for segmenting a plurality of data according to any one of claims 2 and 3, characterized in that: The multiple input data include multiple measurement data sequences acquired by multiple sensors, the weights are included in multiple layers, one of the layers is associated with one of the measurement data sequences and includes one of the weights for each measurement data item, and the segmentation information includes multiple segmentation sequences, one of the segmentation sequences is associated with one of the measurement data sequences.

6. A method for encoding a plurality of data acquired by a sensor, referred to as input data, characterized in that The method comprises: - obtaining (71) segmentation information (SGI) of the plurality of input data and a data subset to be processed (USS) by applying the segmentation information to the plurality of input data, the segmentation comprising determining a weight value to be applied to the plurality of input data before the plurality of input data are processed by at least one processing device (PROC1, SYNT1, DIAG1) previously configured to produce a processing result according to a criterion for optimizing a processing quality, the weight value being determined according to the criterion and a criterion for optimizing an amount of input data to be processed, one of the segmentation information items of one of the data items being assigned a first value or a second value different from the first value according to the weight value, the data subset to be processed (USS) comprising data of the plurality of input data associated with the segmentation information item equal to the first value, and - Encoding the segment information and the data subset to be processed (72).

7. Coding method according to the preceding claim, characterized in that The obtaining also includes: - obtaining (71) modified weight values ​​(MW), said weights being intended to be applied by said processing device (PROC1, SYNT1, DIAG1) to said input data, said weight values ​​being previously determined according to said criteria for optimizing the quality of the input data processing result and for processing said plurality of input data, said weight values ​​having been modified according to said criteria for processing said subset of data to be processed, - encoding (72) the modified weight value.

8. A method for decoding encoded data, characterized in that The method comprises: - decoding (81) coded data, the coded data comprising segmentation information (SGI) of a plurality of data, referred to as input data, acquired by a sensor, a data subset (USS) to be processed by a processing device, the processing device being configured to apply weights to a plurality of decoded input data and to generate a processing result according to a criterion for optimizing the quality of the processing, one of the segmentation information items of one of the input data items being assigned a first value or a second value different from the first value, the data subset to be processed having been obtained by applying the segmentation information to the plurality of input data, the data subset to be processed comprising data of the plurality of input data associated with one of the segmentation information items equal to the first value, the coded data further comprising modified values ​​of the weights, the modified values ​​having been determined for processing the plurality of segmented input data according to a criterion for optimizing the quality of the processing and a criterion for optimizing the data volume of the data subset to be processed, - constructing (82) a plurality of decoded segmented input data (SQ1DS(t), SQ2DS(t), . . . , SQNDS(t)) from the decoded data subset to be processed and the decoded segment information; - providing (83) the plurality of decoded segment input data and the modified values ​​of the weights to the processing device.

9. A signal carrying coded data, the coded data comprising segmentation information of a plurality of data, referred to as input data, acquired by a sensor, a data subset to be processed obtained by applying the segmentation information to the plurality of input data, one of the segmentation information items of one of the input data items being assigned a first value or a second value different from the first value, the data subset to be processed comprising data of the plurality of input data associated with the segmentation information item equal to the first value, the data subset to be processed being intended to be decoded and then used to reconstruct a plurality of decoded segmented input data from the decoded segmentation information in order to be processed by a processing device, the processing device being configured to generate a processing result according to a criterion for optimizing processing quality by applying weights to the plurality of input data, The encoded data also includes modified values ​​of the weights, which have been determined for processing multiple segmented input data based on criteria for optimizing processing quality and criteria for optimizing the data amount of the data subset to be processed, and are intended to be used by the processing device to update the weights before processing multiple reconstructed decoded segmented input data.

10. A device (100) for segmenting a plurality of data acquired by a sensor, referred to as input data, the device being configured to implement: - determining weight values ​​to be applied to the plurality of input data before the plurality of input data are processed by at least one processing device (PROC1, SYNT1, DIAG1), the at least one processing device (PROC1, SYNT1, DIAG1) being configured to generate processing results according to a criterion for optimizing the quality of the processing results of the input data, the weight values ​​being determined according to the criterion and a criterion for optimizing the amount of input data to be processed, - determining segment information of the plurality of input data, wherein one of the segment information items of one of the data items is assigned a first value or a second value different from the first value according to the weight, and - obtaining a data subset to be processed by applying the determined segment information to the plurality of input data, the data subset to be processed comprising data in the plurality of input data associated with the segment information item equal to the first value.

11. A device (200) for encoding a plurality of data acquired by a sensor, referred to as input data, characterized in that The device (200) is configured to implement: - obtaining the segmentation information and the data subset to be processed by applying segmentation information (SGI) of the plurality of input data to the plurality of input data, the segmentation comprising determining a weight value to be applied to the plurality of input data before the plurality of input data are processed by at least one processing device (PROC1, SYNT1, DIAG1), the at least one processing device (PROC1, SYNT1, DIAG1) previously configured to produce a processing result according to a criterion for optimizing a processing quality, the weight value being determined according to the criterion and another criterion for optimizing an amount of input data to be processed, one of the segmentation information items of one of the data items being assigned a first value or a second value different from the first value according to the weight value, the data subset to be processed comprising data of the plurality of input data associated with the segmentation information item equal to the first value, and - Encoding the segment information and the data subset to be processed.

12. A device (300) for decoding encoded data, characterized in that The device (300) is configured to implement: - decoding encoded data, the encoded data comprising segmentation information (SGI) of a plurality of data referred to as input data acquired by a sensor, a data subset (USS) to be processed by a processing device, the processing device being configured to apply weights to a plurality of decoded input data and to generate a processing result according to a criterion for optimizing the quality of the processing, one of the segmentation information items of one of the input data items being assigned a first value or a second value different from the first value, the data subset to be processed having been obtained by applying the segmentation information to the plurality of input data, the data subset to be processed comprising data of the plurality of input data associated with one of the segmentation information items equal to the first value, the encoded data further comprising modified values ​​of the weights, the modified values ​​having been determined for processing the plurality of segmented input data according to a criterion for optimizing the quality of the processing and a criterion for optimizing the data volume of the data subset to be processed, - constructing a plurality of decoded segmented input data (SQ1DS(t), SQ2DS(t), ..., SQNDS(t)) from the decoded data subset to be processed and the decoded segment information; - providing the plurality of decoded segment input data and the modified values ​​of the weights to the processing device.

13. A system(S), characterized in that, The system comprises: - a device (100) for segmenting a plurality of data according to claim 10, configured to generate a plurality of segmented data; - a device for encoding (200) said plurality of segmented data according to claim 11; - a device for decoding (300) encoded data according to claim 12; and - A processing device (PROC2, SYNT2, DIAG2) configured to process a plurality of decoded segment data reconstructed by the decoding device (300).

14. A computer program product comprising program code instructions for implementing the method according to any one of claims 1 to 8 when the program code instructions are executed by a processor.

Images