Training of ML models for monitoring a person

Abstract

A method is provided for training a machine learning ML model for monitoring people based on a data feed capable of describing people. The method is performed by a training data provider (i). The method includes: obtaining (40) a data feed capable of describing people; selecting (42) anonymization levels from a plurality of anonymization levels; anonymizing (44) the data feed according to the selected anonymization level to obtain a processed data feed; and sending (47) the processed data feed as training data for training a central ML model in a central node. Different anonymization levels are available and changes to the levels can be requested by the central node.

Description

Technical Field

[0001] This disclosure relates to the field of implementing the training of machine learning (ML) models for monitoring people, and more particularly to providing data feeds for training, wherein the level of anonymization is dynamically selected. Background Technology

[0002] New technologies have created new opportunities. For example, advancements in digital cameras and communication technologies have made it possible to use video surveillance to monitor people at a relatively low cost. This can be particularly useful for the elderly or disabled, allowing them to enjoy a significantly improved quality of life by living in their own homes rather than in staffed care facilities. Video data can also be used for, for example, people counting.

[0003] Video surveillance can be useful, but it raises privacy concerns. Almost no one likes using video surveillance for continuous monitoring to see when someone needs help.

[0004] One way to reduce privacy concerns is to use machine learning (ML) models to determine the status of people being monitored, instead of manual monitoring. However, this requires training the ML model based on video data, which necessitates feeding the model video data. As part of the training process, this video data used for training sometimes needs to be processed manually, which raises privacy concerns for the people captured in the video data. Summary of the Invention

[0005] One objective is to provide an improved balance between privacy and training data requirements when training data is provided based on data feeds that can characterize people.

[0006] According to a first aspect, a method is provided for training a machine learning (ML) model for monitoring people based on a data feed capable of describing people. The method is performed by a training data provider. The method includes: obtaining a data feed capable of describing people; selecting an anonymization level from a plurality of anonymization levels; anonymizing the data feed according to the selected anonymization level to obtain a processed data feed; sending the processed data feed as training data for training a central ML model in a central node; and receiving an instruction from the central node to increase or decrease the anonymization level. The method is repeated, wherein the next iteration is selected based on the instruction to increase or decrease the anonymization level.

[0007] Anonymization levels, in ascending order of anonymization, can include: facial blurring, replacing the face with a computer-generated facial image, and blurring the entire body.

[0008] The method may further include: determining tags associated with the data feed; and including tags associated with the processed data feed.

[0009] Labels can indicate when a person is close to a fall.

[0010] Labels can be determined based on the inference results of a local ML model, which is set up in the same location as the training data provider.

[0011] According to a second aspect, a training data provider is provided for training a machine learning (ML) model used to monitor people based on a data feed capable of describing a person. The training data provider includes: a processor; and a memory storing instructions that, when executed by the processor, cause the training data provider to perform the following operations: obtain a data feed capable of describing a person; select an anonymization level from a plurality of anonymization levels; anonymize the data feed according to the selected anonymization level to obtain a processed data feed; send the processed data feed as training data for training a central ML model in a central node; and receive an instruction from the central node to increase or decrease the anonymization level; wherein, in the case of repeating the instructions, the next iteration of the selected instruction is based on the instruction to increase or decrease the anonymization level.

[0012] Anonymization levels, in ascending order of anonymization, can include: facial blurring, replacing the face with a computer-generated facial image, and blurring the entire body.

[0013] The training data provider may also include instructions that, when executed by the processor, cause the training data provider to perform the following operations: determine the labels associated with the data feed; and include the labels associated with the processed data feed.

[0014] Labels can indicate when a person is close to a fall.

[0015] The defined instructions may include instructions that, when executed by the processor, cause the training data provider to determine labels based on the inference results of a local ML model, which is located in the same place as the training data provider.

[0016] According to a third aspect, a computer program is provided for training a machine learning (ML) model for monitoring people based on a data feed capable of describing people. The computer program includes computer program code that, when executed on a training data provider, causes the training data provider to perform the following operations: obtain a data feed capable of describing people; select an anonymization level from a plurality of anonymization levels; anonymize the data feed according to the selected anonymization level to obtain a processed data feed; send the processed data feed as training data for training a central ML model in a central node; receive instructions from the central node to increase or decrease the anonymization level; and repeat the computer program code, wherein the next iteration of the selected computer program code is based on the instructions to increase or decrease the anonymization level.

[0017] According to a fourth aspect, a computer program product is provided, the computer program product comprising a computer program according to a third aspect and a computer-readable device for storing the computer program.

[0018] Generally, all terms used in the claims should be interpreted according to their ordinary meaning in the technical field, unless otherwise expressly defined herein. All references to “a / an / the element, instrument, component, device, step, etc.” should be interpreted as referring to at least one instance of the element, instrument, component, device, step, etc., unless otherwise expressly stated. Unless expressly stated otherwise, the steps of any method disclosed herein need not be performed in the exact order disclosed. Attached Figure Description

[0019] Various aspects and implementation methods will now be described by way of example with reference to the accompanying drawings, in which:

[0020] Figure 1 This is a schematic diagram illustrating an environment in which the implementation methods presented herein can be applied;

[0021] Figure 2 This is a flowchart illustrating an implementation of a method for training an ML model used to monitor people based on data feeds capable of depicting people;

[0022] Figure 3 It is shown Figure 1 A schematic diagram of the components of the training data provider; and

[0023] Figure 4 An example of a computer program product that includes a computer-readable device is shown. Detailed Implementation

[0024] Various aspects of this disclosure will now be described more fully below with reference to the accompanying drawings, in which certain embodiments of the invention are illustrated. However, these aspects may be embodied in many different forms and should not be construed as limiting; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey to those skilled in the art the scope of all aspects of the invention. Throughout the description, the same reference numerals refer to the same elements.

[0025] The implementation presented in this paper provides an improved method for anonymizing data feeds used as training data for ML models. Specifically, one of several anonymization levels is selected. In this way, the amount of anonymization can be tailored for a specific purpose, so that anonymization is not excessive and hinders training, while anonymization actively improves the privacy of the people depicted in the data feed as much as possible.

[0026] Figure 1 This is a schematic diagram illustrating an environment in which the embodiments presented herein can be applied. The person 5 to be monitored is present in physical space 14 at least part of the time. Physical space 14 can be, for example, a room, apartment, residence, office, etc. Monitoring device 2 is configured to monitor person 5 based on sensor device 3, which is part of monitoring device 2 or locally connected to monitoring device 2. Monitoring device 2 is also used to capture data of person 5 for the purpose of training local ML model 4 and central ML model 9. Sensor device 3 provides data feeds capable of depicting person 5, such as as image sequences (i.e., video sequences). Sensor device 3 can be implemented as an infrared (IR) camera, video camera, lidar, radar, or any other suitable imaging technology. Additional sensor devices (not shown) may also be provided to provide corresponding data feeds. Training data provider 1 is used to provide training data for training central ML model 9, as described in more detail below. Training data provider 1 can be provided as part of monitoring device 2 or separately. In any case, training data provider 1 is located in the same place as sensor device 3. In this way, sensitive data feeds from sensor device 3 are anonymized by training data provider 1 to prevent privacy-sensitive parts of the data feeds from having to be transmitted remotely.

[0027] Monitoring device 2 includes a local ML model 4. One or more monitoring devices 2 may exist and operate in parallel in the same or complementary scenarios. Monitoring device 2 can be connected to network 6, which can be an Internet Protocol (IP) based network. Network 6 can include, for example, any one or more of a local wireless network, cellular network, wired LAN, WAN (e.g., the Internet), etc. Optionally, a central node 7 containing a central ML model 9 is also connected to network 6.

[0028] The local ML model 4 of monitoring device 2 is used to predict the current or future monitoring status or event based on data feeds from sensor device 3. Specifically, the local ML model 4 is used to infer the outcome of the monitoring of the status or event of person 5, which can be used as a label in the training data. Non-limiting examples of monitoring all human-related statuses or events are: absence, presence, lying in bed, lying on the floor, breathing, near fall event, fall event, pain, etc.

[0029] Figure 2 This is a flowchart illustrating an implementation of a method for training an ML model (e.g., central ML model 9) to monitor people based on data feeds capable of depicting people.

[0030] In step 40, the training data provider 1 obtains a data feed capable of depicting a person. As explained above, the data feed may be based, for example, on data from one or more sensors (e.g., an infrared (IR) camera, video camera, lidar, radar, or any other suitable imaging technology). At this stage, the data feed has not yet been anonymized, and, for example, a face may be visible in the data feed.

[0031] In step 42, the training data provider 1 selects an anonymization level from a plurality of anonymization levels. Anonymization levels may include, for example, in increasing order of anonymization: facial blurring, replacing the face with a computer-generated facial image, and blurring the entire body.

[0032] In the anonymization step 44, the training data provider 1 anonymizes the data feed according to the selected anonymization level, thereby obtaining the processed data feed.

[0033] Alternatively or additionally, face selection can be performed to achieve similar features in terms of facial expressions, which can be a valuable indicator in the training data. Alternatively or additionally, face selection can be based on selecting another face that has similar features to the person in the data feed in terms of hair color, hair length, skin color, etc.

[0034] Optionally, the raw data feed (without anonymization) is stored to allow training data with reduced anonymization to be sent at a later stage if needed.

[0035] In the optional label determination step 45, the training data provider 1 determines labels associated with the data feed. For example, the labels could indicate relatively rare events, such as a person nearly falling. This type of event can be anonymized in the training data and is still valuable because training can be based, for example, on the movement characteristics of a person's body and can be determined with little reliance on facial expressions. Because such events are rare, it makes it possible to provide a large amount of data, and any approach provided, such as that presented in the embodiments herein, is highly valuable.

[0036] Labels can be determined, for example, based on the inference results of a local ML model, where the local ML model is set up in the same location as the training data provider 1.

[0037] In step 46, which optionally includes labels, the training data provider 1 includes labels associated with the processed data feed (from step 45).

[0038] In step 47, when sending processed data, training data provider 1 sends processed (i.e., anonymized and optionally labeled) data feeds to be used as training data for training the central ML model in the training central node.

[0039] In the optional step 48 of receiving adjustment instructions, the training data provider 1 receives instructions from the central node to increase or decrease the anonymization level.

[0040] Then, the method is repeated, and when step 48 is executed, in the next iteration of step 42, an indication based on increasing or decreasing the anonymization level is selected, thus implementing the feedback loop. In this way, the optimization level is dynamically adjusted according to the needs of the central node.

[0041] Using the implementation methods presented herein, the level of anonymization can be adjusted to achieve a balance between the level of detail required for training and the impact on the privacy of the people depicted in the data feed. In other words, the amount of privacy-sensitive data forming part of the training data is reduced compared to the scenario where training data should be usable for training all types of ML models. On the other hand, the level of detail provided in the training data is increased where successful training requires.

[0042] For example, headcount counting, detecting absence / presence of people, or detecting near-fall events do not require large amounts of privacy-sensitive data, such as facial data.

[0043] Figure 3 It is shown Figure 1A schematic diagram of the components of the training data provider 1 is provided. It should be noted that when the training data provider 1 is implemented in a host device such as the monitoring device 2, one or more of the aforementioned components may be shared with the host device. The processor 60 is provided using any combination of one or more of a suitable central processing unit (CPU), multiprocessor, microcontroller, digital signal processor (DSP), etc., capable of executing software instructions 67 stored in memory 64, which can therefore be a computer program product. Alternatively, the processor 60 may be implemented using an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc. The processor 60 may be configured to execute the above-described components. Figure 2 The method described.

[0044] The memory 64 may be any combination of random access memory (RAM) and / or read-only memory (ROM). The memory 64 also includes a permanent storage device, which may be any single memory or combination of magnetic storage, optical storage, solid-state storage, or even remotely mounted memory.

[0045] A data memory 66 is also provided for reading and / or storing data during the execution of software instructions in the processor 60. The data memory 66 may be any combination of RAM and / or ROM.

[0046] The training data provider 1 also includes an I / O interface 62 for communicating with external and / or internal entities. For example, the I / O interface 62 allows the training data provider 1 to communicate with the network 6. Optionally, the I / O interface 62 also includes a user interface.

[0047] Other components of the training data provider 1 have been omitted to avoid obscuring the concepts presented in this paper.

[0048] Figure 4 An example of a computer program product 90 including a computer-readable device is shown. A computer program 91 can be stored on this computer-readable device, which can cause a processor to perform methods according to the embodiments described herein. In this example, the computer program product is in the form of removable solid-state memory, such as a Universal Serial Bus (USB) drive. As explained above, the computer program product can also be embodied in the memory of a device, such as… Figure 3 Computer program product 64. Although computer program 91 is schematically shown herein as part of removable solid-state storage, computer program may be stored in any manner suitable for computer program product (e.g., another type of removable solid-state storage, or optical disc, such as CD (compressed disc), DVD (Digital Universal Disc), or Blu-ray disc).

[0049] A set of implementation methods, listed in Roman numerals, will now be presented.

[0050] i. A method for training a machine learning ML model for monitoring a person based on a data feed capable of describing a person, the method being performed by a training data provider, the method comprising:

[0051] Obtain data feeds capable of describing the person;

[0052] Choose an anonymization level from multiple anonymization levels;

[0053] The data feed is anonymized according to the selected anonymization level to obtain a processed data feed; and

[0054] The processed data is sent as training data for training the central ML model in the central node.

[0055] ii. The method according to embodiment i further includes:

[0056] Receive instructions from the central node to increase or decrease the anonymization level;

[0057] Furthermore, the method is repeated, wherein the selection of the next iteration is based on the indication to increase or decrease the level of anonymization.

[0058] iii. The method according to any one of the foregoing embodiments, wherein the anonymization level includes, in order of increasing anonymization: facial blurring, replacing the face with a computer-generated facial image, and blurring the entire body.

[0059] iv.

[0060] v. The method according to any one of the foregoing embodiments further includes:

[0061] Determine the tag associated with the data feed; and

[0062] This includes the tags associated with the data feed being processed.

[0063] vi. The method according to embodiment v, wherein the label indicates the person's approach to a fall event.

[0064] vii. The method according to embodiment v or vi, wherein the determination of the label is based on the inference result of a local ML model, the local ML model being located in the same place as the training data provider.

[0065] viii. A training data provider for training a machine learning ML model used to monitor a person based on a data feed capable of describing a person, the training data provider comprising:

[0066] Processor; and

[0067] A memory for storing instructions that, when executed by the processor, cause the training data provider to perform the following operations:

[0068] Obtain data feeds capable of describing the person;

[0069] Choose an anonymization level from multiple anonymization levels;

[0070] The data feed is anonymized according to the selected anonymization level to obtain a processed data feed; and

[0071] The processed data is sent as training data for training the central ML model in the central node.

[0072] ix. The training data provider according to embodiment viii further includes instructions that, when executed by the processor, cause the training data provider to perform the following operations:

[0073] Receive instructions from the central node to increase or decrease the anonymization level;

[0074] And the instructions are repeated, wherein the next iteration of the selected instructions is based on the indication to increase or decrease the level of anonymization.

[0075] x. The training data provider according to embodiment viii or ix, wherein the anonymization levels, in the order of increasing anonymization, include: facial blurring, replacing the face with a computer-generated facial image, and blurring the entire body.

[0076] xi.

[0077] xii. The training data provider according to any one of embodiments viii to xi further includes instructions that, when executed by the processor, cause the training data provider to perform the following operations:

[0078] Determine the tag associated with the data feed; and

[0079] This includes the tags associated with the data feed being processed.

[0080] xiii. The training data provider according to embodiment xii, wherein the label indicates an approach fall event of the person.

[0081] xiv. The training data provider according to embodiment xii or xiii, wherein the determined instructions include instructions, when executed by the processor, to cause the training data provider to determine the label based on the inference results of a local ML model, the local ML model being located in the same place as the training data provider.

[0082] xv. A computer program for training a machine learning (ML) model for monitoring a person based on a data feed capable of describing a person, the computer program comprising computer program code that, when executed on a training data provider, causes the training data provider to perform the following operations:

[0083] Obtain data feeds capable of describing the person;

[0084] Choose an anonymization level from multiple anonymization levels;

[0085] The data feed is anonymized according to the selected anonymization level to obtain a processed data feed; and

[0086] The processed data is sent as training data for training the central ML model in the central node.

[0087] xvi. A computer program product comprising a computer program according to embodiment xv and a computer-readable device storing the computer program.

[0088] The foregoing has primarily described various aspects of this disclosure with reference to several embodiments. However, as will be readily understood by those skilled in the art, other embodiments besides those disclosed above are equally possible within the scope of the invention as defined by the appended claims. Therefore, although various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for illustrative purposes and are not intended to be limiting, wherein the true scope and spirit are indicated by the appended claims.

Claims

1. A method for enabling training of a machine learning, ML, model for monitoring a person based on a data feed depicting the person, the method being performed by a training data provider (1), the method comprising: obtaining (40) a data feed depicting the person; selecting (42) a level of anonymization from a plurality of levels of anonymization; anonymizing (44) the data feed according to the selected level of anonymization, resulting in a processed data feed; sending (47) the processed data feed as training data for training a central ML model in a central node; and receiving (48) an indication from the central node to increase or decrease the level of anonymization; and wherein the method is repeated, wherein a next iteration of the selecting (42) is based on the indication to increase or decrease the level of anonymization.

2. The method of claim 1, wherein, the levels of anonymization comprise, in order of increasing anonymization: face blur, replacing faces with computer-generated face images, whole-body blur.

3. The method of one of claims 1-2, further comprising: determining (45) a label associated with the data feed; and including (46) the label associated with the processed data feed. the label indicates a near-fall event of the person.

4. The method of claim 3, wherein, the determining (45) of the label is based on an inference result of a local ML model, the local ML model being disposed at the same site as the training data provider (1).

5. The method of claim 3, wherein, 6. A training data provider (1) for enabling training of a machine learning, ML, model for monitoring a person based on a data feed depicting the person, the training data provider (1) comprising: a processor (60); and a memory (64) storing instructions (67) that, when executed by the processor, cause the training data provider (1) to: obtain a data feed depicting the person; select a level of anonymization from a plurality of levels of anonymization; anonymize the data feed according to the selected level of anonymization, resulting in a processed data feed; send the processed data feed as training data for training a central ML model in a central node; and receive an indication from the central node to increase or decrease the level of anonymization; and repeat the instructions, wherein a next iteration of the instructions of the selecting is based on the indication to increase or decrease the level of anonymization. the levels of anonymization comprise, in order of increasing anonymization: face blur, replacing faces with computer-generated face images, whole-body blur.

8. The training data provider (1) of one of claims 6-7, further comprising instructions (67) that, when executed by the processor, cause the training data provider (1) to:

7. The training data provider (1) of claim 6, wherein determine a label associated with the data feed; and include the label associated with the processed data feed. the label indicates a near-fall event of the person. ​ 9. The training data provider (1) of claim 8, wherein ​ 10. The training data provider (1) of claim 8, wherein The instructions for determining comprise instructions (67) that, when executed by the processor, cause the training data provider (1) to determine the label based on an inference result of a local ML model, the local ML model being located at the same place as the training data provider (1).

11. A computer program product (64, 90) comprising a computer program (67, 91) for enabling training of a machine learning, ML, model for monitoring a person based on a data feed capable of portraying the person and a computer readable means on which the computer program is stored, the computer program comprising computer program code which, when executed on a training data provider (1), causes the training data provider (1) to: obtain a data feed capable of portraying the person; select an anonymization level from a plurality of anonymization levels; anonymize the data feed according to the selected anonymization level, resulting in a processed data feed; send the processed data feed as training data for training a central ML model in a central node; and receive an indication from the central node to increase or decrease the anonymization level; and repeat the computer program code, wherein a next iteration of the selected computer program code is based on the indication to increase or decrease the anonymization level.

Images