Information processing apparatus, method for controlling information processing apparatus, and storage medium

Abstract

To estimate to which subject each area in an image belongs in a more suitable mode even in a situation where objects in a shielding relationship exist in the image.SOLUTION: The attribute estimation unit 113 estimates to which of a plurality of subjects a first region corresponding to at least a part of a subject in an image in which the plurality of subjects are captured belongs. The machine learning model applied to the attribute estimation unit 113 is trained using a loss function that calculates a loss value based on at least a second region in which two or more subjects among the plurality of subjects overlap.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a control method for an information processing device, and a program. [Background technology]

[0002] A technology has been proposed in which a computer or other machine uses images as training data to train a model based on so-called machine learning, and the model is then made to perform various recognition tasks. Examples of recognition tasks include a detection task for detecting body parts (head, face, upper body, whole body, etc.) of a person from an image, and a tracking task for searching for and tracking a specific subject from an image. If such a technology can be applied to identify an object region in an image, it can be expected to improve operability, for example, by enabling an imaging device to focus on the region or adjust the exposure of the region to a more optimal state. Furthermore, if the object region can be identified through a recognition task, it can be used for a variety of applications, not just imaging devices.

[0003] Neural networks (NNs) are known as models for performing the above-mentioned recognition tasks. Among NNs, there are deep NNs (DNNs) that are adapted to deeper layers. In particular, deep convolutional neural networks (DCNNs), which are constructed by repeating multiple convolutional layers and pooling layers, have attracted attention for their high performance (detection accuracy and detection performance). Furthermore, in recent years, a technology called VIT (Vision Transformer), which incorporates an attention mechanism into image recognition, has also been attracting attention.

[0004] When using the autofocus (AF) function of an imaging device, situations can be imagined in which the focus needs to be adjusted to a local region of a subject, such as a human head or upper body, an animal's eyes, or the nose of an airplane. Therefore, in a recognition task applied to AF, a local region is detected for each of a series of potential candidate objects captured in an image. When multiple objects are captured in an image, a determination is made as to which object the detected local region belongs. To enable such a determination, a method has been proposed that uses a DCNN to estimate which object a local region belongs to using a score map. Patent Document 1 discloses a technology that outputs a map showing the positions of a person's joints, as well as information indicating which person in the image each detected joint belongs to. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 7422456 [Patent Document 2] Japanese Patent Application Publication No. 2022-19339 Summary of the Invention [Problem to be solved by the invention]

[0006] On the other hand, with the technology disclosed in Patent Document 1, when objects are in an occluding relationship with each other, it may be difficult to separate into person-specific scores because the occluded object appears less frequently in the image than the object that occludes it.To address this issue, Patent Document 2 discloses a technology for detecting objects that takes into account the occlusion relationship when a main subject is in an occluding relationship with another subject that has similar appearance features or posture. However, the technology disclosed in Patent Document 2 uses DCNN to perform learning that optimizes the score for a main subject that is in an occluding relationship, and while it is possible to identify local regions that belong to the main subject, it is not configured to optimize scores for other objects other than the main subject. Therefore, it is difficult to say that the technology disclosed in Patent Document 2 is suitable for associating local regions that belong to objects that are in an occluding relationship with other objects and have small presence areas.

[0007] In consideration of the above problems, the present invention aims to make it possible to more suitably estimate which subject each area in an image belongs to, even in situations where an object that is in an occluding relationship is present in the image. [Means for solving the problem]

[0008] The information processing device according to the present invention includes an estimation means for estimating to which of a plurality of subjects a first region corresponding to at least a part of a subject in an image of the plurality of subjects belongs, and the estimation means is characterized in that it performs learning using a loss function that calculates a loss value based at least on a second region in which two or more of the plurality of subjects overlap. [Effects of the Invention]

[0009] According to the present invention, even in a situation where an object in an occluding relationship exists in the image, it is possible to more suitably estimate to which subject each region in the image belongs. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an imaging system. [Figure 2] 1 is a flowchart illustrating an example of a process related to learning of a machine learning model. [Figure 3] FIG. 10 is a diagram illustrating an example of a method for generating an attribute correct answer map. [Figure 4] FIG. 10 is a diagram illustrating an example of an attribute score map. [Figure 5] FIG. 10 is a diagram illustrating an example of a method for generating an attribute correct answer map. [Figure 6] 1 is a flowchart illustrating an example of a process related to learning of a machine learning model. [Figure 7] 10A and 10B are diagrams illustrating an example of a method for generating attribute correct maps of foreground and background objects. [Figure 8] 1 is a flowchart illustrating an example of a process related to learning of a machine learning model. [Figure 9] FIG. 10 is a diagram showing an example of a method for generating a correct attribute map of a background region. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claimed invention. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted. Furthermore, to make the description of each embodiment easier to understand, the present disclosure will focus on an imaging system that uses an imaging device such as a digital camera to detect a person's head and face (hereinafter also referred to as a local region), upper body, and entire body (hereinafter also referred to as a whole body region) as a subject. However, the subject is not necessarily limited to a person. For example, a vehicle may be used as a subject, with the nose or body thereof serving as a local region or a whole body region, or an animal may be used as a subject, with the head or tail thereof serving as a local region or a whole body region, and the technology according to the present disclosure may be applied. Furthermore, in this disclosure, when there is no particular distinction between still images and moving images, they may be simply referred to as images. Therefore, when the term "image" is used, it can be applied to both still images and moving images unless otherwise restricted.

[0012] First Embodiment As a first embodiment of the present disclosure, an example of a method for learning a machine learning model that detects regions of body parts such as the head and upper body of a person in an input image by learning based on so-called machine learning will be described. Specifically, in a situation where there are overlapping regions between multiple people in an image, a method for learning will be described in which a loss value appropriate for an occlusion state is calculated so as to reduce the variance of scores associated within regions of the same person, including local regions and whole-body regions.

[0013] An example of the hardware configuration of an information processing device 110 according to this embodiment will be described with reference to FIG. A CPU (Central Processing Unit) 101 controls the overall operation of the information processing device 110 by expanding and executing a control program stored in a ROM (Read Only Memory) 103. A RAM (Random Access Memory) 102 is used as a temporary storage area such as a main memory or work area for the CPU 101. The RAM 102 is also used as an area into which the control program is expanded so that the CPU 101 can execute it. The input unit 105 corresponds to an input interface that accepts input from the user and is realized by, for example, an input device such as a keyboard or touch panel, and is also capable of accepting image input, etc. The display unit 106 corresponds to an output interface that presents various information to the user and is realized by, for example, an output device such as a liquid crystal display, and is capable of displaying various data and the results of various processes. Furthermore, the information processing device 110 can communicate with other devices by connecting to a predetermined network via the communication unit 104. For example, the information processing device 110 may accept image input from other devices, acquire machine learning models, receive instructions from a user, and the like, and may output various processing results to other devices via the communication unit 104. The storage unit 107 is a storage area for storing various data, and may be used, for example, as a storage area for storing a machine learning model according to this embodiment, the details of which will be described later. The storage unit 107 may be realized by a storage device such as an HDD (Hard Disk Drive), a flash memory, or various optical media.

[0014] Referring to FIG. 1(b), an example of the functional configuration of the information processing device 110 according to this embodiment will be described, focusing in particular on the configuration of the part that identifies an area belonging to a specific object (e.g., a person) in a target image using a trained machine learning model. The image acquisition unit 111 receives input of image data of moving images or still images in response to an instruction from a user. The position estimation unit 112 detects local regions and whole-body regions from the target image using a machine learning model (hereinafter also referred to as an area detector) that has been trained to detect areas belonging to a predetermined object in the image, and estimates the center position and size of the object to be detected. The area detector used may be a machine learning model that has been trained individually using a prepared dataset, or a machine learning model based on a known object detection technology (e.g., YoLo, ViT, etc.). The attribute estimation unit 113 estimates a score map for identifying to which person each of the local regions and whole body regions detected by the position estimation unit 112 using the region detector belongs. The association processing unit 114 identifies local regions and whole-body regions belonging to each person in the target image based on the score map acquired by the attribute estimation unit 113. Note that a known process (for example, the method disclosed in Patent Document 1) may be applied to the process related to identifying each region in the image. In addition, an area belonging to a specified object in an image (in other words, at least a part of an area of ​​a specified object in an image), such as a local area or a whole-body area, detected by the position estimation unit 112, corresponds to an example of a first area.

[0015] 1(c), an example of the functional configuration of a learning device 121 related to learning of a machine learning model used in the above-mentioned information processing device 110 will be described. The learning device 121 executes an estimation process using learning data as input for a machine learning model applied to the information processing device 110, and uses the result to cause a learning unit 122 to learn the machine learning model as follows.

[0016] Hereinafter, with reference to Figures 2 to 4, as an example of processing related to learning of a machine learning model by the learning unit 122, a method will be described in which, when two or more people captured in an image overlap, a loss value is calculated according to the size of the area where the two or more people overlap.

[0017] First, an example of processing related to learning of a machine learning model applied to the attribute estimation unit 113 according to this embodiment will be described with reference to FIG.

[0018] In S200, the learning unit 122 receives, as a processing target, an image (hereinafter also referred to as an input image) acquired by the image acquisition unit 111. The acquired image may be both an image in which a person is captured as a subject, and an image in which a subject other than a person is captured, such as a landscape such as the sky or a mountain, an animal, or a vehicle. In S201, the learning unit 122 acquires information about the area of ​​a person in an image, which is used in the map creation process described in S202. Here, it is assumed that the information about the area of ​​a person in an image is a mask area in which the pixel value of the area where the person is present, including the whole-body area, is "1" and the pixel value of the other area is "0." It is also assumed that the local area and the whole-body area are defined as rectangles whose center coordinates, width, and height are held as parameters.

[0019] In S202, the learning unit 122 generates, from the information acquired in S201, a score map (hereinafter also referred to as a correct answer map) to be used as a correct answer of the estimation result by the machine learning model when the machine learning model is learned. Here, an example of a method for generating a correct answer map will be described with reference to Fig. 3. For example, Fig. 3(a) shows an example of an image 300 from which a correct answer map is generated. Specifically, image 300 shows an example of an image captured of a scene in which two people overlap (a scene in which one person partially obscures the other person). Here, an example of a method for generating a correct answer map 307 shown in Fig. 3(c) and an overlap region map 308 shown in Fig. 3(d) using image 300 shown in Fig. 3(a) as an input image will be described.

[0020] (Process 1) The learning unit 122 acquires the whole body regions 303 and 305 and the mask regions 302 and 304 shown in FIG. 3(a) as person region information. (Process 2) The learning unit 122 calculates the area where the acquired whole body areas 303 and 305 overlap each other, and defines this overlapping area as an overlapping area 306 shown in FIG. 3(b). (Process 3) The learning unit 122 generates a correct answer map 307, as shown in FIG. 3(c), so that a different map value (pixel value) is assigned to each of the acquired mask areas 302 and 304. In the example shown in FIG. 3(c), the correct answer map 307 assigns a map value to each area, with the background portion 301 having a value of "0," the mask area 302 of the foreground person having a value of "1," and the mask area 304 of the background person having a value of "2." Furthermore, the learning unit 122 generates an overlapping area map 308, as shown in FIG. 3(d), based on the overlapping area 306. In the example shown in FIG. 3(d), the overlapping area map 308 assigns a map value to each area, with the overlapping area 306 having a value of "1" and the other areas having a value of "0." An area where two or more subjects overlap, such as overlap area 306, is an example of a second area.

[0021] In S203, the learning unit 122 causes the machine learning model to be learned (the machine learning model applied to the attribute estimation unit 113) to estimate a score map (hereinafter also referred to as an attribute score map) for identifying to which person each area in the image 300 belongs. Fig. 4 shows an example of an attribute score map 401 estimated by the machine learning model applied to the attribute estimation unit 113. In the example shown in Fig. 4, the attribute score map 401 assigns a higher score to an area in the image 300 where a person exists than to other areas.

[0022] In S204, the learning unit 122 calculates a loss for the attribute score map 401 estimated in S203. Specifically, the learning unit 122 uses, for model training, a loss function for training so that the scores assigned to person regions in the attribute score map 401 are the same or similar values ​​within the same person region, and a loss function for training so that the scores are different values ​​between different person regions.

[0023] The loss function that calculates the loss value (PullLoss) for learning to have the same or similar values ​​within the same person area is expressed by the relational expression shown below as equation (1-1). In equation (1-1), N indicates the number of people in the image, and the score est(i) indicates the value of pixel position i in the estimated score map. j is a set of the jth person (1≦j≦N; here, the number of people in the image, N=2). The definitions of N and est(i) are common to all embodiments of the present disclosure, including this embodiment. The learning unit 122 weights PullLoss according to the proportion of each person region to the person region in the image. By performing such weighting, the smaller the person region, the larger the value added to PullLoss, and the larger the person region, the smaller the value added to PullLoss.

[0024]

number

[0025] The loss function for calculating the loss value (PushLoss) for learning to have different values ​​between different person regions is expressed by the relational expression shown below as Equation (1-2). In Equation (1-2), β is an empirically obtained hyperparameter. Moreover, OverlapAreaValue is the size of the overlap region 306 or the ratio of the overlap region 306 to the map size of the overlap region map 308. The learning unit 122 weights the loss calculated from the score value using the above parameter, i.e., the sum of β and OverlapAreaValue. By performing such weighting, the loss for an attribute score map having an overlap region is larger than the loss for an attribute score map having no overlap region.

[0026]

number

[0027] The loss function for calculating PushLoss may be expressed by the following relational expression (1-3). In expression (1-3), W o is a hyperparameter obtained empirically, similar to β. The difference between equation (1-3) and equation (1-2) is that OverlapAreaValue is added as PushLoss to the loss calculated from the ground truth map, but the purpose of weighting is essentially the same in both equations.

[0028]

number

[0029] The loss function applied in S204 is expressed as the relational expression shown below as Equation (1-4), which is the sum of Equation (1-1) and Equation (1-2) or Equation (1-3). In Equation (1-4), α is an empirically obtained hyperparameter.

[0030]

number

[0031] In S205, the learning unit 122 updates the connection weighting coefficients (parameters) between layers of the target machine learning model based on the loss expressed by Equation (1-4). The parameter update is performed based on back propagation using, for example, Momentum SGD or the like.

[0032] In S206, the learning unit 122 determines whether or not a predetermined condition related to the end of learning of the machine learning model is met. If the learning unit 122 determines in S206 that the predetermined condition is not satisfied, the process proceeds to S200. In this case, the learning unit 122 acquires the next input image in S200 and executes the processes from S201 onwards again for that input image. Furthermore, if the learning unit 122 determines in S206 that the predetermined condition is satisfied, it ends the series of processes shown in FIG. 2, that is, the processes related to learning the machine learning model. Note that the condition for terminating the learning of the machine learning model in S206 may be, for example, a condition specified in advance by the user. As a specific example, the condition for terminating the learning may be that the estimation accuracy of the machine learning model is equal to or greater than a threshold. As another example, the condition for terminating the learning may be that the number of times the learning process is repeatedly executed is equal to or greater than a threshold, or that the time required for the learning process is equal to or greater than a threshold. In addition, in this embodiment, for the sake of simplicity, images are used one by one for learning, but this does not limit the processing of the learning device 121. For example, it is also possible to use multiple images as a mini-batch for learning. In this case, for example, the learning unit 122 may update the learning parameters based on the sum of the losses of multiple images.

[0033] In the above, in this embodiment, a method has been described in which a loss function is used to calculate a loss value according to the area of ​​overlapping regions of people in training a model that estimates an attribute score map. By using this method, it is possible to calculate the loss for people whose presence area is small due to overlapping regions in the same way as for occluding people. As a result, when applying a machine learning model that estimates which local regions and whole-body regions belong to which people, it is expected that the estimation accuracy in intersection scenes where overlapping regions occur will be improved compared to when training is performed using conventional methods.

[0034] (Modification of the first embodiment) 2 has been described focusing on the case where the information acquired in the process of S201 is "a whole body region and a mask region." In contrast, as a modification of the first embodiment, an example of processing when the information acquired in the process of S201 is "a local region and a whole body region" will be described below focusing particularly on the process of S202.

[0035] (Process 1) The learning unit 122 acquires local regions 500 and 502 and whole-body regions 501 and 503 as person region information for an image 515 shown in FIG. 5(a). (Process 2) As shown in FIG. 5( b), learning unit 122 generates local region circle 514, whose radius is an arbitrary number of surrounding pixels, based on local region center position 513 within local region 500. Learning unit 122 also generates whole body region circle 504, whose radius is an arbitrary number of surrounding pixels, based on whole body region center position 505 within whole body region 501. Learning unit 122 then calculates common tangent 506 to local region circle 514 and whole body region circle 504, and defines the internal region surrounded by local region circle 514, whole body region circle 504, and common tangent 506 as part mask region 507. Learning unit 122 may also define line segment region 508 of an arbitrary thickness connecting local region center position 513 and whole body region center position 505, instead of part mask region 507. Note that in the following description, part mask region 507 will be used. (Process 3) Learning unit 122 performs substantially the same process as (Process 2) for local region 500 and whole body region 501 on local region 502 and whole body region 503 to define part mask region 509. (Process 4) Learning unit 122 generates correct answer map 510, as shown in Fig. 5(d), so that a different map value (pixel value) is assigned to each of acquired part mask areas 507 and 509. Learning unit 122 also generates overlapping area map 512, as shown in Fig. 5(d), from the area where part mask areas 507 and 509 overlap each other (overlapping area 511). The subsequent processes (the processes from S203 onward shown in FIG. 2) are substantially the same as those in the above-described embodiment.

[0036] <Second embodiment> In the first embodiment, if a different map value is used for each subject, it is possible to estimate which subject each region in an image belongs to without being limited to a specific value. Therefore, a correct answer value is not set, and the model is trained so that the variance of the scores for each subject region is reduced. On the other hand, learning without setting a correct answer value may be difficult to achieve convergence. In view of this situation, a second embodiment of the present disclosure will describe an example of a method for training a machine learning model so that the map values ​​in a person region converge within an arbitrary value range.

[0037] An example of processing related to learning of a machine learning model applied to the attribute estimation unit 113 according to this embodiment will be described with reference to Fig. 6. Note that the processing of S600 to S602, S604, S606, and S607 shown in Fig. 6 is substantially the same as the processing of S200 to S203, S205, and S206 shown in Fig. 2, and therefore detailed description thereof will be omitted. Hereinafter, the processing of S603 and S604 will be described.

[0038] In S603, the learning unit 122 generates a foreground object attribute score map corresponding to a region where an object that occludes other objects (hereinafter also referred to as an occluding object) or an object where no occlusion occurs (hereinafter also referred to as a foreground object region) exists. The learning unit 122 also generates a rear object region attribute score map corresponding to a region where an object that is occluded by other objects that do not fall within the foreground region (hereinafter also referred to as an occluded object) exists (hereinafter also referred to as a rear object region). Here, an example of a map generation method in the processing of S603 will be described, focusing on a case where a mask region exemplified in FIG. 7(a) is acquired as person region information in S601.

[0039] (Process 1) The learning unit 122 generates a foreground object attribute score map 704 as shown in Fig. 7(b) from mask regions 701 and 702 of a person, which is a foreground object, out of the person region information. In the example shown in Fig. 7(b), the foreground object attribute score map 704 assigns a map value to each region, with the value in the mask regions 701 and 702 being "1" and the value in the other regions being "0." (Process 2) The learning unit 122 generates a rear object exclusion region 707 from the intersection of the mask regions 702 and 703. Thereafter, the learning unit 122 generates a rear object attribute score map 705 as shown in FIG. 7(c) using the mask region 703 and the rear object exclusion region 707. In the example shown in FIG. 7(c), the rear object attribute score map 705 assigns a map value to each region, with a value of "1" in a mask region 706 that does not include the rear object exclusion region 707 of the mask region 703, and a value of "0" in the other regions.

[0040] In S604, the learning unit 122 calculates the losses for the foreground object attribute score map and the rear object attribute score map, in addition to calculating the PushLoss and the PullLoss.

[0041] The loss function for calculating the loss value (FrontLoss) for the foreground object attribute score map is expressed by the relational expression shown below as equation (1-5). In equation (1-5), Score_th indicates the threshold value of the map value to be lost, and is determined in advance. In FrontLoss, the average value of the absolute difference between the threshold and the map value in the area of ​​the foreground object attribute score map where the map value is less than the threshold is calculated as the loss. This makes it possible to converge the learning of the machine learning model so that the map value in the area where the foreground object exists becomes a value greater than the threshold.

[0042]

number

[0043] The loss function for calculating the loss value (BackLoss) for the rear object attribute score map is expressed by the relational expression shown below as equation (1-6). In BackLoss, the average value of the difference between the threshold and the map value in the area of ​​the rear object attribute score map where the map value is greater than the threshold is calculated as the loss. This makes it possible to converge the learning of the machine learning model so that the map value in the area where the rear object exists, excluding the overlapping area, becomes a value less than the threshold.

[0044]

number

[0045] The loss function applied in S604 is expressed as the sum of PushLoss, PullLoss, Equation (1-5), and Equation (1-6) by the relational expression shown below as Equation (1-7). In Equation (1-7), α, γ, and σ are empirically obtained hyperparameters.

[0046]

number

[0047] In the present embodiment, an example of a method for adding a loss function using thresholds for map values ​​for foreground objects and rear objects in training a model that estimates an attribute score map has been described. As described above, in this method, when calculating the loss value applied in training a machine learning model, the loss (FrontLoss) for occluding objects or objects that are not occluded and the loss (BackLoss) for occluded objects are taken into account. By applying such a method, it becomes possible to perform training such that the map values ​​converge to a specific value range in training a machine learning model, which is expected to have the effect of accelerating the convergence of training.

[0048] <Third embodiment> In the third embodiment of the present disclosure, an example of a method of applying a loss function related to an attribute score map of an area where no person exists (hereinafter, background area) in addition to the loss functions described in the first and second embodiments will be described.

[0049] An example of processing related to learning of a machine learning model applied to the attribute estimation unit 113 according to this embodiment will be described with reference to Fig. 8. Note that the processing of S800 to S803, S805, S807, and S808 shown in Fig. 8 is substantially similar to the processing of S600 to S604, S606, and S607 shown in Fig. 6, and therefore detailed description thereof will be omitted. Hereinafter, the processing of S804 and S806 will be described.

[0050] In S804, the learning unit 122 generates an attribute score map of the background region. Here, an example of a map generation method in the processing of S804 will be described, focusing on the case where the mask region illustrated in Fig. 9(a) is acquired as person region information in S801.

[0051] The learning unit 122 generates a background region attribute score map 905 as shown in Fig. 9(b) from a background region 904, which is part of the person region information and corresponds to a region in the image 900 other than the person mask regions 901 to 903. In the example shown in Fig. 9(b), the background region attribute score map 905 assigns a map value to each region, with the value in the background region 904 being "1" and the value in the mask regions 901 to 903 being "0".

[0052] In S806, the learning unit 122 calculates a loss (BackGroundLoss) related to the background region attribute map in addition to PushLoss, PullLoss, FrontLoss, and BackLoss.

[0053] The loss function for calculating BackgroundLoss is expressed by the relational expression shown below as Equation (1-8). In Equation (1-8), value_th is a threshold value corresponding to the value at which the map value in the background region converges, and is determined in advance. In BackgroundLoss, the average value of the absolute difference between value_th and the map value is calculated as the loss so that the map value in the background region converges to value_th. This makes it possible to converge the map value in the background region to a fixed value. Therefore, for example, by training a machine learning model so that the map value in the person region becomes a value other than the map value in the background region, it becomes possible to separate the attributes of the person region and the background region. Note that value_th may be the same value as Score_th in the second embodiment.

[0054]

number

[0055] The loss function applied in S806 is expressed as the sum of PushLoss, PullLoss, FrontLss, BackLoss, and equation (1-8), as the relational expression shown below as equation (1-9). In equation (1-9), α, γ, σ, and ε are empirically obtained hyperparameters.

[0056]

number

[0057] In the above, in this embodiment, an example of a method for adding a loss function that causes map values ​​in background regions to converge to a unique value in training a model that estimates an attribute score map has been described. As described above, in this method, when calculating the loss value applied in training a machine learning model, the convergence of scores in regions where no subject is present to a predetermined score is taken into account. By applying such a method, it is possible to encourage training so that map values ​​in background regions converge to a specific value in training a machine learning model, which is expected to have the effect of promoting separation of map values ​​between person regions and other regions.

[0058] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0059] The disclosure of this embodiment also includes the following configurations, methods, and programs. (Configuration 1) An estimation means for estimating to which of a plurality of subjects a first region corresponding to at least a part of a subject in an image in which the plurality of subjects are captured belongs, The information processing device, wherein the estimation means performs learning using a loss function that calculates a loss value based at least on a second region where two or more of the plurality of subjects overlap. (Configuration 2) The information processing device according to configuration 1, wherein the subject is a person, and the first region is a region that includes at least a part of a plurality of parts of the person. (Configuration 3) The information processing device according to configuration 1 or 2, wherein the estimation means is trained to output different scores for each of the plurality of subjects. (Configuration 4) The information processing device described in Configuration 3, characterized in that the estimation means is trained to output different scores between two or more of the first regions belonging to the two or more overlapping subjects. (Configuration 5) An information processing device characterized by having: an estimation means for estimating to which of a plurality of subjects a first region corresponding to at least a part of a subject in an image of the plurality of subjects belongs; and a learning means for training the estimation means using a loss function that calculates a loss value based at least on a second region in which two or more of the plurality of subjects overlap. (Configuration 6) The information processing device according to configuration 5, wherein the loss function weights the loss value depending on the size of the second region. (Configuration 7) The information processing device described in configuration 5 or 6, characterized in that the loss function weights the loss value so as to reduce the difference between the scores output for two or more of the first regions belonging to a common subject. (Configuration 8) The information processing device described in any one of configurations 5 to 7, characterized in that the loss function calculates the loss value by taking into account the loss for an object that is not occluded by other objects and the loss for an object that is occluded by other objects. (Configuration 9) An information processing device according to any one of configurations 3 to 6, characterized in that the loss function calculates the loss value by taking into account that the score in an area where the subject does not exist converges to a predetermined score. (Method 1) A control method for an information processing device, comprising an estimation step of estimating to which of the multiple subjects a first region corresponding to at least a part of a subject in an image of the multiple subjects belongs, and wherein the machine learning model applied in the estimation step is trained using a loss function that calculates a loss value based at least on a second region where two or more of the multiple subjects overlap. (Method 2) A control method for an information processing device, comprising: an estimation step of estimating to which of a plurality of subjects a first region corresponding to at least a portion of a subject in an image of the plurality of subjects belongs; and a learning means for performing learning of a machine learning model applied in the estimation step using a loss function that calculates a loss value based at least on a second region in which two or more of the plurality of subjects overlap. (Program 1) A program for causing a computer to function as an information processing device, characterized in that it has an estimation means for estimating to which of the multiple subjects a first region corresponding to at least a part of a subject in an image of multiple subjects belongs, and the estimation means performs learning using a loss function that calculates a loss value based at least on a second region where two or more of the multiple subjects overlap. (Program 2) A program for causing a computer to function as an information processing device, characterized by having an estimation means for estimating to which of a plurality of subjects a first region corresponding to at least a part of a subject in an image of the plurality of subjects belongs, and a learning means for training the estimation means using a loss function that calculates a loss value based at least on a second region where two or more of the plurality of subjects overlap. [Explanation of symbols]

[0060] 110 Information processing equipment 113 Attribute estimation part 121 Learning Device 122 Learning Department

Claims

1. an estimation means for estimating to which of a plurality of subjects a first region corresponding to at least a part of a subject in an image in which the plurality of subjects are captured belongs; The estimation means performs learning using a loss function that calculates a loss value based at least on a second region where two or more of the plurality of subjects overlap. An information processing device characterized by:

2. the subject is a person, The first region is a region that includes at least a part of a plurality of parts of the person.

2. The information processing device according to claim 1, wherein:

3. The information processing apparatus according to claim 1 , wherein the estimation means is trained to output different scores for each of the plurality of subjects.

4. The information processing apparatus according to claim 3 , wherein the estimation means is trained to output different scores between two or more of the first regions belonging to the two or more overlapping subjects, respectively.

5. an estimation means for estimating to which of a plurality of subjects a first region corresponding to at least a part of a subject in an image captured of the plurality of subjects belongs; a learning means for learning the estimation means using a loss function that calculates a loss value based at least on a second region where two or more of the plurality of subjects overlap; An information processing device comprising:

6. The information processing device according to claim 5 , wherein the loss function weights the loss value in accordance with the size of the second region.

7. The information processing device according to claim 5 , wherein the loss function weights the loss value so as to reduce a difference between scores output for two or more of the first regions belonging to a common subject.

8. The information processing device according to claim 5 , wherein the loss function calculates the loss value by taking into account a loss for an object that is not occluded by another object and a loss for an object that is occluded by another object.

9. The information processing apparatus according to claim 3 , wherein the loss function calculates the loss value by taking into consideration that the score in the area where the subject does not exist converges to a preset score.

10. A control method for an information processing device, comprising: an estimation step of estimating to which of a plurality of subjects a first region corresponding to at least a part of a subject in an image in which the plurality of subjects are captured belongs; The machine learning model applied in the estimation step is trained using a loss function that calculates a loss value based at least on a second region where two or more of the plurality of subjects overlap.

10. A method for controlling an information processing device, comprising:

11. A control method for an information processing device, comprising: an estimation step of estimating to which of a plurality of subjects a first region corresponding to at least a part of a subject in an image captured of the plurality of subjects belongs; a learning means for learning a machine learning model to be applied in the estimation step using a loss function that calculates a loss value based at least on a second region in which two or more of the plurality of subjects overlap; 10. A method for controlling an information processing device, comprising:

12. Computer, an estimation means for estimating to which of a plurality of subjects a first region corresponding to at least a part of a subject in an image in which the plurality of subjects are captured belongs; The estimation means performs learning using a loss function that calculates a loss value based at least on a second region where two or more of the plurality of subjects overlap. A program for causing an information processing device to function as the information processing device.

13. Computer, an estimation means for estimating to which of a plurality of subjects a first region corresponding to at least a part of a subject in an image captured of the plurality of subjects belongs; a learning means for learning the estimation means using a loss function that calculates a loss value based at least on a second region where two or more of the plurality of subjects overlap; A program for causing an information processing device to function, comprising:

Images