Image processing apparatus, image processing method, and computer program

Abstract

To provide a technique for improving detection performance for a boundary region in an image with less processing sources.SOLUTION: Error calculation for obtaining an error between a calculation result of a trained model to which a learning image is inputted and a teacher image corresponding to the learning image is performed, and training of the trained model is performed based on the error. An error corresponding to a boundary area of an object in the teacher image is increased more than those in the other areas to perform the error calculation.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a learning technique for a learning model. [Background technology]

[0002] There are technologies that allow machines such as computers to learn and recognize the contents of data such as images and sounds. The purpose of the recognition process is called a recognition task, and the mathematical model for learning and executing the recognition task is called a recognition model.

[0003] Recognition tasks include, for example, object detection tasks for detecting specific objects (faces, eyes, heads, animals, vehicles, etc.) from an image. Recognition tasks also include region detection tasks, which are called semantic region segmentation and perform object detection on a pixel-by-pixel basis in an image. There are also various other recognition tasks, such as object category recognition tasks for determining the category (human, animal, vehicle, etc.) of an object (subject) in an image, a tracking task for searching for and tracking a specific subject, and a scene type recognition task for determining the type of scene (city, mountain, coast, etc.). Hereinafter, recognition tasks may be referred to as tasks.

[0004] Neural networks are known as a technology for learning and executing the above tasks. In particular, a deep (large number of layers) multi-layer neural network is also called a deep neural network (DNN). DNN stands for Deep Neural Network. In particular, a deep convolutional neural network is called a deep convolutional neural network (DCNN). DCNN stands for Deep Convolutional Neural Network. DCNN has been attracting attention in recent years due to its high performance (recognition accuracy, recognition performance). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6872502 [Non-patent literature]

[0006] [Non-Patent Document 1] Yin, H. (2021) “Improved semantic segmentation method using edge features for winter wheat spatial distribution extraction from Gaofen-2 images” Summary of the Invention [Problem to be solved by the invention]

[0007] For example, in a human skin region detection task, which is a task for detecting a human skin region in an image, the detection performance of the boundary region between skin and non-skin, such as the outline of the eyes or mouth, or the hairline, may not improve. This is because the proportion of the region corresponding to the boundary between skin and non-skin in the learning data is low compared to the region that looks like skin or is clearly not skin. When the inferred human skin region information is used to correct the color of human skin in an image, the insufficient detection performance of the boundary region as described above becomes a problem. If the eyes or hair are mistakenly detected as a skin region, the eyes and hair will also become target regions for the color correction calculation, and the actual skin region may be corrected to an incorrect color. This problem becomes more serious when the proportion of the boundary region in the image is greater than the non-boundary region, such as when the face area is small.

[0008] In addition, in an object detection task to detect the center point of a human face in an image, there are cases where not only the center of the human face but also the surrounding area is erroneously detected with a high score. When the center position of a human face is used for facial organ point detection, if the detection result spreads to an area larger than the correct answer definition area as described above, an error occurs in the estimation of the center position, which affects the subsequent facial organ point detection.

[0009] Therefore, in a recognition task, it is important to improve the detection performance in the boundary region between the correct definition region and the non-correct definition region. The methods disclosed in Patent Document 1 and Non-Patent Document 1 are methods for improving the detection performance of the boundary region in a region detection task. However, there is a problem that a dedicated neural network is required to extract edge information of the original image, and the amount of model parameters of the neural network increases. The present invention provides a technology for improving the detection performance of the boundary region in an image with fewer processing resources. [Means for solving the problem]

[0010] One aspect of the present invention comprises a calculation means for performing an error calculation to obtain an error between the calculation result of a learning model to which a learning image is input and a teacher image corresponding to the learning image, and a learning means for learning the learning model based on the error, wherein the calculation means performs the error calculation by making the error corresponding to the boundary region of an object in the teacher image larger than that of other regions. Effect of the Invention

[0011] According to the present invention, it is possible to improve the performance of detecting boundary regions in an image while using fewer processing resources. [Brief description of the drawings]

[0012] [Figure 1] FIG. 2 is a block diagram showing an example of the hardware configuration of a computer device applicable to the image processing device 200. [Diagram 2] FIG. 2 is a block diagram showing an example of the functional arrangement of an image processing device 200. [Diagram 3] 1A is a diagram showing an example of a teacher image, FIG. 1B is a diagram showing the result of boundary region extraction, and FIG. 1C is a diagram showing a boundary line and a region surrounding the boundary line. [Figure 4] 6 is a flowchart of a process performed by the image processing device 200 for learning a human skin region detection task. [Diagram 5]13 is a diagram showing the ratio of the area (number of pixels) of the boundary region to the non-boundary region when the face size in the image is large (top row) and when it is small (bottom row). [Figure 6] (a) is a diagram showing an example of a training image, (b) is a diagram showing an example of a teacher image (correct image), (c) is a diagram explaining a case where the correct definition region in the detection result is trained to appear wider than the correct definition region defined in the teacher image, and (d) is a diagram explaining a case where peak values ​​are trained to appear over a wide range. [Figure 7] 4 is a flowchart of a process performed by the image processing device 200 for learning an object detection task. [Figure 8] (a) shows the result of binarizing the teacher image, and (b) shows the boundary region. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0014] [First embodiment] The image processing device according to the present embodiment performs an error calculation to obtain an error between the calculation result of a learning model to which a learning image is input and a teacher image corresponding to the learning image, and performs learning of the learning model based on the error. At that time, the image processing device according to the present embodiment performs the error calculation by emphasizing the error corresponding to the boundary region in the teacher image.

[0015] An example of the functional configuration of the image processing device 200 according to this embodiment is shown in the block diagram of FIG. 2. In this embodiment, the process of learning performed by the image processing device 200 to execute a human skin region detection task will be described. The human skin region detection task is a task for detecting a human skin region (human skin region) from an input image. The human skin region detection task is a task for outputting "1" for pixels that belong to the human skin region in the input image, and outputting "0" for pixels that belong to regions other than the human skin region (non-human skin region) in the input image. The process of the image processing device 200 to learn to execute the human skin region detection task will be described with reference to the flowchart of FIG. 4.

[0016] In step S401, the detection unit 203 acquires a learning image including a human skin region and a teacher image (correct answer image) corresponding to the learning image from the storage unit 201. If a pixel position L(x,y) in the learning image belongs to a human skin region, the pixel value at a pixel position T(x,y) in the teacher image corresponding to the learning image is "1". If a pixel position L(x,y) in the learning image belongs to a non-human skin region, the pixel value at a pixel position T(x,y) in the teacher image corresponding to the learning image is "0". In other words, the teacher image is a map in which the label "1" is assigned to pixels corresponding to human skin regions and the label "0" is assigned to pixels corresponding to non-human skin regions.

[0017] The detection unit 203 inputs the learning image acquired from the storage unit 201 into a learning model, which is a "recognition model for recognizing an object from an input image," and performs arithmetic processing of the learning model to acquire an estimation result (detection result) of a human skin region in the learning image. Various models can be applied to the learning model, but in this embodiment, a case in which a neural network such as DCNN (Deep Covolutional Neural Network) is used as the learning model will be described. DCNN is a neural network that performs a task by gradually summarizing local features of an input image by repeating a convolution layer and a pooling layer, for example, and obtaining information that is robust against deformation and positional displacement.

[0018] In step S402, the extraction unit 202 extracts the boundary between the human skin region and the non-human skin region (i.e., the boundary between regions with different assigned labels) as the boundary region from the teacher image acquired from the storage unit 201. An example of the teacher image is shown in FIG. 3(a). In FIG. 3(a), the white region indicates the human skin region, and the black region indicates the non-human skin region. FIG. 3(b) shows the result of extracting the boundary region from the teacher image of FIG. 3(a). In FIG. 3(b), the white region indicates the boundary region, and the black region indicates the region other than the boundary region (non-boundary region). That is, the extraction unit 202 extracts the edge of the human skin region (white region) / non-human skin region (black region) in the teacher image as the boundary region. In order to extract the edge of the binarized teacher image as shown in FIG. 3(a), a known edge detection method such as the Canny method, the Laplacian method, the Roberts method, the Prewitt method, or the Sobel method may be used. In the following, a set of pixels belonging to the boundary region in the teacher image is referred to as B.

[0019] In step S403, the loss calculation unit 204 performs an error calculation to obtain the error between the teacher image acquired in step S401 and the detection result acquired in step S401 as a loss, and in this case, the error corresponding to the boundary region in the teacher image is emphasized (the error is made larger than other regions) to perform the error calculation. There are various loss functions used for the error calculation, such as the sum of squares error and the cross entropy error, and an appropriate one may be selected according to the characteristics of the task. For example, when the mean square error is used as the loss function, the loss calculation unit 204 performs the error calculation according to the following formulas (1) and (2).

[0020]

number

[0021] Here, E(y, y') is the error (loss) between the teacher image y and the detection result y', n is the total number of pixels in the teacher image / detection result, wi is the loss weight corresponding to the i-th pixel in the teacher image / detection result, yi is the pixel value of the i-th pixel in the teacher image, and yi' is the pixel value of the i-th pixel in the detection result.

[0022] Here, as shown in formula (2), the loss weight wi is preset to have a value of "1" when the i-th pixel pi in the teacher image does not belong to set B, and to have a value of "w" (>1) when the pixel pi belongs to set B. Note that the value of the loss weight wi described here is an example, and it is sufficient that the loss weight wi is set to have a value W1 (>0) when the pixel pi does not belong to set B, and to have a value W2 (>W1) when the pixel pi belongs to set B.

[0023] That is, the loss calculation unit 204 calculates the error between the detection result and the teacher image as the sum of weighted errors for each pixel between the detection result and the teacher image. At this time, the weight for the error between a pixel belonging to a boundary area in the teacher image and a corresponding pixel in the detection result is larger than the weight for the error between a pixel not belonging to a boundary area in the teacher image and a corresponding pixel in the detection result. Note that, although an example in which the mean square error is used as the loss function has been shown above, the same applies when a loss function other than the mean square error is used.

[0024] In step S404, the learning unit 205 performs a learning process of the learning model by updating the parameters of the learning model so as to reduce the error obtained in step S403. In this embodiment, since DCNN is used as the learning model, in this case, the learning unit 205 performs a learning process of the DCNN by updating the weights of the DCNN so as to reduce the error obtained in step S403. For example, the learning unit 205 may use an optimization method such as gradient descent (SGD) or Adam to update the weights of the DCNN so as to minimize the error according to a hyperparameter such as a learning rate determined in advance.

[0025] In this embodiment, in step S403, the loss function is designed so that the penalty for a boundary region prediction error is large, and the learning model is updated to more strongly correct the boundary region prediction error. This allows the learning model to be trained to improve the detection performance of boundary regions, i.e., the contours of the eyes and mouth, and the hairline.

[0026] In step S405, the learning unit 205 determines whether or not a learning end condition has been satisfied. The learning end condition can be various conditions and is not limited to a specific condition. Examples of the learning end condition that can be applied include "error is less than a threshold", "difference between the previous error and the current error is less than a threshold", "number of times learning (processing of steps S401 to S404) is repeated is equal to or greater than a threshold", "time elapsed since learning started is equal to or greater than a specified time", etc.

[0027] If it is determined that the learning end condition is met, the process according to the flowchart in FIG. 4 ends, and if the learning end condition is not met, the process proceeds to step S401.

[0028] The purpose of learning in this embodiment is to update the parameters of the learning model to suitable parameters so that the detection result of the detection unit 203 becomes as close as possible to the teacher image, but at this time, the detection performance of boundary areas such as the outlines of the eyes and mouth and the hairline may become poor. In this embodiment, the parameters of the learning model are updated to more strongly correct prediction errors in the boundary area, so that the detection performance in the boundary area between skin and non-skin can be improved.

[0029] In this embodiment, the case where the learning model is learned on an image-by-image basis has been described, but the present invention is not limited to this. For example, the learning images may be grouped into several mini-batches for learning. In addition, the parameters of the learning model learned as in this embodiment are used to accurately detect an object from an input image. The parameters of the learning model and a program for detecting an object using the parameters are stored in the memory of the device, and the CPU executes the detection process using the stored data.

[0030] <Modification of the first embodiment> In the first embodiment, a boundary region is extracted from a teacher image, and the boundary region extracted at this time is an edge between regions of different labels, that is, a region corresponding to a boundary line, as shown in Fig. 3(b). In this case, learning of the boundary portion is strengthened by weighting in the loss function, but learning of the region around the boundary line is not strengthened.

[0031] Depending on the purpose of the task, it may be better to learn not only the boundary line but also the surrounding area of ​​the boundary line, as shown in Fig. 3(c). For example, in the case of the boundary area between hair and skin, the detection performance may be improved by strengthening the learning of the surrounding skin area as well, rather than strengthening the learning of only the contour line. In this modified example, the learning of the area surrounding the boundary line is strengthened.

[0032] In this modification, the same processes as those in the first embodiment are performed in the steps other than step S402 in the flow chart of FIG. 4, but the process described below is performed in step S402.

[0033] In step S402, the extraction unit 202 extracts, as a boundary region, a region including the boundary between the human skin region and the non-human skin region and the peripheral region of the boundary, from the teacher image acquired from the storage unit 201. Hereinafter, a method using morphological gradient calculation will be described as an example, but the method is not limited to this.

[0034] The morphological gradient operation is a method for extracting the boundary region of a binary image by taking the difference between an expanded image obtained by expanding the original binary image and a reduced image obtained by reducing the original binary image. At this time, the thickness of the resulting boundary line can be controlled by changing the strength of expansion and reduction. This makes it possible to extract the boundary line between regions of different labels and the boundary line region including the surrounding area of ​​the boundary line from the teacher image. In Fig. 3(c), the white area indicates the boundary region, and the black area indicates the area other than the boundary region.

[0035] When extracting the peripheral area of ​​the boundary, only the peripheral area inside the boundary may be extracted. This can improve the detection performance for detection targets that are prone to detection errors inside the boundary. This can be done by extracting the difference between the original binary image and the reduced image, or by other methods. Also, only the peripheral area outside the boundary may be extracted. This can improve the detection performance for detection targets that are prone to detection errors outside the boundary. This can be done by extracting the difference between the dilated image and the original image, or by other methods.

[0036] Also, the width of the boundary region can be extracted thinly at first, and gradually increased by repeating the processes of steps S401 to S405. In this case, the width can be changed according to a predetermined rule, or can be changed dynamically according to the magnitude of the loss calculated in step S403. The width is increased until the detection performance exceeds a predetermined threshold as a detection result.

[0037] [Second embodiment] In the following embodiments including this embodiment, the differences from the first embodiment will be described, and unless otherwise specified below, it is assumed that the embodiments are the same as the first embodiment. In the first embodiment, as shown in the example of formula (2), a uniform value is set for the loss weight for the boundary region and the loss weight for the non-boundary region. However, there are cases where a more suitable loss function is obtained by determining the loss weight according to the area ratio (pixel number ratio) between the boundary region and the non-boundary region.

[0038] An example is shown in Figure 5. Figure 5 shows the ratio of the area (number of pixels) of the boundary region to the non-boundary region when the face size in the image is large (upper row) and when it is small (lower row). As shown in the upper row of Figure 5, when the face size is large, the area occupied by the boundary region is smaller than the area occupied by the non-boundary region, but as shown in the lower row of Figure 5, when the face size is small, the area occupied by the boundary region is larger than the area of ​​the non-boundary region. In this case, the loss weight for the boundary region is set according to the area ratio of the boundary region to the non-boundary region in the image, rather than setting it uniformly for all images, in order to evenly balance the learning of the entire learning data.

[0039] In this modification, the same processing as in the first embodiment is performed in the steps other than step S403 in the flowchart of Fig. 4, but the processing described below is performed in step S403. In step S403, the loss calculation unit 204 sets the loss weight wi according to the following equation (3) instead of equation (2).

[0040]

number

[0041] Here, S1 indicates the area of ​​the boundary region, and S2 indicates the area of ​​the non-boundary region. When the i-th pixel pi in the teacher image does not belong to set B, the loss calculation unit 204 sets the loss weight wi to a value of "1". When the pixel pi belongs to set B, the loss calculation unit 204 sets the loss weight wi to a value obtained by multiplying the value "w" (>1) by the area ratio of the boundary region to the non-boundary region. That is, when the area of ​​the boundary region is larger than the area of ​​the non-boundary region, a value smaller than the value "w" is set to the loss weight wi, and when the area of ​​the boundary region is smaller than the area of ​​the non-boundary region, a value larger than the value "w" is set to the loss weight wi. Therefore, the overall loss can be set so that the loss of the boundary region is not buried in the loss of the non-boundary region. When the area of ​​the boundary region is large, wi is set to a small value, so that the loss of the boundary region becomes excessive and the learning of the non-boundary region can be prevented from being hindered. In the above, an example in which the mean square error is used as the loss function is shown, but the same applies when a loss function other than the mean square error is used. In this way, by determining the weight of the loss for the boundary region according to the area ratio between the boundary region and the non-boundary region, it is possible to equalize the balance of learning for the entire training data.

[0042] [Third embodiment] In the first and second embodiments, a configuration in which a boundary region in a teacher image is utilized in a region detection task has been described. In the present embodiment, a configuration in which a boundary region is utilized in an object detection task will be described.

[0043] In this embodiment, an example of improving learning performance in a human face detection task is shown. The human face detection task aims to detect the center point of a human face in an input image. At this time, an example of a learning image stored in the storage unit 201 is shown in FIG. 6(a), and an example of a teacher image (correct answer image) corresponding to the learning image is shown in FIG. 6(b).

[0044] In Fig. 6(b), the center point of the area of ​​the person's face in the teacher image is white, and the further away from the center point the blacker the area becomes. In reality, pixels displayed as white have a pixel value of "1," and pixels displayed as black have a pixel value of "0." Furthermore, the pixel value of a pixel displayed as whiter is closer to "1," and the pixel value of a pixel displayed as black is closer to "0."

[0045] Here, the purpose of learning in this embodiment is to update the parameters of the learning model to suitable parameters so that the detection result of the detection unit 203 is as close as possible to the teacher image. However, at this time, the boundary periphery between the correct definition area (non-black area) and the non-correct definition area (black area) may not be correctly learned. For example, as shown in FIG. 6(c), the correct definition area in the detection result may be learned to appear wider than the correct definition area defined in the teacher image. Alternatively, as shown in FIG. 6(d), even if the spread of the correct definition area in the detection result is equivalent to the correct definition area in the teacher image, the peak value may be learned to appear over a wide range. If the correct definition area spreads to a large area as described above, it leads to an error in estimating the center position. In particular, when multiple people are captured in one image and the distance between the faces is close, the face detection results of multiple people may be linked, making it more difficult to estimate the center position. Therefore, the purpose of this embodiment is to improve the detection performance of the boundary area between the correct definition area and the non-correct definition area in the object detection task.

[0046] The process performed by the image processing device 200 to learn an object detection task will be described with reference to the flowchart in Fig. 7. In Fig. 7, the same process steps as those shown in Fig. 4 are given the same step numbers, and the description of these process steps will be omitted.

[0047] In step S701, the detection unit 203 acquires a training image including a human face region and a teacher image (correct image) corresponding to the training image from the storage unit 201. The detection unit 203 then inputs the training image acquired from the storage unit 201 into a training model, which is a "recognition model for recognizing an object from an input image," and performs arithmetic processing of the training model to acquire an estimation result (detection result) of the human face region in the training image.

[0048] In step S702, the extraction unit 202 binarizes the teacher image. The binarization threshold may be freely determined depending on the purpose of learning. In this embodiment, the following description will be given with the threshold set to 0. The result of binarizing the teacher image in FIG. 6(b) with the threshold set to 0 is shown in FIG. 8(a).

[0049] In step S703, the extraction unit 202 extracts, as a boundary region, the boundary between a region with a pixel value of "0" and a region with a pixel value of "1" from the teacher image binarized in step S702 (binarized teacher image). For example, when extracting a boundary region using a morphological gradient operation as in the second embodiment, the boundary region (white region) shown in FIG. 8(b) can be obtained by taking the difference between an expanded image obtained by performing expansion processing on the binarized teacher image in FIG. 8(a) and a reduced image obtained by performing reduction processing on the binarized teacher image in FIG. 8(a).

[0050] As described in the second embodiment, the thickness and radius of the obtained boundary region can be controlled by changing the strength of expansion and contraction. For example, when the detection result tends to spread excessively larger than the correct definition region of the teacher image as shown in FIG. 6(c), the strength of expansion may be increased to extract a thicker boundary region outside the correct definition region. Also, when the detection peak value is over a wide range as shown in FIG. 6(d), the strength of contraction may be increased to extract a thicker boundary region inside the correct definition region. Also, both of the above may be combined. The method for extracting the boundary region is not limited to the morphological gradient calculation.

[0051] Since the learning of the boundary region extracted in step S703 is strengthened more than that of other regions due to weighting of the loss, it is possible to suppress the spread of the detection results or the spread of the detection peak values.

[0052] In this way, according to this embodiment, the boundary region detection performance of the object detection task can be improved by binarizing the teacher image, extracting the boundary region, and weighting the loss in the boundary region.

[0053] In the first to third embodiments, an example of a configuration in which an error calculation is performed to obtain an error between a calculation result of a learning model to which a learning image is input and a teacher image corresponding to the learning image, with the error corresponding to a boundary region in the teacher image being emphasized, is merely described. In other words, as long as the same effect can be obtained, the calculation formula and parameters for the error calculation are not limited to a specific format.

[0054] [Fourth embodiment] Of the functional units of the image processing device 200 shown in Fig. 2, the other functional units except for the storage unit 201 may be implemented by hardware or by software (computer program). In the latter case, a computer device capable of executing the software is applicable to the image processing device 200. An example of the hardware configuration of a computer device applicable to the image processing device 200 will be described with reference to the block diagram of Fig. 1. Note that the hardware configuration shown in Fig. 1 is an example of the hardware configuration of a computer device applicable to the image processing device 200, and can be modified / changed as appropriate.

[0055] The processor 101 is a processor such as a CPU or an MPU, and executes various processes using computer programs and data stored in the memory 102. As a result, the processor 101 controls the operation of the entire image processing device 200, and executes or controls various processes described as processes executed by the image processing device 200.

[0056] The memory 102 has an area for storing computer programs and data loaded from the storage device 103, and an area for storing information (computer programs and data) acquired from the outside via the input interface 104. The memory 102 further has a work area used when the processor 101 executes various processes. In this way, the memory 102 can provide various areas as needed.

[0057] The storage device 103 is a large-capacity information storage device such as a hard disk drive device. The storage device 103 stores an OS (operating system), computer programs and data for causing the processor 101 to execute or control various processes described as processes executed by the image processing device 200, and the like. The computer programs and data stored in the storage device 103 are loaded into the memory 102 as appropriate under the control of the processor 101.

[0058] The storage device 103 may include a recording medium such as a CD-ROM or a DVD-ROM, and a drive device for reading and writing information from and to the recording medium. The storage device 103 may also be a memory device that is detachable from the image processing device, such as a USB. The storage unit 201 in FIG. 2 can be implemented using the storage device 103 and the memory 102.

[0059] The input interface 104 may include various interfaces for inputting information to the image processing device 200. For example, the input interface 104 is an interface for receiving information transmitted from an external device. Also, for example, the input interface 104 is an interface for receiving instructions and information input by a user operating an operation unit (such as a keyboard, a mouse, or a touch panel screen).

[0060] The output interface 105 may include various interfaces for outputting information from the image processing device 200. For example, the output interface 105 is an interface for transmitting information to an external device via a network such as a LAN or the Internet. The processor 101, the memory 102, the storage device 103, the input interface 104, and the output interface 105 are all connected to a system bus 106.

[0061] The numerical values, processing timing, processing order, processing subject, data (information) acquisition method / destination / source / storage location, etc. used in each of the above embodiments are given as examples to provide a concrete explanation, and are not intended to be limited to these examples.

[0062] In addition, a part or all of the embodiments described above may be used in appropriate combination. In addition, a part or all of the embodiments described above may be used selectively.

[0063] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0064] The invention of this specification includes the following image processing device, image processing method, and computer program. (Item 1) A calculation means for performing an error calculation to obtain an error between a calculation result of a learning model to which a learning image is input and a teacher image corresponding to the learning image; a learning means for learning the learning model based on the error; Equipped with The calculation means performs the error calculation by making the error corresponding to the boundary region of the object in the teacher image larger than the error in other regions. 13. An image processing device comprising: (Item 2) The image processing device described in item 1 is characterized in that the calculation means calculates the error between the calculation result and the teacher image as a weighted sum of the errors for each pixel between the calculation result and the teacher image, and a weight for the error between a pixel belonging to a boundary area in the teacher image and a corresponding pixel in the calculation result is greater than a weight for the error between a pixel not belonging to a boundary area in the teacher image and a corresponding pixel in the calculation result. (Item 3) The image processing device according to item 2, characterized in that the calculation means sets a value corresponding to the area ratio of the boundary area and the non-boundary area in the teacher image as a weight for the error between a pixel belonging to the boundary area in the teacher image and a corresponding pixel in the calculation result. (Item 4) The image processing device according to any one of items 1 to 3, characterized in that the learning means learns the learning model by updating parameters of the learning model so as to reduce an error between the calculation result and the teacher image. (Item 5) 5. The image processing device according to any one of items 1 to 4, wherein the boundary region in the teacher image is a boundary line region between regions with different assigned labels. (Item 6) 5. The image processing device according to any one of items 1 to 4, wherein the boundary region in the teacher image is a region including a boundary line between regions with different assigned labels and a peripheral region of the boundary line. (Item 7) 7. The image processing device according to any one of items 1 to 6, wherein the learning means dynamically changes the width of the boundary region. (Item 8) The image processing device according to any one of items 1 to 7, characterized in that the calculation means performs the error calculation by emphasizing an error corresponding to a boundary area in a binarized teacher image obtained by binarizing the teacher image. (Item 9) An input means for inputting an image; a detection means for detecting an object from the input image using parameters of a learning model trained by making an error corresponding to a boundary region of the object included in the image larger than that of other regions; An image processing device comprising: (Item 10) An image processing method performed by an image processing device, comprising: a calculation step in which a calculation means of the image processing device performs an error calculation to obtain an error between a calculation result of a learning model to which a learning image is input and a teacher image corresponding to the learning image; a learning step in which a learning means of the image processing device learns the learning model based on the error; Equipped with In the calculation step, the error calculation is performed by making the error corresponding to the boundary region of the object in the teaching image larger than that of other regions. 13. An image processing method comprising: (Item 11) an input step of inputting an image; a detection step of detecting the object from the input image using parameters of a learning model trained by making an error corresponding to a boundary region of the object included in the image larger than that of other regions; An image processing method comprising: (Item 12) A computer program for causing a computer to function as each of the means of the image processing device according to any one of items 1 to 9.

[0065] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0066] 200: Image processing device 201: Memory unit 202: Extraction unit 203: Detection unit 204: Loss calculation unit 205: Learning unit

Claims

1. A calculation means for performing an error calculation to obtain an error between a calculation result of a learning model to which a learning image is input and a teacher image corresponding to the learning image; a learning means for learning the learning model based on the error; Equipped with The calculation means performs the error calculation by making the error corresponding to the boundary region of the object in the teacher image larger than the error in other regions.

13. An image processing device comprising:

2. The image processing device according to claim 1, characterized in that the calculation means calculates the error between the calculation result and the teacher image as a weighted sum of the errors for each pixel between the calculation result and the teacher image, and a weight for the error between a pixel belonging to a boundary area in the teacher image and a corresponding pixel in the calculation result is greater than a weight for the error between a pixel not belonging to a boundary area in the teacher image and a corresponding pixel in the calculation result.

3. 3. The image processing device according to claim 2, wherein the calculation means sets a value corresponding to an area ratio of a boundary area to a non-boundary area in the teacher image as a weight for an error between a pixel belonging to a boundary area in the teacher image and a corresponding pixel in the calculation result.

4. 2. The image processing apparatus according to claim 1, wherein the learning means performs learning of the learning model by updating parameters of the learning model so as to reduce an error between the calculation result and the teacher image.

5. The image processing apparatus according to claim 1 , wherein the boundary region in the teacher image is a boundary region between regions that are assigned different labels.

6. The image processing apparatus according to claim 1 , wherein the boundary region in the teacher image is a region including a boundary line between regions having different assigned labels and a peripheral region of the boundary line.

7. 2. The image processing apparatus according to claim 1, wherein the learning means dynamically changes the width of the boundary region.

8. 2. The image processing apparatus according to claim 1, wherein said calculation means performs said error calculation by emphasizing an error corresponding to a boundary region in a binarized teacher image obtained by binarizing said teacher image.

9. An input means for inputting an image; a detection means for detecting an object from the input image using parameters of a learning model trained by making an error corresponding to a boundary region of the object included in the image larger than that of other regions; An image processing device comprising:

10. An image processing method performed by an image processing device, comprising: a calculation step in which a calculation means of the image processing device performs an error calculation to obtain an error between a calculation result of a learning model to which a learning image is input and a teacher image corresponding to the learning image; a learning step in which a learning means of the image processing device learns the learning model based on the error; Equipped with In the calculation step, the error calculation is performed by making the error corresponding to the boundary region of the object in the teacher image larger than that of other regions.

13. An image processing method comprising:

11. An input step of inputting an image; a detection step of detecting the object from the input image using parameters of a learning model trained by making an error corresponding to a boundary region of the object included in the image larger than that of other regions; An image processing method comprising:

12. A computer program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 9.

Images