Information processing apparatus and method
By generating scaled images to align object and anchor box dimensions, the training of learning models in anchor box-based object detection is enhanced, addressing the challenge of cumbersome hyperparameter adjustments.
Patent Information
- Application Number
- JP2024086655
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-05-28
- Publication Date
- 2025-12-10
AI Technical Summary
Anchor box-based object detection technologies face challenges in adjusting hyperparameters such as scale and aspect ratio, which are cumbersome and often cannot be changed from default values, leading to inaccurate training of learning models.
Generate multiple second images by reducing or enlarging first images along the x-axis and y-axis directions to align the scale and shape of objects with anchor boxes, allowing the learning model to proceed accurately without adjusting hyperparameters.
This approach enables accurate training of learning models by reducing discrepancies in scale and shape between objects and anchor boxes, improving anchor box-based object detection technology.
Smart Images
Figure 2025179725000001_ABST
Abstract
Description
[Technical Field]
[0001] The present disclosure relates to an information processing device and method. [Background technology]
[0002] Conventionally, techniques related to anchor box-based object detection have been known. For example, Patent Document 1 discloses a parameter determination device that determines parameters of multiple anchor boxes based on the distribution of parameters of bounding boxes. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Patent No. 7327508 Summary of the Invention [Problem to be solved by the invention]
[0004] In anchor box-based object detection, it is desirable to adjust the hyperparameters (e.g., scale and aspect ratio) of the anchor box according to the bounding box, i.e., the shape of the object to be detected in the image. However, adjusting the hyperparameters of the anchor box is generally cumbersome. Furthermore, depending on the learning model used, there are cases where the hyperparameters cannot be changed from their default values, or where changing them from their default values is not recommended. Therefore, there is room for improvement in anchor box-based object detection technology.
[0005] In view of the above, an object of the present disclosure is to improve techniques related to anchor box-based object detection. [Means for solving the problem]
[0006] An information processing device according to an embodiment of the present disclosure includes: An information processing device that trains a learning model that detects an object included in an input image using a plurality of first images to which correct labels indicating the object are respectively assigned, The system includes a control unit that generates a plurality of second images by performing a first image processing on each of the plurality of first images to reduce them in a first direction along one of the x-axis and y-axis, and trains the learning model using the plurality of second images.
[0007] According to one embodiment of the present disclosure, a method comprises: A method executed by an information processing device for training a learning model that detects an object included in an input image using a plurality of first images each having a correct answer label indicating the object, the method comprising: generating a plurality of second images by performing a first image processing on the plurality of first images to reduce the size of the plurality of first images in a first direction along one of the x-axis and the y-axis; and training the learning model using the plurality of second images. [Effects of the Invention]
[0008] According to one embodiment of the present disclosure, techniques for anchor box-based object detection are improved. [Brief explanation of the drawings]
[0009] [Figure 1] 1 is a block diagram illustrating a schematic configuration of an information processing device according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a schematic diagram of a first image including multiple objects. [Figure 3] FIG. 2 is a schematic diagram showing an object and an anchor box on a first image. [Figure 4] 10 is a flowchart showing a first operation of the information processing device. [Figure 5] FIG. 2 is a schematic diagram showing an object and an anchor box on a second image that has been subjected to first image processing. [Figure 6] FIG. 2 is a schematic diagram showing an object and an anchor box on a second image that has been subjected to first image processing and second image processing. [Figure 7] 10 is a flowchart showing a second operation of the information processing device. [Figure 8] FIG. 10 is a schematic diagram of a resultant image in which a marker is superimposed on an object. [Figure 9] FIG. 10 is a diagram showing a first example of a screen displayed on an information processing device according to another embodiment. [Figure 10] FIG. 10 is a diagram showing a second example of a screen displayed on an information processing device according to another embodiment. DETAILED DESCRIPTION OF THE INVENTION
[0010] Hereinafter, embodiments of the present disclosure will be described.
[0011] (Outline of the embodiment) An overview of an information processing device 10 according to an embodiment of the present disclosure will be described with reference to FIG. 1. The information processing device 10 is a computer used to train a learning model that detects objects included in an input image. In this embodiment, an example will be described in which, for example, multiple trays are stacked vertically and an image generated by capturing an image from the side with a camera is used as the input image. The learning model is trained to detect each tray included in the input image as an object.
[0012] The learning model may be, for example, a Faster R-CNN (Region-Based Convolutional Neural Network), but is not limited to this, and may be any anchor box-based object detection model. Training of the learning model employs supervised learning using multiple first images, each of which is assigned a correct label indicating an object. The correct label includes a bounding box indicating the position and shape of the object in the first image. The bounding box may be, for example, a rectangular region defined by a combination of its center coordinates, its length (number of pixels) in the x-axis direction, and its length (number of pixels) in the y-axis direction. The correct label is not limited to a bounding box and may further include any information used in training the learning model. For example, when a learning model is employed that also estimates the class of the detected object (e.g., the name or type of the object), the correct label may further include information indicating the class of the object in the first image. The correct label is assumed to have been assigned to the first image in advance by manual or automatic annotation. For example, as shown in FIG. 2, in this embodiment, the first image 20A may include multiple objects 21A arranged in the y-axis direction (the vertical direction in the figure). Each object 21A on the first image 20A has a substantially rectangular shape with long and short sides. The long sides are substantially parallel to the x-axis direction (the horizontal direction in the figure), and the short sides are substantially parallel to the y-axis direction (the vertical direction in the figure). Typically, in manual annotation or automatic annotation, a rectangular bounding box is set that surrounds the object 21A on the first image 20A or that substantially matches the shape of the object 21A.
[0013] When training a learning model, first, each point on a feature map generated from an input image is set as an anchor. Typically, the set multiple anchors are arranged at equal intervals in a grid pattern on the input image. Next, an anchor box is set for each anchor. Generally, to accurately train the learning model, hyperparameters (scale and aspect ratio) related to the anchor box are adjusted according to the scale and shape of the target object in the image to be detected. However, in this embodiment, the hyperparameters (scale and aspect ratio) related to the anchor box are not changed from their default values. Typically, the default values for scale are "128 pixels," "256 pixels," and "512 pixels." The default values for aspect ratio are "1:1," "2:1," and "1:2." In this case, 3 × 3 = 9 anchor boxes are set for one anchor. Then, a Region Proposal Network (RPN) included in the learning model is trained to learn whether the inside of each anchor box is an object or not, and the difference in position and shape between the anchor box with the object inside and the bounding box indicated by the correct label. As the training progresses, the parameters of the RPN included in the learning model are fitted so that the position and shape of the anchor box with the object inside approaches the position and shape of the bounding box.
[0014] With reference to FIG. 3 , for comparison with the present embodiment described later, a case will be considered in which a learning model is trained using the first image 20A as is. For simplicity, only one object 21A and one anchor 22A are illustrated in FIG. 3 . Furthermore, only three anchor boxes 23A, each with the same scale but different aspect ratios, are illustrated. In the example shown in FIG. 3 , the object 21A has a shape that is longer in the x-axis direction (left-right direction in the figure) than the anchor box 23A, and the scale of the object 21A and the anchor box 23A in the x-axis direction differs relatively greatly. Similarly, the object 21A has a shape that is shorter in the y-axis direction (up-down direction in the figure) than the anchor box 23A, and the scale of the object 21A and the anchor box 23A in the y-axis direction differs relatively greatly. Thus, in the example shown in FIG. 3 , the scale and / or shape of the object 21A differ relatively greatly from those of the anchor box 23A, whose scale and aspect ratio are set to the default values. If there is a large discrepancy in scale and / or shape between the object 21A and the anchor box 23A, training of the learning model does not necessarily proceed with high accuracy.
[0015] In contrast, in this embodiment, the information processing device 10 generates a plurality of second images by performing a first image process on a plurality of first images 20A, which reduces the images in a first direction along one of the x-axis and the y-axis (e.g., the x-axis direction), a second image process on the images in a second direction along the other of the x-axis and the y-axis (e.g., the y-axis direction), or a combination of these processes.The information processing device 10 then trains a learning model using the plurality of second images.
[0016] For example, as shown in FIG. 5 or 6, objects 21B and 21C included in second images 20B and 20C obtained by reducing a first image 20A in the x-axis direction have a shorter length in the x-axis direction than object 21A included in the first image 20A. Also, as shown in FIG. 6, object 21C included in second image 20C obtained by enlarging a first image 20A in the y-axis direction has a longer length in the y-axis direction than object 21A included in the first image 20A. Therefore, according to this embodiment, the discrepancy in scale and / or shape between the objects in the images used to train the learning model and the anchor boxes is reduced, increasing the likelihood that training of the learning model will proceed accurately. Therefore, anchor box-based object detection technology is improved in that training of the learning model can proceed accurately without adjusting hyperparameters related to the anchor boxes according to the scale and shape of the objects in the images used to train the learning model.
[0017] Next, each component of the information processing device 10 will be described in detail with reference to FIG.
[0018] (Configuration of information processing device) The information processing device 10 includes a communication unit 11, an output unit 12, an input unit 13, a storage unit 14, and a control unit 15.
[0019] The communication unit 11 includes an interface for communicating with an external device, and the interface may be compatible with any wired or wireless communication standard.
[0020] The output unit 12 includes one or more output devices that output information. The output devices are, for example, but not limited to, a display or a speaker. Alternatively, the output unit 12 may include an interface for connecting an external output device.
[0021] The input unit 13 includes one or more input devices that detect user input. Examples of the input devices include, but are not limited to, physical keys, capacitive keys, pointing devices such as a mouse, a touch screen integrated with the display of the output unit 12, or a microphone that accepts voice input. Alternatively, the input unit 13 may include an input interface that detects user input via an external input device.
[0022] The storage unit 14 includes one or more memories. The memories may be, for example, semiconductor memories, magnetic memories, optical memories, or the like, but are not limited to these. Each memory included in the storage unit 14 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 14 stores any information used in the operation of the information processing device 10. For example, the storage unit 14 may store system programs, application programs, embedded software, and the like.
[0023] In this embodiment, the storage unit 14 stores a learning model for detecting an object included in an input image. As described above, the learning model is, for example, Faster R-CNN, but is not limited to this and may be any anchor box-based object detection model.
[0024] The control unit 15 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The processor may be, for example, a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process, but is not limited to these. The programmable circuit may be, for example, an FPGA (Field-Programmable Gate Array), but is not limited to this. The dedicated circuit may be, for example, an ASIC (Application Specific Integrated Circuit), but is not limited to this. Hereinafter, the processor, programmable circuit, and dedicated circuit will be referred to as "processor, etc." without any particular distinction being made between them. The control unit 15 controls the overall operation of the information processing device 10.
[0025] (Operation flow of information processing device) The flow of the first operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 4. In summary, the first operation is an operation for training a learning model using a second image.
[0026] Step S100: The control unit 15 generates multiple second images by performing a first image process to reduce the multiple first images, each of which has been assigned a correct label indicating an object, in a first direction along one of the x-axis and the y-axis (here, the x-axis direction), a second image process to expand the multiple first images in a second direction along the other of the x-axis and the y-axis (here, the y-axis direction), or a combination of these processes. Note that the second image 20B shown in FIG. 5 is a schematic representation of an image generated by performing the first image process on the first image 20A. The second image 20C shown in FIG. 6 is a schematic representation of an image generated by performing the first image process and the second image process on the first image 20A. For ease of explanation, it is assumed here that multiple second images 20B or 20C are generated in step S100. However, in step S100, the control unit 15 may generate multiple second images by performing only the second image process on the multiple first images 20A.
[0027] The scaling ratio in the first image processing (hereinafter also referred to as the "first scaling ratio P") may be predetermined or may be arbitrarily determined by the user. Alternatively, the control unit 15 may automatically determine the first scaling ratio P. For example, the control unit 15 may calculate an average value of the lengths along the first direction of the bounding boxes indicated by the correct labels assigned to the multiple first images 20A. The control unit 15 may then calculate the first scaling ratio P by dividing any specified value of the scale of the anchor box (e.g., "128 pixels," "256 pixels," or "512 pixels") by the calculated average value. For example, if the calculated average value is 1280 pixels, the control unit 15 may determine the first scaling ratio P to be 10%, 20%, or 50%. With this configuration, as shown in, for example, Figure 5 or Figure 6, the probability that the length in the first direction (x-axis direction) of the object 21B or 21C contained in the second image 20B or 20C will approximately match the specified value of the scale of the anchor box increases.
[0028] The scaling ratio in the second image processing (hereinafter also referred to as the "second scaling ratio Q") may be predetermined or may be arbitrarily determined by the user. Alternatively, the control unit 15 may automatically determine the second scaling ratio Q. For example, the control unit 15 may calculate an average value of the lengths along the second direction of the bounding boxes indicated by the correct labels assigned to the multiple first images 20A. The control unit 15 may then calculate the second scaling ratio Q by dividing any specified value of the scale of the anchor box (e.g., "128 pixels," "256 pixels," or "512 pixels") by the calculated average value. For example, if the calculated average value is 64 pixels, the control unit 15 may determine the second scaling ratio Q to be 200%, 400%, or 800%. This configuration increases the likelihood that the length of the object 21C included in the second image 20C in the second direction (y-axis direction) will approximately match the specified value of the scale of the anchor box, as shown in FIG. 6, for example.
[0029] Alternatively, the control unit 15 may automatically determine the second enlargement / reduction ratio Q so that the aspect ratio of the object 21C included in the second image 20C substantially matches one of the specified values of the aspect ratio of the anchor box ("1:1," "2:1," or "1:2"). This configuration increases the likelihood that the scale and aspect ratio of the object 21C included in the second image 20C substantially matches the specified values of the scale and aspect ratio of the anchor box.
[0030] When the first image 20A is enlarged or reduced by the first image processing and / or the second image processing, the bounding box indicated in the correct label assigned to the first image 20A is also enlarged or reduced in the same manner. Therefore, the correct label assigned to the second image 20B or 20C includes a bounding box indicating the position and shape of the object 21B or 21C included in the second image 20B or 20C.
[0031] Step S101: The control unit 15 trains a learning model using the generated second images 20B or 20C.
[0032] In detail, the control unit 15 generates a feature map from the second image 20B or 20C. A model such as VGG16 is used to generate the feature map, but this is not limiting and any method can be employed. Next, the control unit 15 sets each point on the generated feature map as an anchor 22B or 22C. The set anchors 22B or 22C are arranged at equal intervals like a grid point on the second image 20B or 20C. Next, the control unit 15 sets an anchor box 23B or 23C for each anchor 22B or 22C. In this embodiment, nine anchor boxes 23B or 23C are set for one anchor 22B or 23C as described above. The control unit 15 then trains the RPN included in the learning model based on a comparison between each anchor box 23B or 23C and a bounding box that has been reduced in the first direction and / or expanded in the second direction by the first image processing and / or the second image processing. In detail, the control unit 15 trains the RPN included in the learning model to learn whether the inside of each anchor box 23B or 23C is the object 21B or 21C, and the difference in position and shape between the anchor box 23B or 23C inside which the object 21B or 21C is located and the bounding box indicated by the correct label. As the training of the learning model progresses, the parameters of the RPN included in the learning model are fitted so that the position and shape of the anchor box 23B or 23C inside which the object 21B or 21C is located approaches the position and shape of the bounding box.
[0033] Step S102: The control unit 15 acquires the learning model to which the parameters have been fitted in step S101 as a trained learning model.
[0034] A flow of the second operation of the information processing device 10 according to this embodiment will be described with reference to FIG. 7. The second operation is, in outline, an operation of detecting an object included in an input image using a trained learning model. The input image will be described as being acquired from an external device such as a camera or a computer via the communication unit 11 of the information processing device 10. Unlike the first image used to train the learning model, the input image does not have a correct answer label attached.
[0035] Step S200: When inputting an input image to the trained learning model, the control unit 15 performs the first image processing and / or the second image processing on the input image before inputting it to the trained learning model.
[0036] In an embodiment in which at least the first image processing is performed in step S100, the control unit 15 performs at least the first image processing on the input image before inputting it to the trained learning model in step S200. Similarly, in an embodiment in which at least the second image processing is performed in step S100, the control unit 15 performs at least the second image processing on the input image before inputting it to the trained learning model in step S200.
[0037] Step S201: The control unit 15 detects an object included in an input image before the first image processing and / or the second image processing is performed, based on the output of the trained learning model.
[0038] The output of the trained learning model may include information directly indicating the position and shape of an object included in the input image before the first image processing and / or the second image processing is performed. Alternatively, the output of the trained learning model may include information indicating the position and shape of an object included in the input image after the first image processing and / or the second image processing is performed. In such a case, the control unit 15 may identify the position and shape of the object included in the input image before the first image processing and / or the second image processing is performed by correcting the position and shape of the object included in the input image after the first image processing and / or the second image processing is performed based on the above-mentioned first enlargement / reduction ratio P and / or second enlargement / reduction ratio Q. Specifically, the control unit 15 identifies the position and shape of the object contained in the input image before the first image processing and / or the second image processing is performed by enlarging the input image (and the position and shape of the object) in a first direction (e.g., the x-axis direction) by the reciprocal 1 / P (e.g., 500%) of the first scaling factor P (e.g., 20%) and / or by reducing the input image in a second direction (e.g., the y-axis direction) by the reciprocal 1 / Q (e.g., 25%) of the second scaling factor Q (e.g., 400%), thereby determining the position and shape of the object contained in the input image before the first image processing and / or the second image processing is performed.
[0039] Step S202: The control unit 15 generates and outputs a resultant image by superimposing markers indicating each detected object on the input image before the first image processing and / or the second image processing is performed.
[0040] For example, in the result image 30 shown in FIG. 8, a marker 31 is superimposed on each detected object. The marker 31 may be, for example, an image including only a contour line, or an image in which the inside of the contour line is filled with a predetermined transparency. The marker 31 may also include information indicating the number of detected objects. In the example shown in FIG. 8, a total of seven objects are detected, and each marker 31 includes a number from "1" to "7" as the number of detections.
[0041] As described above, the information processing device 10 according to this embodiment generates a plurality of second images by performing a first image process for reducing a plurality of first images 20A in a first direction along one of the x-axis and the y-axis, a second image process for enlarging a plurality of first images 20A in a second direction along the other of the x-axis and the y-axis, or a combination of these processes.The information processing device 10 then trains a learning model using the plurality of second images.
[0042] This configuration reduces the discrepancy in scale and / or shape between the anchor boxes and the objects in the images used to train the learning model, increasing the likelihood that the training of the learning model will proceed accurately. Therefore, anchor box-based object detection technology is improved in that the training of the learning model can proceed accurately without adjusting hyperparameters related to the anchor boxes depending on the scale and shape of the objects in the images used to train the learning model.
[0043] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art may make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each component or step can be rearranged so as not to be logically inconsistent, and multiple components or steps can be combined or divided into one.
[0044] In the above-described embodiment, the input image to be input to the trained learning model is acquired from an external device such as a camera via the communication unit 11 of the information processing device 10. However, an embodiment in which the information processing device 10 includes a camera is also possible. In such a case, the user stacks multiple trays, which are the target objects, vertically and captures an image of the entire multiple trays as a subject from the side using the camera of the information processing device 10. When capturing an image, the control unit 15 may output information regarding the posture of the subject when capturing an image. For example, the control unit 15 may output information prompting the user to hold the camera horizontally based on information from a gyro sensor or the like included in the camera. The control unit 15 may also output information prompting the user to move the camera closer to or farther away from the subject so that the proportion of the subject (multiple trays) occupying the entire image falls within a predetermined range.
[0045] In the above-described embodiment, the control unit 15 generates multiple second images by performing a first image process, a second image process, or a combination thereof on multiple first images 20A, each of which has been assigned a correct label indicating an object. An embodiment is also possible in which the user can select the image process to be performed on the multiple first images 20A. For example, when performing step S100, the control unit 15 may display, on the display of the output unit 12, graphical user interfaces (GUIs) 41 and 42, as shown in FIG. 9, that allow the user to select the first image process and the second image process. Subsequently, upon detecting a user operation on a start button 43, for example, the control unit 15 may identify one or two image processes (i.e., the first image process, the second image process, or a combination thereof) selected by the user from the first image process and the second image process. Then, in step S100, the control unit 15 may generate multiple second images by applying one or two image processing operations selected by the user (first image processing, second image processing, or a combination of these) to multiple first images 20A, each of which has been assigned a correct label indicating the object.
[0046] Furthermore, in the above embodiment in which the user can select image processing to be performed on the multiple first images 20A, the control unit 15 may determine whether to display the above-described GUI based on a comparison between the scale (length along the first direction and / or the second direction) of the bounding box indicated in the correct labels assigned to the multiple first images 20A and any specified value of the anchor box scale (e.g., “128 pixels,” “256 pixels,” or “512 pixels”). For example, the control unit 15 may calculate an average value of the lengths along the first direction and / or the second direction of the bounding boxes indicated in the correct labels assigned to the multiple first images 20A. Then, if the calculated average value and any specified value of the anchor box scale significantly deviate from a predetermined criterion (e.g., if there is a difference of a predetermined number of pixels or more between the two, or if one is a predetermined multiple or more of the other), the control unit 15 may display the above-described GUIs 41 and 42.
[0047] Furthermore, when displaying the above-mentioned GUIs 41 and 42, the control unit 15 may further display a message 44 indicating that there is a large discrepancy in scale between the object and the anchor box, a message 45 indicating that the learning model can be trained efficiently by performing the first image processing, the second image processing, or a combination thereof, and / or an image 46 that allows comparison of the object (e.g., 21B or 21C) and the anchor box (e.g., 23B or 23C) on the second image (e.g., 20B or 20C) generated by applying one or two image processing operations selected by the user (i.e., the first image processing, the second image processing, or a combination thereof) to the first image 20A.
[0048] In the above-described embodiment, an example has been described in which the user can arbitrarily determine the first scaling ratio P in the first image processing and / or the second scaling ratio Q in the second image processing. Any UI (User Interface) can be employed to allow the user to determine the first scaling ratio P and / or the second scaling ratio Q. For example, the control unit 15 may accept a user operation specifying the numerical values of the first scaling ratio P and / or the second scaling ratio Q. Alternatively, the control unit 15 may accept a user operation to scale the first image 20A displayed on the screen in the first direction and / or the second direction, as shown in FIG. 10 . In such a case, the control unit 15 may display the first image 20A, the anchor 22A, and the anchor box 23A on the display of the output unit 12. The control unit 15 may then scale the first image 20A around the anchor 22A in accordance with the user operation, while leaving the anchor box 23A displayed without scaling. With this configuration, the user can determine an appropriate first scaling ratio P and / or second scaling ratio Q by scaling the first image 20A so that the scale and / or shape of the object 21A approaches the anchor box 23A.
[0049] In the above-described embodiment, an example in which Faster R-CNN is used as a specific example of the learning model has been described. However, the present invention is not limited to this example, and the learning model may also be, for example, Mask R-CNN. In this case, the learning model detects objects included in an input image and performs instance segmentation of the objects on a pixel-by-pixel basis. Training of the learning model employs supervised learning using a plurality of first images 20A, each of which is assigned a correct label indicating the object. Here, the correct label includes not only a bounding box indicating the position and shape of the object in the first image 20A, but also a segmentation mask (hereinafter also referred to as a "correct mask") indicating the object in the first image 20A on a pixel-by-pixel basis. The correct mask may be defined by any method, such as a polygon, a binary mask, or polygon coordinates. The correct label, including the bounding box and the correct mask, is assigned to the first image 20A in advance by manual annotation or automatic annotation. The shape of the correct mask does not necessarily have to strictly match the shape of the object. Typically, in manual annotation or automatic annotation, a correct mask is set that has a shape that approximately matches that of the object 20A on the first image 20A.
[0050] When Mask R-CNN is used as the learning model in the present disclosure, when first image 20A is scaled by the first image processing and / or second image processing in step S100 described above, the bounding box and the ground truth mask indicated in the ground truth label assigned to first image 20A are also scaled. Therefore, the ground truth label assigned to second image 20B or 20C includes a bounding box indicating the position and shape of object 21B or 21C included in second image 20B or 20C, and a ground truth mask indicating object 21B or 21C in pixel units.
[0051] Furthermore, in the above-mentioned step S101, the control unit 15 trains the mask head network so as to learn the difference in position and shape between the segmentation mask (hereinafter also referred to as "estimated mask") generated by the mask head included in the learning model and the correct mask indicated in the correct label. As the training of the learning model progresses, the parameters of the mask head network are fitted so that the position and shape of the estimated mask approach the position and shape of the correct mask.
[0052] Also, an embodiment is possible in which, for example, a general-purpose computer functions as the information processing system 10 according to the above-described embodiment. Specifically, a program describing the processing content for realizing each function of the information processing system 10 according to the above-described embodiment is stored in the memory of the computer, and the program is read and executed by a processor or the like of the computer. Therefore, the present disclosure can also be realized as a program executable by a processor or the like, or a non-transitory computer-readable medium storing the program. [Explanation of symbols]
[0053] 10. Information processing equipment 11 Communications Department 12 Output section 13 Input section 14 Storage section 15 Control Unit 20A 1st image 20B, 20C 2nd image 21A, 21B, 21C Object 22A, 22B, 22C Anchor 23A, 23B, 23C Anchor Box 30 Result images 31 Marker
Claims
1. An information processing device that trains a learning model that detects an object included in an input image using a plurality of first images to which correct labels indicating the object are respectively assigned, the information processing device comprising: a control unit that generates a plurality of second images by performing a first image process for reducing the plurality of first images in a first direction along one of the x-axis and the y-axis, a second image process for enlarging the plurality of first images in a second direction along the other of the x-axis and the y-axis, or a combination thereof, and trains the learning model using the plurality of second images; An information processing device comprising:
2. 2. The information processing device according to claim 1, The control unit When inputting an input image into the trained learning model, the input image is subjected to the first image processing, the second image processing, or a combination thereof, and then inputted into the trained learning model; Detecting the object included in the input image before the first image processing is performed based on an output of the trained learning model; an information processing device that generates a resultant image by superimposing a marker indicating the detected object on the input image before the first image processing, the second image processing, or a combination thereof is applied.
3. 2. The information processing device according to claim 1, The learning model includes a Region Proposal Network (RPN), the correct label includes a bounding box indicating the position and shape of the object on the first image; The control unit setting a plurality of anchor boxes for each of the plurality of second images; An information processing device that trains the RPN of the learning model based on a comparison between the multiple anchor boxes set in each of the multiple second images and the bounding box reduced in the first direction by the first image processing.
4. 4. The information processing device according to claim 3, An information processing device, wherein the length in the first direction of the object included in each of the plurality of second images approximately matches the scale of each of the anchor boxes set in each of the plurality of second images.
5. 2. The information processing device according to claim 1, The information processing device wherein the control unit generates the plurality of second images by performing both the first image processing and the second image processing on the plurality of first images.
6. 6. The information processing device according to claim 5, An information processing device, wherein the aspect ratio of the object included in each of the plurality of second images is approximately 1:1, 2:1, or 1:
2.
7. A method executed by an information processing device for training a learning model that detects an object included in an input image using a plurality of first images each having a correct answer label indicating the object, the method comprising: generating a plurality of second images by performing a first image process for reducing the plurality of first images in a first direction along one of the x-axis and the y-axis, a second image process for enlarging the plurality of first images in a second direction along the other of the x-axis and the y-axis, or a combination thereof; training the learning model using the plurality of second images; A method comprising:
Citation Information
Patent Citations
Parameter determination device, parameter determination method, and program
JP7327508B2