Apparatus, method, and program
By generating modified images to simulate water surface conditions and train a learning model, the apparatus improves object detection accuracy on water surfaces by addressing interference from reflected light and droplets.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- YOKOGAWA ELECTRIC CORP
- Filing Date
- 2023-03-30
- Publication Date
- 2026-05-15
AI Technical Summary
Existing image processing systems struggle with accurately detecting objects on water surfaces due to interference from reflected light and water droplets, which affect the performance of learning models.
An apparatus and method that generate modified images by applying effects to change the number, shape, position, and size of light-emitting regions and water droplets, and simulate varying weather and oceanographic conditions to train a learning model for improved object detection.
Enhances the accuracy and efficiency of object detection on water surfaces by training the learning model with diverse and realistic images, reducing interference from water reflections and droplets.
Smart Images
Figure 0007859366000001 
Figure 0007859366000002 
Figure 0007859366000003
Abstract
Description
Technical Field
[0001] The present invention relates to an apparatus, a method, and a program.
Background Art
[0002] Patent Documents 1 to 8 describe "a processing means for generating a plurality of second images including a human face by performing a predetermined processing on the first image, and by inputting the plurality of second images into the input layer, setting weight values between each of the units belonging to different processing layers by learning using the plurality of second images as teacher images" (Claim 1 of Patent Document 1), etc. [Prior Art Documents] [Patent Documents] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-221840 [Patent Document 2] Japanese Patent Application Laid-Open No. 2021-117548 [Patent Document 3] Japanese Patent Application Laid-Open No. 2021-120914 [Patent Document 4] Japanese Patent Application Laid-Open No. 2020-197833 [Patent Document 5] International Publication No. 2021 / 130888 [Patent Document 6] International Publication No. 2021 / 130995 [Patent Document 7] International Publication No. 2021 / 177324 [Patent Document 8] Japanese Patent Application Laid-Open No. 2019-32782
Summary of the Invention
[0003] In a first aspect of the present invention, there is provided an apparatus including: an acquisition unit that acquires a first image captured on water; an image generation unit that generates a second image by applying an image effect that changes at least one of the number or shape of light emission regions having a predetermined luminance or more to the first image; and a learning processing unit that performs a learning process of a learning model for detecting a detection target object in an image using the learning data including the second image.
[0004] In the above-described apparatus, the image generation unit may make at least one of the shapes or sizes of two or more light-emitting regions generated in the second image different.
[0005] In any of the above-described devices, the image generation unit may further apply an image effect that changes at least one of the position or size of the light-emitting region in the first image to generate the second image.
[0006] In any of the above-described apparatus, the first image is an image captured through at least one light-transmitting member, and the image generation unit may further apply an image effect to the first image that changes at least one of the position, number, and size of water droplets attached to the light-transmitting member to generate the second image.
[0007] In any of the above-described apparatus, the image generation unit may further apply an image effect to the first image that corresponds to imaging conditions in which at least one of the meteorological conditions and oceanographic conditions differs from the imaging conditions of the first image, thereby generating the second image.
[0008] In any of the above-described devices, the image generation unit may apply the image effect to at least a portion of the region of the object to be detected in the first image.
[0009] In any of the above-described devices, the image generation unit may generate a plurality of second images that are different from each other from a single first image.
[0010] In any of the above-described devices, the first image is accompanied by a label indicating whether or not the object to be detected is present in the first image, and the image generation unit may add a label with the same content as the first image to the second image.
[0011] In any of the above-described devices, the learning processing unit may further use the learning data including the first image to train the learning model.
[0012] In any of the above-described devices, a determination unit may be provided that uses the learned model, which has been trained by the learning processing unit, to determine whether or not the object to be detected exists in the first image newly acquired by the acquisition unit.
[0013] A second aspect of the present invention provides a method comprising: an acquisition step of acquiring a first image captured on water; an image generation step of generating a second image by applying an image effect to the first image that changes at least one of the number or shape of light-emitting regions with a predetermined brightness or higher; and a learning processing step of performing a learning process on a learning model that detects objects to be detected in an image using learning data including the second image.
[0014] In a third aspect of the present invention, a program is provided that causes a computer to function as an acquisition unit that acquires a first image captured on water, an image generation unit that generates a second image by applying an image effect to the first image that changes at least one of the number or shape of light-emitting regions with a predetermined brightness or higher, and a learning processing unit that performs learning processing of a learning model for detecting objects to be detected in an image using learning data including the second image.
[0015] It should be noted that the above summary of the invention does not list all the necessary features of the present invention. Furthermore, subcombinations of these features may also constitute an invention. [Brief explanation of the drawing]
[0016] [Figure 1] A detection system 1 according to an embodiment is shown. [Figure 2] This shows the operation of device 3 during the learning phase of learning model 35. [Figure 3] This shows the operation of device 3 during the operational phase of the learning model 35. [Figure 4] Examples of a computer 2200 in which multiple aspects of the present invention may be embodied in whole or in part are shown. [Modes for carrying out the invention]
[0017] Hereinafter, the present invention will be described through embodiments of the invention. However, the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential for the solution means of the invention.
[0018] <1. Detection system 1> FIG. 1 shows a detection system 1 according to this embodiment. The detection system 1 performs a learning process of a learning model 35 for detecting a detection target object in an image, and detects the detection target object using the learned learning model 35, and includes one or more cameras 2 and a device 3. The detection target object may be an object that can exist on the water surface (for example, on the sea surface), and may be, for example, a buoy, a person, a small ship, or the like.
[0019] <1.1. Camera 2> Each camera 2 performs imaging on the water (for example, on the sea) to generate a first image. In this embodiment, as an example, each camera 2 will be described as imaging a still image for each reference interval. However, it may be configured to image a still image according to an operator's operation, or may be configured to always image a moving image. When the camera 2 images a still image, the first image may be a still image, and when the camera 2 images a moving image, the first image may be a frame in the moving image.
[0020] Each camera 2 may be arranged on a ship, or may be arranged on a waterside or a water structure. The ship on which the camera 2 is arranged may be a ship larger than the detection target object. The waterside or water structure may be, for example, a lighthouse, a buoy (also referred to as a float), a harbor, a bridge, a breakwater, or the like. The imaging area of each camera 2 may be fixed or variable.
[0021] Each camera 2 may capture a first image through at least one light-transmitting member 10. In other words, the first image to be captured may be an image captured through the light-transmitting member 10. The light-transmitting member 10 may be a structure separate from the camera 2 and may be different from the lens of the camera 2. The light-transmitting member 10 may be disposed in front of the camera 2 to enable imaging while preventing water droplets from adhering to the camera 2. When the camera 2 is disposed in the steering room on a ship, the light-transmitting member 10 may be the front window or rear window of the steering room. When the camera 2 is disposed on a waterside or waterborne structure, the light-transmitting member 10 may be a window provided on the structure.
[0022] Each camera 2 may be a visible light camera, or may be a camera for infrared rays or ultraviolet rays (e.g., X-rays). When the detection system 1 includes a plurality of cameras 2, each camera 2 may have the same type (e.g., manufacturer or model number) as each other, or may be different. Each camera 2 may supply the first image to the device 3.
[0023] <1.2. Device 3> The device 3 includes an acquisition unit 30, a display unit 31, a labeling unit 32, an image generation unit 33, an image storage unit 34, a learning model 35, a learning processing unit 36, a supply unit 37, and a determination unit 38.
[0024] <1.2.1. Acquisition unit 30> The acquisition unit 30 acquires the first image captured on water. The acquisition unit 30 may acquire the first image from each camera 2. In this embodiment, as an example, the acquisition unit 30 is connected to each camera 2 by wire or wirelessly, and may acquire the first image each time the first image is captured by the camera 2. The first image may be attached with metadata indicating the identification information of the camera (also referred to as camera ID) of the camera 2 that is the imaging source. The acquisition unit 30 may supply the acquired first image to the display unit 31, the image storage unit 34, the image generation unit 33, and the supply unit 37.
[0025] <1.2.2. Display unit 31> The display unit 31 displays various information. For example, the display unit 31 may display the first image acquired by the acquisition unit 30.
[0026] <1.2.3. Label attachment section 32> The labeling unit 32 adds a label to the first image supplied from the camera 2 indicating whether or not a target object is present in the image. The labeling unit 32 may add labels in response to the operator's actions, such as checking the first image displayed on the display unit 31 or the field of view of the camera 2. For example, the labeling unit 32 may add a label to the first image indicating the presence of a target object for a period of time during which the operator continues to perform actions indicating the presence of a target object in the first image. The labeling unit 32 may also add a label to the first image indicating the absence of a target object for a period of time during which the operator continues to perform actions indicating the absence of a target object in the first image, or for a period during which the operator does not perform actions indicating the presence of a target object in the first image. The labeling unit 32 may add a label to the first image for each camera 2.
[0027] The labeling unit 32 may add a label to the first image acquired by the acquisition unit 30. In this embodiment, as an example, the labeling unit 32 may add a label to the first image supplied from the acquisition unit 30 to at least the image storage unit 34 and the image generation unit 33.
[0028] <1.2.4. Image generation unit 33> The image generation unit 33 generates a second image by applying an image effect (also referred to as an image effect) to the first image. The image generation unit 33 may apply the image effect to the entire area of the first image. The image generation unit 33 may apply an image effect to the first image that changes at least one of the number or shape of light-emitting regions that are above a predetermined brightness (also referred to as a reference brightness).
[0029] The luminescent region may be the so-called overexposed region. The reference brightness may be the maximum brightness (also called the saturation brightness), or it may be any brightness lower than the maximum brightness.
[0030] Changing the number of light-emitting regions means either increasing or decreasing the number of light-emitting regions. The image generation unit 33 may decrease the number of light-emitting regions by lowering the brightness of the light-emitting regions in the first image, or it may increase the number of light-emitting regions by increasing the brightness of at least some of the regions in the first image that are below the reference brightness (also called non-light-emitting regions). If the reference brightness is lower than the maximum brightness, the image generation unit 33 may make the brightness different between two or more light-emitting regions generated in the second image.
[0031] Changing the shape of the light-emitting region means changing the shape of the light-emitting region to a shape that is neither identical nor similar. The image generation unit 33 may change the shape of the light-emitting region by increasing the brightness of the non-light-emitting region adjacent to the light-emitting region in the first image, or by decreasing the brightness of the non-light-emitting region adjacent to the light-emitting region in the first image. When generating multiple light-emitting regions in the second image, the image generation unit 33 may make at least one of the shapes or sizes of the two or more light-emitting regions generated in the second image different.
[0032] The image generation unit 33 may further apply an image effect that changes at least one of the position or size of the light-emitting regions in the first image to generate a second image. Changing the position of the light-emitting regions means changing their position in the image while maintaining the shape of the light-emitting regions. The image generation unit 33 may change the position of the light-emitting regions by lowering the brightness of the light-emitting regions in the first image to make them non-light-emitting regions, and by increasing the brightness of the non-light-emitting regions to make them light-emitting regions. Changing the size of the light-emitting regions means increasing or decreasing the area of the light-emitting regions. For example, in this embodiment, the image generation unit 33 may generate a second image by randomly changing the number, shape, and size of the light-emitting regions in the first image.
[0033] The image generation unit 33 may further apply an image effect to the first image that changes at least one of the position, number, and size of water droplets attached to the light-transmitting member 10 to generate a second image. Changing the position of water droplets attached to the light-transmitting member 10 may mean changing the position in the image while maintaining the shape of the water droplets. The image generation unit 33 may remove water droplets from the position where they were captured attached to the light-transmitting member 10 in the first image, and add water droplets to other positions to generate a second image. Changing the number of water droplets may mean increasing or decreasing the number of water droplets. The image generation unit 33 may add water droplets to the area of the light-transmitting member 10 in the first image to generate a second image, or it may remove water droplets from the area of the light-transmitting member 10 to generate a second image. Changing the size of water droplets may mean increasing or decreasing the area of the water droplets. The image generation unit 33 may generate a second image by changing the shape of the water droplet to a different shape with a different area, or it may generate a second image by increasing or decreasing the area while maintaining the similar shape of the water droplet.
[0034] The image generation unit 33 may further apply an image effect to the first image according to imaging conditions in which at least one of the meteorological conditions and oceanographic conditions differs from the imaging conditions of the first image, in order to generate a second image. Meteorological conditions may be conditions determined by meteorological elements such as temperature, atmospheric pressure, wind direction, wind speed, precipitation (snowfall) amount, cloud cover, solar radiation, and visibility, and are also called meteorological conditions. Oceanographic conditions may be conditions determined by natural phenomena occurring in the sea (for example, sea surface wind, significant wave height, wind waves, swells, ocean currents, etc.), and are also called oceanographic conditions.
[0035] Applying an image effect to the first image that corresponds to imaging conditions different from the imaging conditions of the first image means making the first image an image taken under imaging conditions different from the imaging conditions of the first image. Applying an image effect to the first image that corresponds to imaging conditions different from the imaging conditions of the first image (weather conditions) means making the first image, taken under first weather conditions, an image taken under second weather conditions different from the first weather conditions. Applying an image effect to the first image that corresponds to imaging conditions different from the imaging conditions of the first image (sea conditions) means making the first image, taken under first sea conditions, an image taken under second sea conditions different from the first sea conditions. The image generation unit 33 may generate the second image by applying an image effect to the first image that corresponds to imaging conditions under which the detected object may be captured less clearly than under the imaging conditions of the first image. For example, if imaging is being taken under sunny and calm conditions, the image generation unit 33 may apply an image effect to the first image corresponding to imaging conditions where there is rainfall, high waves, and a lot of sea spray, thereby generating a second image with low brightness and rain and sea spray droplets adhering to the light-transmitting member 10.
[0036] The image generation unit 33 may generate multiple second images that are different from each other from a single first image. The image generation unit 33 may generate multiple different second images by applying the same type of image effect to the same first image, or it may generate multiple different second images by applying different types of image effects to the same first image.
[0037] The image generation unit 33 may add a label to each generated second image that has the same content as the original first image. A label with the same content as the first image may be a label indicating that an object to be detected is present, if such a label was added to the first image, or a label indicating that an object to be detected is not present, if such a label was added to the first image. The image generation unit 33 may add metadata indicating the same camera ID as the original first image to each generated second image. The image generation unit 33 may supply each generated second image to the image storage unit 34.
[0038] <1.2.5. Image Storage Unit 34> The image storage unit 34 stores each first image supplied from the acquisition unit 30 and each second image supplied from the image generation unit 33.
[0039] <1.2.6. Learning Model 35> The learning model 35 detects objects to be detected in an image. The learning model 35 may output a detection result indicating whether or not objects to be detected have been detected in the image, depending on the image data that is input. In this embodiment, as an example, the learning model 35 may output the detection result along with the camera ID when an image with metadata indicating the camera ID is input. The learning model 35 may output the detection result including the location information of the objects to be detected in the image, depending on whether objects to be detected have been detected in the image.
[0040] The learning model 35 may be trained by the learning processing unit 36. The learning model 35 may be a model obtained by machine learning and may be stored in a memory unit (not shown), but is not limited to this.
[0041] <1.2.7. Learning Processing Unit 36> The learning processing unit 36 uses the learning data including the second image to perform the learning process of the learning model 35. The learning processing unit 36 may further use the learning data including the first image to train the learning model 35. The learning processing unit 36 may perform the learning process using the first and second images stored in the image storage unit 34. The learning processing unit 36 may perform the learning process using machine learning, such as deep learning, as an example.
[0042] <1.2.8. Supply section 37> The supply unit 37 supplies a first image to the learning model 35. The supply unit 37 may supply the learning model 35, which has undergone learning processing by the learning processing unit 36, with a first image newly acquired by the acquisition unit 30. As a result, the learning model 35 may output a detection result indicating whether or not an object to be detected has been detected in the first image data. The first image supplied from the supply unit 37 to the learning model 35 does not need to be labeled by the labeling unit 32.
[0043] <1.2.9. Judgment section 38> The determination unit 38 uses the learning model 35, which has been trained by the learning processing unit 36, to determine whether or not a target object exists in the first image newly acquired by the acquisition unit 30. The determination unit 38 may make a determination based on the detection result output from the learning model 35 in response to the supply unit 37 supplying the learning model 35 with the first image.
[0044] The determination unit 38 may display the determination result on the display unit 31. This may provide notification when it is determined that an object to be detected exists in the first image. The determination unit 38 may include the location information of the object to be detected in the determination result and display it. This may provide notification of the location of the object to be detected in the first image.
[0045] The determination unit 38 may output the camera ID of the camera 2 that captured the first image, along with the determination result that an object to be detected exists in the first image. For example, the determination unit 38 may output the camera ID output from the learning model 35 in association with the detection result that an object to be detected exists in the first image, along with the detection result. This allows the system to be informed of which camera 2's field of view the object to be detected is located within.
[0046] The determination unit 38 may supply the labeling unit 32 with a determination result indicating that an object to be detected exists in the first image, along with the camera ID of the camera 2 that captured the first image. As a result, the labeling unit 32 may add a label to the first image for the duration that the determination result indicating the presence of an object to be detected in the first image persists. The learning process of the learning model 35 may then be further executed using the first and second images, which have been labeled according to the determination result of the determination unit 38.
[0047] According to the apparatus 3 described above, a second image is generated by applying an image effect that changes at least one of the number or shape of the light-emitting regions to the first image captured on the water surface. The learning model 35, which detects objects to be detected within the image, is then trained using training data including the second image. Therefore, since the learning model 35 can be trained using the second image, which has flicker due to reflected light from the water surface added by the image effect, the accuracy of object detection on the water surface by the learning model 35 can be improved.
[0048] Furthermore, since at least one of the shapes or sizes differs between the two or more luminescent regions generated in the second image, a second image with diverse luminescent regions is generated. Therefore, the detection accuracy of objects on the water surface by the learning model 35 can be further improved.
[0049] Furthermore, since an image effect that changes at least one of the position or size of the luminescent region in the first image is applied to generate the second image, a second image with a more diverse range of luminescent regions is generated. Therefore, the detection accuracy of objects on the water surface by the learning model 35 can be further improved.
[0050] Furthermore, an image effect is applied to change at least one of the position, number, and size of the water droplets adhering to the light-transmitting member 10, thereby generating a second image. Consequently, the learning model 35 can be trained using the second image, which incorporates the water droplets adhering to the light-transmitting member 10 through an image effect, further improving the detection accuracy of objects on the water surface by the learning model 35.
[0051] Furthermore, the second image is generated by applying image effects to the first image that correspond to imaging conditions where at least one of the weather and oceanographic conditions differs from those of the first image. Therefore, since the learning model 35 can be trained using the second image to which image effects from various weather and oceanographic conditions have been added, the detection accuracy of objects on the water by the learning model 35 can be further improved.
[0052] Furthermore, since multiple second images are generated from a single first image, the learning efficiency of the learning model 35 can be improved. Consequently, the detection accuracy of objects on the water surface by the learning model 35 can be further enhanced.
[0053] Furthermore, since the same label as the first image is added to the second image, the operator is saved the trouble of having to check the second image and add a label. In addition, since the learning process can be performed using a label that accurately indicates whether or not the target object is present, the accuracy of detecting targets on the water surface by the learning model 35 can be further improved.
[0054] Furthermore, since the learning model 35 is trained using training data including the first image, the training efficiency of the learning model 35 can be improved compared to when training is performed using only the second image. Therefore, the detection accuracy of objects on the water surface by the learning model 35 can be further improved.
[0055] Furthermore, since the learning model 35, which has undergone training, is used to determine whether or not a target object exists in the newly acquired first image, it is possible to detect targets on the water with high accuracy.
[0056] <2. Operation of Device 3> <2.1. Actions during the learning phase> Figure 2 shows the operation of device 3 during the learning phase of the learning model 35. Device 3 performs the learning process of the learning model 35 by processing steps S11 to S21.
[0057] In step S11, the acquisition unit 30 acquires a first image captured on the water. In this embodiment, as an example, the acquisition unit 30 may acquire the first image from each camera 2.
[0058] In step S13, the labeling unit 32 adds a label to the first image acquired in step S11 indicating the presence or absence of the object to be detected. The labeling unit 32 may add labels according to the operator's actions.
[0059] In step S15, the image generation unit 33 generates a second image by applying an image effect to the first image. The applied image effect may be an image effect that changes at least one of the number or shape of the light-emitting regions. In addition, the applied image effect may be an image effect that changes at least one of the position or size of the light-emitting regions in the first image, or an image effect that changes at least one of the position, number, and size of water droplets attached to the light-transmitting member 10, or an image effect that corresponds to imaging conditions in which at least one of the weather conditions and ocean conditions differs from the imaging conditions of the first image. The image generation unit 33 may generate the second image using the first image acquired in the most recent step S11 as a base. The image generation unit 33 may generate one or more second images each time the processing of step S15 is performed. The image generation unit 33 may label each generated second image in the same way as the original first image.
[0060] In step S17, the image generation unit 33 may determine whether the number of second images generated from a single first image has reached a first criterion number. The first criterion number may be set in advance to any value. If it is determined that the number of second images generated from a single first image is less than the first criterion number (step S17; No), the process may proceed to step S15. In this case, in the process of step S15, a new second image may be generated based on the same first image as in the previous step S15. If it is determined that the number of second images generated from a single first image is equal to or greater than the first criterion number (step S17; Yes), the process may proceed to step S19.
[0061] In step S19, the image generation unit 33 may determine whether the total number of generated second images has reached the second criterion number. The image generation unit 33 may determine whether the total number of second images generated from multiple first images has reached the second criterion number. The second criterion number may be set in advance to any value greater than the first criterion number, for example, it may be an integer multiple of the first criterion number. If it is determined that the total number of second images generated from multiple first images is less than the second criterion number (step S19; No), the process may proceed to step S11. In this case, a new first image may be obtained in the process of step S11. If it is determined that the total number of second images generated from multiple first images is equal to or greater than the second criterion number (step S19; Yes), the process may proceed to step S21.
[0062] In step S21, the learning processing unit 36 uses the training data, including the generated second image, to train the learning model 35. The learning processing unit 36 may further train the learning model 35 using the training data, including the first image. The learning processing unit 36 may use the content of the labels attached to each image as training data to perform the training. In this embodiment, as an example, the learning processing unit 36 may train the learning model 35 so that objects are detected in images labeled as having an object present, and objects are not detected in images labeled as not having an object present.
[0063] <2.2. Operation during the operational phase> Figure 3 shows the operation of device 3 during the operational phase of the learning model 35. Device 3 determines whether or not a target object exists within the field of view of camera 2 by performing the processes from step S31 to step S35. Note that the learning process of the learning model 35 may be completed at the start of operation.
[0064] In step S31, the acquisition unit 30 acquires a first image captured on the water. The acquisition unit 30 may acquire the first image in the same manner as in step S11 described above. The acquired first image may be displayed on the display unit 31.
[0065] In step S33, the supply unit 37 supplies the first image to the learning model 35. The supply unit 37 may supply the first image data newly acquired by the acquisition unit 30 in step S31 to the learning model 35. As a result, the learning model 35 may output a detection result indicating whether or not an object to be detected has been detected in the image of the first image data.
[0066] In step S35, the determination unit 38 uses the learning model 35 to determine whether or not a target object exists in the newly acquired first image. The determination unit 38 may make the determination based on the detection result output from the learning model 35 in response to the supply of the first image to the learning model 35 in step S33. The determination unit 38 may display the determination result on the display unit 31. After the processing of step S35 is completed, the process may proceed to step S31 described above.
[0067] <3. Variant> In the above embodiment, the device 3 was described as comprising a display unit 31, a labeling unit 32, an image storage unit 34, a supply unit 37, and a determination unit 38, but it is not necessary to include any of these. For example, if the device 3 does not include a labeling unit 32, the acquisition unit 30 may acquire a first image with a label already attached. If the device 3 does not include a supply unit 37 and a determination unit 38, the learned model 35 that has undergone learning processing may be output to an external device for use.
[0068] Furthermore, although the image generation unit 33 was described as applying the image effect to the entire area of the first image, the image effect may be applied to at least a portion of the area of the object to be detected in the first image. This makes it possible to change the ease of detection of the object in the second image. Therefore, by training the learning model 35 using such a second image, the detection accuracy of the object on the water by the learning model 35 can be further improved. Note that the area of the object to be detected in the first image may be specified by the operator. If the learning model 35 has already been generated, the area of the object to be detected in the first image may be specified by the position information of the object to be detected included in the detection result by the learning model 35.
[0069] Furthermore, although the acquisition unit 30 was described as acquiring the first image from camera 2, it may also acquire the first image from a server that stores the first image, or it may acquire the first image from a relay device that collects the first image from multiple cameras 2.
[0070] Furthermore, various embodiments of the present invention may be described with reference to flowcharts and block diagrams, where a block may represent (1) a stage in a process in which an operation is performed or (2) a section of a device having the role of performing an operation. Specific stages and sections may be implemented by dedicated circuits, programmable circuits supplied with computer-readable instructions stored on a computer-readable medium, and / or processors supplied with computer-readable instructions stored on a computer-readable medium. Dedicated circuits may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. Programmable circuits may include reconfigurable hardware circuits, including logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logic operations, flip-flops, registers, memory elements such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc.
[0071] Computer-readable media may include any tangible device capable of storing instructions to be executed by a suitable device, and as a result, computer-readable media having instructions stored therein will comprise a product containing instructions that can be executed to create means for performing operations specified in a flowchart or block diagram. Examples of computer-readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital versatile disk (DVD), Blu-ray (RTM) disk, memory stick, integrated circuit card, etc.
[0072] Computer-readable instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk®, Java®, C++, and traditional procedural programming languages such as the C programming language or similar programming languages.
[0073] Computer-readable instructions may be provided locally or via a wide area network (WAN), such as a local area network (LAN) or the internet, to a processor or programmable circuit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, and these instructions may be executed to create means for performing operations specified in a flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.
[0074] Figure 4 shows an example of a computer 2200 in which multiple aspects of the present invention may be embodied in whole or in part. A program installed on the computer 2200 can cause the computer 2200 to function as an operation or one or more sections of an apparatus according to an embodiment of the present invention, or to execute such operation or one or more sections, and / or to cause the computer 2200 to execute a process or a stage of such process according to an embodiment of the present invention. Such a program may be executed by the CPU 2212 to cause the computer 2200 to perform a particular operation associated with some or all of the blocks in the flowcharts and block diagrams described herein.
[0075] The computer 2200 according to this embodiment includes a CPU 2212, RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected by a host controller 2210. The computer 2200 also includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes legacy input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.
[0076] The CPU 2212 operates according to programs stored in the ROM 2230 and RAM 2214, thereby controlling each unit. The graphics controller 2216 retrieves image data generated by the CPU 2212 from a frame buffer provided in RAM 2214 or from itself, and displays the image data on the display device 2218.
[0077] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides them to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from the IC card and / or writes programs and data to the IC card.
[0078] The ROM 2230 stores boot programs and / or programs that depend on the computer 2200's hardware, which are executed by the computer 2200 when activated. The input / output chip 2240 may also connect various input / output units to the input / output controller 2220 via parallel ports, serial ports, keyboard ports, mouse ports, etc.
[0079] The program is provided on a computer-readable medium such as a DVD-ROM 2201 or an IC card. The program is read from the computer-readable medium and installed on a hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable medium, and executed by the CPU 2212. The information processing described within these programs is read by the computer 2200, resulting in coordination between the program and the various types of hardware resources described above. The apparatus or method may be configured to realize the manipulation or processing of information in accordance with the use of the computer 2200.
[0080] For example, when communication is performed between a computer 2200 and an external device, the CPU 2212 may execute a communication program loaded into RAM 2214 and, based on the processing described in the communication program, instruct the communication interface 2222 to perform communication processing. Under the control of the CPU 2212, the communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in a recording medium such as RAM 2214, a hard disk drive 2224, a DVD-ROM 2201, or an IC card, transmits the read transmission data to the network, or writes received data received from the network to a reception buffer processing area provided on the recording medium.
[0081] Furthermore, the CPU 2212 may read all or necessary parts of files or databases stored on external storage media such as the hard disk drive 2224, DVD-ROM drive 2226 (DVD-ROM 2201), or IC card into the RAM 2214, and perform various types of processing on the data in the RAM 2214. The CPU 2212 then writes the processed data back to the external storage media.
[0082] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and subjected to information processing. The CPU 2212 may perform various types of processing on the data read from RAM 2214, including various types of operations, information processing, conditional judgments, conditional branching, unconditional branching, information retrieval / replacement, etc., as described throughout this disclosure and specified by the program instruction sequence, and write the results back to RAM 2214. The CPU 2212 may also retrieve information in files, databases, etc., within the recording medium. For example, if multiple entries are stored in the recording medium, each having an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU 2212 may search among the multiple entries for an entry that matches the condition for which the attribute value of the first attribute is specified, read the attribute value of the second attribute stored in that entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.
[0083] The programs or software modules described above may be stored on or near computer 2200 on a computer-readable medium. Alternatively, recording media such as hard disks or RAM provided within a server system connected to a dedicated communication network or the Internet can be used as computer-readable media, thereby providing programs to computer 2200 via the network.
[0084] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be clear from the claims that such modified or improved forms may also be included in the technical scope of the present invention.
[0085] It should be noted that the execution order of operations, procedures, steps, and stages in the apparatus, systems, programs, and methods shown in the claims, specifications, and drawings is not explicitly stated as "before," "prior to," etc., and that these can be implemented in any order unless the output of a previous process is used in a later process. Even if the operation flow in the claims, specifications, and drawings is described using phrases such as "first," "next," etc. for convenience, it does not mean that it is essential to perform the operations in that order. [Explanation of Symbols]
[0086] 1. Detection System 2 cameras 3 equipment 10 Light-transmitting member 30 Acquisition Department 31 Display section 32 Label attachment section 33 Image generation unit 34 Image storage unit 35 Learning Models 36 Learning Processing Unit 37 Supply section 38 Judgment section 2200 Computers 2201 DVD-ROM 2210 Host Controller 2212 CPU 2214 RAM 2216 Graphics Controller 2218 Display Devices 2220 Input / Output Controller 2222 Communication Interface 2224 Hard Disk Drive 2226 DVD-ROM drive 2230 ROM 2240 Input / Output Chip 2242 keyboard
Claims
1. An acquisition unit that acquires a first image captured on the water, An image generation unit generates a second image by applying an image effect to the first image that changes at least one of the number or shape of light-emitting regions with a predetermined brightness or higher, A learning processing unit performs training on a learning model that detects objects to be detected within an image using the training data including the second image, A device equipped with the following features.
2. The apparatus according to claim 1, wherein the image generation unit causes at least one of the shapes or sizes of two or more light-emitting regions generated in the second image to differ.
3. The apparatus according to claim 1, wherein the image generation unit further applies an image effect that changes at least one of the position or size of the light-emitting region in the first image to generate the second image.
4. The first image is an image captured through at least one light-transmitting member, The apparatus according to claim 1, wherein the image generation unit further applies an image effect to the first image that changes at least one of the position, number, and size of water droplets adhering to the light-transmitting member to generate the second image.
5. The apparatus according to claim 1, wherein the image generation unit further applies an image effect to the first image according to imaging conditions which differ from the imaging conditions of the first image, at least one of which is weather conditions and ocean conditions, to generate a second image.
6. The apparatus according to claim 1, wherein the image generation unit applies the image effect to at least a portion of the region of the object to be detected in the first image.
7. The apparatus according to claim 1, wherein the image generation unit generates a plurality of second images from a single first image to which different image effects are applied.
8. The first image is accompanied by a label indicating whether or not the object to be detected is present within the first image. The apparatus according to claim 1, wherein the image generation unit adds a label having the same content as the first image to the second image.
9. The apparatus according to claim 1, wherein the learning processing unit further uses the learning data including the first image to train the learning model.
10. The apparatus according to any one of claims 1 to 9, further comprising a determination unit that determines whether or not the object to be detected exists in the first image newly acquired by the acquisition unit, using the learning model that has been trained by the learning processing unit.
11. The acquisition stage involves acquiring the first image captured on the water, An image generation step of generating a second image by applying an image effect to the first image that changes at least one of the number or shape of light-emitting regions with a predetermined brightness or higher, A learning process step in which a learning model for detecting objects within an image is trained using the training data including the second image, A method for providing this.
12. Computers, An acquisition unit that acquires a first image captured on the water, An image generation unit generates a second image by applying an image effect to the first image that changes at least one of the number or shape of light-emitting regions with a predetermined brightness or higher, A learning processing unit performs training on a learning model that detects objects within an image using the training data including the second image. A program that makes it function as such.