Image provision system and image provision method

Abstract

To provide an image providing system and an image providing method, capable of reducing an amount of data of captured images to be transmitted while maintaining required information.SOLUTION: An image providing system 1 includes: a captured image obtaining unit 11 that obtains a captured image by a camera 51 provided on a moving body 50; a use purpose setting unit 12 that sets a use purpose of the captured image; an image discrimination section 13 that discriminates whether or not an image portion of a specific object associated with the use purpose is included in the captured image; an image processing unit 15 that, when it is discriminated that the image portion of the specific object is included in the captured image by the image discrimination section 13, executes masking processing for reducing an amount of information in areas of the captured images except for a specific area including the image portion of the specific object; and an image transmission unit 16 that transmits the captured image subjected to the masking processing to a predetermined destination 210, 211, 212, 213.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image providing system and an image collection method.

Background Art

[0002] Conventionally, a system for collecting images captured by a drive recorder mounted on a vehicle has been proposed (see, for example, Patent Document 1). The above system transmits a signal to the vehicle, specifying a location and requesting the transmission of image data of that location. The vehicle extracts image data that matches the request from the image data captured and stored by the drive recorder and transmits it to the above system.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As described above, when transmitting a captured image from a moving body such as a vehicle to a destination system, it is desirable to reduce the data amount of the transmitted captured image in consideration of communication costs and communication traffic. However, information that meets the purpose of use of the captured image needs to be maintained in the data. The present invention has been made in view of such a background, and an object thereof is to provide an image providing system and an image providing method capable of reducing the data amount of a captured image transmitted from a moving body while maintaining necessary information.

Means for Solving the Problems

[0005] As a first embodiment for achieving the above objective, an image acquisition unit that acquires images captured by a camera mounted on a mobile body, and for the purpose of using the captured images, a specific object installed on the side of the road on which the mobile body is traveling. Type and Exterior check Contents and A unit for setting the purpose of use, an image determination unit for determining whether or not the captured image contains an image portion of the specified object, and an image processing unit that, when the image determination unit determines that the captured image contains an image portion of the specified object, performs a masking process to reduce the amount of information in areas of the captured image other than the area containing the image portion of the specified object. The purpose of use setting unit An example of an image provision system is one that includes a masking process determination unit that determines the content of the masking process based on the type of specific object included in the above-mentioned purpose of use and the content of the appearance check, and an image transmission unit that transmits the captured image with the masking process applied to a predetermined recipient.

[0006] In the above-described image provision system, the image processing unit may be configured to perform a process to convert areas of the captured image other than the specific area into a gray image as the masking process when the captured image is a color image.

[0007] In the image provision system described above, the image transmission unit may be configured to transmit the captured image, which has undergone the masking process, to the recipient along with information indicating the purpose of use.

[0009] In the above-described image provision system, the purpose of use setting unit may set at least one of a plurality of purposes of use having priority, and the masking processing determination unit may, when the image discrimination unit determines that the captured image contains image portions of a plurality of specific objects associated with one of the plurality of purposes of use, determine the content of the masking processing according to the purpose of use with the highest priority among the purposes of use associated with one of the plurality of specific objects. In the image provision system described above, the appearance check may include checking for scratches on the specified object, checking for obstacles attached to the specified object, and checking for peeling paint on the specified object.

[0010] As a second aspect for achieving the above objective, a computer-based image provision method comprising: an image acquisition step of acquiring images captured by a camera mounted on a mobile body; and, for the purpose of using the captured images, a method for providing images of specific objects installed on the side of the road on which the mobile body is traveling. Type and Exterior check Contents and A step to set the purpose of use, and the captured image, The aforementioned An image discrimination step of determining whether or not an image portion of a specific object is included; and an image processing step of performing a masking process to reduce the amount of information in the area of ​​the captured image other than the specific area containing the image portion of the specific object when the image discrimination step determines that the captured image contains an image portion of the specific object. The purpose of use setting step described above An image provision method includes a masking process determination step, which determines the content of the masking process based on the type of specific object included in the above-mentioned purpose of use and the content of the appearance check, and an image transmission step, which transmits the captured image that has undergone the masking process to a predetermined recipient. [Effects of the Invention]

[0011] According to the above image provision system and image provision method, the amount of data of the captured images to be transmitted can be reduced while maintaining the necessary information. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 is a diagram showing the configuration of the image provision system. [Figure 2] Figure 2 is an explanatory diagram of the settings table for the purpose of using the captured images and the masking process. [Figure 3] Figure 3 is an explanatory diagram of the masking process, which grays out areas other than a specific area that includes the image portion of a specific object. [Figure 4] Figure 4 is an explanatory diagram of the masking process, which involves cropping out areas other than a specific area that includes the image portion of a particular object. [Figure 5]Figure 5 is a flowchart of the process for transmitting captured images. [Modes for carrying out the invention]

[0013] [1. Image Provision System Configuration] Referring to Figures 1 to 4, the configuration of the image provision system 1 of this embodiment will be described. The image provision system 1 is mounted on a vehicle 50. The vehicle 50 is equipped with a camera 51 that photographs the area around the vehicle 50, a GNSS (Global Navigation Satellite System) sensor 53 that detects the position of the vehicle 50, and a radar 52 that detects the positions of objects present around the vehicle 50. The camera 51 is a color camera. The vehicle 50 corresponds to the mobile body of this disclosure.

[0014] Image provision system 1 is a computer system equipped with a processor 10, memory 20, communication unit 30, etc. The processor 10 corresponds to the computer in this disclosure. Image provision system 1 communicates with power company server 210, post office server 211, maintenance company server 212, repair company server 213, etc. via a communication network 200 using the communication unit 30. Power company server 210, post office server 211, maintenance company server 212, and repair company server 213 correspond to the recipients in this disclosure.

[0015] Figure 1 shows a situation in which a vehicle 50 is driving along a road and taking pictures of objects installed on the side of the road with a camera 51. In this embodiment, a utility pole and a mailbox are set as the objects to be photographed, and Figure 1 shows the situation in which the image provision system 1 is taking pictures of a utility pole 100 and a mailbox 110 on the side of the road. The utility pole 100 and the mailbox 110 are installed on the opposite side of the lane in which the vehicle 50 is driving. For the utility pole 100, the presence or absence of obstacles 101 and damage 102 is checked from the captured image, and for the mailbox 110, damage and peeling paint are checked from the captured image.

[0016] The memory 20 stores a control program 21 of the image providing system 1 and usage purpose data 22 in which the usage purpose of the captured image by the camera 51 and the like are recorded. As shown in FIG. 2, the usage purpose data 22 includes the usage purpose (check for damage, check for obstacles) corresponding to the type of the imaging target part (post box, utility pole), the priority of the usage purpose (for example, 5 levels from 1 to 5, with 5 being the highest), the imaging conditions (proximity, remote), the masking method (trimming, graying of the surroundings), and the destination of provision (power company server 210, post office server 211, maintenance company server 212, repair company server 213).

[0017] The processor 10 functions as an imaging image acquisition unit 11, a usage purpose setting unit 12, an image discrimination unit 13, a masking process determination unit 14, an image processing unit 15, and an image transmission unit 16 by reading and executing the control program 21 stored in the memory 20.

[0018] Here, the process executed by the imaging image acquisition unit 11 corresponds to the imaging image acquisition step in the image providing method of the present disclosure, and the process executed by the usage purpose setting unit 12 corresponds to the usage purpose setting step in the image providing method of the present disclosure. The process executed by the image discrimination unit 13 corresponds to the image discrimination step in the image providing method of the present disclosure, and the process executed by the image processing unit 15 corresponds to the image processing step in the image providing method of the present disclosure. The process executed by the image transmission unit 16 corresponds to the image transmission step in the image providing method of the present disclosure.

[0019] The imaging image acquisition unit 11 captures the surroundings of the vehicle 50 by the camera 51 and stores the data of the captured image in the memory 20. The imaging image acquisition unit 11 stores the captured image and the data of the imaging conditions in association with each other in the memory 20. The imaging conditions include, for example, the following (1) to (5). (1) Specifications of the camera 51 (resolution, focal length of the lens, mounting position of the camera 51 on the vehicle 50, etc.). (2) Position of the vehicle 50 at the time of imaging. The position of the vehicle 50 is detected by the GNSS sensor 53. (3) Imaging date and time. (4) The position of the object captured in the image relative to the vehicle 50, and the distance between the object and the vehicle 50. The position of the object relative to the vehicle 50, and the distance between the vehicle 50 and the object are calculated based on the detection information from the radar 52. The position of the object relative to the vehicle 50 may be not only the relative / absolute position in the camera 51 or the coordinate system in real space, but also the "direction" of the object as seen from the vehicle 50 or the camera 51. (5) The position of the object to be monitored on the road and the direction of travel of the vehicle 50 on the road. The direction of travel of the vehicle 50 is recognized by the change in the position of the vehicle 50 detected by the GNSS sensor 53.

[0020] The purpose of use setting unit 12 refers to the purpose of use data 22 and sets the purpose of use of the captured image (in this embodiment, checking for scratches and defects on utility poles and checking for scratches on mailboxes). The image discrimination unit 13 determines whether the captured image contains image portions of specific objects (in this embodiment, utility poles and mailboxes) associated with the purpose of use. The image discrimination unit 13 determines whether the captured image contains image portions of specific objects by searching for image portions of specific objects using known search methods such as template matching and feature extraction.

[0021] The masking process determination unit 14 determines the content of the masking process based on the purpose of use of the captured image set by the purpose of use setting unit 12. For example, in the settings shown in Figure 2, if the purpose of use is to check for obstacles such as utility poles, the masking process of graying out the surrounding area is determined. Also, if the purpose of use is to check for scratches on a mailbox, the masking process of cropping is determined.

[0022] The image processing unit 15 executes the masking process determined by the masking process determination unit 14. Here, Figure 3 is an explanatory diagram of the surrounding gray masking process, where 300 shows the captured image (original image) before the masking process is performed, and 310 shows the captured image (masked image) after the masking process is performed. The original image 300 includes the image portion 301 of a utility pole. The original image 300 is a color image, and the image processing unit 15 converts it to a gray image by averaging the R, B, and G gradations (e.g., 8 bits) of each pixel in the color image and changing them into gray gradation pixels. This reduces the amount of data for each pixel from 24 bits (8 bits × 3) to 8 bits.

[0023] The image processing unit 15 sets a specific area 311 that includes the image portion 301 of the utility pole, and converts the area 312 other than the specific area 311 into a grayscale image. As a result, the amount of information in the area 312 other than the specific area 311 is reduced, and the data size of the masked image 310 is reduced compared to the original image 300. In the masked image 310, some information can be obtained from the grayscale image even for areas other than the specific area 311.

[0024] Figure 4 is an explanatory section for the trimming masking process, and the original image 300 is the same image as in Figure 3. The image processing unit 15 deletes (removes information from) areas 322 other than the specific area 321 which includes the image portion 301 of the utility pole, and generates a masked image 320.

[0025] The image transmission unit 16 transmits the captured image, which has been masked by the image processing unit 15, to the recipients via the communication network 200 using the communication unit 30, along with information indicating the purpose of use and the information on the shooting conditions described above. As shown in Figure 2, when the purpose of use of the captured image is to check for damage to a mailbox, the recipients are the post office server 211 and the repair company server 213. When the purpose of use of the captured image is to check for obstacles on a utility pole, the recipients are the power company server 210 and the maintenance company server 212.

[0026] [3. Processing the transmission of captured images] The procedure for transmitting captured images, as performed by the image provisioning system 1, will be explained according to the flowchart shown in Figure 5. The image provisioning system 1 repeatedly performs the process shown in the flowchart in Figure 5 while the vehicle 50 is in use.

[0027] In step S1 of Figure 5, the purpose of use setting unit 12 refers to the purpose of use data 22 and sets the purpose of use of the captured images in accordance with the request from the recipient of the captured images. In the following step S2, the captured image acquisition unit 11 uses the camera 51 to photograph the area around the vehicle 50 and acquire the captured images. In this embodiment, since the purpose of use of the captured images is to check utility poles and mailboxes, the captured image acquisition unit 11 photographs the left and right sides (towards the shoulder of the road) of the vehicle 50.

[0028] In the next step S3, the image discrimination unit 13 searches the captured image for image portions of specific objects (in this embodiment, utility poles and mailboxes) that match the intended use. Then, in the following step S4, if the image discrimination unit 13 finds that the captured image contains image portions of specific objects, it proceeds to step S5. On the other hand, if the captured image does not contain image portions of specific objects, the image discrimination unit 13 proceeds to step S1, in which case the processing from step S1 onwards is executed again.

[0029] In step S5, the masking processing determination unit 14 determines, based on the image discrimination unit 13, whether or not multiple specific object image portions have been extracted from the captured image. If multiple specific object image portions have been extracted from the captured image, the masking processing determination unit 14 proceeds to step S10; if only one specific object image portion has been extracted from the captured image, it proceeds to step S6.

[0030] In step S6, the masking process determination unit 14 determines the content of the masking process based on the purpose of use of the captured image. For example, as shown in Figure 2, when the purpose of use of the captured image is to check for obstacles such as utility poles, the masking process determination unit determines that the surrounding graying masking process will be performed on the captured image.

[0031] Furthermore, in step S10, the masking process determination unit 14 determines the content of the masking process based on the highest priority purpose of use among the multiple purposes of use corresponding to each specific object. For example, in the situation shown in Figure 1, if an image is acquired that includes the image portion of the utility pole 100 and the image portion of the mailbox 110, and the purposes of use are set to check for scratches on the mailbox and check for obstacles on the utility pole, the masking process determination unit 14 determines that the masking process to be performed on the acquired image is a grayed-out masking process corresponding to the high priority check for obstacles on the utility pole.

[0032] In step S7, the image processing unit 15 performs masking on the captured image according to the masking process determination unit 14. In the next step S8, the image transmission unit 16 transmits the masked captured image, along with usage information indicating the purpose of use and shooting condition information indicating the shooting conditions, to the recipient.

[0033] [4. Other Embodiments] In the above embodiment, the purpose of using the captured images was exemplified as checking the condition of utility poles and mailboxes installed on the road (checking for damage and obstacles). However, the purpose of using the captured images may also be set to check the condition of other types of objects installed on the road, such as traffic lights, road signs, and guardrails. Furthermore, the purpose of using the captured images may also be set for objects other than those installed on the road.

[0034] In the above embodiment, a masking process determination unit 14 is provided to determine the content of the masking process according to the intended use of the captured image. However, the masking process determination unit 14 may be omitted, and the content of the masking process may not be changed.

[0035] In the above embodiment, the purpose of use setting unit 12 may set the purpose of use of the captured images based on the driving conditions of the vehicle 50 (driving speed, driving time, weather, temperature, location). The purpose of use setting unit 12 may set the purpose of use of the captured images as follows, for example. When the vehicle speed exceeds a predetermined speed, it is difficult to photograph objects near the vehicle 50, so the purpose of use is set to check the condition of distant structures such as transmission towers. When it is nighttime, it is difficult to photograph distant objects, so the purpose of use is set to check the condition of mailboxes and other objects located near vehicle 50. Since it is difficult to check for damage to the object under rainfall, the purpose of use will be to check for obstacles attached to the object. When temperatures are low and snow or ice accumulation makes it difficult to check for damage to the object, the purpose of use should be set to check for obstacles attached to the object. When vehicle 50 is driving in a town, it is possible to photograph the target object even at night due to streetlights, so the purpose of use is set to check for damage to the target object. On the other hand, when vehicle 50 is driving in the suburbs where there are no streetlights and it is difficult to photograph the target object, the purpose of use may not be set.

[0036] Furthermore, when driving in the overtaking lane on a multi-lane road, in other words, when driving in a lane other than the lane closest to the sidewalk (the leftmost lane in Japan), it may be acceptable not to specify a purpose of use, considering that vehicles traveling in the driving lane (the leftmost lane) may be captured in the image.

[0037] In the above embodiment, the masking process determination unit 14 performed surrounding graying and trimming as masking processes, but other processes may be performed. For example, the amount of information in areas other than the specific area may be reduced by a process that thins out pixels in areas other than the specific area.

[0038] In the above embodiment, an example was given in which the image provisioning system 1 is arranged separately from the camera 51 mounted on the vehicle 50. However, the image provisioning system 1 may also be configured as a function such as a drive recorder, which is housed in the same housing as the camera.

[0039] In the above embodiment, a vehicle 50 was shown as the mobile body of the disclosure, but the mobile body of the disclosure may be an aircraft, a ship, a drone, a robot, or the like.

[0040] Figure 1 is a schematic diagram showing the configuration of the image provisioning system 1 divided by its main processing content, in order to facilitate understanding of the present invention. The configuration of the image provisioning system 1 may be composed of other divisions. Furthermore, the processing of each component may be performed by one hardware unit or by multiple hardware units. Also, the processing of each component shown in the flowchart in Figure 5 may be performed by one program or by multiple programs.

[0041] [4. Configurations supported by the above embodiments] The above embodiment supports the following configuration: (Configuration 1) An image provision system comprising: an image acquisition unit that acquires images captured by a camera mounted on a mobile body; a purpose setting unit that sets the purpose of use of the captured images; an image determination unit that determines whether or not the captured images include an image portion of a specific object associated with the purpose of use; an image processing unit that, when the image determination unit determines that the captured images include an image portion of the specific object, performs a masking process to reduce the amount of information in areas of the captured images other than the specific area containing the image portion of the specific object; and an image transmission unit that transmits the masked captured images to a predetermined recipient. According to the image provision system of Configuration 1, the image processing unit performs a masking process that reduces the amount of information in areas of the captured image other than the specific area containing the image portion of the specific object. This makes it possible to reduce the amount of data in the captured image transmitted by the image transmission unit while maintaining the information necessary for the intended use.

[0042] (Configuration 2) The image processing unit of the image provision system of Configuration 1, wherein when the captured image is a color image, the image processing unit performs a process to convert areas of the captured image other than the specific area into a gray image. According to the image provision system in Configuration 2, it is possible to transmit captured images containing a certain amount of information even for areas other than the specific area.

[0043] (Configuration 3) The image transmission unit transmits the captured image, which has undergone the masking process, to the recipient along with the information indicating the purpose of use, according to Configuration 1 or Configuration 2. According to the image provision system in Configuration 3, by transmitting information about the purpose of use along with the captured image, it becomes easier for the recipient to utilize the captured image.

[0044] (Configuration 4) An image provision system according to any one of Configurations 1 to 3, comprising a masking process determination unit that determines the content of the masking process based on at least one of the purpose of use and the type of the specified object. According to the image provision system in configuration 4, masking processing suitable for the intended use or type of specific object can be performed on the captured image.

[0045] (Configuration 5) The image provision system according to Configuration 4, wherein the purpose of use setting unit sets at least one of a plurality of purposes of use having priority, and the masking processing determination unit determines the content of the masking process according to the purpose of use having the highest priority among the purposes of use associated with any of the plurality of purposes of use when the image discrimination unit determines that the captured image contains image portions of a plurality of specific objects associated with any of the plurality of purposes of use. According to the image provision system of Configuration 5, when a captured image contains image portions of multiple specific objects, the content of the masking process can be determined so as to maintain the amount of information in the image portion of the specific object with the highest priority for the applicable purpose of use.

[0046] (Configuration 6) An image provision method performed by a computer, comprising: an image acquisition step of acquiring an image captured by a camera mounted on a mobile body; a purpose setting step of setting the purpose of use of the captured image; an image determination step of determining whether or not the captured image includes an image portion of a specific object associated with the purpose of use; an image processing step of performing a masking process to reduce the amount of information in areas of the captured image other than the specific area including the image portion of the specific object when the image determination step determines that the captured image includes an image portion of the specific object; and an image transmission step of transmitting the captured image that has undergone the masking process to a predetermined recipient. By executing the image provision method of Configuration 6 using a computer, the same effects and benefits as the image provision system of Configuration 1 can be obtained. [Explanation of Symbols]

[0047] 1...Image provision system, 10...Processor, 11...Captured image acquisition unit, 12...Purpose of use setting unit, 13...Image discrimination unit, 14...Masking processing determination unit, 15...Image processing unit, 16...Image transmission unit, 20...Memory, 21...Control program, 22...Purpose of use data, 50...Vehicle (moving object), 51...Camera, 52...Radar, 53...GNSS sensor, 100...Utility pole, 101...Obstacle, 102...Scratch, 110...Mailbox, 200...Communication network, 210...Power company server, 211...Post office server, 212...Maintenance company server, 213...Repair company server.

Claims

1. An image acquisition unit that acquires images captured by a camera mounted on a mobile device, The purpose of use of the captured images is defined by a usage purpose setting unit that sets the type of specific object installed on the side of the road on which the moving object is traveling and the content of the visual inspection, An image discrimination unit that determines whether or not the captured image contains an image portion of the specific object, When the image discrimination unit determines that the captured image contains an image portion of the specified object, the image processing unit performs a masking process to reduce the amount of information in areas of the captured image other than the specified area containing the image portion of the specified object. A masking process determination unit determines the content of the masking process based on the type of specific object included in the purpose of use set by the purpose of use setting unit and the content of the appearance check, An image transmission unit that transmits the captured image, which has undergone the masking process, to a predetermined recipient. An image provisioning system equipped with the following features.

2. The image processing unit, when the captured image is a color image, performs the masking process by converting areas of the captured image other than the specified area into a grayscale image. The image provisioning system according to claim 1.

3. The image transmission unit transmits the captured image, which has undergone the masking process, to the recipient along with the usage information indicating the intended use. The image provisioning system according to claim 1 or claim 2.

4. The aforementioned purpose-setting unit sets at least one of the multiple purposes of use having priority, When the image discrimination unit determines that the captured image contains image portions of multiple specific objects associated with one of the multiple purposes of use, the masking processing determination unit determines the content of the masking process according to the purpose of use with the highest priority among the multiple purposes of use associated with one of the multiple specific objects. The image provision system according to any one of claims 1 to 3.

5. The aforementioned visual inspection includes checking for scratches on the specified object, checking for any obstructions attached to the specified object, and checking for any peeling paint on the specified object. The image provision system according to any one of claims 1 to 4.

6. A method for providing images performed by a computer, The process involves acquiring images captured by a camera mounted on a mobile device, and The purpose of using the aforementioned captured images is defined in the purpose setting step, which sets the type of specific object installed on the side of the road on which the moving object is traveling and the content of the visual inspection, An image discrimination step to determine whether or not the captured image contains an image portion of the specific object, When the image discrimination step determines that the captured image contains an image portion of the specified object, the image processing step includes performing a masking process to reduce the amount of information in areas of the captured image other than the specified area containing the image portion of the specified object. A masking process determination step in which the content of the masking process is determined based on the type of specific object included in the purpose of use set in the purpose of use step and the content of the appearance check, The image transmission step involves transmitting the captured image, which has undergone the masking process, to a predetermined recipient. Image provision method including

Images