Image processing device, image processing method, image processing program, and recording medium

Abstract

To provide an image processing device capable of performing appropriate image processing for a subject in a captured image.SOLUTION: An image processing device includes: an image acquisition unit that obtains an image captured by an imaging device; a character detection unit that detects a character in the captured image; and an image processing unit that performs processing of making the character unclear when an actual height of the detected character in a vertical direction is lower than a predetermined height or when an actual size of the character is smaller than a predetermined size.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an image processing method, an image processing program, and a recording medium, and particularly relates to an image processing apparatus, an image processing method, an image processing program, and a recording medium that perform image processing based on map information.

Background Art

[0002] An apparatus has been proposed that performs processing for protecting personal information regarding what is shown in an image captured by an imaging device.

[0003] For example, Patent Document 1 discloses an imaging device that detects a person's face, a predetermined object, etc. as personal information included in an image captured by a camera by image recognition, and performs image processing to make the personal information unidentifiable.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] For example, when an image in which the nameplate of an individual's house is captured is publicly posted on the Internet through an SNS (Social Networking Service) or the like, it may be one of the problems that the privacy of the owner or resident of the house is violated.

[0006] In order to prevent the above-mentioned privacy violations, it is conceivable to perform image processing such as blurring the image. However, image processing is also performed on areas in the image where image processing is unnecessary, and information about the original scenery is lost. One of the problems is that information in the original image can be excessively lost.

[0007] This invention has been made in view of the above-mentioned points, and one of its objectives is to provide an image processing apparatus capable of performing appropriate image processing on a subject in a captured image. [Means for solving the problem]

[0008] The invention described in claim 1 is an image processing apparatus comprising: an image acquisition unit that acquires a captured image taken by an imaging device; a character detection unit that detects characters in the captured image; and an image processing unit that performs a process to blur the characters when the actual vertical height of the detected characters is lower than a predetermined height or the actual size of the characters is smaller than a predetermined size.

[0009] The invention described in claim 6 is an image processing method performed by an image processing device, comprising: an image acquisition step of acquiring a captured image taken by an imaging device; a character detection step of detecting characters in the captured image; and an image processing step of performing a process to blur the characters when the actual vertical height of the detected characters is lower than a predetermined height or the actual size of the characters is smaller than a predetermined size.

[0010] The invention described in claim 7 is an image processing program executed by an image processing apparatus equipped with a computer, which causes the computer to perform an image acquisition step of acquiring a captured image taken by an imaging device, a character detection step of detecting characters in the captured image, and an image processing step of performing a process to blur the characters when the actual vertical height of the detected characters is lower than a predetermined height or the actual size of the characters is smaller than a predetermined size.

[0011] Claim 8: A computer-readable storage medium that stores an image processing program which causes an image processing device equipped with a computer to perform an image acquisition step of acquiring an image captured by an imaging device, a character detection step of detecting characters in the image captured, and an image processing step of blurring the characters when the actual vertical height of the detected characters is lower than a predetermined height or the actual size of the characters is smaller than a predetermined size. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows an image processing system 100 according to an embodiment. [Figure 2] This figure shows the front seat area of ​​a vehicle equipped with the terminal device according to the embodiment. [Figure 3] This is a block diagram showing the configuration of the terminal device according to the embodiment. [Figure 4] This figure shows an example of the information included in the photographic information related to the example. [Figure 5] This is a block diagram showing the configuration of an image processing apparatus according to an embodiment. [Figure 6] This figure shows an example of map information related to the embodiment. [Figure 7] This is a top view showing the positional relationship between the shooting location and surrounding features of the vehicle in the embodiment. [Figure 8] This figure shows an example of a reproduced image related to the embodiment. [Figure 9] This figure shows an example of setting attributes used to identify the target area in the embodiment. [Figure 10] This figure shows an example of an image taken in the embodiment. [Figure 11] This figure shows an example of a captured image after image processing according to the embodiment. [Figure 12] This flowchart shows an example of a routine executed by the image processing apparatus according to the embodiment when identifying the attributes of the image processing. [Figure 13]It is a flowchart showing an example of a routine executed when the image processing apparatus according to the embodiment identifies an image processing target area. [Figure 14] It is a flowchart showing an example of a routine executed when the image processing apparatus according to the embodiment performs image processing. [Figure 15] It is a flowchart showing an example of a routine executed when the image processing apparatus according to the embodiment performs image processing. [Figure 16] It is a diagram showing an example of a captured image according to the embodiment. [Figure 17] It is a diagram showing an example of a captured image after image processing according to the embodiment. [Figure 18] It is a flowchart showing an example of a routine executed when the image processing apparatus according to the embodiment performs image processing. [Figure 19] It is a diagram showing an example of a captured image after image processing according to the embodiment.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described in detail. In the following description and the accompanying drawings, the same reference numerals are assigned to substantially the same or equivalent parts.

Embodiment

[0014] While referring to the accompanying drawings, the configuration of an image processing system 100 including an image processing apparatus 10 according to the embodiment will be described.

[0015] FIG. 1 is a diagram showing an overview of the image processing system 100. As shown in FIG. 1, the image processing system 100 includes an image processing apparatus 10 and a terminal device 20 mounted on a vehicle M.

[0016] The image processing device 10 is a server device that communicates with the terminal device 20 via a network NW. The image processing device 10 and the terminal device 20 can send and receive data from each other using a communication protocol such as TCP / IP. The connection between the terminal device 20 and the network NW can be made by mobile communication such as 4G (4th Generation) or 5G (5th Generation), or by wireless communication such as Wi-Fi (registered trademark).

[0017] The image processing device 10 receives an image captured by a camera mounted on the vehicle M from the terminal device 20, identifies a target area in the image to be processed, such as blurring for privacy protection or highlighting landmarks, performs the predetermined image processing on the target area in the image, and stores the image after the image processing.

[0018] The terminal device 20 transmits images captured by a camera mounted on the vehicle M to the image processing device 10.

[0019] The image processing system 100 is a system that provides images that have been processed to protect privacy when a user of the image processing system 100 publishes an image on the internet via a social networking service (SNS) or an electronic map, etc.

[0020] Users of the image processing system 100 can, for example, publish an image in which the part of the picture showing the nameplate of a private residence has been blurred, thus avoiding infringement of the privacy of the homeowner or residents of the house displaying that nameplate.

[0021] Furthermore, for example, if a user of the image processing system 100 does not want the location of the photograph to be identified, they can publish an image in which the portion of the photograph containing the commercial facility has been blurred, thereby protecting their own privacy.

[0022] Figure 2 shows the front seat area of ​​vehicle M. As shown in Figure 2, the terminal device 20 is located on the center console of vehicle M. In this embodiment, the terminal device 20 has a route generation function and a navigation function (so-called car navigation function). The terminal device 20 itself does not have to have a car navigation function. In that case, the terminal device 20 may be connected to a car navigation device installed in vehicle M so as to be able to acquire location information of vehicle M.

[0023] The front camera 11 is mounted on the dashboard DB. The front camera 11 is oriented to capture images in the direction of travel of the vehicle M. When the direction of travel of the vehicle M is forward, the front camera 11 is positioned to capture images in a range that includes the front, right front, and left front of the vehicle M.

[0024] The front camera 11 may be installed anywhere on the vehicle M as long as it can capture images of the area in front of the vehicle M. For example, the front camera 11 may be located behind the rearview mirror RM, that is, on the surface facing the windshield FG, or it may be installed on the outer surface (exterior surface) of the vehicle M, for example, on the hood or front bumper.

[0025] The front camera 11 is connected to the terminal device 20 in a communicative manner and is capable of transmitting the captured video signal to the terminal device 20.

[0026] The GNSS receiver 13 is a device that receives signals (GNSS signals) from GNSS (Global Navigation Satellite System) satellites. The GNSS receiver 13 is, for example, built into the forward camera 11.

[0027] The GNSS receiver 13 may be located anywhere within the vehicle M as long as it can receive GNSS signals. For example, the GNSS receiver 13 may be located near the front camera 11. For example, the GNSS receiver 13 may be located on the dashboard DB.

[0028] The GNSS receiver 13 is connected to the terminal device 20 in a communicative manner and can transmit the received GNSS signals to the terminal device 20. The terminal device 20 uses the GNSS signals to acquire the current location information of the vehicle M.

[0029] Furthermore, the GNSS receiver 13 may be a receiver that supports Real Time Kinematic (RTK) positioning, and may be capable of receiving high-precision position information corrected from GNSS position information.

[0030] The touch panel display 15 consists of a touch panel and a display. The touch panel display 15 is communicatively connected to the terminal device 20. The display of the touch panel display 15 displays images supplied from the terminal device 20. The touch panel of the touch panel display 15 transmits signals to the terminal device 20 indicating input operations by touching the touch panel.

[0031] For example, the touch panel display 15 transmits signals to the terminal device 20 indicating the display and input operations related to route guidance for the vehicle M.

[0032] For example, the touch panel display 15 transmits a signal to the terminal device 20 indicating an input operation for the user to make settings regarding image processing in the image processing system 100. For example, such settings may be for the type of image processing, such as blurring to protect privacy or highlighting parts that should stand out. Alternatively, such settings may be for different privacy protection modes depending on the object of privacy protection, such as a mode to protect the privacy of the owner of a property such as a private residence or a mode to protect the privacy of the user of the image processing system 100.

[0033] For example, the terminal device 20 may accept input operations for setting image processing via voice through a microphone (not shown) provided in the vehicle M.

[0034] Referring to Figures 3 and 4, the configuration and functions of the terminal device 20 installed in vehicle M will be described.

[0035] Figure 3 is a block diagram showing the configuration of the terminal device 20. As shown in Figure 3, the terminal device 20 is configured with its various parts connected via a system bus 21.

[0036] The input unit 23 is an interface that enables communication between the terminal device 20 and the equipment installed in the vehicle M.

[0037] The input unit 23 is an interface that enables communication between the terminal device 20 and the front camera 11.

[0038] The input unit 23 is an interface that connects the terminal device 20 to the GNSS receiver 13. The terminal device 20 receives GNSS signals from the GNSS receiver 13 and obtains information indicating the current position of the vehicle M.

[0039] The input unit 23 is an interface that enables communication between the terminal device 20 and the touch panel of the touch panel display 15.

[0040] The input unit 23 is an interface that enables communication between the terminal device 20 and the gyro sensor GY and the acceleration sensor AC. The gyro sensor GY is a sensor capable of detecting, for example, the lateral angle (attitude), angular velocity, or angular acceleration of the vehicle M. The acceleration sensor AC is a sensor capable of detecting the acceleration in the direction of travel of the vehicle M, i.e., the longitudinal direction, when viewed from above the vehicle M. The acceleration sensor AC can also detect, for example, the lateral (width direction) acceleration perpendicular to the direction of travel of the vehicle M.

[0041] The terminal device 20 acquires information indicating the direction of travel of the vehicle M, for example, based on the transition of the vehicle M's current position information acquired by the GNSS receiver 13, and the direction of acceleration indicated by the sensor signals of the gyro sensor GY and the acceleration sensor AC.

[0042] Furthermore, the terminal device 20 acquires information indicating the direction in which the image captured by the front camera 11 was taken, for example, using information indicating the direction of travel of the vehicle M.

[0043] The storage unit 25 is composed of, for example, a hard disk drive, an SSD (solid state drive), flash memory, etc., and stores various programs executed on the terminal device 20. These programs may be acquired via a network from, for example, another server device, or they may be recorded on a recording medium and read via various drive devices. In other words, the programs stored in the storage unit 25 can be transmitted via a network, and they can also be recorded on a computer-readable recording medium and transferred.

[0044] A map information database (not shown) is constructed in the memory unit 25. The map information database stores map information used for route generation of vehicle M by the terminal device 20, etc.

[0045] The control unit 27 is composed of a CPU (Central Processing Unit) 27A, a ROM (Read Only Memory) 27B, a RAM (Random Access Memory) 27C, etc., and functions as a computer. The CPU 27A reads and executes various programs stored in the ROM 27B and the memory unit 25 to realize various functions.

[0046] The control unit 27 acquires images, which are either video or still images, of the scenery in front of the vehicle M from the front camera 11 via the input unit 23.

[0047] The control unit 27 acquires the current location information of the vehicle M from the GNSS receiver 13 via the input unit 23. This current location information is, for example, information that associates information indicating the time with information indicating the position of the vehicle M at that time. The information indicating the position of the vehicle M may be information indicating its position on a map.

[0048] Furthermore, the control unit 27 generates shooting information by associating the image acquired from the front camera 11 with information indicating the shooting conditions, including the shooting position and shooting direction at the time the image was taken.

[0049] The control unit 27 acquires information indicating the shooting position, which is the position of the camera 11 when the image was taken. For example, the control unit 27 acquires a GNSS signal from the GNSS receiver and acquires information indicating the image shooting position based on the received GNSS signal.

[0050] For example, the control unit 27 obtains the position information of the vehicle M at the time the image was taken from the GNSS receiver 13, and sets the position indicated by this position information as the shooting position. For example, if the mounting position of the front camera 11 and the mounting position of the GNSS receiver 13 are far apart, the control unit 27 may calculate the shooting position by correcting for the difference in mounting positions.

[0051] Furthermore, the shooting position may include not only a horizontal position but also a vertical position. For example, the vertical position is determined by the vehicle height M and the mounting position of the front camera 11, and information indicating the vertical position may be stored in the storage unit 25 in association with the front camera 11.

[0052] For example, the control unit 27 may acquire a QZSS signal (Quasi-Zenith Satellite System, Michibiki) from the GNSS receiver and obtain high-precision position information indicating the image capture location based on the received QZSS signal.

[0053] Furthermore, for example, the control unit 27 may acquire information corrected from GNSS positioning using RTK positioning to obtain a more accurate shooting position.

[0054] The control unit 27 acquires the shooting direction based, for example, on the orientation of the front camera 11 when the image was taken and the direction of travel of the vehicle M, which is obtained based on the acceleration of the vehicle M. For example, if the front camera 11 is fixed to face forward of the vehicle M, the control unit 27 may acquire the direction of travel of the vehicle M as the shooting direction.

[0055] For example, if the front camera 11 is mounted at a variable angle with respect to the direction of travel of the vehicle M, the control unit 27 may obtain information from the front camera 11 indicating the orientation of the front camera 11 at the time of shooting, and obtain the shooting direction based on the direction of travel of the vehicle M and the orientation of the front camera 11.

[0056] Furthermore, the control unit 27 may acquire information indicating the field of view of the front camera 11, along with the image from the front camera 11, in addition to the shooting position and shooting direction. That is, the shooting information may include information indicating the field of view when the image was taken. For example, the information indicating the field of view is held in the front camera 11 and transmitted to the terminal device 20 along with the image.

[0057] Figure 4 shows an example of shooting information generated by the control unit 27 of the terminal device 20. In the example shown in Figure 4, for one of the multiple images acquired by the front camera 11, the frame number of that image is associated with the shooting position, shooting direction, and field of view at the time the image was acquired.

[0058] In the example shown in Figure 4, the shooting position is represented as a three-dimensional coordinate system, including the height. However, the shooting position is not limited to three-dimensional coordinates; it may also be represented as a two-dimensional coordinate system.

[0059] The direction of shooting is expressed by an azimuth angle, for example, with north being 0°.

[0060] The field of view is expressed, for example, as angles in the horizontal, vertical, and diagonal directions. The field of view included in the shooting information is not limited to these; for example, it may only include the horizontal angle.

[0061] The control unit 27 transmits the generated shooting information to the image processing device 10. The control unit 27 may transmit the shooting information for each frame sequentially, as illustrated in Figure 4, or it may transmit the shooting information at predetermined intervals of a certain number of frames or at predetermined intervals of time.

[0062] Referring again to Figure 3, the communication unit 29 is a network adapter such as a NIC (Network Interface Card) connected to a wireless device (not shown). The communication unit 29 communicates with the image processing device 10 according to commands from the control unit 27. For example, the communication unit 29 handles communication when the terminal device 20 transmits captured information to the image processing device 10.

[0063] The output unit 31 is connected to the display of the touch panel display 15 installed in the vehicle M. The output unit 31 is an interface for supplying various information to the display in accordance with commands from the control unit 27.

[0064] For example, the output unit 31 supplies information to the display for displaying a navigation screen. Alternatively, for example, the output unit 31 supplies information to the display for displaying a screen that accepts input for image processing settings, such as the setting of a privacy protection mode by the image processing device 10. For example, the output unit 31 may supply images captured by the front camera 11 to the display.

[0065] Next, the configuration and functions of the image processing device 10 will be described with reference to Figures 5-9.

[0066] Figure 5 is a block diagram showing the configuration of the image processing device 10. As shown in Figure 5, the image processing device 10 is a server device configured with each part connected via a system bus 41.

[0067] The large-capacity storage device 43 is a storage device composed of, for example, a hard disk drive, an SSD (solid state drive), flash memory, etc. The large-capacity storage device 43 stores various programs executed in the image processing device 10. These programs may be obtained, for example, from other server devices via a network, or they may be recorded on a recording medium and read via various drive devices.

[0068] For example, the large-capacity storage device 43 stores an image processing program that the image processing device 10 executes when it identifies a target area in an image that is to be processed.

[0069] For example, the large-capacity storage device 43 stores an image processing program that the image processing device 10 executes when performing image processing on a target area in an image.

[0070] For example, the large-capacity storage device 43 stores an image processing program that the image processing device 10 executes when it processes an area in an image that contains characters that meet predetermined conditions and are highly likely to infringe on privacy. Characters that are highly likely to infringe on privacy are, for example, characters whose height from the ground is below a predetermined height or whose size is below a predetermined size, or characters that match words related to personal names or place names.

[0071] The large-capacity storage device 43 stores a database containing data necessary for executing various programs.

[0072] The large-capacity storage device 43 includes a map information database (hereinafter referred to as the map information DB) 43A. The map information DB 43A stores map information that includes information indicating the location of a feature and the attributes of that feature. In the map information stored in the map information DB 43A, the location information of a feature and the information indicating the respective attributes of the feature are associated with each other.

[0073] For example, the map information stored in map information DB43A is, for instance, map information in which the location information of features is represented in three dimensions. However, in this map information, the location information of features may also be represented in two dimensions.

[0074] In this embodiment, the map information is used to identify the area to be processed for image processing based on the attributes of features in the captured image.

[0075] Figure 6 shows an example of information indicating attributes stored in the map information database. As shown in Figure 6, in the map information database, each feature ID, which is an identifier that identifies a feature, is associated with the attributes of that feature and stored in correspondence.

[0076] As shown in Figure 6, the attributes of geographical features include private attributes such as private homes and individual shops, and highly public attributes such as commercial facilities and public facilities.

[0077] In addition to the example shown in Figure 6, map information may also store utility poles and manholes as geographic features. For example, utility poles often have street signs or other markings that can identify addresses, and if a utility pole is visible in the captured image, it may be necessary to determine whether or not to include it in the image processing of this embodiment. Similarly, manholes often have characters or figures that can identify the municipality where they are installed, and for example, if you want to prevent the location of the photograph from being identified, it may be necessary to determine whether or not to include them in the image processing of this embodiment.

[0078] The large-capacity storage device 43 includes a dictionary database (hereinafter referred to as the dictionary DB) 43B. The dictionary DB 43B stores predetermined words that are referenced when the image processing device 10 performs processing on characters depicted in an image.

[0079] For example, dictionary DB43B stores a name dictionary containing a large number of names. For instance, when the image processing device 10 performs a process to blur the text on a nameplate of a private residence captured in an image, this name dictionary is used to determine whether or not a name included in the name dictionary is present in the text of the image.

[0080] For example, dictionary DB43B stores a place name dictionary containing a large number of place names. For instance, when the image processing device 10 performs processing to blur the text portion of place names, such as on street signs, that are captured in an image, this place name dictionary is used to determine whether or not a place name included in the place name dictionary is included in the text in the image.

[0081] The control unit 45 is composed of a CPU (Central Processing Unit) 45A, a ROM (Read Only Memory) 45B, a RAM (Random Access Memory) 45C, etc., and functions as a computer. The CPU 45A reads and executes various programs stored in the ROM 45B and the mass storage device 43 to realize various functions.

[0082] The control unit 45 reads and executes an image processing program stored in the large-capacity storage device 43 to identify the target area in the image that is to be processed.

[0083] The control unit 45 reads and executes an image processing program stored in the large-capacity storage device 43, thereby performing image processing on the target area in the image.

[0084] The control unit 45 reads and executes an image processing program stored in the large-capacity storage device 43 to blur areas in an image that contain characters that meet predetermined conditions that make them highly likely to infringe on privacy. Characters that are highly likely to infringe on privacy include, for example, characters whose height from the ground is below a predetermined height or whose size is below a predetermined size, or characters that match words related to personal names or place names.

[0085] The communication unit 47 is a network adapter such as a NIC (Network Interface Card) connected to a wireless device (not shown). The communication unit 47 communicates between the image processing device 10 and the outside world according to commands from the control unit 45.

[0086] For example, the communication unit 47 communicates when the image processing device 10 acquires imaging information, including an image captured by a front camera 11 mounted on the vehicle M, the position and direction of the image, from the terminal device 20.

[0087] The functions of the control unit 45 of the image processing device 10 will be described in detail below, with reference to Figures 7 to 11.

[0088] The control unit 45 acquires imaging information from the terminal device 20, including the image captured by the front camera 11 as an imaging device, the position and direction of the image, and functions as an imaging information acquisition unit.

[0089] [Identifying attributes] The control unit 45 functions as an attribute identification unit that identifies the attributes of a feature in a captured image based on map information indicating the location and attributes of the feature, and acquired photographic information.

[0090] The control unit 45, acting as an attribute identification unit, generates a reconstructed image based on map information stored in the map information DB 43A, which reproduces an image taken under conditions including the shooting location and shooting direction indicated by the shooting information. By comparing the features that were the subject in the captured image with the features in the reconstructed image, it identifies the attributes of the features that were the subject in the captured image.

[0091] Figure 7 is a schematic top view illustrating an example of the positional relationship between vehicle M and geographical features located around vehicle M's current position. In Figure 7, geographical features are represented by rectangular shapes, and each feature is assigned a feature ID. In the example shown in Figure 7, utility pole PP is located between feature F1 and feature F2.

[0092] Figure 8 schematically shows an example of a reconstructed image generated by the control unit 45 when identifying attributes. In the example shown in Figure 8, when the vehicle M is at the current position shown in Figure 7, the image processing device 10 receives shooting information including an image taken by the front camera 11, and the reconstructed image IM is shown, which is generated based on the said shooting information.

[0093] For example, when the control unit 45 receives shooting information from the terminal device 20, it refers to the map information DB 43A in the large-capacity storage device 43 and generates a reproduced image by reproducing the size and angle at which the feature in the image is captured when photographed under the conditions of shooting position, shooting direction, and field of view indicated by the shooting information, such as as shown in Figure 4. As described above, since attributes are associated with each feature in the map information DB 43A, when the reproduced image is generated, the attributes of the feature are associated with the feature in the reproduced image.

[0094] The control unit 45 generates a reconstructed image using image generation techniques such as 3D rendering based on, for example, three-dimensional map information stored in the map information DB 43A. Alternatively, the control unit 45 may generate a reconstructed image based on, for example, a two-dimensional map.

[0095] In the example shown in Figure 8, a reconstructed image is generated that shows a feature with feature ID F2 and attribute as a private residence, and a feature with feature ID F5 and attribute as a commercial facility, prominently in the foreground, with features with feature IDs F1 and F4 and attribute as private residences visible in the background, appearing smaller.

[0096] The control unit 45 can identify the attributes of the subject object in the captured image by comparing the subject object in the captured image included in the shooting information with the subject object in the reconstructed image as shown in Figure 8.

[0097] [Identifying the target area] Furthermore, the control unit 45 functions as a target area identification unit that identifies areas in the captured image where features whose attributes, as identified by the attribute identification unit, are predetermined, as target areas for image processing.

[0098] In identifying the target area, for example, if the purpose is to protect privacy, the area to be blurred is identified. For example, if the purpose is to make characters, buildings, etc. that serve as landmarks when referring to an electronic map with an attached image stand out, the area to be emphasized is identified.

[0099] The following explanation will focus primarily on cases where the purpose is to protect privacy.

[0100] Examples of privacy protections include the privacy of the owners of the objects depicted in the photographs, or the privacy of the users who utilize the photographs.

[0101] For example, if the privacy of property owners is to be protected, it is necessary to prevent the publication of images showing personal information such as the exterior of a private house or names displayed on a nameplate. In this case, private attributes such as private houses and privately owned shops are set as attributes to identify the target area based on the attributes of the property.

[0102] In other words, by selecting an area containing features with predefined private attributes as the target area and applying image processing to that area, the privacy of the feature's owner can be protected.

[0103] Furthermore, if the privacy of users who utilize the captured images is to be protected, for example, it is necessary to prevent the location of the photograph from being identified by the publication of images that include public facilities or street signs. In this case, attributes that are highly public, such as commercial facilities and public facilities, would be set as attributes to identify the target area based on the attributes of the features.

[0104] In other words, by designating an area containing features with predefined high public attributes as the target area and applying image processing to that area, the privacy of users who utilize the captured images can be protected.

[0105] The attributes used to identify the target area may be pre-set in the image processing device 10, or they may be set or changed by the user via the touch panel display 15, for example. For example, these predetermined attributes may be stored in the mass storage device 43 along with information indicating the privacy protection target and information indicating the content of the image processing.

[0106] Figure 9 shows an example of setting attributes used to identify a target area. In the example shown in Figure 9, "Privacy Protection Mode" indicates different attribute settings depending on the object of privacy protection. In the example shown in Figure 9, "Mode 1" is used when the privacy of the object owner is to be protected, and "Mode 2" is used when the privacy of the user using the captured image is to be protected.

[0107] Furthermore, the "protection scope level" shown in Figure 9 indicates the breadth of privacy protection, or in other words, the degree of protection. In the example shown in Figure 9, the higher the protection scope level, the more attributes are set as attributes used to identify the target area.

[0108] Furthermore, while setting more attributes enhances privacy protection, it also enlarges the area of ​​the image identified as the target region. Consequently, if blurring is applied, the range of information lost within the image becomes wider.

[0109] For example, in the setting example shown in Figure 9, either Mode 1 or Mode 2 may be used alone, or a combination of Mode 1 and Mode 2 settings may be used. For example, Mode 1 may be set to Level 3 and Mode 2 to Level 1.

[0110] In addition to modes 1 and 2 illustrated in Figure 9, a mode may be set for the purpose of highlighting landmark areas in the image. For example, the target area identified as a landmark area by this mode can be enhanced with image processing.

[0111] Figure 10 shows an example of a captured image, illustrating how the target area is identified within the captured image FL1000. Figure 10 shows an example where the target area is identified in order to protect the privacy of the property owner.

[0112] Each feature in the captured image FL1000 is associated with its attributes. Based on these attributes, the target area is identified in the captured image FL1000.

[0113] In Figure 10, the area enclosed by the dashed line represents the target area AR1 for image processing. In the example shown in Figure 10, the area containing a feature with the attribute "private residence" is identified as the target area AR1.

[0114] Furthermore, in Figure 10, the area enclosed by the dashed line represents region AR2, which is an area that is excluded from image processing, even if it contains a feature. In the example shown in Figure 10, the area containing a feature with the attribute "commercial facility" is region AR2, which is excluded from processing.

[0115] Furthermore, when the control unit 45 identifies a target area, it may exclude from the target area any area in which an object different from a feature with a predetermined attribute is depicted, even if that object has a different attribute than the feature with a predetermined attribute.

[0116] As a distinct object, for example, a utility pole often has address signs such as street signs attached to it, making it useful as an image for route guidance using real-world images. In this case, if image processing is performed to blur the area in which the utility pole is visible, useful information will be lost. Therefore, by excluding the area in which the utility pole is visible from the target area, it is possible to prevent the loss of useful information from the captured image. In the example shown in Figure 10, the area in which the utility pole is visible within the target area AR1 may be excluded from the target area.

[0117] For example, the map information stored in the map information DB43A may include location information for utility poles, and the control unit 45 may use this location information to identify utility poles within an area of ​​the captured image where a feature with a predetermined attribute is visible, and exclude the area containing the utility pole from the target area. For example, if the map information does not include location information for utility poles, the control unit 45 may detect a utility pole within an area where a feature with a predetermined attribute is visible using image recognition and exclude the area containing the utility pole from the target area.

[0118] [Image processing methods] Furthermore, the control unit 45 functions as an image processing unit that performs processing to blur or enhance at least a portion of the target area in the captured image.

[0119] Examples of methods for blurring at least a portion of the target area include image processing such as mosaic processing, blurring, and masking.

[0120] The control unit 45 performs a process to blur the parts of the image that contain text, such as nameplates, within the target area. For example, if a private residence is captured in the image, this prevents the name of the homeowner, which is personal information, from being published on the internet, thereby protecting the homeowner's privacy.

[0121] Furthermore, the control unit 45 may, for example, perform a process to blur the entire target area. In this case, for example, the entire building, which is a predetermined feature in the captured image, can be blurred, thereby preventing, for example, the exterior of a private residence from being published on the internet.

[0122] Figure 11 shows an example of a captured image after image processing that blurs the entire target area. In the example shown in Figure 11, the target area AR1, which shows a building with the attribute of being a private residence, is blurred.

[0123] On the other hand, examples of image processing that emphasizes the target area include processing to increase the contrast of the target area in the captured image, or processing to increase the brightness or saturation.

[0124] [Character detection] Furthermore, the control unit 45 functions, for example, as a character detection unit that detects characters in the captured image. The control unit 45 detects characters in the captured image using known character recognition technologies such as OCR (Optical Character Recognition) or AI-based character recognition (AI-OCR). For example, the control unit 45 as a character detection unit may detect characters within a target area.

[0125] [Image processing related to text] The control unit 45, acting as an image processing unit, may, for example, perform a process to blur the area in the image that contains characters that meet predetermined conditions.

[0126] For example, the control unit 45, acting as an image processing unit, performs a process to blur the character if the actual vertical height of the character detected by the character detection unit is lower than a predetermined height or if the actual size of the character is smaller than a predetermined size.

[0127] The control unit 45 calculates the vertical height of a character based on the distance from the shooting position where the captured image was taken to the character detected by the character detection unit.

[0128] For example, the control unit 45 acquires information indicating the shooting location of the captured image along with the captured image, and calculates the actual distance from the shooting location to the detected character based on the shooting location and map information. Based on the calculated actual distance, the control unit 45 converts the vertical height of the character in the image from the ground to the actual height of the character.

[0129] In this embodiment, if the actual vertical height of the characters detected in the captured image is higher than a predetermined height, it is highly likely that the characters are intended to be seen by an unspecified number of people, such as on a store sign. If the actual height is less than or equal to the predetermined height, it is highly likely that the characters are written on a nameplate.

[0130] For example, the height of a nameplate from the ground is often 2m or less. On the other hand, when the height of letters from the ground exceeds 2m, they are often letters on a sign. Therefore, the control unit 45 may perform a process to blur the letters if the calculated height of the letters is 2m or less, and control the letters not to blur them if they exceed 2m.

[0131] This method prevents the personal names written on nameplates from being published on the internet, and ensures that the text on signs and other objects is not obscured, thereby preventing excessive information loss from the captured images.

[0132] Furthermore, the control unit 45 calculates the size of the character based on the distance from the shooting position where the captured image was taken to the character detected by the character detection unit.

[0133] For example, the control unit 45 acquires information indicating the shooting location of the captured image along with the captured image, and calculates the actual distance from the shooting location to the detected character based on the shooting location and map information. Based on the calculated actual distance, the control unit 45 converts the size of the character in the image to the actual size of the character.

[0134] In this embodiment, if the actual size of the characters detected in the captured image is larger than a predetermined size, it is highly likely that they are characters intended to be seen by an unspecified number of people, such as on a store sign. If the actual size of the characters is less than or equal to the predetermined size, it is highly likely that they are characters written on a nameplate.

[0135] For example, the letters on nameplates are often 15 cm or less in width in the height direction, while the letters on signs are often larger. Therefore, the control unit 45 may perform a process to blur the calculated letter size if the width in the height direction is 15 cm or less, and not blur the letter if the width in the height direction is greater than 15 cm.

[0136] By doing so, for example, it is possible to prevent the personal names written on nameplates from being published on the internet, and to ensure that the text on signs and other objects is not blurred, thereby preventing excessive information loss from the captured images.

[0137] For example, if the vertical height of the detected character is lower than a predetermined height (determination by height) or if the actual size of the character is smaller than a predetermined size (determination by size), the character may be blurred if either of these conditions is met. Alternatively, the determination by height and the determination by size may be combined, and the character may be blurred if both conditions are met.

[0138] The control unit 45, acting as an image processing unit, may perform a process to blur characters detected in the captured image if those characters match a predetermined word. The predetermined word may be, for example, a person's name or a place name.

[0139] The control unit 45 may, for example, perform a process to blur characters if a string of one or more characters detected in the captured image matches either a person's name or a place name stored in the dictionary DB 43B of the mass storage device 43. This allows characters that are not person's names or place names, such as "Caution: Children Crossing," to remain in the image.

[0140] For example, characters matching a given word may be part of a string of characters. For instance, only the "Takahashi" part of "Takahashi Clinic" may be considered a person's name and subject to blurring.

[0141] Furthermore, the determination of whether or not a given word matches may be applied in combination with the determination based on the height of the characters or the size of the characters as described above.

[0142] For example, if the sign for "Takahashi Clinic" is positioned higher or written in larger letters than the nameplate, applying the above-mentioned judgment based on letter height or letter size will exclude the "Takahashi Clinic" text on the sign from image processing, allowing it to remain in the captured image without becoming blurry.

[0143] In this way, by targeting characters that meet certain conditions for image processing to blur them, and excluding other characters, it is possible to prevent excessive loss of information in the captured image.

[0144] For example, the control unit 45 may identify the area to be processed by the image processing described above, and if the characters within that area match a predetermined word, it may perform a process to blur the characters.

[0145] Furthermore, for example, the control unit 45 may perform a process to blur the character if the vertical height of the character detected within the target area is lower than a predetermined height, or if the actual size of the character is smaller than a predetermined size, and the character matches a predetermined word.

[0146] Referring to Figures 12 to 18, the control routines executed by the control unit 45 of the image processing device 10 in this embodiment will be described.

[0147] Figure 12 is a flowchart showing an example of an attribute identification routine RT1, which is executed by the control unit 45 when identifying the attributes of features in a captured image. For example, the control unit 45 starts the attribute identification routine RT1 when it detects that the terminal device 20 has started transmitting image capture information.

[0148] When the control unit 45 starts the attribute identification routine RT1, it receives and acquires new shooting information from the terminal device 20, including the captured image taken by the front camera 11, the shooting position of the captured image, and the shooting direction (step S101). In step S101, for example, shooting information including one frame image, the shooting position when the frame image was acquired, and the shooting direction is acquired. For example, as shown in Figure 4, the shooting information may include information indicating the field of view in which the captured image was taken.

[0149] After step S101 is executed, the control unit 45 refers to the map information stored in the map information DB 43A (step S102). In step S102, the control unit 45 reads the map information for the area including the shooting location indicated by the shooting information acquired in step S101.

[0150] After step S102 is executed, the control unit 45 generates a reconstructed image (step S103) that reproduces an image taken under conditions including the shooting position and shooting direction indicated by the shooting information acquired in step S101, based on the map information read in step S102. In step S103, for example, a reconstructed image like the one shown in Figure 8 is generated. Also in step 103, the attributes of features in the reconstructed image are identified by the attributes associated with the features included in the map information.

[0151] After step S103 is completed, the control unit 45 compares the captured image with the reconstructed image (step S104).

[0152] After step S104 is executed, the control unit 45 identifies the attributes of the features in the captured image (step S105). In step S105, for example, the attributes of the features in the captured image are identified by comparing the features that were the subject in the captured image with the features in the reconstructed image.

[0153] After step S105 is executed, the control unit 45 stores an image with information indicating the attributes identified in step S104 (step S106).

[0154] After step S106 is executed, the control unit 45 terminates the attribute identification routine RT1 and starts a new attribute identification routine RT1.

[0155] Figure 13 is a flowchart showing an example of a target area identification routine RT2, which is executed by the control unit 45 when identifying the target area for image processing. For example, when the attribute identification routine RT1 is started, the control unit 45 starts the target area identification routine RT2.

[0156] When the control unit 45 starts the target area identification routine RT2, it acquires a new captured image with the attributes of the features attached (step S201). In step S201, the control unit 45 reads out an image that has been stored with the attributes of the features attached to the features in the captured image, for example, in step S106 of the attribute identification routine RT1.

[0157] After step S201 is executed, the control unit 45 refers to the setting of attributes for identifying the target area (step S202). In step S202, the control unit 45 refers to attributes that are pre-set according to the privacy protection target or protection scope level, for example, as illustrated in Figure 9.

[0158] For example, if the privacy to be protected concerns the privacy of the property owner, then private attributes such as a private residence are set as attributes to identify the area in question.

[0159] For example, if the privacy to be protected is that of users who use the captured images, then attributes that are highly public, such as commercial facilities, are set as attributes to identify the target area.

[0160] After step S202 is executed, the control unit 45 identifies the area in the acquired captured image that contains features with attributes corresponding to the settings referenced in step S202 as the target area (step S203).

[0161] After step S203 is executed, the control unit 45 stores an image with information about the target area (step S204). In step S204, an image with information indicating the target area, such as the one illustrated in Figure 10, is stored in the large-capacity storage device 43.

[0162] After step S204 is executed, the control unit 45 terminates the target area identification routine RT2 and starts a new target area identification routine RT2.

[0163] Figure 14 is a flowchart showing an example of an image processing routine RT3, which is executed by the control unit 45 when performing image processing. For example, when the target area identification routine RT2 is started, the control unit 45 starts the image processing routine RT3.

[0164] When the control unit 45 starts the image processing routine RT3, it acquires a captured image with information about the target area attached (step S301). In step S301, for example, the control unit 45 reads the image stored in step S204 of the target area identification routine RT2.

[0165] After step S301 is executed, the control unit 45 refers to the image processing settings (step S302). In step S302, for example, the control unit 45 refers to the mass storage device 43 and reads information indicating whether the image processing to be performed is an image processing that blurs at least a part of the target area or an image processing that enhances the target area.

[0166] After step S302 is executed, the control unit 45 performs image processing on the target area (step S303). In step S303, for example, the control unit 45 performs image processing corresponding to the image processing content referenced in step S302. In step S303, for example, the control unit 45 performs processing to blur the entire target area or processing to emphasize the target area.

[0167] After step S303 is executed, the control unit 45 stores the captured image after image processing (step S304).

[0168] After step S304 is executed, the control unit 45 terminates the image processing routine RT3 and starts a new image processing routine RT3.

[0169] The image stored in step S304 of the image processing routine RT3 is available to the user. For example, a user of the image processing system 100 can publish an image with the target area processed on the internet via social media or an electronic map without infringing on the privacy being protected.

[0170] Figure 15 is a flowchart showing an example of an image processing routine RT4, which is executed by the control unit 45 when performing image processing to blur characters. For example, when the control unit 45 starts transmitting a captured image or captured information from the terminal device 20, it starts the image processing routine RT4.

[0171] When the control unit 45 starts the image processing routine RT4, it acquires the captured image taken by the front camera 11 (step S401). In step S401, for example, the control unit 45 receives the captured image along with the position in which the image was taken from the terminal device 20.

[0172] After step S401 is performed, the control unit 45 uses character recognition technology to detect characters in the captured image and determines whether or not characters exist in the captured image (step S402). In step S402, for example, the area in which characters are captured is identified as the character area LT.

[0173] Figure 16 shows an example where the area containing text within the captured image FL1000 is identified as the text area LT. As shown in Figure 16, the area containing text on a nameplate of a private house, the area containing the text "〇〇 Clinic" on a sign, and the area containing the text "〇〇〇 Store" on a sign are identified as the text area LT.

[0174] In step S402, if the control unit 45 determines that no characters exist in the captured image (step S402: NO), it terminates the image processing routine RT4 and starts a new image processing routine RT4.

[0175] In step S402, the control unit 45 determines that characters exist in the captured image (step S402: YES), and calculates the vertical height of the detected characters from the ground (hereinafter also referred to as the height position) and the size of the characters (step S403). In step S403, the control unit 45 calculates the height and size of the characters based on the distance from the shooting position where the image was taken to the detected characters, for example, as described above.

[0176] After step S403 is executed, the control unit 45 determines whether the height of the characters calculated in step S403 is lower than a predetermined height (step S404). In step S404, considering that the height of a nameplate from the ground is often 2m or less, the predetermined height from the ground is determined to be, for example, 2m.

[0177] In step S404, if it is determined that the calculated character height is lower than a predetermined height (step S404: YES), the control unit 45 determines whether the calculated character size is smaller than a predetermined size (step S405). In step S405, considering that the size of characters written on a nameplate is often 15 cm or less in width in the height direction, the predetermined size is determined to be, for example, 15 cm in width in the height direction.

[0178] In step S405, if it is determined that the calculated character size is smaller than a predetermined size (step S405: YES), the control unit 45 performs a process to blur the character detected in step S402 (step S406). In step S406, for example, image processing such as mosaic processing, blurring processing or masking processing is applied to the detected character portion.

[0179] Figure 17 shows an example of a process applied to the text area LT within the captured image FL1000, where the text has been blurred. In the example shown in Figure 17, the area of ​​the text area LT that shows the text written on a nameplate of a private house has been blurred.

[0180] In the example shown in Figure 17, for instance, the letters written on a nameplate of a private residence are subject to image processing to blur because their actual vertical height is below a predetermined height and their actual size is below a predetermined size. On the other hand, for example, the letters that read "〇〇 Clinic" are not subject to image processing to blur because, although their actual vertical height is below a predetermined height, their size is larger than a predetermined size.

[0181] In the example shown in Figure 17, the letters on the sign that read "XXX Store" are not subject to image processing to blur them because their actual vertical height is greater than the specified height and their actual size is greater than the specified size.

[0182] After step S406 is executed, the control unit 45 stores the image, which has undergone image processing to blur the characters, in the mass storage device 43 (step S407). In step S407, for example, the image that has undergone image processing may be saved in a dedicated folder that can be uploaded by the user.

[0183] If the control unit 45 determines in step S404 that the height position of the characters is not lower than a predetermined height (step S404: NO) or in step S405 that the size of the characters is not smaller than a predetermined size (step S405: NO), then in step S407, it stores an image that has not been processed by this routine.

[0184] In step S407, for example, images that have not undergone the image processing may be saved in a dedicated folder that users can upload, just like images that have undergone image processing. Images that have not undergone image processing in step S407 can be said to be images that have been confirmed to be uploadable by users according to the criteria of this routine by going through step S404 or step S405.

[0185] After step S407 is executed, the control unit 45 terminates the image processing routine RT4 and starts the image processing routine RT4 again.

[0186] The image processing routine RT4 may be executed on a captured image to which information about the target region has been added, for example, via the target region identification routine RT2. For example, in step S401, a captured image to which information about the target region has been added may be acquired, and in step S402, instead of determining whether or not characters exist in the captured image, a determination may be made as to whether or not characters exist within the target region.

[0187] By doing so, the text area to be processed can be identified efficiently and accurately. For example, in the example shown in Figure 17, by determining the vertical height and size of the text only for the target area AR1, the text "〇〇 Clinic" located outside the target area can be excluded from image processing without having to identify the text area.

[0188] This routine ensures that areas of the captured image that are highly likely to infringe on privacy, such as areas containing nameplates, are reliably blurred (blurred). Furthermore, areas that are not highly likely to infringe on privacy, such as signs positioned higher than the nameplate or signs with larger lettering than the nameplate, can be excluded from blurring. Therefore, privacy can be protected without unnecessarily damaging the scenery or information in the captured image.

[0189] Figure 18 is a flowchart showing an example of an image processing routine RT5, which is executed by the control unit 45 when performing image processing to blur characters. For example, when the target area identification routine RT2 is started, the control unit 45 starts the image processing routine RT5.

[0190] When the control unit 45 starts the image processing routine RT5, it acquires an image with information about the target area attached, similar to the case of the image processing routine RT3 (step S501).

[0191] After step S501 is executed, the control unit 45 uses character recognition technology to detect characters within the target area and determines whether or not characters exist within the target area (step S502).

[0192] In step S502, if it is determined that no characters exist within the target area (step S502: NO), the control unit 45 terminates the image processing routine RT5 and starts a new image processing routine RT5.

[0193] In step S502, if it is determined that a character exists within the target area (step S502: YES), the control unit 45 refers to the dictionary DB 43B of the mass storage device 43 (step S503) and determines whether the character detected in step S502 matches a word included in the dictionary DB 43B (step S504).

[0194] In step S504, if it is determined that the character matches a word included in the dictionary DB43B (step S504: YES), the control unit 45 performs a process to blur the character detected in step S502 (step S505).

[0195] In step S505, for example, as illustrated in Figure 17, image processing such as mosaic processing is applied so that only the part of the name "〇〇" that is written becomes blurred in areas where the letters written on a nameplate of a private house are visible or where the letters "〇〇 Clinic" are visible on a sign.

[0196] After step S505 is executed, the control unit 45 stores the image, which has undergone image processing to blur the characters, in the mass storage device 43 (step S506). In step S506, for example, the image that has undergone image processing may be saved in a dedicated folder that can be uploaded by the user.

[0197] In step S504, the control unit 45 determines that the image does not match a word included in the dictionary DB43B (step S504: NO), and in step S506, it stores the image that has not been processed by this routine.

[0198] In step S506, for example, images that have not undergone the image processing may also be saved in a dedicated folder that users can upload, just like images that have undergone the image processing.

[0199] After step S506 is executed, the control unit 45 terminates the image processing routine RT5 and starts the image processing routine RT5 again.

[0200] The image processing routine RT5 may, for example, be executed on captured images that do not have information about the target area attached.

[0201] Furthermore, the image processing routine RT5 may be executed in combination with the image processing routine RT4. For example, steps S503 and S504 in the image processing routine RT5, which determine whether or not a word matches a word included in the dictionary, may be executed before or after the determination regarding the height or size of the character in the image processing routine RT4, that is, before step S403 or after step S405. In this way, only characters that are more likely to infringe on privacy can be subject to blurring with greater accuracy, while other characters can be excluded from blurring, thereby preventing damage to the information of the original image.

[0202] Furthermore, the image processing routine described using Figures 12-18 may be executed when a user intends to publish an image via social media, etc., or it may be executed sequentially or periodically when a captured image is acquired and saved to an upload folder.

[0203] In this embodiment, image processing may be performed to highlight landmarks, characters, etc., in order to make them stand out, thereby emphasizing the target area or at least a portion of the captured image.

[0204] Furthermore, the image enhancement processing may be performed in combination with the blurring processing described above. For example, in order to protect the privacy of the owners of the objects captured in the image, image processing may be performed to blur areas containing private homes or names, while image processing may be performed to enhance areas containing landmarks or areas with text such as street signs or billboards.

[0205] Figure 19 shows an example where the captured image FL1000 was processed to blur the area of ​​the nameplate, and to enhance the building of a commercial facility that serves as a useful landmark for route guidance using real-world images.

[0206] As described in detail above, the image processing apparatus 10 of this embodiment includes: an image information acquisition unit that acquires image information including an image captured by an imaging device, the shooting position of the image, and the shooting direction; an attribute identification unit that identifies the attributes of a feature in the image based on map information including the location of a feature and information indicating the attributes of the feature, and the image information; and a target area identification unit that identifies an area in the image where a feature whose attribute identified by the attribute identification unit is a predetermined attribute is photographed, as a target area for image processing.

[0207] This configuration allows for the identification of areas requiring image processing, for example, to protect privacy, based on the attributes of the features. For instance, areas outside the target region can retain the original image's appearance and information even after image processing.

[0208] Furthermore, the image processing apparatus 10 of this embodiment includes an image acquisition unit that acquires a captured image taken by an imaging device, a character detection unit that detects characters in the captured image, and an image processing unit that performs a process to blur the characters when the actual vertical height of the detected characters is lower than a predetermined height or the actual size of the characters is smaller than a predetermined size.

[0209] With this configuration, for example, the text portion of a nameplate, such as a sign, can be blurred, while parts of the text on a sign that do not require blurring can be excluded from the processing. Therefore, the original appearance and information of the image can be maintained in the excluded parts even after image processing.

[0210] Therefore, the image processing apparatus 10 of this embodiment can provide an image processing apparatus capable of performing appropriate image processing on a subject in a captured image. For example, the image processing apparatus 10 of this embodiment can provide an image processing apparatus that prevents excessive loss of information in a captured image due to image processing to protect the privacy of the subject in the image.

[0211] The configurations and routines in the above-described embodiments are merely illustrative examples and can be appropriately selected and modified depending on the application and other factors.

[0212] For example, the terminal device 20 may have the functions of the image processing device 10 in the above embodiment and the information stored in the memory unit, and the terminal device 20 may perform image processing such as identifying attributes and target areas, and blurring the text portion.

[0213] For example, the terminal device 20 may be a configuration in which a terminal device having a similar configuration to the terminal device 20 is integrated with a front camera 11 and a touch panel display 15. Specifically, for example, the terminal device 20 may be a terminal device such as a smartphone, tablet, or PC with a camera that has an application that performs the same functions as the terminal device 20 described above. In this case, the terminal device 20 may be mounted on the dashboard DB, for example, by a cradle, so that the built-in camera can capture images of the front of the vehicle M through the windshield of the vehicle M.

[0214] Furthermore, the terminal device 20 may be configured not to display a screen to the driver of vehicle M. For example, the terminal device 20 may have a configuration similar to a drive recorder, or it may be a device integrated with the front camera 11. Specifically, the terminal device 20 may be a device that incorporates hardware within the housing of the front camera 11 that performs the function of transmitting the captured image or captured information of the terminal device 20 to the image processing device 10. In this case, the terminal device 20 may not display or output the privacy protection mode selection input screen or the like as described above.

[0215] Although an example has been described in which the forward-facing camera 11 and terminal device 20, which serve as imaging devices, are mounted on a vehicle, the imaging device and terminal device of the present invention may also move together with other moving objects such as bicycles, motorcycles, or ships. Furthermore, the moving object may be a person. For example, the terminal device 20 may have a built-in camera like a smartphone, which is held by a person, and image processing may be performed by an image processing device on images taken, for example, while walking.

[0216] Furthermore, the imaging device of the present invention is not limited to one that moves with the moving object; for example, it may be a fixed-point camera fixed at a predetermined location along a road. [Explanation of Symbols]

[0217] 10 Image Processing Device 20 Terminal devices 23 Input section 25 Memory section 27 Control Unit 29 Communications Department 31 Output section 43 Mass storage 43A Map Information Database 43B Dictionary DB 45 Control Unit 47 Communications Department 100 Image Processing Systems M Vehicle

Claims

1. An image acquisition unit that acquires images captured by an imaging device, A character detection unit for detecting characters in the captured image, An image processing apparatus having an image processing unit that performs a process to blur the detected character when the vertical height of the character from the ground is lower than a predetermined height.

2. An image acquisition unit that acquires an image captured by an imaging device, A character detection unit for detecting characters in the captured image, The system includes an image processing unit that performs a process to blur the detected character if the vertical height of the character from the ground is lower than a predetermined height or if the actual size of the character is smaller than a predetermined size. The image processing unit is an image processing device that acquires the vertical height or the size of the character based on the distance from the shooting position where the captured image was taken to the character.

3. An image acquisition unit that acquires an image captured by an imaging device, A shooting information acquisition unit that acquires shooting information including the shooting position and shooting direction of the aforementioned captured image, An attribute identification unit that identifies the attributes of a feature in a captured image based on map information including the location of the feature and information indicating the attributes of the feature, and the aforementioned photographic information, A target area identification unit identifies the area in the captured image in which a feature whose attribute identified by the attribute identification unit is a predetermined attribute is depicted as a target area for image processing, A character detection unit that detects characters within the target region identified by the target region identification unit in the captured image, An image processing unit performs a process to blur the detected character if the vertical height of the character from the ground is lower than a predetermined height or if the actual size of the character is smaller than a predetermined size. An image processing device having

4. The image processing apparatus according to any one of claims 1 to 3, wherein the image processing unit performs a process to blur the characters when the characters detected by the character detection unit match a predetermined word.

5. The image processing apparatus according to claim 4, wherein the predetermined word is a word relating to a person's name or a place name.

6. An image processing method performed by an image processing device, An image acquisition step in which an image captured by an imaging device is acquired, A character detection step for detecting characters in the captured image, An image processing method comprising: an image processing step of performing an image processing to blur the detected character if the vertical height of the character from the ground is lower than a predetermined height.

7. An image processing program executed by an image processing device equipped with a computer, wherein the computer, An image acquisition step in which an image captured by an imaging device is acquired, A character detection step for detecting characters in the captured image, Image processing step: If the vertical height of the detected character from the ground is lower than a predetermined height, the process of blurring the character is performed. An image processing program to execute this.

8. An image processing method performed by an image processing device, An image acquisition step in which an image captured by an imaging device is acquired, A character detection step for detecting characters in the captured image, The process includes an image processing step of blurring the detected character if the vertical height of the character from the ground is lower than a predetermined height or if the actual size of the character is smaller than a predetermined size, An image processing method that, in the image processing step, obtains the vertical height or the size of the character based on the distance from the shooting position where the captured image was taken to the character.

9. An image processing program executed by an image processing device comprising a computer, wherein the computer: An image acquisition step in which an image captured by an imaging device is acquired, A character detection step for detecting characters in the captured image, If the vertical height of the detected character from the ground is lower than a predetermined height or the actual size of the character is smaller than a predetermined size, the system performs an image processing step that blurs the character. An image processing program that, in the image processing step, causes to obtain the vertical height or the size of the character based on the distance from the shooting position where the captured image was taken to the character.

10. An image processing method performed by an image processing device, An image acquisition step in which an image captured by an imaging device is acquired, A shooting information acquisition step, which acquires shooting information including the shooting position and shooting direction of the aforementioned captured image, An attribute identification step that identifies the attributes of a feature in a captured image based on map information including the location of the feature and information indicating the attributes of the feature, and the aforementioned photographic information, A target area identification step involves identifying, in the aforementioned captured image, the area in which a feature whose attribute identified in the attribute identification step is a predetermined attribute is depicted as the target area for image processing. A character detection step for detecting characters within the target region identified by the target region identification step in the captured image, Image processing steps to blur the characters if the vertical height of the detected characters from the ground is lower than a predetermined height or if the actual size of the characters is smaller than a predetermined size; Image processing methods including [specific details omitted].

11. An image processing program executed by an image processing device comprising a computer, wherein the computer: An image acquisition step in which an image captured by an imaging device is acquired, A shooting information acquisition step, which acquires shooting information including the shooting position and shooting direction of the aforementioned captured image, An attribute identification step that identifies the attributes of a feature in a captured image based on map information including the location of the feature and information indicating the attributes of the feature, and the aforementioned photographic information, A target area identification step involves identifying, in the aforementioned captured image, the area in which a feature whose attribute identified in the attribute identification step is a predetermined attribute is depicted as the target area for image processing. A character detection step for detecting characters within the target region identified by the target region identification step in the captured image, Image processing steps to blur the characters if the vertical height of the detected characters from the ground is lower than a predetermined height or if the actual size of the characters is smaller than a predetermined size; An image processing program to execute this.

12. In an image processing device equipped with a computer, An image acquisition step in which an image captured by an imaging device is acquired, A character detection step for detecting characters in the captured image, Image processing step: If the vertical height of the detected character from the ground is lower than a predetermined height, the process of blurring the character is performed. A computer-readable recording medium that stores an image processing program that executes the image.

Images