Information processing device, information processing method, and information processing program

Abstract

To provide a vehicle information acquisition system that solves a problem of not being able to display a photographed vehicle and a registered vehicle using comparable information.SOLUTION: A vehicle information acquisition system comprises: an acquisition unit that acquires an image showing at least a portion of a license plate attached to a vehicle; a character extraction unit that extracts a character from a portion of the license plate in the image; a vehicle information acquisition unit that acquires vehicle information corresponding to an extracted character from a storage unit which stores vehicle information and a character of a registered vehicle in association with each other; and a display control unit that displays the acquired vehicle information on a display unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a vehicle information acquisition system for acquiring vehicle information, a vehicle information acquisition method, a vehicle information acquisition program, and a storage medium for storing the vehicle information acquisition program. [Background technology]

[0002] An example of a system that reads the number printed on a vehicle license plate and uses the characters that are read will be described.

[0003] Patent Document 1 describes a system that extracts characters from an image of a vehicle's license plate and searches a database (DB). The system displays the data hit as a result of the search on a display device. Patent Document 2 describes a system that reads and temporarily stores vehicle numbers and other information. Furthermore, when the system receives an instruction to end continuous reading, it sequentially searches the vehicle DB for the temporarily stored numbers. The system then displays the search results on a display. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 11-296785 [Patent Document 2] Patent Publication No. 2000-322421 Summary of the Invention [Problem to be solved by the invention]

[0005] However, while the above-mentioned system can display the search results for license plates, it has the problem of being unable to display information that allows a comparison between the photographed vehicle and the registered vehicle. As a result, users of the above-mentioned system are unable to know that the license plate is counterfeit, etc. Therefore, an object of the present invention is to provide a vehicle information acquisition system that solves the problem of being unable to display information that allows a comparison between the photographed vehicle and the registered vehicle. [Means for solving the problem]

[0006] The first information processing device of the present invention comprises an acquisition unit that acquires an image of a vehicle photographed by a terminal, an extraction unit that extracts a character string from the area of ​​the license plate included in the image, a vehicle information acquisition unit that acquires information about a registered vehicle associated with the character string, and an output unit that outputs information about the registered vehicle if the character string extracted from the image is any of a plurality of pre-stored registered information, and outputs information indicating that the character string extracted from the image is not any of the pre-stored registered information if the character string extracted from the image is not any of the plurality of pre-stored registered information. In a second information processing method of the present invention, a computer includes a vehicle information acquisition unit that acquires an image of a vehicle taken by a terminal, extracts a character string from the area of ​​the license plate included in the image, and acquires information about the registered vehicle associated with the character string; and if the character string extracted from the image is any of a plurality of pre-stored registered information, outputs information about the registered vehicle, and if the character string extracted from the image is not any of the plurality of pre-stored registered information, outputs information indicating that the character string extracted from the image is not the pre-stored registered information. The third information processing program of the present invention causes a computer to perform the following processes: acquiring an image of a vehicle photographed by a terminal; extracting a character string from the area of ​​the license plate included in the image; acquiring information about the registered vehicle associated with the character string; if the character string extracted from the image is any of a plurality of pre-stored registration information, outputting information about the registered vehicle based on the character string; and if the character string extracted from the image is not any of the plurality of pre-stored registration information, outputting information indicating that the character string extracted from the image is not the pre-stored registration information. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a vehicle information acquisition system that can display information that allows a comparison between a photographed vehicle and a registered vehicle. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram illustrating a configuration example of a first embodiment. [Figure 2] FIG. 10 is an explanatory diagram illustrating photographing a license plate. [Figure 3] 10 is an example of information stored in the storage unit 23. [Figure 4] 10 is an example of information stored in the storage unit 23. [Figure 5] 10 is an example of a display by the display unit 13. [Figure 6] 4 is a flowchart illustrating an operation example of the first embodiment. [Figure 7] FIG. 10 is a block diagram showing a configuration example of a second embodiment. [Figure 8] FIG. 2 is an explanatory diagram for explaining photographing of a vehicle. [Figure 9] 10 is an example of information stored in the storage unit 25. [Figure 10] 10 is an example of a display by the display unit 13. [Figure 11] 10 is a flowchart illustrating an operation example of the second embodiment. [Figure 12]FIG. 10 is a block diagram showing a configuration example of a third embodiment. [Figure 13] 10 is an example of a display by a display unit. [Figure 14] 1 is an example of a hardware configuration. [Figure 15] 1 is an example of a hardware configuration. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the components described in the following embodiments are merely examples and are not intended to limit the technical scope of the present invention.

[0010] (First embodiment)

[0011] (Configuration of the first embodiment) A configuration example of the first embodiment will be described. FIG. 1 is a diagram showing an example of the configuration of a vehicle information acquisition system 1000. In the first embodiment, it is assumed that the present invention is used in a special area where entry of vehicles is restricted when an event or the like is held. Entry into the special area is prohibited for vehicles other than those of event-related personnel and local residents. Therefore, a person guarding the special area patrols the special area and confirms that vehicles parked within the special area are vehicles of pre-registered related personnel (hereinafter referred to as registered vehicles). Alternatively, the person guarding the special area checks vehicles at checkpoints set up near the border between the special area and other areas. At this time, the person guarding the special area checks whether the characters displayed on the license plate attached to the vehicle are those of a registered vehicle.

[0012] Number plates are also called license plates. The characters displayed on the number plates are also called vehicle registration numbers or vehicle numbers. The characters include numbers and symbols.

[0013] Here, a license plate is a vehicle registration plate, vehicle number plate, or sign that is required to be attached to a vehicle under traffic laws and other regulations. Normally, license plates display characters that are unique to each vehicle. Therefore, if the characters on the license plate match the pre-registered characters, the vehicle is authorized to enter the special area. However, someone attempting to illegally enter the special area may forge a license plate. In this case, the person guarding the special area may mistake a vehicle that is not authorized to enter as a registered vehicle.

[0014] Therefore, the vehicle information acquisition system 1000 of the first embodiment employs the following configuration: In the following description, the user of the vehicle information acquisition system 1000 and the user of the mobile terminal 100 are described as guarding a special area.

[0015] The vehicle information acquisition system 1000 includes a mobile terminal 100 and a server 200. The mobile terminal 100 is communicably connected to the server 200 via a wireless or wired network. The mobile terminal 100 includes an acquisition unit 10, a character extraction unit 11, a display control unit 12, a transmission unit 14, a reception unit 15, and a display unit 13. The server 200 includes a reception unit 20, a vehicle information acquisition unit 21, a transmission unit 22, and a storage unit 23.

[0016] The acquisition unit 10 in the mobile terminal 100 acquires an image that captures at least a portion of the license plate attached to the vehicle. The image may be captured by a photographing unit (not shown) included in the mobile terminal 100. The photographing unit (not shown) may be a camera having, for example, a CCD (Charge Coupled Device) sensor, a CMOS (Complementary Meta Oxide Semiconductor) sensor, an image processor, or the like. The photographing unit (not shown) generates image data by converting received light into image data.

[0017] FIG. 2 is an explanatory diagram illustrating the photographing of a license plate by a photographing unit within the mobile terminal 100. The user of the mobile terminal 100 operates the mobile terminal 100 while pointing the photographing unit (not shown) towards the photographing vehicle 1 to photograph the license plate attached to the photographing vehicle 1. The acquisition unit 10 acquires the image generated by the photographing unit (not shown). The image acquired by the acquisition unit 10 needs to show at least a part of the license plate. Images 2 and 3 are examples of images acquired by the acquisition unit 10. Image 2 shows the entire license plate 5 attached to the photographing vehicle 1. On the other hand, image 3 shows only a part of the license plate 5.

[0018] The character extraction unit 11 extracts characters from the license plate portion in the image acquired by the acquisition unit 10. The character extraction unit 11 may identify the license plate portion in the image by pattern recognition or the like. The character extraction unit 11 performs OCR (Optical Character Recognition) processing on the image acquired from the acquisition unit 10 to extract characters.

[0019] For example, when image 2 in FIG. 2 is acquired, the character extraction unit 11 extracts the characters "Shinagawa," "300," "sa," and "12-34." The character extraction unit determines which item the extracted characters correspond to based on the position data of the extracted characters and the position data of pre-stored items. The item position data may be stored in the storage unit 23. The position indicated by the position data may be an absolute position in the image, or the relative position of each character or item. An item indicates, for example, the content of information required to be displayed on a vehicle by traffic laws. For example, the items include "Transport Bureau, etc.", "Type," "Business Use," and "Designated Number." The character extraction unit 11 does not have to determine the item corresponding to the characters.

[0020] The transmitting unit 14 in the mobile terminal 100 transmits the characters extracted by the character extracting unit 11 and the items corresponding to the characters to the server 200. The receiving unit 20 in the server 200 receives the characters and items transmitted by the transmitting unit 14.

[0021] The storage unit 23 in the server 200 stores the vehicle information of the registered vehicle in association with the characters.

[0022] Here, vehicle information may be any information related to a vehicle. For example, vehicle information may be information recorded on a vehicle inspection certificate, such as the chassis number, model, body shape, and owner information, information displayed on a license plate attached to the vehicle, an image of the vehicle, and features of the image. An example of information stored in the memory unit 23 is shown in FIG. 3. The characters are divided into the following categories: "Transport Bureau, etc.", "Type," "Business Use," and "Designation Number." The characters contained in these categories are required to be displayed on license plates under traffic laws. The vehicle information also includes "Vehicle Model," "Vehicle Name," "Manufacturer," "Year," and "Color." Note that the information stored in the memory unit 23 is not limited to that shown in FIG. 3.

[0023] From Figure 3, the characters "Shinagawa", "300", "sa", and "12-34" correspond to the vehicle information "Standard passenger car", "Purlis", "Toyoda", "H23", and "White". Also, the characters "Shinagawa", "350", "ha", and "34-56" correspond to the vehicle information "Standard passenger car", "Ellie", "H25", and "Red". The characters "Kashiwa", "550", "mi", and "56-78" correspond to the vehicle information "Compact passenger car", "Aldo", "Susuki", "H20", and "Black".

[0024] Furthermore, the storage unit 23 stores the vehicle information and the comparison image in association with each other. An example of the information stored in the storage unit 23 is shown in FIG. 4. In FIG. 4, the vehicle information "Standard passenger car", "Purlis", "Toyoda", "H23", and "White" corresponds to the comparison image "aaa.jpg". Furthermore, the vehicle information "Standard passenger car", "Purlis", "Toyoda", "H23", and "Black" corresponds to the comparison image "bbb.jpg". The vehicle information "Standard passenger car", "Purlis", "Toyoda", "H23", and "Gray" corresponds to the comparison image "ccc.jpg".

[0025] The image file format is not limited to JPEG (Joint Photographic Experts Group). For example, the image file format may be GIF (Graphics Interchange Format), PNG (Portable Network Graphics), BMP (Microsoft Windows (registered trademark) Bitmap Image), etc.

[0026] The comparison image includes an image of a vehicle that satisfies the vehicle information. In other words, the comparison image includes an image of a vehicle whose vehicle information, such as model, name, manufacturer, year, and color, matches that of a registered vehicle. For example, the comparison image "aaa.jpg" includes an image of a vehicle whose model is a passenger car, whose name is Purlis, whose manufacturer is Toyoda, whose year is H23, and whose color is white. The comparison image may also be an image of a vehicle that appears in a catalog, product introduction site, etc. The comparison image may also be an image of a registered vehicle.

[0027] The vehicle information acquisition unit 21 receives the characters from the receiving unit 20. Then, the vehicle information acquisition unit 21 acquires vehicle information corresponding to the extracted characters from the storage unit 23. That is, the vehicle information acquisition unit 21 acquires vehicle information corresponding to characters that match the extracted characters from the storage unit 23. For example, assume that the extracted characters are "Shinagawa", "300", "sa", and "12-34", and the storage unit 23 stores the information shown in FIG. 3. In this case, the vehicle information acquisition unit 21 refers to the storage unit 23 and acquires the vehicle information "Standard Passenger Car", "Purlis", "Toyoda", "H23", and "White" that corresponds to the characters "Shinagawa", "300", "sa", and "12-34".

[0028] At this time, the vehicle information acquisition unit 21 may check for each item whether the extracted characters match the characters stored in the storage unit 23. For example, the vehicle information acquisition unit 21 determines whether the extracted characters corresponding to the item "Transport Bureau, etc." match the characters stored in the storage unit 23 corresponding to the item "Transport Bureau, etc." Similarly, the vehicle information acquisition unit 21 determines whether the characters associated with the items "Type," "Business Use," and "Designation Number" match for each item. Then, if the characters match for all items, the vehicle information corresponding to the extracted characters may be acquired from the storage unit 23.

[0029] Furthermore, the vehicle information acquisition unit 21 acquires items corresponding to the vehicle information from the storage unit 23. For example, the vehicle information acquisition unit 21 associates "vehicle model" with "standard passenger car," "vehicle name" with "Purlis," "manufacturer" with "Toyoda," "year of manufacture" with "H23," and "color" with "white," and acquires these. Note that the vehicle information acquisition unit 21 does not have to acquire items from the storage unit 23.

[0030] Furthermore, vehicle information acquisition unit 21 acquires, together with the vehicle information, a comparison image corresponding to the vehicle information from storage unit 23. That is, vehicle information acquisition unit 21 acquires, from storage unit 23, a comparison image corresponding to the vehicle information acquired by storage unit 23. For example, as shown in FIG. 4, if vehicle information acquisition unit 21 acquires "Standard Passenger Car," "Purlis," "Toyoda," "H23," and "White" as vehicle information, it acquires comparison image "aaa.jpg" from storage unit 23. Transmission unit 22 in server 200 transmits the vehicle information, items, and comparison image acquired by vehicle information acquisition unit 21 to mobile terminal 100.

[0031] The receiving unit 15 in the mobile terminal 100 receives the vehicle information, items, and comparison images transmitted from the server 200. The display control unit 12 causes the display unit 13 to display the vehicle information and items acquired by the vehicle information acquisition unit 21. At this time, the display control unit 12 causes the display unit 13 to display the comparison images together with the vehicle information. Note that the display control unit 12 does not have to display the items on the display unit 13.

[0032] The display unit 13 displays vehicle information, items, and a comparison image under the control of the display control unit 12. An example of a display by the display unit 13 is shown in FIG. 5. In FIG. 5, the display unit 13 of the mobile terminal 100 displays the comparison image "aaa.jpg," vehicle information, and items. As described above, the vehicle shown in the comparison image "aaa.jpg" is a standard passenger car, has a name "Purulis," is a manufacturer "Toyoda," is a year of manufacture "H23," and is white in color. In addition, the vehicle information displayed by the display unit 13 in FIG. 5 is "standard passenger car," "Purulis," "Toyoda," "H23," and "white." In addition, the items displayed by the display unit 13 are "car type," "car name," "manufacturer," "year of manufacture," and "color."

[0033] The display by the display unit 13 is not limited to that shown in Fig. 4. For example, the display unit 13 does not have to display any items. The display unit 13 may also display the image acquired by the acquisition unit 10 (an image of the photographed vehicle 1) along with the above display. The display unit 13 may also display the characters extracted by the character extraction unit 11, i.e., the characters displayed on the license plate of the photographed vehicle 1.

[0034] (Operation of the first embodiment) Next, a description will be given of the operation of the vehicle information acquisition system 1000. Fig. 6 is a sequence diagram illustrating the operation of the vehicle information acquisition system 1000.

[0035] First, the acquisition unit 10 in the mobile terminal 100 acquires an image that shows at least a part of the license plate attached to the vehicle (S1). The image may be captured by a photographing unit (not shown) included in the mobile terminal 100. The photographing unit (not shown) may be a camera having, for example, a CCD sensor, a CMOS sensor, an image processor, or the like. The photographing unit (not shown) generates an image by converting received light into data.

[0036] For example, the acquisition unit 10 may acquire an image after the user of the mobile terminal 100 starts a predetermined application. For example, when the user starts the application, an image capturing unit (not shown) captures an image of the license plate. Then, the acquisition unit 10 acquires the image from the image capturing unit (not shown).

[0037] Next, character extraction unit 11 extracts characters from the license plate portion of the image acquired by acquisition unit 10 (S2). Character extraction unit 11 performs OCR processing on the image acquired from acquisition unit 10 to extract characters. Character extraction unit 11 also determines items corresponding to the extracted characters. Thereafter, transmission unit 14 transmits the characters and items extracted by character extraction unit 11 to server 200.

[0038] The receiving unit 20 in the server 200 receives the characters and items transmitted by the transmitting unit 14. The vehicle information acquiring unit 21 receives the characters and items from the receiving unit 20. Then, the vehicle information acquiring unit 21 acquires vehicle information corresponding to the extracted characters from the storage unit 23 (S3). At this time, the vehicle information acquiring unit 21 acquires the items corresponding to the vehicle information. Furthermore, the vehicle information acquiring unit 21 acquires a comparison image corresponding to the acquired vehicle information from the storage unit 23 (S4). Note that the vehicle information acquiring unit 21 does not have to acquire the items from the storage unit 23.

[0039] Transmitting unit 22 in server 200 transmits the vehicle information, items, and comparison images acquired by vehicle information acquiring unit 21 to mobile terminal 100. Then, receiving unit 15 in mobile terminal 100 receives the vehicle information, items, and comparison images transmitted from server 200.

[0040] The display control unit 12 causes the display unit 13 to display the comparative image together with the vehicle information and items received by the receiving unit 15. Note that the display control unit 12 does not have to cause the display unit 13 to display the items.

[0041] As described above, the vehicle information acquisition system 1000 of this embodiment can display vehicle information for registered vehicles whose license plate characters match those of the photographed vehicle. Furthermore, the vehicle information acquisition system 1000 can display an image of a vehicle that satisfies the vehicle information. This allows a user of the mobile terminal 100 to easily grasp the appearance of a vehicle with the name "Purlis" by looking at the displayed image, even if the user does not know what the vehicle looks like. Therefore, by comparing the image displayed by the mobile terminal 100 with the appearance, etc., of the photographed vehicle, the user can efficiently determine whether the photographed vehicle matches the registered vehicle.

[0042] Note that server 200 may have character extraction unit 11. In this case, transmission unit 14 in mobile terminal 100 transmits the image acquired by acquisition unit 10 to server 200. Then, character extraction unit 11 in server 200 extracts characters from the license plate portion in the image received from mobile terminal 100, and transfers the extracted characters to vehicle information acquisition unit 21.

[0043] Furthermore, although storage unit 23 has separate data associating characters with vehicle information and data associating vehicle information with comparative images, storage unit 23 may have one data associating characters, vehicle information, and comparative images. In this case, vehicle information acquisition unit 21 can acquire vehicle information corresponding to characters and comparative images at the same time, thereby improving the processing speed of vehicle information acquisition system 1000.

[0044] Furthermore, the vehicle information acquisition unit 21 may calculate the similarity between the extracted characters and the characters stored in the memory unit 23, and acquire vehicle information corresponding to the character with the highest similarity. Alternatively, the vehicle information acquisition unit 21 may acquire all vehicle information corresponding to characters whose similarity is equal to or greater than a threshold. In this case, the display control unit 12 may display all vehicle information acquired by the vehicle information acquisition unit 21 on the display unit 13. The display control unit 12 may also display the vehicle information on the display unit 13 in order of similarity. This allows the user to compare the registered vehicle with the photographed vehicle even when the characters displayed on the license plate in the image are unclear or when some of the characters are not captured in the image.

[0045] Furthermore, if the storage unit 23 does not store any characters whose similarity to the extracted characters is equal to or greater than the threshold value, or any characters identical to the extracted characters, the vehicle information acquisition unit 21 may transmit a message to that effect to the mobile device 100 via the transmission unit 22. The display control unit 12 may then display a message on the display unit 13 indicating that the photographed vehicle is not registered. This allows the user to know that the photographed vehicle is not a registered vehicle. Alternatively, the display control unit 12 may display a message on the display unit 13 encouraging the user to take another photograph. This allows the user to take another photograph of the same vehicle and check whether it is registered. These messages may be stored in advance in a storage unit (not shown) within the mobile device 100, or may be transmitted to the mobile device 100 by the server 200.

[0046] Furthermore, the storage unit 23 may store information about the owner of the registered vehicle in association with the characters. For example, the storage unit 23 may register a facial image of the owner of the registered vehicle in association with the characters. In this case, the vehicle information acquisition unit 21 may acquire the facial image along with the vehicle information. Furthermore, the display control unit 12 may display the facial image on the display unit 13 along with the vehicle information. This allows the user to determine whether the photographed vehicle matches the registered vehicle by checking the owner of the photographed vehicle, even if the vehicle with a forged license plate attached has the same model, name, color, etc. as the registered vehicle.

[0047] In addition, although the server 200 has the storage unit 23 in this embodiment, the storage unit in the present invention may be realized by a plurality of hardware configurations. For example, a storage device in the server 200 may store data associating characters with vehicle information, and a storage device in another server that can communicate with the server 200 may store data associating vehicle information with comparison images.

[0048] The information displayed on license plates is regulated by the Road Transport Vehicle Act in Japan, but may differ in other countries. Therefore, the information displayed on license plates may also differ depending on the country. Taking these points into consideration, the content displayed on license plates is not limited to that described in this embodiment.

[0049] Second Embodiment

[0050] (Configuration of the second embodiment) An example of the configuration of the second embodiment will be described. Fig. 7 is a diagram showing an example of the configuration of a vehicle information acquisition system 1001. The vehicle information acquisition system 1001 differs from the vehicle information acquisition system 1000 in that it includes a vehicle information extraction unit 16, a collation unit 24, and a storage unit 25. The rest of the configuration is the same as in the first embodiment, so description thereof will be omitted where appropriate.

[0051] The acquisition unit 10 acquires an image that includes at least a part of the vehicle in addition to an image that includes at least a part of the license plate. The image that includes at least a part of the vehicle may be captured by a capturing unit (not shown) included in the mobile terminal 100.

[0052] FIG. 8 is an explanatory diagram illustrating the photographing of the photographed vehicle 1 by the photographing unit in the mobile terminal 101. The user of the mobile terminal 101 operates the mobile terminal 101 with the photographing unit (not shown) pointed toward the photographed vehicle 1 to photograph the photographed vehicle 1. The acquisition unit 10 then acquires the image generated by the photographing unit (not shown). The image acquired by the acquisition unit 10 needs to show at least a part of the photographed vehicle 1. However, it is preferable that the image show parts of the vehicle with many irregularities and parts that vary depending on the vehicle model. For example, it is preferable that the image show the vehicle's headlights, grille, bumper, emblem, ornament, wheels, etc.

[0053] Image 4 and image 6 are examples of images acquired by the acquisition unit 10. Image 4 shows the entire photographed vehicle 1. On the other hand, image 6 shows only a part of the photographed vehicle 1.

[0054] The vehicle information extraction unit 16 extracts vehicle information of a vehicle from an image that shows at least a portion of the vehicle. In this embodiment, the vehicle information is, for example, a feature of the image. The vehicle information extraction unit 16 extracts the feature from the vehicle portion in the image acquired by the acquisition unit 10. The vehicle information extraction unit 16 may identify the vehicle portion in the image by pattern recognition or the like. The feature extraction may be realized by a method such as SIFT (Scale-Invariant Feature Transform) or HOG (Histograms of Oriented Gradients). Here, for example, it is assumed that the vehicle information extraction unit 16 extracts feature data X from the vehicle portion in the image.

[0055] The transmission unit 14 transmits the characters “Shinagawa”, “300”, “sa”, and “12-34” extracted by the character extraction unit 11 and the feature amount data X extracted by the vehicle information extraction unit 16 to the server 201.

[0056] The storage unit 25 in the server 201 stores vehicle information of the registered vehicle in association with characters displayed on the license plate of the registered vehicle. In addition to the information stored in the storage unit 23, the storage unit 25 stores image features as vehicle information. The image is an image of an actual registered vehicle. Note that the image may not be an image of the registered vehicle, but may be an image of a vehicle having the same model, name, color, etc. as the registered vehicle. For example, the image may be an image of a vehicle published in a catalog, product introduction site, etc.

[0057] An example of data stored in the storage unit 25 is shown in Fig. 9. As can be seen from Fig. 9, the characters "Shinagawa", "300", "sa", and "12-34" correspond to feature amount data X. The characters "Shinagawa", "350", "ha", and "34-56" correspond to feature amount data Y. The characters "Kashiwa", "550", "mi", and "56-78" correspond to feature amount data Z. The vehicle information other than the features is the same as that in Fig. 3, and therefore will not be described here.

[0058] The vehicle information acquisition unit 21 acquires, from the storage unit 25, vehicle features corresponding to the characters "Shinagawa", "300", "sa", and "12-34" received by the receiving unit 20. As can be seen from FIG. 9, the vehicle feature corresponding to the characters "Shinagawa", "300", "sa", and "12-34" is feature amount data X, and therefore the vehicle information acquisition unit 21 acquires the feature amount data X, which is the vehicle feature. Furthermore, the vehicle information acquisition unit 21 acquires "standard passenger car", "Purlis", "Toyoda", "H23", and "white", which are vehicle information corresponding to the characters "Shinagawa", "300", "sa", and "12-34".

[0059] The matching unit 24 acquires the vehicle information extracted by the vehicle information extraction unit 16 from the receiving unit 20. The matching unit 24 also acquires the vehicle information acquired by the vehicle information acquisition unit 21. Then, the matching unit 24 compares the extracted vehicle information with the vehicle information acquired by the vehicle information acquisition unit 21.

[0060] Thereafter, the matching unit 24 transmits the matching result to the mobile terminal 101 via the transmission unit 22. The matching result is, for example, information on whether the matching was successful or not, the degree of match, etc. Note that even if the extracted vehicle information does not completely match the vehicle information acquired by the vehicle information acquisition unit, the matching unit 24 may output a matching result indicating successful matching. For example, if the degree of match between the extracted vehicle information and the vehicle information acquired by the vehicle information acquisition unit is equal to or greater than a predetermined threshold, the matching unit 24 may output a matching result indicating successful matching.

[0061] For example, in the above example, the matching unit 24 acquires feature amount data X of an image of the photographed vehicle 1. Furthermore, the matching unit 24 acquires feature amount data X of an image of a registered vehicle. Then, the matching unit 24 matches these acquired feature amounts and causes the transmission unit 22 to transmit the matching result.

[0062] The transmission unit 22 transmits the vehicle information acquired by the vehicle information acquisition unit 21 and the matching result by the matching unit 24 to the mobile terminal 100. For example, the transmission unit 22 transmits the characters "Standard Passenger Car," "Purlis," "Toyoda," "H23," and "White" acquired by the vehicle information acquisition unit 21 and the matching result. Note that the transmission unit 22 may also transmit the feature amount acquired by the vehicle information acquisition unit 21.

[0063] The display control unit 12 causes the display unit 13 to display the matching result by the matching unit 24 together with the vehicle information received by the receiving unit 15 from the server 201. An example of a display by the display unit 13 is shown in Fig. 10. As shown in Fig. 10, the display unit 13 displays the matching result "Matching successful" along with the characters "Standard passenger car," "Purlis," "Toyoda," "H23," and "White."

[0064] (Operation of the second embodiment) Next, a description will be given of the operation of the vehicle information acquisition system 1001. Fig. 11 is a sequence diagram illustrating the operation of the vehicle information acquisition system 1001. Descriptions of operations similar to those of the vehicle information acquisition system 1000 shown in Fig. 10 will be omitted where appropriate.

[0065] After characters are extracted from an image showing at least a part of the license plate (S2), the acquisition unit 10 in the mobile terminal 100 acquires an image showing at least a part of the vehicle. This image may be captured by a capture unit (not shown) included in the mobile terminal 100, similar to the image acquired in S1. Note that if at least a part of the vehicle is captured in the image acquired in S1, the acquisition unit 10 does not need to acquire the image again. Furthermore, the acquisition unit 10 may acquire an image showing at least a part of the vehicle before acquiring the image showing at least a part of the license plate (S1).

[0066] Then, the vehicle information extraction unit 16 extracts vehicle information of the vehicle from the image in which at least a portion of the vehicle is captured (S6). The vehicle information in this embodiment is, for example, image feature amounts. The vehicle information extraction unit 16 extracts feature amounts from the vehicle portion in the image acquired by the acquisition unit 10. The vehicle information extraction unit 16 may identify the vehicle portion in the image by pattern recognition or the like. The extraction of feature amounts may be achieved by a method such as SIFT (Scale-Invariant Feature Transform) or HOG (Histograms of Oriented Gradients). Note that the order of S1, S2, and S6 is not limited to this.

[0067] The transmitting unit 14 transmits the characters extracted by the character extracting unit 11 and the vehicle information extracted by the vehicle information extracting unit 16 to the server 201. The receiving unit 20 in the server 200 receives the characters and vehicle information transmitted by the transmitting unit 14.

[0068] The vehicle information acquisition unit 21 acquires vehicle information corresponding to the extracted characters from the storage unit 25 (S3). The vehicle information acquired by the vehicle information acquisition unit 21 is, for example, a feature amount of an image.

[0069] The matching unit 24 acquires the vehicle information extracted by the vehicle information extraction unit 16 and the vehicle information acquired by the vehicle information acquisition unit 21. Then, the matching unit 24 matches the extracted vehicle information with the vehicle information acquired by the vehicle information acquisition unit (S7). Thereafter, the matching unit 24 causes the transmission unit 22 to transmit the matching result to the mobile terminal 101.

[0070] The matching result is, for example, information on whether matching was successful or not, the degree of match, etc. Note that even if the extracted vehicle information does not completely match the vehicle information acquired by the vehicle information acquisition unit, the matching unit 24 may output a matching result indicating successful matching. For example, if the degree of match between the extracted vehicle information and the vehicle information acquired by the vehicle information acquisition unit is equal to or greater than a predetermined threshold, the matching unit 24 may output a matching result indicating successful matching.

[0071] The display control unit 12 causes the display unit 13 to display the result of the matching by the matching unit 24 together with the vehicle information received by the receiving unit 15 from the server 201 (S8).

[0072] As described above, the vehicle information acquisition system 1001 of this embodiment can compare the vehicle information of the photographed vehicle with the vehicle information of the registered vehicle and display the comparison result. This allows the user of the mobile terminal 100 to know whether the registered vehicle and the photographed vehicle match in terms other than the license plate, without having to compare them visually. In other words, the user can more efficiently determine whether the registered vehicle and the photographed vehicle match.

[0073] In this embodiment, the vehicle information extraction unit 16 acquires feature amounts from an image of the photographed vehicle 1, but the vehicle information acquired from the image is not limited to feature amounts. For example, the vehicle information may be the ratio of the lengths of the characteristic parts of the vehicle. The characteristic parts of the vehicle may be, for example, the wheelbase, vehicle height, overall length, etc. The lengths of the characteristic parts of the vehicle may be extracted by the vehicle information extraction unit 16 through image recognition processing. Alternatively, the vehicle information extraction unit 16 may extract the lengths of the characteristic parts of the vehicle from information input by the user via an input unit (not shown). The input unit (not shown) may be a mouse, touch panel, keyboard, etc.

[0074] At this time, the vehicle information extraction unit 16 extracts the length ratio of the characteristic parts by dividing the lengths of the two or more extracted characteristic parts. The mobile terminal 101 transmits the extracted length ratio of the characteristic parts to the server 201. The storage unit 25 in the server 201 stores the length ratio of the characteristic parts of the vehicle as vehicle information. Then, the vehicle information acquisition unit 21 acquires the length ratio of the characteristic parts corresponding to the extracted characters from the storage unit 25. The comparison unit 24 compares the length ratio of the characteristic parts extracted by the vehicle information extraction unit 16 with the length ratio of the characteristic parts acquired by the vehicle information acquisition unit 21.

[0075] Furthermore, the vehicle information extraction unit 16 may extract vehicle information for each item. For example, the vehicle information extraction unit 16 may extract the feature amount of the entire vehicle and the length of the characteristic part of the vehicle. In this case, the matching unit 24 may match the vehicle features for each item. Then, the matching unit 24 may calculate an overall degree of match based on the degree of match for each item and output the matching result.

[0076] (Third embodiment)

[0077] (Configuration of the third embodiment) 12 is a diagram showing an example of the configuration of a vehicle information acquisition system 1002. The vehicle information acquisition system 1002 includes an acquisition unit 300, a character extraction unit 400, a vehicle information acquisition unit 500, and a display control unit 600.

[0078] The acquisition unit 300 acquires an image that captures at least a portion of the license plate attached to the vehicle. The image may be captured by a camera (not shown). The camera (not shown) may be, for example, a camera having a CCD (Charge Coupled Device) sensor, a CMOS (Complementary Meta Oxide Semiconductor) sensor, an image processor, or the like. The camera (not shown) generates image data by converting received light into image data. The camera (not shown) may be located within the vehicle information acquisition system 1002 or in an external system.

[0079] The character extraction unit 400 extracts characters from the license plate portion of the image acquired by the acquisition unit 300. The character extraction unit 400 performs OCR (Optical Character Recognition) processing on the image acquired from the acquisition unit 300 to extract characters.

[0080] The vehicle information acquisition unit 500 acquires vehicle information corresponding to the extracted characters from a storage unit (not shown) that stores vehicle information of registered vehicles in association with characters. The storage unit (not shown) may be located within the vehicle information acquisition system 1002 or in an external system. The vehicle information may be any information related to a vehicle. For example, the vehicle information may be information displayed on the license plate attached to the vehicle (chassis number), information written on the vehicle inspection certificate of the vehicle, an image of the vehicle, feature quantities of the image, information about the owner of the vehicle, etc.

[0081] The display control unit 600 displays on a display unit (not shown) the vehicle information acquired by the vehicle information acquisition unit 500. The display unit (not shown) may be present within the vehicle information acquisition system 1002 or in an external system.

[0082] An example of display by the display unit is shown in Fig. 13. In Fig. 13, the display unit 700 displays the vehicle type "standard passenger car" and the vehicle color "white" as vehicle information of the registered vehicle. However, the vehicle information displayed by the display unit is not limited to the vehicle type and vehicle color.

[0083] The vehicle information acquisition system 1002 of this embodiment can display vehicle information for registered vehicles determined based on the license plate of the photographed vehicle. That is, a user of the vehicle information acquisition system 1002 can compare the photographed vehicle with the registered vehicle by comparing the appearance, etc., of the photographed vehicle with the displayed vehicle information of the registered vehicle. In other words, this embodiment can provide a vehicle information acquisition system that can display information that allows a comparison between the photographed vehicle and the registered vehicle.

[0084] (Hardware configuration) Next, an example of the hardware configuration of the present invention will be described. Figures 14 and 15 are diagrams showing an example of the hardware configuration of vehicle information acquisition systems 1000 to 1002. A computer 1003 shown in Figure 14 and a mobile terminal shown in Figure 15 are connected to each other wirelessly or via a wire so as to be able to communicate with each other.

[0085] The computer 1003 may be any general-purpose computer. For example, the computer 1003 may be a laptop PC, a desktop PC, a server, a workstation, a blade server, etc. The computer 1003 may also be communicatively connected to a server or another computer.

[0086] 14, the computer 1003 includes a processor 1402, a memory 1404, a storage 1406, a north bridge 1408 connected to the memory 1404, and an expansion slot 1410. The computer 1003 also includes a south bridge 1412 connected to a communication interface 1414 and the storage 1406. These components are connected to each other by a bus and fixed to the motherboard.

[0087] The processor 1402 reads and executes instructions stored in the memory 1404, the storage 1406, etc. The processor 1402 may execute instructions to display an image on a display device (not shown).

[0088] The processor 1402 reads and executes programs stored in the memory 1404 and storage 1406. The processor 1402 may be a circuit such as a CPU (Central Processing Unit) or ASICs (Application Specific Integrated Circuits). The processor 1402 also realizes the functions of the vehicle information acquisition unit 500 and the matching unit 24. That is, the processor 1402 realizes the functions of the vehicle information acquisition unit 500 and the matching unit 24 by reading out the programs stored in the memory 1404 and storage 1406. The processor 1402 may also realize the function of the vehicle information acquisition unit 21.

[0089] The memory 1404 is a computer-readable storage medium that stores information. The memory 1404 may be a volatile storage medium such as RAM, or a non-volatile storage medium such as ROM. The memory 1404 may also be an optical disk, a magnetic disk, or the like. The storage 1406 may be a large-capacity computer-readable storage medium. The storage 1406 may be, for example, a floppy disk, a hard disk drive, an optical disk, a flash memory, or the like. The computer program stored in the computer-readable storage medium includes instructions that are executed to realize the above-described embodiments. The memory 1404 also realizes the storage unit 26. The storage unit 26 may be the storage unit 23 of the first embodiment or the storage unit 25 of the second embodiment.

[0090] The northbridge 1408 connects the computer's brains, such as the CPU and memory, and controls the timing and speed of data transfer. The northbridge 1408 mainly mediates data between high-speed devices. On the other hand, the southbridge 1412 mediates data between relatively slower devices. For example, the northbridge 1408 may be connected to the memory 1404, the processor 1402, and an expansion slot 1410 that can accept an expansion card (not shown). The southbridge 1412 may also be connected to the storage 1406 and a communication interface 1414.

[0091] The communication interface 1414 may include various communication ports such as USB, Bluetooth (registered trademark), Ethernet (registered trademark), and wireless Ethernet. These communication ports may also be connected to one or more input / output devices, such as communication devices such as a keyboard, a pointing device, a scanner, a wireless LAN (Local Area Network) adapter, and a router. That is, the computer 1003 may be connected to the mobile terminal 1004 via the communication interface 1414. The communication interface 1414 may also include a wireless LAN adapter. The communication interface 1414 realizes the functions of the receiving unit 20 and the transmitting unit 22.

[0092] Next, a description will be given of the mobile terminal 1004 shown in Fig. 15. The mobile terminal 1004 may be a mobile terminal such as a mobile phone, a smartphone, or a PDA (personal digital assistant).

[0093] The mobile terminal 1004 may include a display 1454, a transceiver 1468, an audio codec 1460, a control interface 1458, and a display interface 1456. The mobile terminal 1004 may also include a communication interface 1466, a processor 1452, a GPS (Global Positioning System) receiving module 1470, additional memory 1474, memory 1464, and an external interface 1462.

[0094] The display 1454 may be, for example, a TFT LCD (Thin-film Transistor Liquid Crystal Display), an OLED (Organic Light Emitting Diode) display, or other display. The display may also be connected to the control interface 1458 or the display interface 1456. The processor 1452 may exchange information with a user of the mobile terminal 1004 via the control interface 1458 or the display interface 1456. The display 1454 implements the function of the display unit 13. The display 1454 may also implement the function of the display unit 700.

[0095] The display interface 1456 may be configured by a circuit. The display interface 1456 controls a display to display images and other information to a user. The display interface 1456 implements the functions of the display control unit 600. The display interface 1456 may also implement the functions of the display control unit 12.

[0096] The control interface 1458 receives commands from a user and may convert the received commands for passing to the processor 1452.

[0097] Mobile terminal 1004 may also include an external interface 1462 that allows external devices to communicate with processor 1402. External interface 1462 may provide wired or wireless communication for mobile terminal 1004.

[0098] The expansion memory 1474 connects to the mobile terminal 1004 via the expansion interface 1472. The expansion memory 1474 may also include, for example, a SIMM (Single In Line Memory Module) card interface. The expansion memory 1474 may also provide additional storage space for the mobile terminal 1004 to store applications and other information. The expansion memory 1474 may also store information related to security. The expansion memory 1474 may function as a security management device for the mobile terminal 1004. For example, the expansion memory 1474 is programmed with instructions that ensure secure use of the mobile terminal 1004.

[0099] The mobile terminal 1004 performs wireless communication via the communication interface 1466. That is, the mobile terminal 1004 may communicate with the computer 1003 via the communication interface 1466. The communication interface 1466 realizes the functions of the transmitter 14 and the receiver 15.

[0100] The communication interface 1466 may include digital signal processing circuitry and may implement communication under a mode or protocol such as Global System for Mobile Communication (GMS), Short Message Service (SMS), or Enhanced Messaging Service (EMS). Alternatively, the communication interface 1466 may implement communication under a mode or protocol such as Multimedia Messaging Service (MMS), Code Division Multiple Access (CDMA), or Time Division Multiple Access (TDMA).

[0101] Furthermore, communication interface 1466 may realize communications under modes and protocols such as Personal Digital Cellular (PDC), Wideband Code Division Multiple Access (WCDMA), CDMA2000, and General Packet Radio Service (GPRS). Such communications may occur, for example, via transceiver 1468. Short-range communications, such as communications using Bluetooth, Wi-Fi (registered trademark), and the like, may also occur via transceiver 1468.

[0102] The GPS receiver module 1470 may provide data for use by applications running on the mobile terminal 1004. Such data may be, for example, location-related or navigation-related data.

[0103] Mobile terminal 1004 may also perform voice communication via audio codec 1460. Audio codec 1460 receives voice information from a user and converts it into usable digital information. Audio codec 1460 then generates audio that can be heard by the user. The information handled by audio codec 1460 may include voice calls, recorded sounds (voice messages, music files, etc.), sounds generated by applications on mobile terminal 1004, etc.

[0104] The processor 1452 and memory 1464 are similar to those of the computer 1003, and therefore a description thereof will be omitted. The processor 1452 realizes the functions of the acquisition unit 800, the character extraction unit 400, and the vehicle information extraction unit 16.

[0105] The vehicle information acquisition systems 1000 to 1002 may be realized by either the computer 1003 or the mobile terminal 1004. In this case, for example, the processor 1402 in the computer 1003 may also realize the functions of the acquisition unit 800, the character extraction unit 400, the vehicle information extraction unit 16, and the display control unit 600. Furthermore, for example, the processor 1452 may also realize the functions of the vehicle information acquisition unit 21 and the matching unit 24.

[0106] The above-described embodiments are merely examples of the present invention, and various modifications can be made within the scope of the spirit of the present invention as defined in the claims. [Explanation of symbols]

[0107] 1 vehicle 2 images 3 images 4 images 5. License Plate 6 images 10 Acquisition Department 11 Character extraction section 12 Display control unit 13 Display section 14 Transmitter 15 Receiving unit 16 Vehicle information extraction unit 20 Receiving unit 21 Vehicle information acquisition unit 22 Transmitter 23 Memory section 24 Matching unit 25 Memory section 26 Memory section 100 mobile devices 101 Mobile Devices 200 servers 201 Server 300 Acquisition Department 400 Character extraction part 500 Vehicle information acquisition unit 600 Display control unit 700 Display 1000 Vehicle Information Acquisition System 1001 Vehicle Information Acquisition System 1002 Vehicle information acquisition system 1003 Computer 1004 Mobile devices 1402 processor 1404 memory 1406 Storage 1408 Northbridge 1410 expansion slot 1412 Southbridge 1414 Communication Interface 1452 processor 1454 Display 1456 Display Interface 1458 Control Interface 1460 Audio Codec 1466 Communication Interface 1468 Transceiver 1462 External Interface 1464 memory 1470 GPS Receiver Module 1472 Expansion Interface 1474 additional memory

Claims

1. an acquisition unit that acquires an image of a vehicle taken by a terminal; an extraction unit that extracts a character string from an area of ​​the license plate included in the image; a vehicle information acquisition unit that acquires information on a registered vehicle associated with the character string; an output unit that outputs information about the registered vehicle when the character string extracted from the image is any one of a plurality of pieces of pre-stored registration information, and outputs information indicating that the character string extracted from the image is not any one of the plurality of pieces of pre-stored registration information when the character string extracted from the image is not any one of the plurality of pieces of pre-stored registration information; Equipped with the vehicle information acquisition unit acquires, from a storage unit, information about the registered vehicle and a comparison image corresponding to the information about the registered vehicle; The output unit outputs the comparison image together with the information on the registered vehicle to a display device. Information processing device.

2. the output unit outputs the character string extracted from the image together with the image to a display device. The information processing device according to claim 1 .

3. When the character string extracted from the image is not a pre-stored character string, the output unit outputs, to the terminal, information indicating that the character string extracted from the image is not a pre-stored character string.

3. The information processing device according to claim 1.

4. the output unit outputs, to a display device, information about the registered vehicle that differs depending on the character string extracted from the image. The information processing device according to claim 1 .

5. a storage unit that stores information about the registered vehicle in association with a character string on the license plate; the vehicle information acquisition unit acquires information about the registered vehicle from the storage unit based on the character string extracted from the area of ​​the license plate; The information processing device according to claim 1 .

6. The character string extracted from the image includes characters indicating the transport bureau, type, business type, and designated number. The information processing device according to claim 1 .

7. a vehicle information extraction unit that extracts feature amounts of a portion related to the vehicle from the image; a matching unit that matches the extracted feature amount with a feature amount of an image related to the registered vehicle stored in a storage unit, The output unit further outputs the collation result by the collation unit to a display device. The information processing device according to claim 1 .

8. The computer The terminal acquires an image of the vehicle, Extracting a character string from an area of ​​the license plate included in the image; Acquires information about a registered vehicle associated with the character string and a comparison image corresponding to the information about the registered vehicle from a storage unit; If the character string extracted from the image is any one of a plurality of pieces of pre-stored registration information, information relating to the registered vehicle and the comparison image are output to a display device, and if the character string extracted from the image is not any one of the plurality of pieces of pre-stored registration information, information indicating that the character string extracted from the image is not the pre-stored registration information is output. Information processing methods.

9. On the computer, A process of acquiring an image of the vehicle by the terminal; A process of extracting a character string from an area of ​​the license plate included in the image; A process of acquiring, from a storage unit, information on a registered vehicle associated with the character string and a comparison image corresponding to the information on the registered vehicle; a process of outputting information about the registered vehicle and the comparison image to a display device based on the character string when the character string extracted from the image is one of a plurality of pieces of pre-stored registration information, and outputting information indicating that the character string extracted from the image is not one of the plurality of pieces of pre-stored registration information when the character string extracted from the image is not one of the plurality of pieces of pre-stored registration information; An information processing program that executes the above.

Images