Search system and search method

Abstract

To provide a search system capable of searching for a reference image estimated to be obtained by photographing a vehicle that is similar to a searching object vehicle at a high speed on the basis of an image of the searching object vehicle.SOLUTION: A search system comprises: a processing device that extracts a feature amount in regard to a number plate of a vehicle contained in an image obtained by photographing a vehicle; an image accumulation database that collects and records a reference image obtained by photographing a traveling vehicle from a camera installed on a road side; and a terminal device that receives an input of a search image obtained by photographing a search object vehicle, and acquires the reference image estimated to be obtained by photographing the vehicle that is similar to the search object vehicle from the image accumulation database.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004] , , , , , , , ,

[0005] , , ,

[0001] The present disclosure relates to a vehicle search system and a search method.

Background Art

[0002] In recent years, number plate reading devices (image processing devices) that acquire number plate information from images of vehicle number plates have been installed at various locations such as toll roads, general roads, and parking lots. The images captured by the number plate reading device and the read number plate information are aggregated, for example, in a single central device (database) and used for vehicle monitoring and tracking. For example, Patent Document 1 describes a vehicle tracking system that searches for images and information of vehicles that match the number plate information of a target vehicle among a large number of vehicle data stored in a central device and displays them on a user's terminal device (the in-vehicle device of the tracking vehicle).

[0003] As an image search technique for searching for reference images similar to an arbitrary image among a large number of reference images, for example, the CBIR (Content-Based Image Retrieval) method is known. The CBIR method is a technique for searching for images with a high degree of match (similarity) based on feature amounts such as shapes and colors extracted from images (see, for example, Non-Patent Document 1). Further, as another image search technique, there is the TBIR (Text-Based Image Retrieval) method. The TBIR method is a technique in which text information is associated with and recorded in advance for reference images, and keyword searches are performed based on this text information.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005] [Non-Patent Document 1] "Development of technology to instantly search for a target image from a large number of images," [online], Fujitsu Laboratories Ltd., [Accessed January 16, 2023], Internet <URL: https: / / pr.fujitsu.com / jp / news / 2016 / 02 / 2-1.html> [Overview of the project] [Problems that the invention aims to solve]

[0006] The search cost is lower with the TBIR method than with the CBIR method. However, with the TBIR method, the type and accuracy of the text information used for searching may not match, making it unsuitable for search systems that handle large amounts of images. Also, the search time with the CBIR method depends on the performance of the hardware. In other words, to achieve a high-speed search system with the CBIR method, high-performance hardware is required. This could lead to a high cost for the search system, making practical implementation difficult.

[0007] Furthermore, both conventional CBIR and RBIR methods are techniques for searching for similar images. In other words, conventional image search technology can search for images of vehicles that have a similar shape or color to the vehicle being searched, but it is difficult to estimate whether an image is of the same vehicle being searched.

[0008] The purpose of this disclosure is to provide a search system and search method that can quickly search for reference images that are presumed to be of the same vehicle as the vehicle being searched, based on images of the vehicle being searched. [Means for solving the problem]

[0009] According to one aspect of this disclosure, the search system comprises: a processing device for extracting feature quantities relating to the license plate of a vehicle contained in an image of the vehicle; an image storage database for collecting and recording reference images of moving vehicles taken from a camera installed on the roadside; and a terminal device for receiving input of a search image of a vehicle to be searched and obtaining a reference image from the image storage database that is presumed to be of the same vehicle as the vehicle to be searched, wherein the processing device comprises a feature quantity extraction unit for extracting feature quantities relating to the license plate of a vehicle from the image, and the feature quantity extraction unit for obtaining the reference The terminal device includes a reference data generation unit that generates reference data containing reference features extracted from a search image and records it in a feature database, and a search request unit that requests the feature database to search for reference data having reference features similar to the search features extracted from the search image by the feature extraction unit, and the terminal device includes a search image acquisition unit that receives the input of the search image and transmits it to the processing unit, and a reference image acquisition unit that acquires a reference image from the image storage database that corresponds to the search result of the reference data acquired from the feature database.

[0010] According to one aspect of the present disclosure, the search method is a search method using a search system comprising: a processing device that extracts feature quantities relating to the license plate of a vehicle contained in an image of a vehicle; an image storage database that collects and records reference images of moving vehicles taken from a camera installed on the roadside; and a terminal device that receives input of a search image of a vehicle to be searched and obtains from the image storage database a reference image that is presumed to be of the same vehicle as the vehicle to be searched, wherein the processing device extracts reference feature quantities relating to the license plate of a vehicle contained in the reference image; and the processing device The process includes the steps of: generating reference data including reference features and recording it in a feature database; the terminal device receiving input of the search image and transmitting it to the processing device; the processing device extracting search features relating to the license plate of a vehicle included in the search image; the processing device requesting the feature database to search for reference data having reference features similar to the search features; and the terminal device obtaining a reference image from the image storage database corresponding to the search result of the reference data obtained from the feature database. [Effects of the Invention]

[0011] According to the above embodiment, reference images of the same vehicle as the vehicle being searched can be quickly searched based on the image of the vehicle being searched. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows the overall configuration of a search system according to one embodiment. [Figure 2] This is a block diagram showing the functional configuration of a processing apparatus according to one embodiment. [Figure 3] Block diagram showing the functional configuration of a terminal device according to one embodiment. [Figure 4] A sequence diagram showing an example of data storage processing according to one embodiment. [Figure 5]It is a first diagram for explaining data accumulation processing according to an embodiment. [Figure 6] It is a second diagram for explaining data accumulation processing according to an embodiment. [Figure 7] It is a sequence diagram showing an example of image search processing according to an embodiment. [Figure 8] It is a diagram for explaining image search processing according to an embodiment.

Mode for Carrying Out the Invention

[0013] (Overall Configuration of Search System) FIG. 1 is a diagram showing the overall configuration of a search system according to an embodiment. The search system 1 according to the present embodiment is a system for searching for images of vehicles photographed by roadside devices 5 installed at toll booths on toll roads, roadside of general roads, etc. The user of the search system 1 is, for example, a regulatory agency (such as the police) that conducts searches and tracking of violating vehicles, stolen vehicles, etc.

[0014] The roadside device 5 includes a camera 51 and a sensor 52.

[0015] The camera 51 photographs vehicles traveling on the road. The camera 51 is installed at a position and orientation such that at least the vehicle number plate can be clearly photographed. When the road has a plurality of lanes, one camera 51 may be installed for each lane, and each camera 51 may photograph only the vehicles traveling in the lane that is its photographing target. Alternatively, one camera 51 may be installed for a plurality of lanes, and one camera 51 may photograph vehicles traveling in a plurality of lanes.

[0016] The sensor 52 is, for example, a vehicle detector that detects vehicles traveling on the road. The camera 51 may be configured to perform photographing at the timing when the vehicle detector detects a vehicle.

[0017] As shown in FIG. 1, the search system 1 includes an image storage database (DB) 2, a processing device 3, a feature database (DB) 4, and a terminal device 6.

[0018] The image storage DB 2 is a database that collects and records the image D1 of the traveling vehicle taken by the camera 51 of the roadside device 5. The image D1 stored in the image storage DB 2 is also described as "reference image D1". Further, the reference image D1 may be attached with additional information D11 such as the shooting date and time and the shooting location where the camera 51 captured the reference image D1.

[0019] The processing device 3 extracts reference features related to the license plate of the vehicle included in the reference image D1.

[0020] The feature DB 4 is a database for storing and searching the reference features extracted by the processing device 3.

[0021] The terminal device 6 is communicably connected to the image storage DB 2, the processing device 3, and the feature DB 4 of the search system 1 via a network such as the Internet. The terminal device 6 receives an input of a search image D2 of the search target vehicle taken by the user, and acquires a reference image D1 estimated to be of the same vehicle as the search target vehicle from the image storage DB 2. Details of the process by which the terminal device 6 acquires the reference image D1 will be described later.

[0022] (Functional configuration of the processing device) FIG. 2 is a block diagram showing the functional configuration of the processing device according to an embodiment. As shown in FIG. 2, the processing device 3 includes a processor 31, a memory 32, a storage 33, and a communication interface 34. *

[0023] The processor 31 functions as a feature extraction unit 311, a reference data generation unit 312, and a search request unit 313 by operating according to a predetermined program.

[0024] The feature extraction unit 311 extracts features related to the vehicle's license plate from the image. These license plate features include, for example, license plate information (place name, classification number, symbol, serial number) which is the string of characters displayed on the license plate. The features may also include the license plate size and color.

[0025] The reference data generation unit 312 generates reference data D3, which includes the reference features D13 extracted by the feature extraction unit 311 from the reference image D1, and records it in the feature DB4.

[0026] The search request unit 313 requests the feature database 4 to search for reference data D3 that has a reference feature D13 similar to the search feature D21 extracted by the feature extraction unit 311 from the search image D2.

[0027] The predetermined program executed by the processor 31 is stored on a computer-readable recording medium. A computer-readable recording medium refers to a magnetic disk, magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, etc. Alternatively, this computer program may be distributed to a computer via a communication line, and the computer that receives the distribution may execute the program. Furthermore, this program may be intended to implement only a part of the functions described above. Moreover, it may be a program that can implement the above functions in combination with a program already recorded in the computer system, a so-called differential file (differential program).

[0028] Memory 32 has a memory area necessary for the operation of the processor 31.

[0029] The storage 33 is a so-called auxiliary storage device, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). The storage 33 stores data that each part of the processor 31 acquires, generates, and references during processing.

[0030] The communication interface 34 is an interface for sending and receiving various data and control signals between the image storage DB2, the feature data DB4, and the terminal device 6.

[0031] (Functional configuration of terminal devices) Figure 3 is a block diagram showing the functional configuration of a terminal device according to one embodiment. As shown in Figure 3, the terminal device 6 comprises a processor 61, memory 62, storage 63, input / output interface 64, and communication interface 65.

[0032] The processor 61 performs the functions of a search image acquisition unit 611 and a reference image acquisition unit 612 by operating according to a predetermined program.

[0033] The search image acquisition unit 611 receives the search image D2 from the user and transmits it to the processing unit 3.

[0034] The reference image acquisition unit 612 acquires a reference image D1 from the image storage DB2 that corresponds to the search result of the reference data D3 acquired from the feature database DB4.

[0035] The predetermined program executed by the processor 61 is stored on a computer-readable recording medium. A computer-readable recording medium refers to a magnetic disk, magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, etc. Alternatively, this computer program may be distributed to a computer via a communication line, and the computer that receives the distribution may execute the program. Furthermore, this program may be intended to implement only a part of the functions described above. Moreover, it may be a program that can implement the above functions in combination with a program already recorded in the computer system, a so-called differential file (differential program).

[0036] Memory 62 has a memory area necessary for the operation of the processor 61.

[0037] The storage 63 is a so-called auxiliary storage device, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). The storage 63 stores data that each part of the processor 61 acquires, generates, or references during processing.

[0038] The input / output interface 64 is a connection interface for connecting to devices such as a display device 66 and an input device 67. The display device 66 is a monitor such as a liquid crystal display. The input device 67 is an input device such as a keyboard or mouse. Note that the display device 66 and the input device 67 may be an integrated device, such as a touch panel.

[0039] The communication interface 65 is an interface for sending and receiving various data and control signals between the image storage DB2, the processing unit 3, and the feature quantity DB4.

[0040] (Regarding the storage process of reference images and reference data) Figure 4 is a sequence diagram showing an example of data storage processing according to one embodiment. Figure 5 is a first diagram illustrating a data storage process according to one embodiment. Figure 6 is a second diagram illustrating the data storage process according to one embodiment. The following describes the process of accumulating reference image D1 and reference data D3 in search system 1, with reference to Figures 4 to 6.

[0041] First, the camera 51 of the roadside device 5 photographs a moving vehicle (step S101) and transmits the captured image D1 to the image storage DB2 (step S102). At this time, as shown in Figure 5, the camera 51 transmits additional information D11, including the date and time the image D1 was taken and the location where the image D1 was taken (the location where the camera 51 is installed), to the image storage DB2 along with the image D1.

[0042] Next, as shown in Figures 4 and 5, the image storage DB2 records the image (reference image) D1 and additional information D11 received from the roadside device 5 (camera 51) with an index D12 (step S103). The index D12 is information used to link the reference image D1 with the additional information D11 and the reference feature quantity D13 extracted from the reference image D1, and is a code or string that has common elements in these data D1, D11, and D13. Having common elements means, for example, that some or all of them are composed of the same code or string.

[0043] Furthermore, the image storage DB2 transmits the reference image D1 and index D12 to the processing unit 3 (step S104).

[0044] Then, the feature extraction unit 311 of the processing device 3 performs predetermined image processing on the reference image D1 to extract the reference feature D13 (step S105). In this embodiment, the reference feature D13 is text information indicating license plate information (place name, classification number, symbol, serial number), plate size, plate color, etc., as illustrated in Figure 6. The plate size is represented by text information such as "large plate," "medium plate," or "small plate." The plate color is represented by text information such as white background (private passenger car), green background (commercial passenger car), yellow background (private light vehicle). Note that the technology for reading license plate information etc. from the image may utilize the technology of a known license plate reader.

[0045] Next, the reference data generation unit 312 of the processing unit 3 generates reference data D3, which includes the index D12 of the reference image D1 and the reference feature D13 extracted from the reference image D1 by the feature extraction unit 311, and sends it to the feature DB4 for recording (step S106). The feature DB4 records the received reference data D3 (step S107).

[0046] Furthermore, if the reference image D1 contains multiple vehicle license plates, the feature extraction unit 311 extracts a reference feature D13 for each of these multiple license plates. At this time, the reference data generation unit 312 generates reference data D3 that includes the multiple reference feature D13s.

[0047] The search system 1 executes a series of processes shown in Figures 4 to 6 each time the camera 51 of the roadside device 5 captures an image D1. Note that Figure 4 shows an example where the processing unit 3 generates reference data D3 and records it in the feature database DB4 each time it receives a reference image D1, but the system is not limited to this. In other embodiments, the processing unit 3 may acquire multiple newly accumulated reference images D1 in the image storage DB2 at a predetermined timing, and simultaneously create the reference data D3 corresponding to each reference image D1 and record it in the feature database DB4. The predetermined timing could be, for example, after a certain period of time has elapsed or after a certain number of reference images D1 have been accumulated. Alternatively, the predetermined timing could be when the processing unit 3 is not performing other resource-intensive processes and has sufficient processing resources (for example, when the resources used by other processes are less than X%).

[0048] (Regarding image search processing) Figure 7 is a sequence diagram showing an example of image search processing according to one embodiment. Figure 8 is a diagram illustrating an image search process according to one embodiment. The following describes the image search process in search system 1, with reference to Figures 7 and 8.

[0049] For example, a user may want to identify when and where (which route) a vehicle to be searched, such as a traffic violator or a stolen vehicle, traveled. In this case, the user prepares an image of the vehicle to be searched (search image D2) in advance, as illustrated in Figure 8. When the search image acquisition unit 611 of the terminal device 6 acquires the search image D2 entered by the user (step S201), it makes a search request to the processing device 3 (step S202). In other embodiments, the search image D2 may not be entered manually by the user, but automatically by another system that is linked to the system. For example, the other system may be a traffic violation monitoring system, and an image of a traffic violation vehicle captured by the surveillance camera of the traffic violation monitoring system may be automatically entered into the terminal device 6 as the search image D2.

[0050] When the feature extraction unit 311 of the processing device 3 receives the search image D2 from the terminal device 6, it performs a process to extract features (search features D21) from this image (step S203). This process is the same as step S105 in Figure 4.

[0051] Furthermore, the search request unit 313 of the processing unit 3 transmits the search feature D21 extracted by the feature extraction unit 311 to the feature DB4 to request a search for similar reference data D3 (step S204). At this time, the search request unit 313 also specifies to the feature DB4 the destination of the search results (the terminal device 6 which is the source of the search request).

[0052] Next, the feature database DB4 searches for a reference feature D13 (reference data D3) that is similar to the search feature D21 received from the processing unit 3 (step S205).

[0053] If there is dirt on camera 51 or the license plate, some characters on the license plate may not be readable or may be misread from reference image D1 or search image D2. Therefore, feature DB4 considers the possibility of missing or misrecognized features (some characters) and searches not only reference feature D13 that exactly matches search feature D21, but also reference feature D13 that is similar to search feature D21.

[0054] For example, let's explain using a sequence of characters in license plate information as an example. Feature DB4 is the sequence of characters "12-3X" in the search feature D21. If the last character ("X") cannot be read, reference feature D13 that matches the characters other than the missing part "X", i.e., reference feature D13 with a sequence number from "12-30" to "12-39", may be searched as similar. Alternatively, feature DB4 may search for reference feature D13 that has a matching rate with the text information contained in the search feature D21 that is above a predetermined threshold, and consider it similar to the search feature D21. Feature DB4 may also use other known search algorithms to search for reference feature D13 that are similar to the search feature D21.

[0055] For example, if a single camera 51 captures multiple lanes, the reference image D1 captured by this camera 51 may contain the license plates of multiple vehicles. In other words, the reference data D3 may contain reference features D13 for each of the multiple license plates. In this case, if any one of the reference features D13 in the reference data D3 is similar to the search feature D21, the feature DB4 will also include this reference data D3 in the search results as similar data. This allows reference images D1 that capture the license plates of multiple vehicles simultaneously to be treated appropriately as search targets.

[0056] Furthermore, the feature database 4 transmits the search results for the reference feature D13 to the terminal device 6 (step S206). The feature database 4 transmits reference data D3, which includes similar reference feature D13 and its index D12, to the terminal device 6 as a search result. In addition, if the feature database 4 searches for similar reference feature D13 based on the match rate, it may transmit the search results including the match rate, as shown in Figure 8.

[0057] The reference image acquisition unit 612 of the terminal device 6 displays the search results (list of reference data D3) received from the feature quantity DB4 on the display device 66. The user selects and specifies the item from the search results displayed on the display device 66 that is estimated to be the same vehicle as the vehicle being searched. The reference image acquisition unit 612 of the terminal device 6 requests the image storage DB2 to provide the reference image D1 corresponding to the reference data D3 (reference feature quantity D13) selected by the user (step S207).

[0058] For example, the list of reference data D3, which is the search result, is marked with a checkbox. The user checks the reference feature D13 and the match rate for each reference data D3, checks (selects) the checkbox for the reference data D3 of the vehicle that is presumed to be the same as the vehicle being searched for, and presses the image acquisition button. The user may select one or more data. Then, the reference image acquisition unit 612 specifies the index D12 of the reference data D3 selected by the user and requests the image storage DB2 to send a reference image D1 that matches this index.

[0059] The image storage DB2 reads the reference image D1, which has the same index as the index D12 specified by the terminal device 6, and transmits it to the terminal device 6 (step S208). At this time, the image storage DB2 may also transmit additional information D11 along with the reference image D1, as shown in the example in Figure 8.

[0060] The reference image acquisition unit 612 of the terminal device 6 displays the reference image D1 (and additional information D11) received from the image storage DB2 on the display device 66 (step S209). The user checks the reference image D1 to confirm whether the vehicle shown in the image is the same as the vehicle being searched for. In addition, if additional information D11 (information on the date and time of shooting and the location of shooting) is available, the user can find out when and where the vehicle being searched passed through.

[0061] (Effect, Action) As described above, the search system 1 according to this embodiment includes a processing device 3 that extracts feature quantities related to the vehicle's license plate from an image, an image storage DB 2 that collects and records a reference image D1 from a camera 51, and a terminal device 6 that receives input of a search image D2 from a user and obtains a reference image D1 from the image storage DB 2 that is presumed to be of the same vehicle as the vehicle to be searched. The processing device 3 includes a feature quantity extraction unit 311 that extracts feature quantities from an image, a reference data generation unit 312 that generates reference data D3 including reference feature quantities D13 extracted by the feature quantity extraction unit 311 from the reference image D1 and records it in the feature quantity DB 4, and a search request unit 313 that requests the feature quantity DB 4 to search for reference data D3 having reference feature quantities D13 similar to the search feature quantities D21 extracted by the feature quantity extraction unit 311 from the search image D2. The terminal device 6 includes a search image acquisition unit 611 that receives input of a search image D2 from the user and transmits it to the processing unit 3, and a reference image acquisition unit 612 that acquires a reference image D1 corresponding to the search result of the reference data D3 acquired from the feature quantity DB4 from the image storage DB2.

[0062] As described above, conventional image search methods (CBIR method) that input search images and search for similar reference images have the problem of long search times in search systems that handle a large number of reference images. However, as described above, the search system 1 according to this embodiment extracts features limited to license plate information from reference image D1 and search image D2 and performs the search, so it can significantly reduce processing time compared to conventional image search methods that match features extracted from the entire image. Therefore, the search system 1 can perform image searches at a faster speed than conventional image search methods.

[0063] Furthermore, conventional image search methods (CBIR method, TBIR method) extract features from the entire image, which sometimes resulted in a large number of search results including different vehicles (reference images) that were only similar in shape or color to the vehicle being searched for. Consequently, users had to visually inspect these numerous search results to find the vehicle identical to the one being searched for. In contrast, the search system 1 according to this embodiment performs searches by comparing features limited to license plate information. Therefore, even if a vehicle is similar in shape or color to the vehicle being searched for, vehicles with different license plates are less likely to be included in the search results. Consequently, the search system 1 can appropriately search only for vehicles that are presumed to be identical to the vehicle being searched for.

[0064] Furthermore, the feature extraction unit 311 of the processing unit 3 extracts text information, including the string of characters written on the license plate, from the reference image D1 and the search image D2 as reference feature D13 and search feature D21.

[0065] In this way, the search system 1 only needs to perform text information searches, making it possible to significantly speed up the search process compared to conventional image search processes. Furthermore, when the search system 1 performs a search, it only needs to send and receive text information, namely the reference feature D13 and the search feature D21, in communication between the processing unit 3 and the feature database DB4, and between the feature database DB4 and the terminal device 6. This allows for faster communication between these devices during the search process and reduces the load on the network.

[0066] Furthermore, the image storage DB2 records the reference image D1 with an index D12 attached, the reference data generation unit 312 of the processing device 3 generates reference data D3 by attaching the index D12 of the reference image D1 to the reference feature quantity D13 extracted from the reference image D1, and the reference image acquisition unit 612 of the terminal device 6 acquires the reference image D1 corresponding to the index D12 of the reference data D3 included in the search results from the image storage DB2.

[0067] In this way, the search system 1 can manage the reference image D1 and its reference feature D13, which are recorded separately in the image storage DB2 and feature DB4, by linking them using index D12. Furthermore, the image storage DB2 only needs to read and provide the reference image D1 corresponding to the index D12 specified by the terminal device 6, thus eliminating the need for image search functions and other such features. This significantly reduces the processing load on the image storage DB2 and makes it possible to configure the image storage DB2 at a low cost.

[0068] Furthermore, the reference data generation unit 312 of the processing unit 3 generates reference data D3 that includes the reference feature quantities D13 for each of the multiple license plates when the reference image D1 contains the license plates of multiple vehicles.

[0069] In this way, when the search system 1 photographs multiple lanes with a single camera 51, it can also appropriately treat reference images D1, which simultaneously capture the license plates of multiple vehicles, as search targets.

[0070] As described above, embodiments relating to this disclosure have been explained, but these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0071] <Note> The search system and search method described in the above-described embodiment can be understood, for example, as follows:

[0072] (1) According to the first embodiment, the search system 1 comprises a processing device 3 that extracts feature quantities relating to the license plate of a vehicle contained in an image of the vehicle, an image storage database 2 that collects and records reference images D1 of moving vehicles from a camera 51 installed on the roadside, and a terminal device 6 that receives input of a search image D2 of a vehicle to be searched and obtains a reference image D1 from the image storage database 2 that is presumed to be of the same vehicle as the vehicle to be searched. The processing device 3 comprises a feature quantity extraction unit 311 that extracts feature quantities relating to the license plate of a vehicle from an image, a reference data generation unit 312 that generates reference data D3 including reference feature quantities D13 extracted by the feature quantity extraction unit 311 from the reference image D1 and records it in the feature quantity database 4, and a search request unit 313 that requests the feature quantity database 4 to search for reference data D3 having reference feature quantities D13 similar to search feature quantities D21 extracted by the feature quantity extraction unit 311 from the search image D2. The terminal device 6 includes a search image acquisition unit 611 that receives the input of a search image D2 and transmits it to the processing unit 3, and a reference image acquisition unit 612 that acquires a reference image D1 corresponding to the search result of the reference data D3 acquired from the feature database 4 from the image storage database 2.

[0073] Conventional image search methods (CBIR method), which input search images and search for similar reference images, have the problem of long search times in search systems that handle a large number of reference images. However, as described above, the search system 1 according to this embodiment extracts features limited to license plate information from reference image D1 and search image D2 and performs the search, so it can significantly reduce processing time compared to conventional image search methods that match features extracted from the entire image. Therefore, the search system 1 can perform image searches at a faster speed than conventional image search methods.

[0074] Furthermore, conventional image search methods (CBIR method, TBIR method) extract features from the entire image, which sometimes resulted in a large number of search results including different vehicles (reference images) that were only similar in shape or color to the vehicle being searched for. Consequently, users had to visually inspect these numerous search results to find the vehicle identical to the one being searched for. In contrast, the search system 1 according to this embodiment performs searches by comparing features limited to license plate information. Therefore, even if a vehicle is similar in shape or color to the vehicle being searched for, vehicles with different license plates are less likely to be included in the search results. Consequently, the search system 1 can appropriately search only for vehicles that are presumed to be identical to the vehicle being searched for.

[0075] (2) According to the second embodiment, in the search system 1 according to the first embodiment, the feature extraction unit 311 of the processing device 3 extracts license plate information, which is a string of characters written on the license plate, from the reference image D1 and the search image D2 as reference feature D13 and search feature D21.

[0076] In this way, the search system 1 only needs to perform text information searches, making it possible to significantly speed up the search process compared to conventional image search processes. Furthermore, when the search system 1 performs a search, it only needs to send and receive text information, namely the reference feature D13 and the search feature D21, in communication between the processing unit 3 and the feature database DB4, and between the feature database DB4 and the terminal device 6. This allows for faster communication between these devices during the search process and reduces the load on the network.

[0077] (3) According to the third embodiment, in the search system 1 according to the first or second embodiment, the image storage database 2 records a reference image D1 with an index D12 attached to it, the reference data generation unit 312 of the processing device 3 generates reference data D3 by attaching the index D12 of the reference image D1 to a reference feature quantity D13 extracted from the reference image D1, and the reference image acquisition unit 612 of the terminal device 6 acquires the reference image D1 corresponding to the index D12 of the reference data D3 included in the search results from the image storage database 2.

[0078] In this way, the search system 1 can manage the reference image D1 and its reference feature D13, which are recorded separately in the image storage DB2 and feature DB4, by linking them using index D12. Furthermore, the image storage DB2 only needs to read and provide the reference image D1 corresponding to the index D12 specified by the terminal device 6, thus eliminating the need for image search functions and other such features. This significantly reduces the processing load on the image storage DB2 and makes it possible to configure the image storage DB2 at a low cost.

[0079] (4) According to the fourth embodiment, in the search system 1 according to any one of the first to third embodiments, the reference data generation unit 312 of the processing unit 3 generates reference data D3 that includes the reference feature quantities D13 of each of the multiple license plates when the reference image D1 includes the license plates of multiple vehicles.

[0080] In this way, when the search system 1 photographs multiple lanes with a single camera 51, it can also appropriately treat reference images D1, which simultaneously capture the license plates of multiple vehicles, as search targets.

[0081] (5) According to the fifth aspect, the search method is a search method using a search system 1 comprising: a processing device 3 that extracts feature quantities relating to the license plate of a vehicle contained in an image of a vehicle; an image storage database 2 that collects and records reference images D1 of moving vehicles from a camera 51 installed on the roadside; and a terminal device 6 that receives input of a search image D2 of a vehicle to be searched from a user and obtains a reference image D1 from the image storage database 2 that is presumed to be of the same vehicle as the vehicle to be searched, wherein the processing device 3 extracts reference feature quantities D13 relating to the license plate of a vehicle contained in the reference image D1; and the processing device 3 The process includes the steps of: generating reference data D3 containing the search feature D13 and recording it in the feature database 4; the terminal device 6 receiving input of a search image D2 from a user and transmitting it to the processing device 3; the processing device 3 extracting search feature D21 related to the license plate of a vehicle contained in the search image D2; the processing device 3 requesting the feature database 4 to search for reference data D3 having a reference feature D13 similar to the search feature D21; and the terminal device 6 obtaining a reference image D1 corresponding to the search result of the reference data D3 obtained from the feature database 4 from the image storage database 2. [Explanation of symbols]

[0082] 1. Search System 2. Image storage database (DB) 3 Processing Unit 31 processors 311 Feature Extraction Unit 312 Reference Data Generation Unit 313 Search Request Section 32 memory 33 Storage 34 Communication Interfaces 4. Feature Database (DB) 5 Roadside equipment 51 Camera 52 sensors 6 Terminal devices 61 processors 611 Image acquisition unit for search 612 Reference image acquisition unit 62 memory 63 storage 64 input / output interfaces 65 Communication Interfaces 66 Display device 67 Input device D1 Reference Image D2 Search Images D3 Reference Data

Claims

1. A processing device for extracting features related to the license plate of a vehicle from an image of the vehicle, An image storage database that collects and records reference images of moving vehicles taken from cameras installed along the roadside, A terminal device that receives input of search images taken of the vehicle to be searched, and obtains reference images from the image storage database that are presumed to be of the same vehicle as the vehicle to be searched, Equipped with, The aforementioned processing apparatus is A feature extraction unit extracts feature quantities, which are text information related to the vehicle's license plate, from an image. A reference data generation unit generates reference data consisting only of text information containing reference features extracted from the reference image by the feature extraction unit and records it in a feature database. A search request unit requests the feature database to search for reference data having reference features that partially match or have a matching rate equal to or greater than a predetermined threshold with the search features extracted by the feature extraction unit from the search image. It has, The aforementioned terminal device is A search image acquisition unit that receives the input of the search image and transmits it to the processing device, A reference image acquisition unit obtains search results from the feature database for reference data having reference features similar to the search features of the search image, and obtains a reference image from the image storage database that corresponds to the reference data included in the search results of the feature database. Having, Search system.

2. The feature extraction unit of the processing apparatus extracts license plate information, which is a string of characters written on a license plate, from the reference image and the search image as the reference feature and the search feature, respectively. The search system according to claim 1.

3. The aforementioned image storage database records the reference images with an index, The reference data generation unit of the processing apparatus generates reference data by assigning an index of the reference image to the reference feature quantities extracted from the reference image. The reference image acquisition unit of the terminal device acquires a reference image from the image storage database that corresponds to the index of the reference data included in the search results. The search system according to claim 1.

4. The reference data generation unit of the processing apparatus generates reference data that includes reference feature quantities for each of the multiple license plates when the reference image includes multiple license plates. A search system according to any one of claims 1 to 3.

5. A processing device that extracts feature quantities, which are text information about the license plate of a vehicle, from an image of a vehicle, An image storage database that collects and records reference images of moving vehicles taken from cameras installed along the roadside, A terminal device that receives input of search images taken of the vehicle to be searched, and obtains reference images from the image storage database that are presumed to be of the same vehicle as the vehicle to be searched, A search method using a search system comprising: The processing device includes the steps of extracting reference features relating to the license plate of a vehicle included in the reference image, The processing device generates reference data consisting only of text information including the reference features and records it in a feature database. The terminal device receives the input of the search image and transmits it to the processing device. The processing device includes the steps of extracting search features related to the license plate of a vehicle included in the search image, The processing device requests the feature database to search for reference data having reference features that partially match the search features or whose matching rate is above a predetermined threshold, The terminal device obtains search results from the feature database for reference data having reference features similar to the search features of the search image, and obtains a reference image from the image storage database that corresponds to the reference data included in the search results of the feature database. A search method that has [a certain feature].

Images