Vehicle recognition device, vehicle recognition method, and vehicle recognition program

Abstract

To enable even a computer with reduced processing capability to detect vehicle information such as the number of passengers.SOLUTION: A number detection unit 111 detects a license plate of a vehicle from one of a plurality of pieces of image data obtained by photographing a vehicle with a camera 40. A target detection unit 112 extracts, as a processed image, image data in which the license plate is detected by the number detection unit 111 among the pieces of image data. A target detection unit 112 detects at least one target of an emblem displayed on the vehicle and a passenger of the vehicle from the processed image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a technique for collecting information about a traveling vehicle. [Background technology]

[0002] There is a technology that identifies information such as the number of passengers in a vehicle from video data captured of the vehicle. Patent Document 1 describes that the number of passengers in a vehicle is identified by detecting the faces of passengers using a face detection technology. This technology will make it possible to offer discounts on tolls based on the number of passengers, and to determine whether or not a vehicle can pass based on the number of passengers. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2017 / 158647 Summary of the Invention [Problem to be solved by the invention]

[0004] In the method described in Patent Document 1, when a vehicle is moving at high speed, image data for many frames in which the vehicle is captured is processed, and therefore a device with high processing capabilities is required. The present disclosure aims to enable vehicle information such as the number of passengers to be detected even with a computer having limited processing power. [Means for solving the problem]

[0005] A vehicle recognition device according to the present disclosure includes: a number detection unit that detects a license plate of the vehicle from any one of a plurality of image data obtained by photographing the vehicle with a camera; an object detection unit that extracts, as a processed image, image data in which the license plate has been detected by the number detection unit from among the plurality of image data, and detects at least one object between a mark displayed on the vehicle and a passenger in the vehicle from the processed image; Equipped with. [Effects of the Invention]

[0006] In this disclosure, image data in which license plates are detected is extracted as a processed image, and objects are detected from the processed image. This eliminates the need to perform object detection processing on image data for an unnecessarily large number of frames. As a result, vehicle information such as the number of occupants can be detected even on a computer with limited processing power. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a configuration diagram of a vehicle recognition system 100 according to a first embodiment. [Figure 2] 1 is a configuration diagram of a vehicle recognition device 10 according to a first embodiment. [Figure 3] FIG. 2 is a configuration diagram of a central device 20 according to the first embodiment. [Figure 4] 4 is a flowchart of a vehicle recognition process according to the first embodiment. [Figure 5] FIG. 2 is an explanatory diagram of a vehicle database 231 according to the first embodiment. [Figure 6] 10 is a flowchart of a search process according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Embodiment 1 ***Configuration Description*** The configuration of a vehicle recognition system 100 according to the first embodiment will be described with reference to FIG. The vehicle recognition system 100 includes a plurality of vehicle recognition devices 10, a central device 20, and one or more search terminals 30. Each vehicle recognition device 10 and the central device 20 are connected via a transmission path 91. The central device 20 and each search terminal 30 are connected via a transmission path 92.

[0009] The vehicle recognition device 10 is a computer installed near a road. The central device 20 is a computer installed in a server room or the like. The search terminal 30 is a terminal such as a PC used by a user. PC is an abbreviation for personal computer.

[0010] The configuration of the vehicle recognition device 10 according to the first embodiment will be described with reference to FIG. The vehicle recognition device 10 includes hardware such as a processor 11, a memory 12, a storage 13, and a communication interface 14. The processor 11 is connected to other hardware via signal lines and controls the other hardware.

[0011] The vehicle recognition device 10 includes, as functional components, a license plate detection unit 111, an object detection unit 112, a vehicle identification unit 113, and an information management unit 114. The functions of each functional component of the vehicle recognition device 10 are realized by software. The storage 13 stores a program that realizes the function of each functional component of the vehicle recognition device 10. The program is read into the memory 12 by the processor 11 and executed by the processor 11. In this way, the function of each functional component of the vehicle recognition device 10 is realized.

[0012] The vehicle recognition device 10 is connected to a camera 40 via a communication interface 14. The camera 40 is installed so as to capture an image of the road on which the vehicle is traveling. For example, the camera 40 is installed so as to capture an image of the road from ahead in the direction of travel of the vehicle. This makes it possible to capture an image of the vehicle from the front. An illumination device 41 is installed near the camera 40 to illuminate the photographing area of ​​the camera 40 as needed.

[0013] The configuration of the central device 20 according to the first embodiment will be described with reference to FIG. The central device 20 includes the following hardware components: a processor 21, a memory 22, a storage 23, and a communication interface 24. The processor 21 is connected to other hardware components via signal lines and controls these other hardware components.

[0014] The central device 20 includes, as a functional component, a search unit 211. The functions of the functional components of the vehicle recognition device 10 are realized by software. The storage 23 stores programs that realize the functions of each functional component of the central device 20. These programs are read into the memory 22 by the processor 21 and executed by the processor 21. As a result, the functions of each functional component of the central device 20 are realized.

[0015] The storage 23 realizes the function of a vehicle database 231 .

[0016] The processors 11 and 21 are ICs that perform processing. IC stands for Integrated Circuit. Specific examples of the processors 11 and 21 are a CPU, a DSP, and a GPU. CPU stands for Central Processing Unit. DSP stands for Digital Signal Processor. GPU stands for Graphics Processing Unit.

[0017] The memories 12 and 22 are storage devices that temporarily store data. Specific examples of the memories 12 and 22 are SRAM and DRAM. SRAM stands for Static Random Access Memory. DRAM stands for Dynamic Random Access Memory.

[0018] The storages 13 and 23 are storage devices that store data. Specific examples of the storages 13 and 23 are HDDs. HDD stands for Hard Disk Drive. The storages 13 and 23 may also be portable recording media such as SD (registered trademark) memory cards, CompactFlash (registered trademark), NAND flash, flexible disks, optical disks, compact disks, Blu-ray (registered trademark) disks, and DVDs. SD stands for Secure Digital. DVD stands for Digital Versatile Disk.

[0019] The communication interfaces 14 and 24 are interfaces for communicating with external devices. Specific examples of the communication interfaces 14 and 24 are Ethernet (registered trademark), USB, and HDMI (registered trademark) ports. USB is an abbreviation for Universal Serial Bus. HDMI is an abbreviation for High-Definition Multimedia Interface.

[0020] 2 shows only one processor 11. However, there may be multiple processors 11, and the multiple processors 11 may cooperate to execute programs that realize the respective functions of the vehicle recognition device 10. Similarly, in FIG. 3, there is only one processor 21 shown. However, there may be multiple processors 21, and the multiple processors 21 may cooperate to execute programs that realize the respective functions of the central device 20.

[0021] ***Explanation of Operation*** The operation of the vehicle recognition system 100 according to the first embodiment will be described with reference to FIGS. The operation procedure of vehicle recognition system 100 according to embodiment 1 corresponds to the vehicle recognition method according to embodiment 1. Furthermore, the program that realizes the operation of vehicle recognition system 100 according to embodiment 1 corresponds to the vehicle recognition program according to embodiment 1.

[0022] The operation of the vehicle recognition system 100 according to the first embodiment includes a vehicle recognition process and a search process. The vehicle recognition process is executed by the vehicle recognition device 10. The search process is executed by the central device 20.

[0023] The vehicle recognition process according to the first embodiment will be described with reference to FIG. (Step S11: Video acquisition process) The number detection unit 111 acquires, via the communication interface 14, image data of a new frame that constitutes the video data obtained by the camera 40 capturing an image of the road.

[0024] (Step S12: Number detection process) The number detection unit 111 detects the license plate of the vehicle from the image data acquired in step S11. If the license plate is detected, the number detection unit 111 advances the process to step S13. On the other hand, if the license plate is not detected, the number detection unit 111 returns the process to step S11.

[0025] Here, the image data acquired in step S11 may or may not include a vehicle. Even if the image data acquired in step S11 includes a vehicle, the vehicle may only be small or only a portion of the vehicle may be included. Even if the image data acquired in step S11 includes a vehicle, the vehicle may not be clearly visible. In step S12, the license plate is detected if the image data includes a large, clear image of the vehicle and the characters on the license plate can be properly detected.

[0026] (Step S13: Image extraction process) The object detection unit 112 extracts, as a processed image, the image data in which the number plate has been detected by the number detection unit 111 in step S12.

[0027] The processes of steps S14 and S15 are executed in parallel.

[0028] (Step S14: Object detection process) The object detection unit 112 detects at least one of the objects, such as a mark displayed on a vehicle and a vehicle occupant, from the processed image extracted in step S13. Specifically, the mark is an elderly person mark indicating that the driver is elderly, a beginner mark indicating that the driver is a beginner, etc. Vehicle occupants are people riding in the vehicle, including the driver. By identifying the vehicle occupants, the number of occupants in the vehicle can be identified.

[0029] At this time, the object detection unit 112 narrows down the detection range of the object in the processed image based on the position where the license plate was detected from the processed image in step S12. Then, the object detection unit 112 detects the object only within the narrowed detection range. Suppose the object is a senior citizen mark. Senior citizen marks are supposed to be affixed in an area between 0.4 meters and 1.2 meters above the ground. Therefore, the object detection unit 112 identifies an area that is expected to be the affixing area based on the position of the license plate, and narrows the detection area to the identified area. Note that the object detection unit 112 may also add a reference range around the affixing area to the detection area. Assume that the object is a passenger. The passenger is captured in the window of the vehicle. Therefore, the object detection unit 112 identifies an area that is assumed to be the window of the vehicle from the position of the license plate, and narrows the detection range to the identified area. Note that the object detection unit 112 may also add a reference range around the area that is assumed to be the window to the detection range.

[0030] As a specific detection method, the object detection unit 112 may use an object detection model configured using a neural network, or may use template matching. The object detection unit 112 may also use other algorithms for detecting objects from image data.

[0031] (Step S15: Vehicle identification process) The vehicle identification unit 113 identifies vehicle information, which is information about the vehicle, from the license plate detected in step S12 and the processed image extracted in step S13. The vehicle information includes information such as the automobile registration number and the vehicle name shown on the license plate. Specifically, the vehicle identification unit 113 detects the vehicle registration number by performing character recognition on the license plate area. The vehicle identification unit 113 also identifies the vehicle name by providing the processed image as an input to a vehicle name inference model. The vehicle name inference model is configured using a neural network or the like, and is a model that takes image data as input and outputs the name of the vehicle captured in the image data. For example, the vehicle name inference model calculates the similarity between the shape of the vehicle captured in the image data and the shape of each vehicle name, and outputs the vehicle name with the highest similarity. The vehicle name inference model may be a model that inputs vehicle feature quantities and outputs a vehicle name. In this case, the vehicle identification unit 113 extracts vehicle feature quantities from the processed image and inputs the extracted feature quantities to the vehicle name inference model. The feature quantities are information that quantifies the vehicle features in the image data.

[0032] The vehicle identification unit 113 may identify the vehicle model as a prerequisite for identifying the vehicle name. The vehicle model is a classification of the vehicle, such as a large vehicle, a standard vehicle, or a light vehicle. In most cases, the vehicle model can be identified from the size of the license plate, the classification number of the vehicle registration number, and the color of the license plate (background color and character color).

[0033] (Step S16: Validity determination process) The object detection unit 112 determines the validity of the detected object based on the shape of the vehicle identified from the vehicle name identified in step S15. Specifically, the object detection unit 112 determines whether the detected object is detected from an appropriate range based on the shape of the vehicle. An appropriate range means a range in which the object can be detected. For example, suppose the object is a passenger. In this case, the object detection unit 112 identifies the window part of the vehicle more accurately than in step S14 based on the shape of the vehicle and the position of the license plate. Then, the object detection unit 112 determines whether the position where the passenger was detected in step S14 is a window part. If the detected position is a window part, the object detection unit 112 determines that the detected passenger is valid. On the other hand, if the detected position is not a window part, the object detection unit 112 determines that the detected passenger is invalid.

[0034] If there is an object determined to be invalid, the object detection unit 112 executes the process of step S14 again with an appropriate range as the detection range, and detects the object from the appropriate range. Then, the object detection unit 112 proceeds to the process of step S17. On the other hand, if there is no object determined to be invalid, the object detection unit 112 proceeds to the process of step S17.

[0035] (Step S17: Information management process) The information management unit 114 associates the vehicle information identified in step S15 with the information on the object detected in step S14 and writes the information to a storage device. In this case, the storage device is the vehicle database 231 of the central device 20. As a result, as shown in Figure 5, vehicle database 231 stores records containing vehicle information such as the vehicle registration number and vehicle name, and information on the object such as whether or not the vehicle has a senior citizen mark, whether or not it has a beginner mark, and the number of passengers.

[0036] The search process according to the first embodiment will be described with reference to FIG. (Step S21: Condition acquisition process) The search unit 211 of the central device 20 acquires search conditions. The search conditions are specified using information included in either the vehicle information or the object information registered in the vehicle database 231. In the case of the vehicle database 231 shown in Fig. 5, the search conditions are specified by at least one of the vehicle registration number, the vehicle name, whether or not the vehicle has a senior citizen mark, whether or not the vehicle has a beginner mark, and the number of passengers. For example, the search unit 211 acquires search conditions input by the search terminal 30. Alternatively, the search unit 211 may acquire search conditions that are set in advance.

[0037] (Step S22: Search process) The search unit 211 of the central device 20 extracts records that satisfy the search conditions acquired in step S21 from the vehicle database 231. Then, the search unit 211 outputs information about the extracted records to the search terminal 30.

[0038] ***Effects of the First Embodiment*** As described above, the vehicle recognition system 100 according to the first embodiment extracts image data in which a license plate is detected as a processed image and detects an object from the processed image. This eliminates the need to perform processing to detect an object on image data of an unnecessarily large number of frames. As a result, even if the vehicle recognition device 10 is implemented using a computer with limited processing power, it is possible to detect an object.

[0039] Many computers with high processing power are large and not suitable for installation in outdoor environments. Therefore, it is often difficult to install them on roadsides or other locations near cameras 40. As a result, when performing object detection processing on image data of many frames capturing images of vehicles, as in the past, it was necessary to perform the detection processing on a computer with high processing power installed in a server room or the like. For example, when several thousand cameras 40 are installed, the image data obtained by each camera 40 is transferred to a computer installed in a server room or the like, and the computer installed in the server room or the like performs the detection processing all at once. Therefore, when detecting objects as in the past, real-time searching of the detection results is impaired. In contrast, in the vehicle recognition system 100 according to the first embodiment, even if the vehicle recognition device 10 is implemented by a computer with limited processing power, it is possible to detect an object. That is, an object can be detected by the vehicle recognition device 10 installed on the roadside or the like near where the camera 40 is installed. As a result, there is no need for the central device 20 installed in a server room or the like to perform the detection process. Therefore, it is possible to improve the real-time nature of searches for detection results.

[0040] The vehicle recognition system 100 according to the first embodiment stores vehicle information and object information in association with each other in the vehicle database 231. This makes it possible to provide services that use object information for each vehicle. For example, it becomes possible to provide services for elderly people or services according to the number of passengers.

[0041] ***Other Configurations*** <Variation 1> In the first embodiment, each functional component is realized by software. However, as a first modification, each functional component may be realized by hardware. The differences between the first embodiment and the first modification will be described below.

[0042] When each functional component is realized by hardware, the vehicle recognition device 10 includes an electronic circuit instead of the processor 11, the memory 12, and the storage 13. The electronic circuit is a dedicated circuit that realizes the functions of each functional component, the memory 12, and the storage 13.

[0043] Possible electronic circuits include single circuits, composite circuits, programmed processors, parallel programmed processors, logic ICs, GAs, ASICs, and FPGAs. GA stands for Gate Array. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field-Programmable Gate Array. Each functional component may be realized by one electronic circuit, or each functional component may be realized by distributing it among a plurality of electronic circuits.

[0044] <Variation 2> As a second modification, some of the functional components may be realized by hardware, and other functional components may be realized by software.

[0045] The processor 11, memory 12, storage 13, and electronic circuitry are collectively referred to as a processing circuit. In other words, the functions of the functional components are realized by the processing circuit.

[0046] Furthermore, the term "unit" in the above description may be read as a "circuit," "step," "procedure," "process," or "processing circuit."

[0047] Various aspects of the present disclosure are summarized below as appendices. (Appendix 1) a number detection unit that detects a license plate of the vehicle from any one of a plurality of image data obtained by photographing the vehicle with a camera; an object detection unit that extracts, as a processed image, image data in which the license plate has been detected by the number detection unit from among the plurality of image data, and detects at least one object between a mark displayed on the vehicle and a passenger in the vehicle from the processed image; A vehicle recognition device comprising: (Appendix 2) The object detection unit narrows a detection range of the object in the processed image based on a position where the license plate is detected from the processed image by the license plate detection unit, and detects the object from the narrowed detection range. 10. A vehicle recognition device according to claim 1. (Appendix 3) The vehicle recognition device further a vehicle identification unit that identifies the name of the vehicle from the processed image; Equipped with The object detection unit determines the validity of the detected object based on the shape of the vehicle identified from the vehicle name identified by the vehicle identification unit. 3. A vehicle recognition device according to claim 1 or 2. (Appendix 4) The vehicle recognition device further a vehicle identification unit that identifies vehicle information, which is information about the vehicle, from the license plate detected by the license plate detection unit; an information management unit that associates the vehicle information identified by the vehicle identification unit with object information, which is information about the object detected by the object detection unit, and writes the information into a storage device; 3. A vehicle recognition device according to claim 1 or 2, comprising: (Appendix 5) The computer detects the license plate of the vehicle from any one of a plurality of image data obtained by photographing the vehicle with the camera, A vehicle recognition method in which a computer extracts image data from multiple image data in which the license plate has been detected as a processed image, and detects at least one of the following objects from the processed image: a mark displayed on the vehicle and an occupant of the vehicle. (Appendix 6) a number detection process for detecting a license plate of the vehicle from any one of a plurality of image data obtained by photographing the vehicle with a camera; an object detection process for extracting, as a processed image, image data in which the license plate has been detected by the number detection process from among the plurality of image data, and detecting at least one object between a mark displayed on the vehicle and a passenger in the vehicle from the processed image; A vehicle recognition program that causes a computer to function as a vehicle recognition device that performs the above.

[0048] The embodiments and modifications of the present disclosure have been described above. Some of these embodiments and modifications may be combined and implemented. Also, one or more of them may be implemented partially. Note that the present disclosure is not limited to the above embodiments and modifications, and various modifications are possible as needed. [Explanation of symbols]

[0049] 100 Vehicle recognition system, 10 Vehicle recognition device, 11 Processor, 12 Memory, 13 Storage, 14 Communication interface, 111 Number detection unit, 112 Object detection unit, 113 Vehicle identification unit, 114 Information management unit, 20 Central unit, 21 Processor, 22 Memory, 23 Storage, 24 Communication interface, 211 Search unit, 231 Vehicle database, 30 Search terminal, 40 Camera, 41 Lighting device.

Claims

1. a number detection unit that detects a license plate of the vehicle from any one of a plurality of image data obtained by photographing the vehicle with a camera; an object detection unit that extracts, as a processed image, image data in which the license plate detection unit has detected the license plate from among the plurality of image data, and detects at least one object from the processed image, including a mark displayed on the vehicle and a passenger in the vehicle; a vehicle identification unit that identifies the name of the vehicle from the processed image; Equipped with the object detection unit identifies a shape of the vehicle identified from the vehicle name identified by the vehicle identification unit, identifies a range in which the object may be detected based on the shape of the vehicle, and determines the validity of the detected object based on the range in which the object may be detected and the position where the object is detected. Vehicle recognition device.

2. The object detection unit narrows a detection range of the object in the processed image based on a position where the license plate is detected from the processed image by the license plate detection unit, and detects the object from the narrowed detection range. The vehicle recognition device according to claim 1 .

3. When the position where the object is detected in determining the validity of the object is not within the range where the object can be detected, the object detection unit sets the range where the object can be detected as the detection range and detects the object again from the set detection range. The vehicle recognition device according to claim 2 .

4. The vehicle recognition device further a vehicle identification unit that identifies vehicle information, which is information about the vehicle, from the license plate detected by the license plate detection unit; an information management unit that associates the vehicle information identified by the vehicle identification unit with object information, which is information about the object detected by the object detection unit, and writes the information into a storage device; The vehicle recognition device according to claim 1 .

5. The computer detects the license plate of the vehicle from any one of a plurality of image data obtained by photographing the vehicle with the camera, a computer extracting, from the plurality of image data, image data in which the license plate has been detected as a processed image, and detecting at least one object from the processed image, the mark displayed on the vehicle and the occupant of the vehicle; Identifying the name of the vehicle from the processed image; Identifying a range in which the object may be detected based on the shape of the vehicle identified from the vehicle name, and determining the validity of the detected object based on the range in which the object may be detected and the position where the object is detected. Vehicle recognition method.

6. Detecting a license plate of the vehicle from any one of a plurality of image data obtained by photographing the vehicle with a camera; extracting, from the plurality of image data, image data in which the license plate has been detected as a processed image, and detecting at least one object from the processed image, including a mark displayed on the vehicle and a passenger in the vehicle; Identifying the name of the vehicle from the processed image; Identifying a range in which the object may be detected based on the shape of the vehicle identified from the vehicle name, and determining the validity of the detected object based on the range in which the object may be detected and the position where the object is detected; A vehicle recognition program that causes a computer to function as a vehicle recognition device that performs the above.

Images