Image recognition system

Abstract

To provide an image recognition system that can improve the accuracy of image recognition. [Solution] An image recognition system according to one embodiment includes an imaging unit, a zoom unit, and an image recognition unit. The imaging unit captures an image. The zoom unit enlarges the image captured by the imaging unit. The image recognition unit recognizes and identifies the object to be recognized from the enlarged image enlarged by the zoom unit.

Description

Technical Field

[0001] The disclosed embodiments relate to an image recognition system.

Background Art

[0002] There is an image recognition system that performs image recognition of a recognition target from an image using a learning model such as image recognition AI (Artificial Intelligence) (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of Invention

Problems to be Solved by the Invention

[0004] However, when image recognition of a recognition target is performed from an image of a subject located at a relatively long distance captured outdoors using an indoor image recognition AI that performs image recognition of a recognition target from an image of a subject located at a relatively short distance captured indoors, the accuracy of image recognition decreases.

[0005] One aspect of the embodiments has been made in view of the above, and an object thereof is to provide an image recognition system capable of improving the accuracy of image recognition.

Means for Solving the Problems

[0006] An image recognition system according to one aspect of the embodiments includes an imaging unit, a zoom unit, and an image recognition unit. The imaging unit captures an image. The zoom unit enlarges the image captured by the imaging unit. The image recognition unit performs image recognition and discriminates a recognition target from the enlarged image enlarged by the zoom unit.

Effects of the Invention

[0007] The image recognition system according to the present invention does not directly recognize the object to be recognized from the captured image, but rather recognizes and identifies the object to be recognized from an enlarged image obtained by expanding the captured image, thereby improving the accuracy of image recognition. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is an explanatory diagram showing an overview of the image recognition method by the image recognition system according to the embodiment. [Figure 2] Figure 2 is a block diagram showing an example of the configuration of an image recognition system according to the embodiment. [Figure 3] Figure 3 is an explanatory diagram showing a specific example of the image recognition procedure according to the embodiment. [Figure 4] Figure 4 is an explanatory diagram showing a specific example of the image recognition procedure according to the embodiment. [Figure 5] Figure 5 is a flowchart showing an example of the processing performed by the image recognition system according to the present invention. [Modes for carrying out the invention]

[0009] The embodiments of the image recognition system will be described in detail below with reference to the attached drawings. However, this invention is not limited to the embodiments described below. Here, we will describe a case in which the image recognition system according to the embodiment recognizes and identifies a person in a parking lot from an image of the entire parking lot.

[0010] Furthermore, the image recognized by the image recognition system according to this embodiment is not limited to images of parking lots, but may be an image of any location. Also, the object of recognition by the image recognition system according to this embodiment is not limited to people, but may be any object other than people.

[0011] Figure 1 is an explanatory diagram illustrating the outline of an image recognition method using an image recognition system 1 according to an embodiment. As shown in Figure 1, the image recognition system 1 includes an imaging device 2 and an image recognition device 3. The imaging device 2 is installed in a position that allows it to oversee the entire parking lot 10.

[0012] The imaging device 2 captures a wide-angle image of the entire parking lot 10. The imaging device 2 is connected to the image recognition device 3 via wired or wireless means for information communication. The imaging device 2 transmits an image of the entire parking lot 10 to the image recognition device 3. In the example shown in Figure 1, the imaging device 2 transmits an image of the parking lot 10 that includes two people 11 and 12 and three vehicles 13, 14, and 15 to the image recognition device 3.

[0013] The image recognition device 3 is equipped with an image recognition AI (Artificial Intelligence) that has been machine-trained to recognize and identify pre-set recognition targets from input images. The image recognition device 3 does not recognize the person to be recognized from the captured image received from the imaging device 2, but rather recognizes and identifies the person 11 to be recognized from an enlarged image P of the captured image.

[0014] The image recognition device 3 may acquire an enlarged image from the imaging device 2, or it may acquire an enlarged image by enlarging the image it receives from the imaging device 2 itself. In this way, the image recognition device 3 does not directly recognize the object to be recognized from the image, but rather recognizes and identifies the object to be recognized from the enlarged image obtained by enlarging the image, thereby improving the accuracy of image recognition.

[0015] Therefore, even if the image recognition device 3 is equipped with an indoor image recognition AI that performs image recognition of a target from an image of a subject located at a relatively close distance and captured indoors, it can still perform image recognition of a target from an image of a subject located at a relatively long distance and captured outdoors. In other words, the image recognition device 3 can perform image recognition and identification of a target with high accuracy from an image captured outdoors without having to retrain the indoor image recognition AI for outdoor use.

[0016] Next, with reference to FIG. 2, a configuration example of the image recognition system 1 according to the embodiment will be described. FIG. 2 is a block diagram showing an example of the configuration of the image recognition system 1 according to the embodiment. As shown in FIG. 2, the image recognition system 1 includes an imaging device 2 and an image recognition device 3.

[0017] The imaging device 2 includes an imaging unit 21, a digital zoom unit 22, and an optical zoom unit 23. The imaging unit 21 is, for example, a solid-state imaging device such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0018] The digital zoom unit 22 is an image processing unit that cuts out a partial image from a captured image imaged by the imaging unit 21 through digital zoom and enlarges the partial image. The optical zoom unit 23 includes a normal lens and a telephoto lens.

[0019] The optical zoom unit 23 can cause the imaging unit 21 to capture an enlarged image enlarged by optical zoom by switching the lens used by the imaging unit 21 from the normal lens to the telephoto lens.

[0020] Since the digital zoom unit 22 generates an enlarged image digitally zoomed through image processing, the processing speed for generating the enlarged image is faster than that of the optical zoom unit 23. However, although the size of the enlarged image enlarged by digital zoom increases, the resolution thereof slightly decreases.

[0021] On the other hand, when causing the imaging unit 21 to capture an enlarged image enlarged by optical zoom, the optical zoom unit 23 needs to mechanically switch the lens used for imaging from the normal lens to the telephoto lens, and thus it takes time to capture the optically zoomed enlarged image. However, the enlarged image enlarged by optical zoom has less reduction in resolution compared to the image enlarged by digital zoom.

[0022] The image recognition device 3 comprises a storage unit 4 and a controller 5. The storage unit 4 is, for example, an information storage device such as a data flash, and stores image data 41, a threshold value 42, a first score 43, and a second score 44. The image data 41 is the data of the captured image that the image recognition device 3 receives and acquires from the imaging device 2.

[0023] The threshold 42, the first score 43, and the second score 44 are numerical values ​​used or calculated by the controller 5 when performing image recognition on the target object. The threshold 42, the first score 43, and the second score 44 will be described later with reference to Figures 3 and 4.

[0024] The controller 5 includes a microcomputer with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and various circuits. The controller 5 includes an image recognition unit 51 that functions by the CPU executing a program stored in ROM using RAM as a working area.

[0025] Furthermore, the image recognition unit 51 of the controller 5 may be composed of hardware such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), either partially or entirely.

[0026] The image recognition unit 51 is the image recognition AI described above. The image recognition AI is equipped with a machine learning model such as a DNN (Deep Neural Network) model. When image data 41 is input, the machine learning model is trained to calculate a score indicating the likelihood that the subject contained in the image data 41 is a pre-defined recognition target. The image recognition unit 51 identifies the subject as a candidate for recognition if the calculated score is above a threshold.

[0027] When the image recognition unit 51 receives an image from the imaging device 2, it performs image recognition on the image to identify candidate targets for recognition. The image recognition unit 51 then requests an enlarged image from the imaging device 2, which is an enlarged version of the recognized target.

[0028] The imaging device 2 digitally zooms in on the image of the target to be recognized by the image recognition unit 51 and enlarges it from the captured image. Then, the imaging device 2 transmits the enlarged image, which has been digitally zoomed in, to the image recognition device 3.

[0029] Furthermore, the image processing that involves extracting and enlarging the recognized image from the captured image using digital zoom may be performed by the image recognition unit 51. In this case, the digital zoom unit 22 is provided in the image recognition unit 51.

[0030] The image recognition unit 51 performs image recognition on the candidate images of the target to be recognized, which have been enlarged by digital zoom. In this way, the image recognition unit 51 performs image recognition on the enlarged images of the target to be recognized, so it can perform image recognition on the target to be recognized with higher accuracy than when image recognition is performed on the image before enlargement.

[0031] Subsequently, the image recognition unit 51 requests the imaging device 2 to capture images of the candidate recognition targets identified by image recognition from the image enlarged by digital zoom using a telephoto lens. Details of the method for narrowing down the candidate recognition targets will be described later with reference to Figures 3 and 4. In response to the request from the image recognition unit 51, the imaging device 2 enlarges the candidate recognition targets identified by image recognition from the image enlarged by digital zoom using a telephoto lens, thereby enlarging the candidate recognition targets by optical zoom.

[0032] The imaging device 2 transmits an image of a candidate object to be recognized, magnified by optical zoom, to the image recognition device 3. The image recognition unit 51 performs image recognition and identifies the object to be recognized from the magnified image of the candidate object to be recognized.

[0033] In this way, the image recognition unit 51 can improve the accuracy of image recognition by performing image recognition and identification on an image enlarged by optical zoom, which has a higher resolution than the image enlarged by digital zoom.

[0034] Next, with reference to Figures 3 and 4, a specific example of the image recognition procedure according to the embodiment will be described. Figures 3 and 4 are explanatory diagrams showing a specific example of the image recognition procedure according to the embodiment. Here, as with Figure 1, we will explain using the example of a case where two people 11 and 12 and three vehicles 13, 14, and 15 are present within the premises of the parking lot 10.

[0035] First, the image recognition unit 51 performs image recognition on a pre-magnified image of the entire parking lot 10 captured by the imaging device 2 to identify candidates for recognition. At this time, the image recognition unit 51 recognizes people 11 and 12 and three vehicles 13, 14, and 15 as candidates for recognition.

[0036] At this point, the image recognition unit 51 does not determine whether or not a candidate is the person to be recognized, but calculates a first score 43 for each of the five candidate recognition targets, indicating the likelihood that the person is the target recognition target. The numbers written in the dotted ellipse within the speech bubble shown in Figure 3 are the first scores 43 calculated for each candidate recognition target.

[0037] In the example shown in Figure 3, the first score 43 for person 11 is 0.5, the first score 43 for person 12 is 0.4, the first score 43 for vehicle 13 is 0.4, the first score 43 for vehicle 14 is 0.2, and the first score 43 for vehicle 13 is 0.3. The image recognition unit 51 stores the calculated first score 43 in the storage unit 4, associating it with each candidate recognition target.

[0038] Subsequently, the image recognition unit 51 requests the imaging device 2 to obtain an enlarged image of a candidate image for recognition, in which the first score 43 is higher than the threshold 42, by digital zooming. Here, as an example, the threshold 42 is set to 0.4.

[0039] In the example shown in Figure 3, the first score 43 was 0.4 or higher, which is the threshold 42, for people 11, 12 and vehicle 13. Therefore, the image recognition unit 51 requests the imaging device 2 to provide enlarged images of people 11, 12 and vehicle 13, obtained by digital zooming.

[0040] The imaging device 2 digitally zooms in and enlarges images of people 11, 12 and vehicle 13 from an image showing the entire parking lot 10. Then, the imaging device 2 transmits the enlarged image, created by digital zoom, to the image recognition device 3.

[0041] In this way, the digital zoom unit 22 of the imaging device 2 selectively enlarges the images of the people 11, 12 and the vehicle 13, which are selected by the image recognition unit 51 from among the five candidate recognition targets, using digital zoom. As a result, the processing load on the digital zoom unit 22 is reduced compared to when all five candidate recognition target images are enlarged using digital zoom.

[0042] Subsequently, the image recognition unit 51 calculates a second score 44 from the image of the candidate to be recognized after it has been enlarged by digital zoom, indicating the likelihood that the recognized candidate is the person being recognized. The numbers written within the solid ellipse in the speech bubble shown in Figure 4 are the second scores 44 calculated for each candidate to be recognized.

[0043] In the example shown in Figure 4, the second score 44 for person 11 is 0.9, the second score 44 for person 12 is 0.8, and the second score 44 for vehicle 13 is 0.2. The image recognition unit 51 stores the calculated second score 44 in the storage unit 4, associating it with each candidate recognition target. For example, the image recognition unit 51 reads the features of the candidate for which a score was calculated from the captured image, and records the candidate's features, the first score 43, and the second score 44 as a set in the storage unit 4.

[0044] In this way, the image recognition unit 51 calculates a second score 44 only for candidates whose first score 43 is 0.4 or higher than the threshold 42, among the candidates for recognition for which a first score 43 has been calculated. As a result, the processing load for calculating the second score 44 is reduced for the image recognition unit 51 compared to calculating the second score 44 for all candidates for recognition for which a first score 43 has been calculated.

[0045] Subsequently, the image recognition unit 51 requests the imaging device 2 to enlarge an image of the candidate recognition target whose second score 44 is higher than the first score 43, using optical zoom. The imaging device 2 captures the candidate recognition target whose second score 44 is higher than the first score 43 and enlarges it using the optical zoom unit 23.

[0046] The optical zoom unit 23 transmits the magnified image, enlarged by optical zoom, to the image recognition device 3. The image recognition unit 51 performs image recognition on the magnified image of the candidate recognition target, which has been enlarged by optical zoom, and identifies the target as a person.

[0047] In this way, the image recognition unit 51 performs image recognition on a candidate image of the recognition target, which has a higher resolution than the image enlarged by digital zoom, and identifies the person. Therefore, even an indoor image recognition AI can improve the accuracy of image recognition.

[0048] Next, referring to Figure 5, the processes performed by the image recognition system 1 according to this embodiment will be described. Figure 5 is a flowchart showing an example of the processes performed by the image recognition system 1 according to this embodiment.

[0049] As shown in Figure 5, in the image recognition system 1, a threshold 42 is set in the image recognition unit 51 (step S101). Then, the imaging device 2 captures an overall image of the target area (step S102). The image recognition device 3 performs image recognition on the overall image captured by the imaging device 2 to identify candidate objects to be recognized (step S103).

[0050] Next, the image recognition device 3 calculates a first score 43 for the candidate recognition target recognized by the image (step S104). After that, the image recognition device 3 determines whether or not there are candidates whose first score 43 is equal to or greater than the threshold 42 (step S105).

[0051] If the image recognition device 3 determines that there are no candidates whose first score 43 is equal to or greater than the threshold 42 (step S105, No), it proceeds to step S102. If the image recognition device 3 determines that there are candidates whose first score 43 is equal to or greater than the threshold 42 (step S105, Yes), it enlarges the candidates whose first score 43 is equal to or greater than the threshold 42 by digital zoom (step S106).

[0052] Subsequently, the image recognition device 3 calculates a second score 44 for the candidate that has been enlarged by digital zoom (step S107). Then, the image recognition device 3 determines whether the calculated second score 44 is higher than the first score 43 for that candidate (step S108).

[0053] If the image recognition device 3 determines that the second score 44 is not higher than the first score 43 (step S108, No), it proceeds to step S111. If the image recognition device 3 determines that the second score 44 is higher than the first score 43 (step S108, Yes), the optical zoom unit 23 magnifies the candidate using optical zoom (step S109). In this way, the image recognition system 1 can speed up image recognition processing by narrowing the target of optical zoom to a subset of candidates (subjects) where the second score 44 is higher than the first score 43.

[0054] Then, the image recognition device 3 recognizes and identifies the object to be recognized from the magnified image of the candidate, which has been enlarged by optical zoom (step S110). Then, the image recognition device 3 determines whether or not image recognition has been completed for all candidates recognized in step S103 (step S111).

[0055] If the image recognition device 3 determines that image recognition of all candidates has not been completed (step S111, No), it moves the process to step S104. If the image recognition device 3 determines that image recognition of all candidates has been completed (step S111, Yes), it terminates the process and restarts the process from step S101. Alternatively, the image recognition device 3 may be configured to start the process from step S102 if it determines that image recognition of all candidates has been completed (step S111, Yes).

[0056] In the embodiment described above, the image recognition unit 51 performed image recognition on the wide-angle image before enlargement to identify candidates for recognition and calculated a first score 43. Then, it enlarged the candidate images with a first score 43 equal to or greater than the threshold 42 using digital zoom to calculate a second score 44. However, this is just one example.

[0057] The image recognition unit 51 may perform image recognition on the wide-angle image before enlargement to identify candidates for recognition, and then calculate a first score 43 for the enlarged image of the recognized candidate obtained by digitally zooming the image of the candidate.

[0058] In this case, the image recognition unit 51 may calculate a second score 44 by optically zooming in on candidate images whose first score 43 is equal to or greater than the threshold 42. Subsequently, the image recognition unit 51 performs image recognition on the optically zoomed candidates to make a distinction. This further improves the reliability of the second score 44, thereby improving the accuracy of image recognition by the image recognition device 3.

[0059] Furthermore, although the above-described embodiment described a case where the imaging device 2 and the image recognition device 3 are separate, the imaging device 2 and the image recognition device 3 may be integrated. In this case, the image recognition device 3 is built into the imaging device 2. Also, when the imaging device 2 and the image recognition device 3 are separate, the server responsible for the functions of the image recognition device 3 may be deployed on a communication network, for example, using the cloud.

[0060] Furthermore, the image recognition system 1 may be configured to directly recognize and identify the object to be recognized from an enlarged image obtained by digital zooming or optical zooming of the entire image of the target area.

[0061] While embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0062] 1. Image Recognition System 2. Imaging device 3. Image recognition device 4 Storage section 5 Controllers 10 Parking 11,12 people Vehicles 13, 14, 15 21 Imaging Department 22 Digital Zoom Section 23 Optical zoom section 41 Image data 42 threshold 43 First score 44 Second score 51 Image Recognition Unit

Claims

1. An imaging unit that captures images, A zoom unit that enlarges the image captured by the aforementioned imaging unit, An image recognition unit that performs image recognition and identifies the object to be recognized from the enlarged image magnified by the zoom unit. An image recognition system that includes this feature.

2. The image recognition unit performs image recognition on the image captured by the imaging unit to identify candidate targets for recognition. The zoom unit crops and enlarges the image of the candidate to be recognized, which has been recognized by the image recognition unit, using digital zoom. The image recognition unit performs image recognition on the candidate image of the target to be recognized, which has been enlarged by the digital zoom. The image recognition system according to claim 1.

3. The imaging unit captures the candidate for recognition, which has been recognized from the image enlarged by the digital zoom, and enlarges it by optical zoom. The image recognition unit performs image recognition and identifies the target to be recognized from the candidate images of the target to be recognized, which have been magnified by the optical zoom. The image recognition system according to claim 2.

4. The image recognition unit calculates a first score indicating the likelihood that the candidate for recognition, recognized from the image before it was enlarged by the digital zoom, is the target for recognition. The zoom unit digitally zooms in and enlarges the image of the candidate recognition target whose first score is equal to or greater than the threshold. The image recognition system according to claim 3.

5. The image recognition unit calculates a second score from the image of the candidate to be recognized after it has been enlarged by the digital zoom, indicating the likelihood that the candidate to be recognized is the target to be recognized. The zoom unit captures images of candidate recognition targets whose second score is higher than the first score and magnifies them by optical zoom. The image recognition system according to claim 4.

Images