Image processing device, image processing method, and image processing program

Abstract

The present invention provides an image processing device capable of appropriately improving the recognition rate according to the size of the object to be recognized. [Solution] The image processing device 100 comprises an input image acquisition unit 111, a storage unit 120 for resize information and target size, a pre-recognition processing unit 112, an image recognition unit 113, and a resize information generation unit 114. The input image acquisition unit 111 acquires an input image captured by a camera. The pre-recognition processing unit 112 resizes the input image to a recognition image based on the resize information at a first time point. The image recognition unit 113 recognizes the recognition image and generates recognition result information including the size to be recognized and the recognition determination probability. The resize information generation unit 114 generates resize information for a second time point after the first time point based on the recognition result information and the target size information.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

Background Art

[0002] Conventionally, a technique for recognizing an input image using a learning result obtained by previously learning an image to be recognized has been proposed. In Patent Document 1, an image processing apparatus that performs image recognition using a discriminator generated by statistical learning with a fixed-size sample image as teacher data is disclosed. In the image processing apparatus disclosed in Patent Document 1, recognition processing is performed using a transformed image obtained by geometrically transforming a target image so that the aspect ratio of the target image including the object to be detected becomes a predetermined ratio set in advance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In an image obtained by learning an object to be recognized, the recognition rate varies depending on the degree of learning according to the size of the target image. Therefore, for example, in the image processing apparatus in Patent Document 1, even if the input image has a size with a low recognition rate, only the aspect ratio is converted, and the recognition rate may not improve.

[0005] The present invention has been made in view of the above problems, and an object thereof is to provide an image processing apparatus, an image processing method, and an image processing program capable of appropriately improving the recognition rate according to the size of the recognition target.

Means for Solving the Problems

[0006] An image processing apparatus according to one aspect of the present invention comprises: an input image acquisition unit that acquires an input image captured by a camera; a storage unit that stores resize information and target size information; a pre-recognition processing unit that resizes the input image to a recognition image based on the resize information at a first time point; an image recognition unit that recognizes the recognition image and generates recognition result information including the size of the object to be recognized and the recognition determination probability; and a resize information generation unit that generates resize information for a second time point after the first time point based on the recognition result information and the target size information.

[0007] An image processing method according to one aspect of the present invention is an image processing method performed by a computer, which acquires an input image captured by a camera, resizes the input image to a recognition image based on resize information at a first time point stored in a storage unit, recognizes the recognition image, generates recognition result information including the size of the object to be recognized and the recognition determination probability, and generates resize information at a second time point after the first time point based on the recognition result information and target size information stored in a storage unit in advance.

[0008] An image processing program according to one aspect of the present invention causes a computer to perform the following steps: acquire an input image captured by a camera; resize the input image to a recognition image based on resize information for a first time point stored in a memory unit; recognize the recognition image and generate recognition result information including the size of the object to be recognized and the probability of recognition; and generate resize information for a second time point after the first time point based on the recognition result information and target size information stored in a memory unit. [Effects of the Invention]

[0009] According to the present invention, it is possible to appropriately improve the recognition rate according to the size of the object to be recognized. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a block diagram showing the schematic configuration of the image processing apparatus according to this embodiment. [Figure 2]Figure 2 is a block diagram showing the functional configuration of the image processing apparatus according to this embodiment. [Figure 3A] Figure 3A shows an example of an input image according to this embodiment. [Figure 3B] Figure 3B shows a comparative example of input image, recognition image, and output image in an image processing device. [Figure 3C] Figure 3C shows examples of input images, recognition images, and output images applied to the image processing apparatus according to this embodiment. [Figure 4A] Figure 4A is a diagram illustrating the relationship between typical training image size and recognition probability as a comparative example. [Figure 4B] Figure 4B is a diagram illustrating the relationship between the learning image size and the recognition determination probability applied in the image processing apparatus according to this embodiment. [Figure 5] Figure 5 shows an example of target size information applied to the image processing apparatus according to this embodiment. [Figure 6] Figure 6 is a flowchart showing an example of processing performed by the image processing apparatus according to this embodiment. [Figure 7] Figure 7 is a flowchart illustrating the prerequisite processing in the image processing apparatus according to this embodiment. [Figure 8] Figure 8 is a flowchart illustrating the resize information generation process in the image processing apparatus according to this embodiment. [Figure 9] Figure 9 is a flowchart illustrating the target size update process in the image processing apparatus according to this embodiment. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will be described below with reference to the drawings. In the drawings, identical parts are denoted by the same reference numerals and their descriptions are omitted.

[0012] (Outline configuration of the image processing device 100) FIG. 1 is a block diagram showing a schematic configuration of an image processing apparatus 100 according to the present embodiment. The image processing apparatus 100 includes a control unit 110, a storage unit 120, and an input / output IF 130 (Interface).

[0013] The image processing apparatus 100 acquires an input image of an image captured by a camera via a sensor 131 and an ISP 132 (Image Signal Processor) of the input / output IF 130. Further, the control unit 110 of the image processing apparatus 100 performs recognition processing on a recognition target included in the input image based on a learning image in which the recognition target has been learned in advance and stored in the storage unit 120. Further, the control unit 110 stores recognition result information, which is a result of recognizing the recognition target, in the storage unit 120. Further, the control unit 110 may output the recognition result information to the outside via an output unit 133.

[0014] Further, the image processing apparatus 100 may be configured as a general-purpose microcomputer including a CPU (Central Processing Unit, control unit 110), a memory (storage unit 120), and an input / output unit (input / output IF 130).

[0015] In this case, a computer program for causing the microcomputer to function as the image processing apparatus 100 may be installed. By executing the computer program, the microcomputer functions as a plurality of information processing circuits included in the image processing apparatus 100.

[0016] In the present embodiment, an example in which a plurality of information processing circuits included in the image processing apparatus 100 are realized by software is shown. Of course, it is also possible to prepare dedicated hardware for executing each of the information processes shown below to configure the information processing circuit. Further, a plurality of information processing circuits may be configured by individual hardware. Specifically, part or all of the information processing functions can be executed by hardware such as a DSP (Digital Signal Processor) or an ASIC (Application Specific Integrated Circuit).

[0017] (Functional Configuration of Image Processing Apparatus 100) FIG. 2 is a block diagram showing the functional configuration of the image processing apparatus 100 according to the present embodiment. As shown in FIG. 2, the control unit 110 of the image processing apparatus 100 includes an input image acquisition unit 111, a pre-recognition processing unit 112, an image recognition unit 113, a resize information generation unit 114, and a target size update unit 115 as functions.

[0018] Further, the control unit 110 operates, for example, an operating system to control the entire image processing apparatus 100. Furthermore, the control unit 110 operates based on a program stored in the storage unit 120 and executes each of the above functions. Note that the program is not limited to the form stored in the storage unit 120, and may be, for example, a configuration stored in a ROM (Read Only Memory) or the like (not shown) within the image processing apparatus 100.

[0019] The storage unit 120 stores, as data, information included in an input image information DB 121 (DB: Database), a resize information DB 122, a recognition image information DB 123, a recognition result information DB 124, a target size information DB 125, and an image recognition model DB 126. Note that the storage unit 120 for storing each of these data may be one or a plurality. For example, one storage unit 120 may be configured to store data in divided areas. Alternatively, the data may be distributed and stored in a plurality of storage devices installed at physically separated locations.

[0020] The input image acquisition unit 111 acquires the input image captured by the camera. Specifically, the input image acquisition unit 111 acquires the image captured by the camera as an input image via the sensor 131 and ISP 132 of the input / output IF 130 and stores it in the input image information DB 121. The input image acquired by the input image acquisition unit 111 of the image processing device 100 according to this embodiment may be a video at a predetermined frame rate generated by the ISP 132. Alternatively, the input image acquired by the input image acquisition unit 111 may be a series of still images generated at predetermined timings in the ISP 132. The input image acquisition unit 111 may output the input image to the recognition preprocessing unit 112 instead of storing it in the input image information DB 121.

[0021] Here, the differences between the recognition processing in a conventional image processing device and the recognition processing in the image processing device 100 according to this embodiment will be explained using Figures 3A to 3C and Figures 4A and 4B.

[0022] Figure 3A shows an example of an input image according to this embodiment. In the example shown in Figure 3A, a diagram is used for explanation in which a person is superimposed on an image where the person to be recognized is gradually approaching. For example, in the diagram shown in Figure 3A, an example is shown in which each image is superimposed when a person moves from the back left to the front right. In Figure 3A, the lines dividing the image into rectangles are to clearly show the position of the person, and the shaded area shows the path the person is moving along. Also, in the example shown in Figure 3A, each image before superimposition contains one image of a person, and by superimposing them, 10 people are shown. That is, in the example shown in Figure 3A, the input image shows an example in which the position and size of a person change as they move from the back left to the bottom right.

[0023] Figure 3B shows examples of input images, recognition images, and output images in a conventional image processing device. Figure 3C shows examples of input images, recognition images, and output images applied to the image processing device 100 according to this embodiment. An input image is an image input to the image processing device, a recognition image is an image used for image recognition within the image processing device, and an output image is an image output from the image processing device. In this embodiment, the previous input image (hereinafter referred to as the previous frame) corresponds, for example, to input image (B) in Figures 3B and 3C, which is the previous frame, input image (A) in Figures 3B and 3C. Similarly, in this embodiment, the next input image, the following input image (hereinafter referred to as the input image after the current one) corresponds to input image (B) in Figures 3B and 3C, which is the input image after the current one, with respect to input image (A) in Figures 3B and 3C. That is, in Figures 3B and 3C, the image progresses in the direction indicated by the arrow.

[0024] In the image processing device, the input image is subjected to predetermined image processing to improve recognition performance, and the result is used as a recognition image. Image recognition is then performed on the recognition image. In image recognition, the recognized object is calculated as the recognition target, and the likelihood of the recognition target being calculated as the recognition judgment probability. Furthermore, the recognition target is recognized as a rectangle, and the width and height of that rectangle are calculated as the recognition target size, and the coordinates of that rectangle are calculated as the recognition target coordinates. The recognition target size can also be considered as the area of ​​that rectangle. Below, the recognition target size will be explained as a single value representing the area of ​​the rectangle, but it can also be implemented as two values: the width and height of the rectangle.

[0025] Currently, recognition probability is considered in three stages: high, medium, and low. In conventional image processing devices, for example, in the example shown in Figure 3B, the size of the person in input image (A) and the person in input image (B) are different. In this case, even if the recognition probability in output image (A) is "high," the recognition probability in output image (B) may be "medium," which is lower than the recognition probability in output image (A).

[0026] On the other hand, in the example of the image processing device 100 according to this embodiment shown in Figure 3C, the resizing process described later is performed on the recognition image, and the recognition determination probability is maintained as "high" even in the output image (B). The image processing device 100 may perform predetermined image processing on the input image to improve recognition performance.

[0027] Figure 4A is a diagram illustrating the relationship between the typical training image size and recognition probability in conventional image processing devices. Figure 4B is a diagram illustrating the relationship between the training image size and recognition probability applied in the image processing device 100 according to this embodiment.

[0028] In Figures 4A and 4B, "Original Image Size" refers to the size of the original image in the training data used for machine learning and deep learning. "Trained Image Size" refers to the image size in the training data obtained by changing the size of the "Original Image Size" in several patterns. "Input Image Size" refers to the size of the image input to the image processing device 100. "Recognition Image Size" refers to the size of the recognition image used for image recognition processing.

[0029] As shown in Figure 4A, the recognition probability is high when the recognition image input to the recognition circuit is close to the size of the training image, but decreases as it deviates from the training image size. Furthermore, the recognition probability is highest when the size of the recognition image is equal to the size of the original image.

[0030] On the other hand, in the example of the image processing device 100 according to this embodiment shown in Figure 4B, it is shown that a high determination probability is maintained even when the recognition image input to the recognition circuit (image recognition unit 113 described later) deviates from the training image size, by performing a predetermined resizing process on the input image. The dashed curve in Figure 4B represents the recognition determination probability in Figure 4A. In other words, the image processing device 100 according to this embodiment can maintain a high recognition determination probability even in continuous images where the position (coordinates) and / or size of the recognition target changes.

[0031] The specific processing performed by the pre-recognition processing unit 112 is described below. The pre-recognition processing unit 112 may perform predetermined image processing on the input image to improve recognition performance. This image processing in the pre-recognition processing unit 112 may include, for example, a process to brighten dark areas across the entire image. Furthermore, the image processing in the pre-recognition processing unit 112 may include edge enhancement processing.

[0032] The pre-recognition processing unit 112 may select a predetermined frame from multiple frames included in the input image to be used as the recognition image. The pre-recognition processing unit 112 may select a predetermined range from the input image to be used as the recognition image. The pre-recognition processing unit 112 may resize the input image to be used as the recognition image.

[0033] Furthermore, if resize information has been generated during the processing of an input image prior to the present, the pre-recognition processing unit 112 performs resize processing on the input image. In other words, the pre-recognition processing unit 112 performs resize processing on the input image based on the resize information at the first time point. In this embodiment, resize information includes information regarding the resize magnification and the resize region. Here, the resize region includes information regarding the width and height of the region to be resized, and information regarding its position. The resize magnification is information indicating the magnification of the target in the resized recognition image relative to the size of the resize region in the input image. In this embodiment, resizing means changing the size of the recognition target included in the recognition image to a size that increases the recognition determination probability. The pre-recognition processing unit 112 sets the size of the resize region by multiplying the width and height by the resize magnification, respectively. The pre-recognition processing unit 112 may also calculate the area by multiplying the width and height of the resize region, and then set the size by multiplying the area by the resize magnification. The pre-recognition processing unit 112 may also perform a predetermined filtering process during resizing to remove noise generated during resizing.

[0034] Specifically, the pre-recognition processing unit 112 changes the size of the recognition target in the resize region included in the recognition image according to the resize magnification included in the resize information. The pre-recognition processing unit 112 also overlays the region containing the resized recognition target onto the recognition image according to the resize region included in the resize information, and updates the recognition image. Specifically, the pre-recognition processing unit 112 resizes the region containing the recognition target in the recognition image by the resize magnification based on the resize magnification and the resize region, and overlays the resized region containing the recognition target, aligned to the correct position, onto the recognition image. The pre-recognition processing unit 112 may also align the center position of the resized region after resizing with the center position of the resized region before resizing. The pre-recognition processing unit 112 may also select a predetermined position in the resize region when aligning the positions. Furthermore, the pre-recognition processing unit 112 stores the resized recognition image in the recognition image information DB 123. The pre-recognition processing unit 112 may output the resized recognition image to the image recognition unit 113 instead of storing it in the recognition image information DB 123.

[0035] Furthermore, if the pre-recognition processing unit 112 has not generated resize information during the processing of an input image prior to the current one, it will not perform resize processing and will store the input image as a recognition image in the recognition image information DB 123. Alternatively, the determination of whether or not resize information was generated during the processing of an input image prior to the current one may be made using a flag indicating whether or not resize information was generated during the processing of an input image prior to the current one. Or, the determination of whether or not resize information was generated during the processing of an input image prior to the current one may be made by providing a resize information table for storing resize information and determining whether or not there is information stored in the resize information table. If the pre-recognition processing unit 112 has not generated resize information during the processing of an input image prior to the current one, it may resize the image at a predetermined magnification. The resize information DB 122 may store a predetermined magnification as an initial value for the resize magnification.

[0036] The image recognition unit 113 performs recognition processing on the image to be recognized based on the image recognition model stored in the image recognition model DB 126 of the storage unit 120, and generates recognition result information. The recognition result information includes the size of the object to be recognized, the coordinates of the object to be recognized, the recognition determination probability, and identification ID information that identifies the object to be recognized included in the input image. The image recognition unit 113 corresponds to the recognition circuit. The training images stored in the image recognition model DB 126 correspond to multiple image data that have been previously trained using machine learning, and are, for example, information stored in the image recognition model DB 126 by the user.

[0037] Identification ID information is, for example, information used to identify whether the object being recognized is a person or a car. Furthermore, the identification ID information assigns a predetermined number to each object being recognized, such as "Person 1," "Person 2," ..., "Car 1," "Car 2," allowing for the identification of the recognized object.

[0038] Furthermore, the image recognition unit 113 stores the recognition result information, which is the result of the recognition process, in the recognition result information DB 124. Alternatively, the image recognition unit 113 may output the recognition result information, which is the result of the recognition process, to the outside via the output unit 133 of the input / output IF 130. Alternatively, the image recognition unit 113 may output the recognition result information, which is the result of the recognition process, to the target size update unit 115 instead of storing it in the recognition result information DB 124.

[0039] The resize information generation unit 114 generates resize information to be used for processing the input image at a later time, based on the recognition result information and the target size information previously stored in the storage unit 120. In other words, the resize information generation unit 114 generates resize information for a second time point, after the first time point, based on the recognition result information and the target size information previously stored in the storage unit 120. The resize information generation unit 114 also stores the generated resize information in the resize information DB 122.

[0040] Figure 5 shows an example of target size information applied to the image processing apparatus 100 according to this embodiment. In the table shown in Figure 5, the size rank is a ranking of the size of the recognition image. Also in the table shown in Figure 5, the minimum size and maximum size indicate the minimum and maximum sizes of the recognition image for each size rank. Also in the table shown in Figure 5, the target size indicates the size for which the recognition judgment probability is high for each size rank. Furthermore, in the table shown in Figure 5, the registered recognition judgment probability indicates the registered recognition judgment probability for the target size of each size rank. The target size information is stored in the target size information DB125. The target size information DB125 may store the minimum size, maximum size, and target size as area, or it may store the width and height separately. The target size information DB125 may have initial values ​​for each value included in the target size information stored in advance. The target size information DB125 may store the initial value of the registered recognition judgment probability as 0. The target size information DB125 may have only one size rank. The target size information DB125 may use the minimum size as the minimum size of the image to be recognized. The target size information DB125 may use the maximum size as the maximum size of the image to be recognized. The target size information DB125 may use the target size as the size to be recognized.

[0041] The resize information generation unit 114 calculates a resize multiplier for the resize information by dividing the target size included in the target size information by the size of the object to be recognized, if the object to be recognized exists in the identification ID information of the recognition result information. In other words, the resize information generation unit 114 calculates a resize multiplier for the resize information at the second time point by dividing the target size included in the target size information by the size of the object to be recognized. The resize information generation unit 114 refers to the size rank information that includes the size of the object to be recognized in the recognition result information in the target size information. The resize information generation unit 114 may also refer to the size rank information that is closest to the size of the object to be recognized in the recognition result information in the target size information. The determination of whether or not an object to be recognized exists is made by whether or not identification ID information exists in the recognition result information. That is, if there is no recognition ID information in the recognition result information, it is determined that there is no object to be recognized.

[0042] Furthermore, the resize information generation unit 114 calculates the resize region of the resize information based on the coordinates of the recognized object if a recognized object exists in the identification ID information of the recognition result information. Specifically, the resize information generation unit 114 sets a region in the input image after the current one in which a recognized object may exist, based on the size and position of the recognized object. For example, as in the example of input image (B) in Figure 3C, the resize information generation unit 114 calculates the center of the recognized object represented as a dashed rectangle and sets the range enlarged by a predetermined size from the center of the recognized object as the resize region. The resize information generation unit 114 may also use a rectangle with a fixed aspect ratio as the resize region. The resize information generation unit 114 may also calculate the width of the resize region by multiplying the width of the recognized object by a predetermined magnification factor.

[0043] The target size update unit 115 updates the target size information if the recognition determination probability of the recognition result information is higher than the registered recognition determination probability registered in the target size information. The target size update unit 115 refers to the size rank information that includes the size to be recognized in the recognition result information in the target size information. In other words, the target size update unit 115 updates the target size in the target size information to the size to be recognized in the recognition result information if the recognition determination probability of the recognition result information is higher than the registered recognition determination probability registered in the target size information. Also, the target size update unit 115 updates the registered recognition determination probability of the target size information to the recognition determination probability of the recognition result information if the recognition determination probability of the recognition result information is higher than the registered recognition determination probability registered in the target size information. The target size update unit 115 stores the updated target size information in the target size information DB 125.

[0044] Furthermore, the target size update unit 115 may generate a new size rank in the target size information corresponding to the size of the recognition target if there is no recognition target in the identification ID information of the recognition result information.

[0045] This allows the image processing device 100 to flexibly apply to images of various sizes, even those not included in the target size information, by registering a new size rank.

[0046] (Outline of the processing flow of the image processing device 100) Next, the processing (image processing method) related to the image processing device 100 will be explained based on the flowcharts in Figures 6 to 9. The series of operations of the image processing device 100 shown in the flowcharts in Figures 6 to 9 start when the image processing device 100 is powered on and end when image processing is completed. In addition, the flowcharts shown in Figures 6 to 9 also end when the power is turned off or when an interrupt occurs indicating the end of processing. Furthermore, in the following explanation of the flowcharts, the same content as described in the above explanation of the image processing device 100 will be omitted or simplified.

[0047] Figure 6 is a flowchart illustrating an example of the processing of the image processing apparatus 100 according to this embodiment. Figure 7 is a flowchart illustrating the pre-recognition processing, which is a subroutine process in the processing of the image processing apparatus 100 according to this embodiment. Figure 8 is a flowchart illustrating the resize information generation processing, which is a subroutine process in the processing of the image processing apparatus 100 according to this embodiment. Figure 9 is a flowchart illustrating the target size update processing, which is a subroutine process in the processing of the image processing apparatus 100 according to this embodiment. First, the flowchart shown in Figure 6 will be explained.

[0048] In step S601, the input image acquisition unit 111 acquires the input image captured by the camera. Specifically, the input image acquisition unit 111 acquires the image captured by the camera as an input image via the sensor 131 and ISP 132 of the input / output IF 130 and stores it in the input image information DB 121. After that, the process proceeds to step S602.

[0049] In step S602, if the pre-recognition processing unit 112 has resize information generated based on an input image prior to the current one, it resizes the input image to a recognition image based on the resize information previously stored in the storage unit 120. The pre-recognition processing unit 112 also stores the resized recognition image in the recognition image information DB 123. The specific processing in step S602 will be explained using the flowchart shown in Figure 7.

[0050] In step S701, the recognition preprocessing unit 112 performs image processing for recognition. Specifically, the recognition preprocessing unit 112 performs predetermined image processing on the input image to improve recognition performance. This image processing in the recognition preprocessing unit 112 may include, for example, a process to brighten dark areas overall. The image processing in the recognition preprocessing unit 112 may also include edge enhancement processing. The recognition preprocessing unit 112 stores the result of the predetermined image processing as a recognition image in the recognition image information DB 123. After that, the process proceeds to step S702.

[0051] In step S702, the pre-recognition processing unit 112 determines whether or not there is resize information from a resize information generation process performed on an input image prior to the current one. If the pre-recognition processing unit 112 determines that there is resize information (step S702: YES), the process proceeds to step S703. On the other hand, if the pre-recognition processing unit 112 determines that there is no resize information (step S702: NO), the subroutine processing ends and the process proceeds to step S603 in Figure 6.

[0052] In step S703, the pre-recognition processing unit 112 resizes the target area. Specifically, the pre-recognition processing unit 112 changes the size of the recognition target included in the recognition image according to the resize magnification included in the resize information. After that, the process proceeds to step S704.

[0053] In step S704, the recognition preprocessing unit 112 updates the recognition image by superimposing the region containing the resized recognition target onto the recognition image according to the resize region included in the resize information. Specifically, the recognition preprocessing unit 112 resizes the region containing the recognition target of the recognition image by the resize magnification based on the resize magnification and resize region, and superimposes the resized region containing the recognition target onto the recognition image. Furthermore, the recognition preprocessing unit 112 stores the resized recognition image in the recognition image information DB 123. After that, the subroutine processing ends and the process proceeds to step S603 in Figure 6.

[0054] In step S603, the image recognition unit 113 performs recognition processing on the image to be recognized based on the training images stored in the image recognition model DB 126 of the storage unit 120, and generates recognition result information. The recognition result information includes the size of the recognition target, the coordinates of the recognition target, the recognition determination probability, and identification ID information that identifies the recognition target included in the input image. The image recognition unit 113 stores the recognition result information, which is the result of the recognition processing, in the recognition result information DB 124. Alternatively, the image recognition unit 113 may output the recognition result information, which is the result of the recognition processing, to the outside via the output unit 133 of the input / output IF 130. After that, the process proceeds to step S604.

[0055] In step S604, the resize information generation unit 114 generates resize information to be used for processing the input image from the present onward, based on the recognition result information and the target size information previously stored in the storage unit 120. The resize information generation unit 114 also stores the generated resize information in the resize information DB 122. The specific processing in step S604 will be explained using the flowchart shown in Figure 8.

[0056] In step S801, the resize information generation unit 114 obtains recognition result information from the recognition result information DB 124. The process then proceeds to step S802.

[0057] In step S802, the resize information generation unit 114 determines whether or not a recognition target exists in the recognition result information's identification ID information. The resize information generation unit 114 determines whether or not a recognition target exists by checking whether or not identification ID information exists in the recognition result information. That is, if no recognition ID information exists in the recognition result information, the resize information generation unit 114 determines that there is no recognition target. In step S802, if the resize information generation unit 114 determines that a recognition target exists in the identification ID information (step S802: YES), the process proceeds to step S803. On the other hand, in step S802, if the resize information generation unit 114 determines that there is no recognition target in the identification ID information (step S802: NO), the subroutine processing ends and the process proceeds to step S605 in Figure 6.

[0058] In step S803, the resize information generation unit 114 calculates the target size of the input image. Specifically, the resize information generation unit 114 calculates the target size of the input image by working backward from the resize magnification of the resize information with respect to the recognition target size. In the target size information, the resize information generation unit 114 refers to the size rank information that includes the recognition target size of the recognition result information. The resize information generation unit 114 may also refer to the size rank information that is closest to the recognition target size of the recognition result information. For example, the resize information generation unit 114 calculates the target size of the input image by dividing the recognition target size by the resize magnification. Note that this calculation of the target size of the input image may be obtained, for example, from information stored in the input image information DB 121. After that, the process proceeds to step S804.

[0059] In step S804, the resize information generation unit 114 calculates the resize multiplier for the resize information by dividing the target size included in the target size information by the recognized target size. The resize information generation unit 114 also stores the calculated resize multiplier in the resize information DB 122. After that, the process proceeds to step S805.

[0060] In step S805, the resize information generation unit 114 calculates the resize region from the coordinates of the recognition target in the recognition result information. Specifically, the resize information generation unit 114 sets a region in subsequent input images where the recognition target may exist, based on the size and position of the recognition target. For example, as in the example of input image (B) in Figure 3C, the resize information generation unit 114 calculates the center of the recognition region represented as a dashed rectangle, and sets the resize region as the range enlarged by a predetermined size from the center of the recognition region. The resize information generation unit 114 may also use a rectangle with a fixed aspect ratio as the resize region. The resize information generation unit 114 may also calculate the width of the resize region by multiplying the width of the recognition region by a predetermined magnification factor. The resize information generation unit 114 also stores the calculated resize region in the resize information DB 122. After that, the subroutine processing ends, and the process proceeds to step S605 in Figure 6.

[0061] In step S605, the target size update unit 115 updates the registration recognition determination probability of the target size information. The specific processing in step S605 will be explained using the flowchart shown in Figure 9.

[0062] In step S901, the target size update unit 115 obtains recognition result information from the recognition result information DB 124. The process then proceeds to step S902.

[0063] In step S902, the target size update unit 115 determines whether or not a size rank corresponding to the recognition target size of the recognition result information exists. If the target size update unit 115 determines in step S902 that a size rank corresponding to the recognition target size of the recognition result information exists in the target size information (step S902: YES), the process proceeds to step S903. On the other hand, if the target size update unit 115 determines in step S902 that a size rank corresponding to the recognition target size of the recognition result information does not exist in the target size information (step S902: NO), the process proceeds to step S904.

[0064] In step S903, the target size update unit 115 determines whether the recognition probability of the recognition target is higher than the registered recognition probability stored in the target size information. If the target size update unit 115 determines in step S903 that the recognition probability of the recognition target is higher than the registered value in the target size information (step S903: YES), the process proceeds to step S904. On the other hand, if the target size update unit 115 determines in step S903 that the recognition probability of the recognition target is not higher than the registered value in the target size information (step S903: NO), the subroutine processing ends and the process proceeds to step S606 in Figure 6.

[0065] In step S904, the target size update unit 115 updates the registered data of the target size information. Specifically, if the recognition judgment probability of the recognition result information is higher than the registered recognition judgment probability registered in the target size information, the target size in the target size information is updated to the recognition target size in the recognition result information. Also, if the recognition judgment probability of the recognition result information is higher than the registered recognition judgment probability registered in the target size information, the target size update unit 115 updates the registered recognition judgment probability of the target size information to the recognition judgment probability of the recognition result information. The target size update unit 115 stores the updated target size information in the target size information DB 125. Furthermore, if there is no size rank corresponding to the recognition target size in the identification ID information of the recognition result information, the target size update unit 115 may generate a new size rank corresponding to the recognition target size in the target size information. After that, the subroutine processing ends and the process proceeds to step S606 in Figure 6.

[0066] In step S606, the control unit 110 determines whether or not image processing has been completed. In this embodiment, image processing is terminated when the input image acquisition unit 111 has finished acquiring the input image. Alternatively, the termination of image processing may be triggered, for example, by an external user inputting a termination process.

[0067] In step S606, if the control unit 110 determines that image processing is complete (step S606: YES), the process ends. On the other hand, if the control unit 110 determines in step S606 that image processing is not complete (step S606: NO), the process returns to step S601, and the process from step S601 is repeated.

[0068] Although the flowchart in Figure 6 shows an example that includes the target size update process, the configuration of the embodiment is not limited to this configuration. For example, the target size update process may not be included in the flowchart shown in Figure 6 and may be performed separately. For example, the target size update process may be a standalone configuration (flowchart) that inputs training image data and performs the target size update process.

[0069] As described above, the following effects and advantages can be obtained according to this embodiment.

[0070] The image processing device 100 includes an input image acquisition unit 111 that acquires an input image captured by a camera. The image processing device 100 also includes a storage unit 120 that stores resize information and target size information. The image processing device 100 also includes a recognition pre-processing unit 112 that resizes the input image into a recognition image based on the resize information at a first time point. The image processing device 100 also includes an image recognition unit 113 that recognizes the recognition image and generates recognition result information including the size of the image to be recognized and the recognition determination probability. The image processing device 100 also includes a resize information generation unit 114 that generates resize information for a second time point after the first time point based on the recognition result information and the target size information.

[0071] As a result, the image processing device 100 can appropriately improve the recognition rate according to the size of the recognition target by resizing the region containing the recognition target using the resize information. More specifically, the image processing device 100 according to this embodiment can maintain a high recognition determination probability in a series of images in which the position (coordinates) and / or size of the recognition target changes.

[0072] Furthermore, in this embodiment, the resize information generation unit 114 calculates the resize multiplier for the resize information at the second time point by dividing the target size included in the target size information by the size of the object to be recognized. This enables the image processing device 100 to perform appropriate resize processing on the object to be recognized in the input image, thereby improving the recognition determination probability.

[0073] Furthermore, in this embodiment, the target size update unit 115 updates the target size information when the recognition determination probability of the recognition result information is higher than the registered recognition determination probability registered in the target size information. As a result, the image processing device 100 can improve its recognition accuracy by updating the registered recognition determination probability of the target size information when the recognition determination probability is high in the image recognition process.

[0074] (Other embodiments) Although these embodiments have been described above, the embodiments are not limited to these, and various modifications are possible within the scope of the gist of the embodiments. Furthermore, it is possible to combine some or all of the various embodiments to create new embodiments.

[0075] The size rank of the target size information used in the image processing device 100 according to this embodiment may be aggregated according to the processing status. For example, the target size information DB 125 initially sets and stores the size rank in any number of stages as target size information. After the target size information is updated by executing image processing more times than a first predetermined value, those with a registration recognition judgment probability higher than a second predetermined value may be extracted. The extracted target size information will be in line with the characteristics of the recognition circuit (image recognition unit 113) of this image processing device 100. Since the recognition target size that results in a high recognition judgment probability varies depending on the recognition circuit, it is possible to set appropriate target size information according to the characteristics of the recognition circuit. The target size information DB 125 may also be set so that the minimum size and maximum size of each size rank are consecutive between adjacent size ranks.

[0076] Furthermore, a computer program (image processing program) that causes a computer to perform the image processing described above, and a computer-readable recording medium on which that program is recorded, are included within the scope of this embodiment. Here, the type of computer-readable recording medium is arbitrary. Also, the computer program described above is not limited to one recorded on the recording medium described above, but may be transmitted via telecommunication lines, wireless or wired communication lines, networks such as the Internet, etc. [Explanation of symbols]

[0077] 100 Image Processing Devices 110 Control Unit 111 Input Image Acquisition Unit 112 Pre-recognition processing unit 113 Image Recognition Unit 114 Resize Information Generation Unit 115 Target Size Update Section 120 Storage section 121 Input Image Information Database 122 Resize Information Database 123 Image information database for recognition 124 Recognition result information DB 125 Target Size Information DB 126 Image Recognition Model Database 130 Input / Output Interfaces 131 Sensors 132 ISP 133 Output section

Claims

1. An input image acquisition unit that acquires an input image captured by a camera, A memory unit that stores resize information and target size information, A recognition preprocessing unit that resizes the input image into a recognition image based on the resize information at the first point in time, An image recognition unit recognizes the aforementioned recognition image and generates recognition result information including the size of the object to be recognized and the recognition determination probability. A resize information generation unit generates the resize information for a second time point after the first time point, based on the recognition result information and the target size information. An image processing device equipped with the following features.

2. The image processing apparatus according to claim 1, wherein the resize information generation unit calculates a value obtained by dividing the target size included in the target size information by the recognized target size as the resize multiplier of the resize information at the second time point.

3. It also includes a target size update section, The image processing apparatus according to claim 1 or 2, wherein the target size update unit updates the target size information when the recognition determination probability of the recognition result information is higher than the registered recognition determination probability registered in the target size information.

4. A computer-based image processing method, The input image captured by the camera is acquired, Based on the resize information at a first time point stored in the memory unit, the input image is resized to a recognition image. The recognition image is recognized, and recognition result information including the size of the object to be recognized and the recognition determination probability is generated. An image processing method that generates resize information for a second time point after the first time point, based on the recognition result information and target size information previously stored in the storage unit.

5. The steps include acquiring an input image captured by a camera, and The steps include: resizing the input image to a recognition image based on the resize information at a first time point stored in the memory unit; The steps include: recognizing the aforementioned recognition image and generating recognition result information including the size of the object to be recognized and the recognition determination probability; An image processing program that causes a computer to perform the steps of generating resize information for a second time point after the first time point, based on the recognition result information and target size information stored in the storage unit in advance.

Images