Image recognition support device, method, and program

Abstract

To support improvement in recognition accuracy by adjusting an input image inputted to an image recognition device in consideration of a recognition result by the image recognition device.SOLUTION: An image recognition support device acquires a recognition result obtained by performing recognition processing of a plurality of objects by an image recognition device to an input image inputted from an imaging device including an image quality adjustment part, to acquire positional information on the input image of the plurality of objects on the basis of the recognition result, calculates a movement direction of each object from a transition of the positional information of each of the plurality of objects between successive image frames, decides a priority order of the plurality of objects on the basis of an approximation degree between a preset movement direction and the calculated movement direction of each object, and decides an image quality adjustment parameter for the input image to which the recognition processing is performed, so as to improve a recognition rate indicating possibility of the recognition result of the object having a high priority in the image recognition device, and sets the decided image quality adjustment parameter to the image quality adjustment part.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to an image recognition support device, method, and program.

Background Art

[0002] Patent Document 1 discloses a technique for detecting the moving speed of an entering object by image recognition and controlling the shutter speed of a surveillance television camera using the information. Further, in the technique of Patent Document 1, the brightness near the entering object detected by using image recognition is detected, and the aperture of the surveillance television camera is controlled using the information. When a plurality of detected objects exist, priorities are assigned to the plurality of detected objects according to the situation of the installation position of the television camera, and the shooting conditions of the camera are determined according to the object with the highest priority.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, for example, when the right side of an image captured by a surveillance television camera is the entrance of a store and the left side is the back of the store, the shooting conditions of the camera are determined according to the object in the area closer to the entrance. Thus, in Patent Document 1, when there are a plurality of objects with different moving speeds in the imaging range of the camera, priorities are assigned to the objects to be photographed according to the situation of the installation position of the camera.

[0005] However, in Patent Document 1, since an area with a higher priority is preset within the imaging range of the camera, the priority of objects outside the area cannot be increased. For example, at a river or a beach, it is required to recognize swimmers in a dangerous state included in the entire imaging range of the camera, and to preferentially detect such swimmers in a dangerous state with high recognition accuracy.

[0006] In view of the above problems, an object of the present disclosure is to provide an image recognition support device, method, and program that can assist in improving recognition accuracy by adjusting an input image input to an image recognition device in consideration of the recognition result by the image recognition device.

Means for Solving the Problems

[0007] An image recognition support device according to the present disclosure includes a recognition result acquisition unit that acquires a recognition result obtained by performing recognition processing of a plurality of objects on an input image input from an imaging device that adjusts image quality using image quality adjustment parameters, a position information acquisition unit that acquires position information of the plurality of objects on the input image based on the recognition result, a movement information calculation unit that calculates a movement direction of each object from transitions of the position information of each of the plurality of objects between consecutive image frames, a priority determination unit that determines the priority of the plurality of objects based on an approximation degree between a preset movement direction and the calculated movement direction of each object, and a image quality adjustment parameter setting unit that determines an image quality adjustment parameter for an input image on which recognition processing is performed so that a recognition rate indicating the certainty of the recognition result of an object with a high priority in the image recognition device is improved, and sets the determined image quality adjustment parameter in the image quality adjustment unit.

[0008] The image recognition support method according to the present disclosure includes a process of acquiring, by an image recognition device, a recognition result of performing recognition processing on a plurality of objects on an input image input from an imaging device including an image quality adjustment unit that adjusts the image quality using image quality adjustment parameters, a process of acquiring, based on the recognition result, position information of the plurality of objects on the input image, a process of calculating, from transitions of the position information of each of the plurality of objects between consecutive image frames, a moving direction of each object, a process of determining a priority order of the plurality of objects based on an approximation degree between a preset moving direction and the calculated moving direction of each object, and a process of determining image quality adjustment parameters for the input image on which the recognition processing is performed so that a recognition rate indicating the certainty of the recognition result of the object with a high priority order in the image recognition device is improved, and a process of setting the determined image quality adjustment parameters in the image quality adjustment unit.

[0009] The image recognition support program according to the present disclosure causes a computer to execute a process of acquiring, by an image recognition device, a recognition result of performing recognition processing on a plurality of objects on an input image input from an imaging device including an image quality adjustment unit that adjusts the image quality using image quality adjustment parameters, a process of acquiring, based on the recognition result, position information of the plurality of objects on the input image, a process of calculating, from transitions of the position information of each of the plurality of objects between consecutive image frames, a moving direction of each object, a process of determining a priority order of the plurality of objects based on an approximation degree between a preset moving direction and the calculated moving direction of each object, and a process of determining image quality adjustment parameters for the input image on which the recognition processing is performed so that a recognition rate indicating the certainty of the recognition result of the object with a high priority order in the image recognition device is improved, and a process of setting the determined image quality adjustment parameters in the image quality adjustment unit.

Advantages of the Invention

[0010] According to the present disclosure, by adjusting the input image input to the image recognition device in consideration of the recognition result by the image recognition device, it becomes possible to support an improvement in recognition accuracy.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0012] Hereinafter, specific embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same reference numerals are assigned to the same elements, and redundant descriptions will be omitted as necessary for clarity of explanation.

[0013] The embodiment relates to an image recognition support device that supports improvement in recognition accuracy in an image recognition system including an imaging device such as a surveillance camera and a recognition device that performs recognition processing of an object on an imaging image captured using the imaging device. Many general image recognition systems have a function of photographing an unspecified number of objects and recognizing a plurality of objects. When there are a plurality of objects in a captured image, there are often priorities to be detected depending on the state of the objects.

[0014] For example, in a surveillance camera system installed at a river or a bathing beach where there are multiple swimmers, it is required to prioritize the detection of swimmers in a dangerous state. In such a system for monitoring swimmers, the priority changes depending on the moving direction of the object. Specifically, in the case of a river, assume a swimmer being washed downstream from upstream, and in the case of a bathing beach, assume a swimmer being washed by an offshore current. It can be determined that a swimmer moving from the shore towards the sea is a swimmer with a high degree of danger. In the embodiment, for example, in a bathing beach, the task is to increase the priority of a swimmer being washed towards the sea to obtain a recognition result in line with the purpose of the image recognition system 100.

[0015] FIG. 1 is a block diagram showing the overall configuration of an image recognition system 100 including an image recognition support device 10 according to an embodiment. As shown in FIG. 1, the image recognition system 100 includes an image recognition support device 10, an imaging device 20, an image recognition device 30, a display device 40, and an input device 50. The image recognition system 100 has a function of displaying an image captured by the imaging device 20 for visual recognition by the user. Also, the provided functions of the image recognition system 100 include a function of recognizing an object included in the captured image and displaying it in a visually recognizable manner to the user.

[0016] Hereinafter, an example in which the image recognition system 100 constitutes a surveillance camera system installed at a river or a bathing beach where there are multiple swimmers will be described. In this example, multiple swimmers are objects to be recognized by the image recognition device 30. Note that in this example, the object is a person, but it is not limited thereto, and it may be an animal such as a dog or a cat, or a vehicle such as a car, a motorcycle, or a ship.

[0017] <Imaging device 20> The imaging device 20 captures a scene including a plurality of objects, and generates display image data and recognition image data from the captured data. FIG. 2 is a block diagram showing a configuration example of the imaging device 20 in FIG. 1. As shown in FIG. 2, the imaging device 20 includes a camera unit 21, a signal processing unit 22, an image output unit 23, and a recognition signal processing unit 24.

[0018] The camera unit 21 includes, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor as an imaging device. Note that the camera unit 21 may be provided with a lens group (imaging optical system) including a zoom lens and a focus lens, an iris diaphragm, a mechanical shutter, and the like. The camera unit 21 level-adjusts and A / D-converts an image signal based on an image formed by the imaging optical system according to a predetermined amplification gain, and sequentially outputs it as shooting data to the signal processing unit 22.

[0019] The signal processing unit 22 acquires the shooting data continuously shot by the camera unit 21, and performs various image processes on the image data in units of frames (hereinafter referred to as image frames). The signal processing unit 22 can generate an image frame conforming to a predetermined display signal format. For example, when generating image data in the full high-definition format, the signal processing unit 22 performs white balance processing, gamma processing, etc. on each image frame, and then executes enlargement / reduction processing to an image size of 1920 x 1080 pixels. The signal processing unit 22 delivers the image data subjected to each process to the image output unit 23.

[0020] The image output unit 23 can acquire the recognition result by the image recognition device 30 described later via the image recognition support device 10. The image output unit 23 can perform a processing process on the image data using the recognition result and generate display image data. The image output unit 23 outputs the generated display image data to the display device 40. The display device 40 is a liquid crystal display device or the like having an image display function. The display device 40 can display a shooting image with high visibility and a display image including the recognition result based on the display image data.

[0021] Note that the image output unit 23 may generate information to be displayed superimposed on the captured image using the recognition result. The display device 40 can, for example, superimpose and display a frame surrounding the recognized object area on the captured image, character information corresponding to the type of recognition target, recognition determination information such as the recognition rate, etc.

[0022] In addition, the image output unit 23 may create auxiliary display information such as menu setting displays used when the user inputs setting information using an input device 50 described later. The image output unit 23 can superimpose a menu setting display on the captured image.

[0023] The image data processed by the signal processing unit 22 is supplied to the recognition signal processing unit 24 in addition to being used for display on the display device 40. The recognition signal processing unit 24 converts the image data processed by the signal processing unit 22 into recognition image data suitable for object recognition processing. The recognition image data is supplied to the image recognition device 30 and used for object recognition. Note that the shooting data generated by the camera unit 21 may be directly supplied to the recognition signal processing unit 24 without passing through the signal processing unit 22. Also, a signal being processed by the signal processing unit 22 may be supplied to the recognition signal processing unit 24.

[0024] The recognition signal processing unit 24 converts the image data into recognition image data by performing image quality adjustment processing based on image quality adjustment parameters. That is, the recognition signal processing unit 24 can also be said to be an image quality adjustment unit that adjusts the image quality using the image quality adjustment parameters. The image recognition support device 10 described later generates image quality adjustment parameters that affect the recognition rate of the object and are used when the recognition signal processing unit 24 performs image quality adjustment processing. Such image quality adjustment parameters include a luminance gain for controlling the luminance (brightness) of the captured image, tone mapping characteristics, an aperture gain for edge enhancement, etc.

[0025] <Image Recognition Device 30> The image recognition device 30 performs image recognition processing on the recognition image supplied from the recognition signal processing unit 24 and outputs the recognition result to the image recognition support device 10. The recognition images are continuously input to the image recognition device 30, and the recognition processing is continuously executed as needed. The recognition result includes the presence or absence of the object, the type of the object, the area or position of the object, and the recognition rate. The presence or absence of the object is information indicating whether the object is recognized, that is, identified, by the image recognition processing on the recognition image. The type of the object is information indicating the type of the recognition object. The area of the object is a set of coordinates defining the range of the area including the recognized recognition object within the recognition image. The area of the object is, for example, a range specified by pixel values in the XY coordinate system. Note that the position of the object is, for example, a representative point such as the center coordinates of the object recognized within the recognition image.

[0026] The recognition rate is an example of the degree of certainty of recognition by image recognition. That is, the recognition rate is numerical information indicating the recognition accuracy of the presence or absence, type, and area of the object recognized by the image recognition processing on the recognition image. The recognition rate may be indicated, for example, from 0 to 100%. Also, for calculating the recognition rate, for example, a threshold value indicating the degree of similarity to the object, the number of passing stages of the discriminator, etc. may be used. When a plurality of recognition objects are recognized, the recognition result may generate a set of the type, area, and recognition rate for each object.

[0027] The image recognition device 30 is hardware or software capable of executing known image recognition processing, or a combination thereof. For example, the image recognition device 30 is realized by executing a known image recognition processing program on a computer. Note that the image recognition device 30 may be redundant in a plurality of computers, and each functional block may be realized by a plurality of computers. Also, the image recognition device 30 may be realized in a form in which each of a client-server system, a cloud computing system, etc. is connected via a communication network. Also, the function of the image recognition device 30 may be provided in the form of SaaS (Software as a Service). Alternatively, the image recognition device 30 may be realized by the same computer as the image recognition support device 10.

[0028] As the image recognition process by the image recognition device 30, a method of storing a plurality of images for each object and recognizing the object using pattern matching can be used. At this time, deep learning (deep neural network learning) may be performed using a plurality of images captured from various angles to create a model for recognizing the object. It is generally known that such an image recognition process has a recognition rate that varies depending on the characteristics of the image, such as the ease of distinguishing the object to be recognized from the background in the recognition image.

[0029] In addition, the image recognition device 30 has a function of tracking each recognized object by a known technique such as motion compensation between image frames. The image recognition device 30 can associate identification information such as an identification number for identifying the same object with the position information and provide it to the image recognition support device 10. The above-described recognition result may also include the identification information.

[0030] <Image recognition support device 10> The image recognition support device 10 determines and sets the image quality adjustment parameters according to the image recognition result for the recognition image output from the image recognition device 30. The recognition signal processing unit 24 further performs image quality adjustment of the recognition image using the set image quality adjustment parameters and supplies the adjusted recognition image to the image recognition device 30. When the image recognition support device 10 obtains the recognition result for the adjusted recognition image, it determines and sets the image quality adjustment parameters again.

[0031] In this way, the image recognition support device 10 performs feedback control of the image quality adjustment parameters according to the recognition result for the recognition image. The image recognition support device 10 repeats the feedback control until the recognition rate of the object in the recognition image by the image recognition support device 10 stabilizes in a state where it is high. This makes it possible to increase the recognition rate of the object in the recognition image.

[0032] Here, the plurality of objects included in the captured image have different shooting conditions such as brightness and background. Therefore, the image quality adjustment parameters for optimal recognition rate often vary for each object. Thus, it is necessary to prioritize the plurality of objects and perform feedback control of the image quality adjustment parameters so that the recognition rate of the object with the highest priority is increased. In the embodiment, the priority is determined by focusing on the moving direction of each object, and image quality adjustment is performed to be advantageous for the recognition process of the object with the highest priority.

[0033] FIG. 3 is a block diagram showing a configuration example of the image recognition support device 10 of FIG. 1. Here, an example is shown in which the image recognition support device 10 is realized by one computer, but it is not limited thereto. The image recognition support device 10 may be redundant in a plurality of computers, and each functional block may be realized by a plurality of computers. Alternatively, all or part of the functions of the image recognition support device 10 may be realized by a general-purpose or dedicated circuit such as a semiconductor device. In these cases, the image recognition support device 10 may be communicably connected to the imaging device 20 and the image recognition device 30 via a communication network.

[0034] The image recognition support device 10 includes a processing unit 1, a storage unit 2, and an IF (InterFace) unit 3. The storage unit 2 includes a non-volatile storage device such as a hard disk and a flash memory, and a memory such as a RAM (Random Access Memory), that is, a volatile storage device. The storage unit 2 stores an image recognition support program, recognition results, and an image quality adjustment history. The image recognition support program is a computer program in which the processing of the image recognition support method according to the embodiment is implemented. Further, the recognition results include the recognition rate, position information, and identification information for identifying the same object for each object. The image quality adjustment history is a history of the image quality adjustment parameters determined and set by the image recognition support device 10. The IF unit 3 is an interface circuit responsible for transmitting and receiving data between the image recognition support device 10 and the outside.

[0035] The processing unit 1 is a processing device that executes each process of the image recognition support device 10. The processing unit 1 is, for example, a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), or a quantum processor (quantum computer control chip). The processing unit 1 causes the image recognition support program to be read from the storage unit 2 into the memory and executes it. Thereby, the processing unit 1 realizes the functions of the recognition result acquisition unit 11, the position information acquisition unit 12, the movement information calculation unit 13, the priority determination unit 14, and the parameter setting unit 15, and generates the above-described image quality adjustment parameters. A part or all of each configuration of the processing unit 1 may be realized by a general-purpose or dedicated circuit realized by, for example, a semiconductor device.

[0036] The recognition result acquisition unit 11 acquires the recognition results of the image recognition processing of a plurality of objects performed by the image recognition device 30 on the recognition image. The position information acquisition unit 12 acquires the position information on the recognition images of the plurality of objects based on the acquired recognition results. As described above, as the position information of the object, for example, the center coordinates of the object recognized in the recognition image can be used.

[0037] The movement information calculation unit 13 calculates the movement direction of each object from the transition of the positions (coordinates) of each of the plurality of objects between consecutive image frames. The movement information calculation unit 13 can calculate the movement direction of each object, for example, using the identification information of each object input from the image recognition device 30 and the position information associated therewith.

[0038] Further, the movement information calculation unit 13 can calculate the speed of movement of each object from the transition of the positions (coordinates) of each of the plurality of objects between consecutive image frames. That is, the movement information calculation unit 13 can obtain the movement vector of each object. The movement vector is a vector that represents the direction and magnitude of the change in the position of the object in the recognition image over time.

[0039] Fig. 4 shows an example of the movement status of the object in the recognition image. In the example shown in Fig. 4, the recognition image P0 shows a bathing beach, assuming that the upper side is the offing and the lower side is the land. When increasing the priority of the swimmers washed away to the offing in the bathing beach, the direction from the land to the offing becomes the preset movement direction (hereinafter referred to as the attention direction).

[0040] The attention direction can be set, for example, based on the setting information input from the input device 50. The input device 50 includes a keyboard, a mouse, a touch panel, etc., and the user performs an input operation of the setting information. The image recognition support device 10 can receive the setting information from the input device 50 via the IF unit 3. In addition, if the installation location of the image recognition system 100 is determined and there is no need to change the attention direction, the attention direction may be preset.

[0041] In Fig. 4, it is assumed that the recognition image P0 contains three persons P1, P2, and P3. The positions of the respective persons P1, P2, and P3 in the first image frame are indicated by broken-line circles. And in the second image frame after a predetermined time interval has elapsed from the first image frame, it is assumed that the respective persons P1, P2, and P3 have moved to the positions indicated by solid-line circles. The movement directions of the respective persons P1 to P3 are indicated by the directions of the black arrows in the recognition image P0. The size of the black arrow represents the speed of movement of the respective persons P1 to P3.

[0042] Returning to Fig. 3, the priority determination unit 14 determines the priorities of a plurality of objects based on the approximation degree between the preset movement direction (attention direction) and the calculated movement directions of the respective objects. Here, the attention direction is set as the Y direction, and the direction orthogonal to the Y direction is set as the X direction. The priority determination unit 14 can obtain the approximation degree of the movement direction of each object with the attention direction, for example, by calculating the ratio of the Y-direction component and the X-direction component of the movement vector of each object. The priority determination unit 14 can, for example, increase the priority of the object with a higher approximation degree.

[0043] In addition, the priority determination unit 14 can calculate the speed of movement of the object using the motion vector. For example, the priority determination unit 14 can calculate the speed of movement of the object using the magnitude of the Y-direction component of the motion vector and a predetermined time interval (number of frames) between the first image frame and the second image frame. The priority determination unit 14 may comprehensively determine the priority according to the detection purpose using the degree of approximation of the moving direction of the object with respect to the Y direction and the speed of movement of the object.

[0044] In the example of FIG. 4, since the order of the degrees of approximation of each of the persons P1 to P3 with respect to the Y direction and the order of the speeds of the Y-direction components are both P2 → P3 → P1, the priority of the object is also P2 → P3 → P1. Note that since the Y-direction component of the motion vector of the person P1 is negative (minus), the person P1 may be excluded from the objects for priority determination.

[0045] In addition, the priority determination unit 14 can also obtain the priority of each subject considering the ratio of the involvement of the degree of approximation and the speed, and determine the priority according to the priority. The involvement ratio can be selected, for example, in consideration of the detection purpose and the environment. An example of the calculation formula of the priority that is the basis of the priority determination when the involvement ratio coefficients of the degree of approximation and the speed are K1 and K2, respectively, is shown below. Priority = K1 × Degree of approximation + K2 × Relative ratio of speed

[0046] Note that the degree of approximation can take a value from 0 to 1. The degree of approximation 1 indicates 100% of the Y component. Also, the relative ratio of speed indicates the ratio with respect to the set upper limit value of the calculated speed of the object. The relative value of the speed can take a value from 0 to 1, and values equal to or higher than a predetermined set upper limit value can be regarded as 1.

[0047] When the range of the priority is from 0 to 1, K1 and K2 are determined so that K1 + K2 = 1. For example, if the ratio of the involvement of the degree of approximation and the speed is 1:1, then K1 = 0.5 and K2 = 0.5. K1 and K2 can be adjusted so that the recognition accuracy of the target object is high.

[0048] For example, for the purpose of monitoring a swimmer being carried away by an offshore current at a beach, the image recognition system 100 needs to determine that it is abnormal for a swimmer who can normally move freely to be carried straight out to sea. In this case, it is conceivable to focus on the degree of approximation of the moving direction rather than the speed of the swimmer's movement. The participation ratio coefficients can be, for example, K1 = 0.7 and K2 = 0.3.

[0049] Also, in a river, there are usually many swimmers moving along the flow of the river. Therefore, for the purpose of monitoring a swimmer being carried away by the river, the image recognition support device 10 needs to extract a dangerous swimmer who is moving fast from among the swimmers moving in the direction of the river flow. In this case, it is conceivable to focus on the speed rather than the moving direction. The participation ratio coefficients can be, for example, K1 = 0.3 and K2 = 0.7.

[0050] The upper limit value of the speed setting can also be set in consideration of the detection purpose and the environment. For example, the speed of an offshore current is said to be at most 2 m / s. Therefore, by setting 2 m / s as the upper limit value of the speed and obtaining the ratio of the speed of the object to the upper limit value, it is effective for discriminating an object being carried away by the offshore current. Also, if the speed of the object is 2 m / s or more, the relative ratio of the speed may be regarded as 1. Also, in the case of detecting a swimmer being carried away by a river, the speed of the river flow at that time can be set as the upper limit value. This is advantageous for improving the detection accuracy of an object being carried away by the river. The above-mentioned upper limit value can be input by the user via the input device 50, for example.

[0051] When a plurality of objects are detected, the speed of the fastest detected object among the speeds of these objects may be set as the upper limit value. This can omit the input of the upper limit value by the user.

[0052] Note that the priority may be determined based on either the approximation or the speed. However, depending on the detection purpose and conditions, there are cases where prioritization based on speed is not effective. For example, in the case of detecting swimmers, if the speeds of all of the plurality of objects are within a safe speed range, prioritization based on speed often has no meaning.

[0053] The parameter setting unit 15 determines image quality adjustment parameters for the input image for which recognition processing is to be performed so as to improve the recognition rate indicating the certainty of recognition of the object with the highest priority in the image recognition device 30.

[0054] Specifically, the parameter setting unit 15 first calculates a control recognition rate that is the basis for feedback control of the image quality adjustment parameters in consideration of the priority. That is, the image quality adjustment parameters function as a control recognition rate calculation unit that calculates the control recognition rate in consideration of the priority using the recognition rate of each object. As the method for calculating the control recognition rate, an optimal method can be used depending on the system, purpose of use, detection environment, etc. For example, as the method for calculating the control recognition rate, the following calculation methods (1) to (3) can be considered. (1) Use the recognition rate of the object with the highest priority. (2) Among the plurality of detected objects, use the average value of the recognition rates of the objects whose priority is, for example, in the top 20%. (3) Calculate the recognition rate in consideration of the weighting coefficient for each priority.

[0055] In the method of (3), for example, the weighting coefficients whose sum is 1 are assigned so that the coefficient with a larger value is for the object with a higher priority. The control recognition rate can be the sum of the products of the recognition rate of each object and the assigned weighting coefficient. For example, in the example of FIG. 4, assuming that the recognition rate of the person P2 with the first priority is 0.8, the recognition rate of the person P3 with the second priority is 0.6, and the weighting coefficients are 0.7 and 0.3, respectively. Here, it is assumed that the person P1 is excluded from the object of priority determination. The control recognition rate is 0.8 × 0.7 + 0.6 × 0.3 = 0.74.

[0056] The parameter setting unit 15 determines the image quality adjustment parameters of the recognition signal processing unit 24 so that the recognition rate for control becomes a high value. FIG. 5 shows an example of the change in the recognition rate accompanying the change in the image quality adjustment parameters. In the example shown in FIG. 5, it is assumed that the image quality adjustment parameter is the luminance gain (digital gain) in the recognition signal processing unit 24. As shown in FIG. 5, the recognition rate increases until the luminance gain is small and the image data of the recognition target becomes a dark value Y1 for the recognition process, and then becomes almost constant between the value Y1, which does not significantly affect the recognition process, and the value Y2, and decreases when the image data is too bright for the recognition process compared to the value Y2. That is, when the luminance gain is changed, the graph of the recognition rate has a characteristic of a mountain shape with a flat top.

[0057] As described above, the image recognition support device 10 stores and manages the recognition rate for each object and the image quality adjustment history in the storage unit 2. That is, it can be said that the storage unit 2 is also a management unit that manages the change in the recognition rate of the object with the highest priority when the image quality adjustment parameters of the recognition signal processing unit 24 are changed. The parameter setting unit 15 obtains the change in the recognition rate for control accompanying the change in the previous image quality adjustment parameters, and can determine new image quality adjustment parameters so that the recognition rate for control is improved with reference to the change in the recognition rate for control.

[0058] Specifically, when the recognition rate for control is improving, the parameter setting unit 15 changes the new image quality adjustment parameter in the increasing direction when the image quality adjustment parameter was increased during the previous adjustment of the image quality adjustment parameter, and in the decreasing direction when it was decreased. That is, when the recognition rate for control is improving, the parameter setting unit 15 sets the direction of change of the image quality adjustment parameter to the same direction as that during the previous adjustment.

[0059] Also, when the recognition rate for control is decreasing, if the image quality adjustment parameter was increased during the previous adjustment of the image quality adjustment parameter, the parameter setting unit 15 changes the new image quality adjustment parameter in the decreasing direction, and if it was decreased, it changes it in the increasing direction. That is, when the recognition rate for control is decreasing, the parameter setting unit 15 sets the direction of change of the image quality adjustment parameter to be opposite to that during the previous adjustment.

[0060] For example, in the region below the value Y1 shown in FIG. 5, when the luminance gain is increased, the recognition rate increases. In this case, by further increasing the luminance gain, it becomes possible to further improve the recognition rate. Also, in the region above the value Y2 shown in FIG. 5, when the luminance gain is decreased, the recognition rate increases. In this case, by further decreasing the luminance gain, the recognition rate can be further improved.

[0061] In this way, the parameter setting unit 15 can determine the image quality adjustment parameter so that the recognition rate becomes near the flat top shown in FIG. 5 by repeating the readjustment of the image quality adjustment parameter. The parameter setting unit 15 can set the determined image quality adjustment parameter to the recognition signal processing unit 24 via the IF unit 3. As a result, the recognition signal processing unit 24 can generate recognition image data in a state where the recognition rate of the recognition process by the image recognition device 30 is high.

[0062] Note that since the image recognition support device 10 performs feedback control of the image quality adjustment parameter based on the recognition rate for control, the image quality adjustment parameter always fluctuates near the top of the recognition rate. To avoid such fluctuations in the image quality adjustment parameter, the parameter setting unit 15 may perform control such as not changing the image quality adjustment parameter once the recognition rate for control has reached a value near the top of FIG. 5, unless the recognition rate for control falls below a predetermined threshold (hereinafter referred to as the readjustment threshold).

[0063] Here, with reference to FIG. 6, an example of the image recognition support method according to the embodiment will be described. FIG. 6 is a flowchart showing the flow of the image recognition support process according to the embodiment. First, the image recognition support device 10 initializes the image adjustment parameters in the recognition signal processing unit 24 (S10). Note that the image adjustment parameters for the initial setting may be set in advance in the recognition signal processing unit 24. In this case, the step of S10 can be omitted.

[0064] Although not shown in FIG. 6, the recognition signal processing unit 24 generates recognition image data using the initially set image quality adjustment parameters and outputs it to the image recognition device 30. The image recognition device 30 performs recognition processing on the input recognition image data and generates a recognition result.

[0065] Also, the image recognition support device 10 acquires setting information regarding the attention direction via the input device 50 (S11). Note that if the installation location of the image recognition system 100 is determined and there is no need to change the attention direction, the attention direction may be set in advance. In this case, the step of S11 can be omitted.

[0066] The image recognition support device 10 acquires the recognition result from the image recognition device 30. Specifically, the image recognition support device 10 acquires at least the recognition rate of each of the plurality of objects recognized in the recognition image and the position information of each object (S12). Next, the image recognition support device 10 calculates the moving direction from the change in the position of each object (S13).

[0067] Then, the image recognition support device 10 determines the priority based on the attention direction and the moving direction of each object (S14). The image recognition support device 10 can determine the priority of the plurality of objects, for example, based on the degree of approximation between the attention direction and the calculated moving direction of each object. Note that the image recognition support device 10 may calculate the speed of movement of each object and use the information on the moving direction and speed of movement of each object to determine the priority.

[0068] Thereafter, the image recognition support device 10 calculates a control recognition rate considering the priority (S15). The control recognition rate can be calculated, for example, by considering a weight coefficient for each priority. Then, the image recognition support device 10 compares the current control recognition rate with the previous control recognition rate (S16).

[0069] If the current control recognition rate has decreased compared to the previous control recognition rate, the process proceeds to step S17. In step S17, the image recognition support device 10 determines whether the current control recognition rate is a value near the peak. If the control recognition rate is near the peak (S17, YES), it is determined whether the control recognition rate is less than or equal to the readjustment threshold (S18). If the control recognition rate is not near the peak (S17, NO), or if the control recognition rate is near the peak but the control recognition rate is less than or equal to the readjustment threshold (S18, YES), the image quality adjustment parameter is changed in the direction opposite to the direction of change of the previous image quality adjustment parameter (S19).

[0070] Then, the image recognition support device 10 saves the current position of the object, the recognition rate, and the changed image quality adjustment parameter (S23). On the other hand, if the control recognition rate is greater than the readjustment threshold (S18, NO), the image recognition support device 10 saves the current position of the object, the recognition rate, and the previous image quality adjustment parameter without changing the image quality adjustment parameter (S23).

[0071] If the current control recognition rate has increased compared to the previous control recognition rate, the process proceeds to step S20. In step S20, the image recognition support device 10 determines whether the current control recognition rate is a value near the peak. If the control recognition rate is near the peak (S20, YES), it is determined whether the control recognition rate is less than or equal to the readjustment threshold (S21). If the control recognition rate is not near the peak (S20, NO), or if the control recognition rate is near the peak but the control recognition rate is less than or equal to the readjustment threshold (S21, YES), the image quality adjustment parameter is changed in the same direction as the direction of change of the previous image quality adjustment parameter (S22).

[0072] Then, the image recognition support device 10 saves the position of the current object, the recognition rate, and the image quality adjustment parameters after the change (S23). On the other hand, when the control recognition rate is greater than the readjustment threshold (S21, NO), the image recognition support device 10 saves the position of the current object, the recognition rate, and the image quality adjustment parameters before the change without changing the image quality adjustment parameters (S23).

[0073] Note that in S16, when the current control recognition rate has not changed from the previous control recognition rate, the process proceeds to step S23, and the position of the current object, the recognition rate, and the image quality adjustment parameters before the change are saved.

[0074] As described above, according to the embodiment, using the recognition result by the image recognition device 30, the moving directions of a plurality of objects in the recognition image are obtained, and the priority is determined based on the moving directions. Then, by adjusting the image quality of the recognition image input to the image recognition device 30 so that the recognition rate of the object with the higher priority becomes higher, it is possible to support the improvement of the recognition accuracy along the detection purpose. For example, when a plurality of objects are included in the recognition image, it is possible to improve the recognition accuracy of the object with the higher priority, which is in a dangerous state, among these objects.

[0075] Each functional block that performs various processes described in the drawings can be configured by a processor, a memory, and other circuits in terms of hardware. Also, it is possible to realize the above-described processes by causing the processor to execute a program. Therefore, these functional blocks can be realized in various forms by only hardware, only software, or a combination thereof, and are not limited to any one of them.

[0076] The above-described program can be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include semiconductor memories (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM). Also, the program may be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable media can supply the program to the computer via wired communication paths such as electric wires and optical fibers, or wireless communication paths.

[0077] In the above embodiment, for the captured data after being captured by the camera unit 21, a recognition image with a high recognition rate is generated by performing image quality adjustment through software operation in the recognition signal processing unit 24, but it is not limited thereto. For example, it is also possible to adjust the brightness of the recognition image by adjusting the amount of light incident on the image sensor using the mechanical shutter or iris diaphragm of the imaging device 20. Specifically, the imaging device 20 may control the mechanical shutter according to the exposure time per frame specified by the image quality adjustment parameter input from the image recognition support device 10 to adjust the charge accumulation time of the image sensor.

[0078] The content of the present disclosure can be used in various fields that utilize image recognition. Some or all of the above embodiments may be described as follows in the appended claims, but are not limited thereto.

[0079] (Appended Claim A1) A recognition result acquisition unit that acquires a recognition result obtained by performing recognition processing of a plurality of objects on an input image input from an imaging device including an image quality adjustment unit that adjusts the image quality using image quality adjustment parameters; A position information acquisition unit that acquires position information of the plurality of objects on the input image based on the recognition result; A movement information calculation unit that calculates the movement direction of each object from the transition of the position information of each of the plurality of objects between consecutive image frames; A priority determination unit that determines the priority of the plurality of objects based on the degree of approximation between a preset movement direction and the calculated movement direction of each object; An image quality adjustment parameter setting unit that determines image quality adjustment parameters for an input image on which recognition processing is performed so that the recognition rate indicating the certainty of the recognition result of an object with a high priority in the image recognition device is improved, and sets the determined image quality adjustment parameters in the image quality adjustment unit; Including An image recognition support device. (Appendix A2) The movement information calculation unit further calculates the speed at which each object moves from the transition of the position information of each of the plurality of objects between consecutive image frames; The priority determination unit determines the priority of the plurality of objects based on the degree of approximation and the relative ratio between a preset moving speed and the calculated moving speed of each object; The image recognition support device according to Appendix A1. (Appendix A3) The apparatus further includes a management unit that manages the change in the recognition rate of the object with the highest priority when the image quality adjustment parameter of the image quality adjustment unit is changed; The image quality parameter setting unit determines the image quality adjustment parameters so that the recognition rate is improved with reference to the change in the recognition rate accompanying the change in the image quality adjustment parameters; The image recognition support device according to Appendix A1 or A2. (Appendix A4) Acquire the recognition rate of each of the plurality of objects in the image recognition device; The control recognition rate calculation unit further calculates a control recognition rate by using the recognition rates of the respective objects and considering the priority order. The image quality adjustment parameter setting unit determines the image quality adjustment parameters so as to improve the control recognition rate. The image recognition support device according to any one of claims A1 to A3. (Appendix A5) The control recognition rate calculation unit calculates the control recognition rate by summing the products of the recognition rates of the respective objects and weight coefficients corresponding to the priority order. The image recognition support device according to claim A4. (Appendix B1) A computer A process of obtaining a recognition result obtained by performing recognition processing of a plurality of objects on an input image input from an imaging device including an image quality adjustment unit that adjusts image quality using image quality adjustment parameters, A process of obtaining position information of the plurality of objects on the input image based on the recognition result, A process of calculating the moving direction of each object from the transition of the position information of each of the plurality of objects between consecutive image frames, A process of determining the priority order of the plurality of objects based on the degree of approximation between a preset moving direction and the calculated moving direction of each object, Determining image quality adjustment parameters for an input image on which recognition processing is performed so that the reliability of the recognition result of an object with a high priority order in the image recognition device is improved, and setting the determined image quality adjustment parameters in the image quality adjustment unit, An image recognition support method for executing. (Appendix C1) A process of obtaining a recognition result obtained by performing recognition processing of a plurality of objects on an input image input from an imaging device including an image quality adjustment unit that adjusts image quality using image quality adjustment parameters, A process of obtaining position information of the plurality of objects on the input image based on the recognition result, A process of calculating the movement direction of each object from the transition of the position information of each of the plurality of objects between consecutive image frames, A process of determining the priority order of the plurality of objects based on the degree of approximation between the preset movement direction and the calculated movement direction of each object, A process of determining the image quality adjustment parameters for the input image for performing the recognition process so that the recognition rate indicating the certainty of the recognition result of the object with the highest priority in the image recognition apparatus is improved, and setting the determined image quality adjustment parameters in the image quality adjustment unit, An image recognition support program for causing a computer to execute.

[0080] Some or all of the elements described in Appendices A2 to A5 subordinate to Appendix A1 (image recognition support apparatus) may also be subordinate to Appendix B1 (image recognition support method) and Appendix C1 (image recognition support program) by the same subordinate relationship.

Explanation of Signs

[0081] 100 Image recognition system 1 Processing unit 2 Storage unit 3 IF unit 10 Image recognition support apparatus 11 Recognition result acquisition unit 12 Position information acquisition unit 13 Movement information calculation unit 14 Priority determination unit 15 Parameter setting unit 20 Imaging device 21 Camera unit 22 Signal processing unit 23 Image output unit 24 Recognition signal processing unit 30 Image recognition apparatus 40 Display device 50 Input device P0 Recognition image P1~P3 Persons

Claims

1. A recognition result acquisition unit that acquires a recognition result obtained by performing recognition processing of a plurality of objects by an image recognition device on an input image input from an imaging device including an image quality adjustment unit that adjusts the image quality using image quality adjustment parameters; A position information acquisition unit that acquires position information of the plurality of objects on the input image based on the recognition result; A movement information calculation unit that calculates the movement direction of each object from the transition of the position information of each of the plurality of objects between consecutive image frames; A priority determination unit that determines the priority of the plurality of objects based on the degree of approximation between a preset movement direction and the calculated movement direction of each object; An image quality adjustment parameter setting unit that determines an image quality adjustment parameter for an input image on which recognition processing is performed so that the recognition rate indicating the certainty of the recognition result of an object with a high priority in the image recognition device is improved, and sets the determined image quality adjustment parameter in the image quality adjustment unit; including an image recognition support device.

2. The movement information calculation unit further calculates the speed at which each object moves from the transition of the position information of each of the plurality of objects between consecutive image frames; The priority determination unit determines the priority of the plurality of objects based on the degree of approximation and the relative ratio between a preset moving speed and the calculated moving speed of each object; The image recognition support device according to claim 1.

3. The apparatus further includes a management unit that manages a change in the recognition rate of the object with a high priority when the image quality adjustment parameter of the image quality adjustment unit is changed; The image quality adjustment parameter setting unit determines the image quality adjustment parameter so that the recognition rate is improved with reference to the change in the recognition rate accompanying the change in the image quality adjustment parameter; The image recognition support device according to claim 1.

4. A computer performs a process of acquiring a recognition result obtained by performing recognition processing of a plurality of objects by an image recognition device on an input image input from an imaging device including an image quality adjustment unit that adjusts the image quality using image quality adjustment parameters; a process of acquiring position information of the plurality of objects on the input image based on the recognition result; a process of calculating the movement direction of each object from the transition of the position information of each of the plurality of objects between consecutive image frames; a process of determining the priority of the plurality of objects based on the degree of approximation between a preset movement direction and the calculated movement direction of each object; A process of determining an image quality adjustment parameter for an input image on which recognition processing is performed so that the recognition rate indicating the certainty of the recognition result of a target object with a high priority in the image recognition apparatus is improved, and setting the determined image quality adjustment parameter in the image quality adjustment unit; An image recognition support method for executing the above. **Claim 5** A process of obtaining a recognition result obtained by performing recognition processing of a plurality of target objects by an image recognition apparatus on an input image input from an imaging apparatus including an image quality adjustment unit that adjusts image quality using an image quality adjustment parameter; A process of obtaining position information of the plurality of target objects on the input image based on the recognition result; A process of calculating a moving direction of each target object from transitions of the position information of each of the plurality of target objects between consecutive image frames; A process of determining the priority order of the plurality of target objects based on the degree of approximation between a preset moving direction and the calculated moving direction of each target object; A process of determining an image quality adjustment parameter for an input image on which recognition processing is performed so that the recognition rate indicating the certainty of the recognition result of a target object with a high priority in the image recognition apparatus is improved, and setting the determined image quality adjustment parameter in the image quality adjustment unit; An image recognition support program for causing a computer to execute the above.

Images