Image recognition support apparatus and method

Abstract

To support improvement of recognition accuracy by adjusting an input image so as to increase the accuracy of recognizing a detection target with high priority.SOLUTION: An image recognition support apparatus includes: a recognition result acquisition unit which acquires recognition results obtained by an image recognition apparatus performing a recognition processing on a plurality of target objects in an input image input from an imaging apparatus equipped with an image quality adjustment unit which adjusts image quality using an image quality adjustment parameter; a position information acquisition unit which acquires position information of the target objects on the input image, based on the recognition results; a priority determination unit which determines priority order of the target objects, based on a relationship between the position information and a priority region for recognition in the input image; and an image quality adjustment parameter setting unit which determines an image quality adjustment parameter for the input image subjected to recognition processing so as to increase a recognition rate which indicates likelihood of recognition results of high-priority target objects in the image recognition apparatus, and sets the determined image quality adjustment parameter to the image quality adjustment unit.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to an image recognition support apparatus and method.

Background Art

[0002] Patent Document 1 discloses a technique for obtaining an image signal suitable for the purpose of object detection by image recognition. In Patent Document 1, a region where a detection target appears during a past predetermined period is defined as an active area, and a region where the detection target does not appear is defined as an inactive area. Then, object detection is performed from only the detection signals of the active areas in the imaging pixels, thereby improving the processing efficiency.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Many image recognition apparatuses have a function of photographing and recognizing an unspecified number of detection targets. When there are a plurality of detection targets, there may be a priority order to be detected depending on the state of the detection targets. In Patent Document 1, only the active area and the inactive area are divided according to the presence or absence of the detection target, and the priority of the detection targets included in each area is not considered.

[0005] In view of the above problems, an object of the present disclosure is to provide an image recognition support apparatus and method capable of adjusting an input image so that the recognition accuracy of a detection target with a high priority is increased.

Means for Solving the Problems

[0006] The image recognition support device according to the present disclosure includes a recognition result acquisition unit that acquires a recognition result obtained by performing recognition processing of a plurality of objects on an input image input from an imaging device including an image quality adjustment unit that adjusts the image quality using image quality adjustment parameters, a position information acquisition unit that acquires position information of the plurality of objects on the input image based on the recognition result, a priority determination unit that determines the priority order of the objects based on the relationship between a priority area to be preferentially recognized in the input image and the position information, an image quality adjustment parameter setting unit that determines image quality adjustment parameters for an input image on which recognition processing is to be performed so that the recognition rate indicating the certainty of the recognition result of an object with a high priority order in the image recognition device is improved, and sets the determined image quality adjustment parameters in the image quality adjustment unit, and includes an image recognition support device.

[0007] The image recognition support method according to the present disclosure includes a computer performing a process of acquiring a recognition result obtained by performing recognition processing of a plurality of objects on an input image input from an imaging device including an image quality adjustment unit that adjusts the image quality using image quality adjustment parameters, a process of acquiring position information of the plurality of objects on the input image based on the recognition result, a process of determining the priority order of the objects based on the relationship between a priority area to be preferentially recognized in the input image and the position information, and a process of determining image quality adjustment parameters for an input image on which recognition processing is to be performed so that the recognition rate indicating the certainty of the recognition result of an object with a high priority order in the image recognition device is improved, and setting the determined image quality adjustment parameters in the image quality adjustment unit.

Effect of the Invention

[0008] According to the present disclosure, it is possible to adjust the input image so that the recognition accuracy of a detection target with a high priority order is increased.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0010] Hereinafter, specific embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same reference numerals are assigned to the same elements, and redundant descriptions are omitted as necessary for clarity of explanation.

[0011] The embodiment relates to an image recognition support device that supports improvement of recognition accuracy in an image recognition system including an imaging device such as a surveillance camera and a recognition device that performs recognition processing of an object on an imaging image captured using the imaging device. Many general image recognition systems have a function of photographing an unspecified number of objects and recognizing a plurality of objects. Such an image recognition system may be used for the purpose of monitoring an area where a range where people should be and a range where people should not be are clearly separated, such as a bathing beach, a ski resort, a sidewalk, or a mountain path. When there are a plurality of objects in a captured image, there are often priorities to be detected depending on the state of the objects.

[0012] This priority may change according to the position information of the object. Examples of cases where the priority changes according to the position information of the object include a surveillance camera system installed at a beach with multiple swimmers or a ski resort with multiple skiers. For example, in a surveillance camera system installed at a beach with multiple swimmers, it is required to detect swimmers in a dangerous state preferentially. Specifically, in the case of a beach, a swimmer who has entered the prohibited swimming area can be judged as a swimmer with a high degree of danger.

[0013] Also, in a surveillance camera system installed at a ski resort with multiple skiers, it is required to detect skiers in a dangerous state preferentially. At a ski resort, a skier who has deviated from the skiing area can be judged as a skier with a high degree of danger. In the embodiment, it is an issue to increase the priority of such a detection target with a high degree of danger to obtain a recognition result in line with the purpose of the image recognition system 100.

[0014] FIG. 1 is a block diagram showing the overall configuration of an image recognition system 100 including an image recognition support device 10 according to an embodiment. As shown in FIG. 1, the image recognition system 100 includes an image recognition support device 10, an imaging device 20, an image recognition device 30, a display device 40, and an input device 50. The image recognition system 100 has a function of displaying an image captured by the imaging device 20 for visual recognition by the user. Further, the provided functions of the image recognition system 100 also include a function of recognizing an object included in the captured image and displaying it in a visually recognizable manner to the user.

[0015] Hereinafter, an example in which the image recognition system 100 constitutes a surveillance camera system installed at a beach with multiple swimmers will be described. In this example, multiple swimmers are objects to be recognized by the image recognition device 30. Note that in this example, the object is a person, but it is not limited to this, and it may be an animal such as a dog or a cat, or a vehicle such as a car, a motorcycle, or a ship.

[0016] <Imaging device 20> The imaging device 20 captures a scene including a plurality of objects and generates display image data and recognition image data from the captured data. FIG. 2 is a block diagram showing a configuration example of the imaging device 20 in FIG. 1. As shown in FIG. 2, the imaging device 20 includes a camera unit 21, a signal processing unit 22, an image output unit 23, and a signal processing unit 24 for recognition.

[0017] The camera unit 21 includes, as an imaging element, for example, a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) sensor, or the like. Note that the camera unit 21 may include a lens group (imaging optical system) including a zoom lens and a focus lens, an iris diaphragm, a mechanical shutter, and the like. The camera unit 21 adjusts the level of an image signal based on an image formed by the imaging optical system according to a predetermined amplification gain, performs A / D conversion, and sequentially outputs the data as captured data to the signal processing unit 22.

[0018] The signal processing unit 22 acquires the captured data continuously captured by the camera unit 21 and performs various image processes on the image data in units of frames (hereinafter referred to as image frames). The signal processing unit 22 can generate an image frame conforming to a predetermined display signal format. For example, when generating image data in the full high-definition format, the signal processing unit 22 performs white balance processing, gamma processing, etc. on each image frame, and then executes enlargement / reduction processing to an image size of 1920×1080 pixels. The signal processing unit 22 delivers the image data subjected to each process to the image output unit 23.

[0019] The image output unit 23 can obtain the recognition result by the image recognition device 30 described later via the image recognition support device 10. The image output unit 23 can perform a processing operation on the image data using the recognition result and generate display image data. The image output unit 23 outputs the generated display image data to the display device 40. The display device 40 is a liquid crystal display device or the like having an image display function. The display device 40 can display a captured image with high visibility and a display image including the recognition result based on the display image data.

[0020] Note that the image output unit 23 may generate information to be displayed superimposed on the captured image using the recognition result. The display device 40 can, for example, superimpose and display a frame surrounding the region of the recognized object, character information corresponding to the type of the recognition target, recognition determination information such as the recognition rate, etc. on the captured image.

[0021] Also, the image output unit 23 may create auxiliary display information such as menu setting display used when the user inputs setting information using the input device 50 described later. The image output unit 23 can superimpose the menu setting display on the captured image.

[0022] The image data processed by the signal processing unit 22 is supplied to the recognition signal processing unit 24 in addition to being used for display on the display device 40. The recognition signal processing unit 24 converts the image data processed by the signal processing unit 22 into recognition image data suitable for the recognition processing of the object. The recognition image data is supplied to the image recognition device 30 and used for the recognition of the object. Note that the captured data generated by the camera unit 21 may be directly supplied to the recognition signal processing unit 24 without passing through the signal processing unit 22. Also, a signal being processed by the signal processing unit 22 may be supplied to the recognition signal processing unit 24.

[0023] The recognition signal processing unit 24 converts the image data into recognition image data by performing image quality adjustment processing based on the image quality adjustment parameters. That is, the recognition signal processing unit 24 can also be said to be an image quality adjustment unit that adjusts the image quality using the image quality adjustment parameters. The image recognition support device 10 described later generates image quality adjustment parameters that affect the recognition rate of the object and are used when the recognition signal processing unit 24 performs image quality adjustment processing. Such image quality adjustment parameters include the luminance gain for controlling the luminance (brightness) of the captured image, tone mapping characteristics, aperture gain for edge enhancement, and the like.

[0024] <Image recognition device 30> The image recognition device 30 performs image recognition processing on the recognition image supplied from the recognition signal processing unit 24 and outputs the recognition result to the image recognition support device 10. The recognition images are continuously input to the image recognition device 30, and the recognition processing is continuously executed as needed. The recognition result includes the presence or absence of the object, the type of the object, the area or position of the object, and the recognition rate. The presence or absence of the object is information indicating whether the object is recognized, that is, identified, by the image recognition processing on the recognition image. The type of the object is information indicating the type of the recognition object. The area of the object is a set of coordinates defining the range of the area including the recognized recognition object within the recognition image. The area of the object is, for example, a range specified by pixel values in the XY coordinate system. Note that the position of the object is, for example, a representative point such as the center coordinates of the object recognized within the recognition image.

[0025] The recognition rate is an example of the degree of certainty of recognition by image recognition. That is, the recognition rate is numerical information indicating the recognition accuracy of the presence or absence, type, and area of the object recognized by the image recognition processing on the recognition image. The recognition rate may be indicated, for example, from 0 to 100%. Also, for calculating the recognition rate, for example, a threshold value indicating the degree of similarity to the object, the number of passing stages of the discriminator, and the like may be used. When a plurality of recognition objects are recognized, the recognition result may generate a set of type, area, and recognition rate for each object.

[0026] The image recognition device 30 is hardware, software, or a combination thereof that can execute known image recognition processing. For example, the image recognition device 30 is realized by executing a known image recognition processing program on a computer. Note that the image recognition device 30 may be redundant across multiple computers, and each functional block may be realized by multiple computers. Also, the image recognition device 30 may be realized in a form where each of a client-server system, a cloud computing system, etc. is connected via a communication network. Further, the functions of the image recognition device 30 may be provided in the form of SaaS (Software as a Service). Alternatively, the image recognition device 30 may be realized on the same computer as the image recognition support device 10.

[0027] As the image recognition processing by the image recognition device 30, a method of storing a plurality of images for each object and recognizing the object using pattern matching can be used. At this time, deep learning (deep neural network learning) may be performed using a plurality of images captured from various angles to create a model for recognizing the object. It is generally known that such image recognition processing has a recognition rate that varies depending on the characteristics of the image, such as the ease of distinguishing the object to be recognized from the background in the recognition image.

[0028] Also, the image recognition device 30 has a function of tracking each recognized object by a known technique such as motion compensation between image frames. The image recognition device 30 can associate identification information such as an identification number for identifying the same object with the position information and provide it to the image recognition support device 10. The above-described recognition result may also include the identification information.

[0029] <Image Recognition Support Device 10> The image recognition support device 10 determines and sets image quality adjustment parameters according to the image recognition result for the recognition image output from the image recognition device 30. The recognition signal processing unit 24 further performs image quality adjustment on the recognition image using the set image quality adjustment parameters, and supplies the adjusted recognition image to the image recognition device 30. When the image recognition support device 10 obtains the recognition result for the adjusted recognition image, it determines and sets the image quality adjustment parameters again.

[0030] In this way, the image recognition support device 10 performs feedback control of the image quality adjustment parameters according to the recognition result for the recognition image. The image recognition support device 10 repeats the feedback control until the recognition rate of the object in the recognition image by the image recognition support device 10 becomes stable in a state where it is high. Thereby, it becomes possible to increase the recognition rate of the object in the recognition image.

[0031] Here, the shooting conditions such as brightness and background of the plurality of objects included in the captured image are different. For this reason, the image quality adjustment parameters for optimizing the recognition rate often differ depending on each object. Therefore, it is necessary to prioritize the plurality of objects and perform feedback control of the image quality adjustment parameters so that the recognition rate of the object with the higher priority becomes higher. For example, by setting a swimmer with a high degree of danger, such as a swimmer who has entered the swimming prohibited area of a beach or is heading in the direction of the swimming prohibited area, as an object with a high priority, and setting the image quality adjustment parameters to increase the recognition rate of the object, it becomes possible to obtain a recognition result that conforms to the purpose of the surveillance camera system. In the embodiment, the priority is determined by paying attention to the position of each object, and image quality adjustment is performed to be advantageous for the recognition process of the object with the higher priority.

[0032] FIG. 3 is a block diagram showing a configuration example of the image recognition support device 10 of FIG. 1. Here, an example is shown in which the image recognition support device 10 is realized by a single computer, but it is not limited thereto. The image recognition support device 10 may be redundant in a plurality of computers, and each functional block may be realized by a plurality of computers. Alternatively, all or part of the functions of the image recognition support device 10 may be realized by a general-purpose or dedicated circuit such as a semiconductor device. In these cases, the image recognition support device 10 may be communicably connected to the imaging device 20 and the image recognition device 30 via a communication network.

[0033] The image recognition support device 10 includes a processing unit 1, a storage unit 2, and an IF (InterFace) unit 3. The storage unit 2 includes a non-volatile storage device such as a hard disk and a flash memory, and a memory such as a RAM (Random Access Memory), that is, a volatile storage device. The storage unit 2 stores an image recognition support program, recognition results, and an image quality adjustment history. The image recognition support program is a computer program in which the processing of the image recognition support method according to the embodiment is implemented. Further, the recognition results include a recognition rate, position information, and identification information for identifying the same object for each object. The image quality adjustment history is a history of image quality adjustment parameters determined and set by the image recognition support device 10. The IF unit 3 is an interface circuit responsible for transmitting and receiving data between the image recognition support device 10 and the outside.

[0034] The processing unit 1 is a processing device that executes each process of the image recognition support device 10. The processing unit 1 is, for example, a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), or a quantum processor (quantum computer control chip). The processing unit 1 causes the memory to read the image recognition support program from the storage unit 2 and executes it. Thereby, the processing unit 1 realizes the functions of the recognition result acquisition unit 11, the position information acquisition unit 12, the motion vector calculation unit 13, the priority determination unit 14, the parameter setting unit 15, and the priority area setting unit 16, and generates the above-described image quality adjustment parameters. A part or all of each configuration of the processing unit 1 may be realized by a general-purpose or dedicated circuit realized by, for example, a semiconductor device.

[0035] The recognition result acquisition unit 11 acquires the recognition results of the image recognition processing of a plurality of objects performed by the image recognition device 30 on the recognition target image. The position information acquisition unit 12 acquires the position information on the recognition target images of the plurality of objects based on the acquired recognition results. As described above, as the position information of the object, for example, the center coordinates of the object recognized in the recognition target image can be used.

[0036] The motion vector calculation unit 13 calculates the motion vector of each object from the transition of the positions (coordinates) of each of the plurality of objects between consecutive image frames. The motion vector calculation unit 13 can obtain the motion vector, for example, by calculating the moving direction and the moving amount of each object between consecutive image frames using the identification information of each object input from the image recognition device 30 and the position information associated therewith. Note that the motion vector can be calculated using known techniques such as motion compensation processing between image frames. Also, using known techniques, the motion vector after the current frame may be predicted based on the calculated motion vector and the like.

[0037] The priority area setting unit 16 sets an area (hereinafter referred to as the priority area) in which the detection target is to be preferentially detected. The priority area can be set, for example, based on the setting information input from the input device 50. The input device 50 includes a keyboard, a mouse, a touch panel, etc., and the user performs an input operation of the setting information. The user can input the setting information for setting the priority area according to the shooting environment, shooting conditions, monitoring purpose, etc. Here, the user who sets the priority area is, for example, a person who manages the image recognition system 100. The image recognition support device 10 can receive the setting information from the input device 50 via the IF unit 3. In addition, when the installation location of the image recognition system 100 is determined and there is no need to change the direction, the priority area may be set in advance.

[0038] In addition, the priority area setting unit 16 can also automatically set the boundary line based on the image recognition result of the recognition image. In an example where a lane where people's entry is generally restricted is set as the priority area and a sidewalk where people usually walk is set as the non-priority area, the boundary line separating the lane and the sidewalk will be an actual existing one such as a curb or a guardrail. For example, the image recognition device 30 can detect the boundary line from the recognition image using general methods such as pattern matching and edge extraction, and input the boundary line information to the image recognition support device 10. The priority area setting unit 16 can automatically set this detected boundary line as the boundary line between the priority area and the non-priority area.

[0039] Also, the priority area setting unit 16 can also set the boundary line in cooperation with an accident management system (not shown) that manages the history of accidents that have occurred in the past within the range photographed by the imaging device 20. For example, in an example where an accident-prone area is set as the priority area, the priority area setting unit 16 can obtain the accident history information from the accident management system, and automatically set the priority area at a location on the road where there is no physical boundary based on the obtained accident history information.

[0040] FIG. 4 shows an example of the positional relationship between the priority area and the object in the recognition image. In the example shown in FIG. 4, it is assumed that the recognition image P0 shows a ski field. In the recognition image P0, the central part is the skiing area, and the left and right sides of the skiing area are the no-skiing areas. In this case, the no-skiing areas become the priority areas, and the skiing area becomes the non-priority area. The line separating the priority area and the non-priority area is called the boundary line. In the example of FIG. 4, the user can set two boundary lines and set the area sandwiched between the two boundary lines as the priority area and the area outside the boundary lines as the non-priority area.

[0041] Returning to FIG. 3, the priority determination unit 14 determines the priority of the object based on the relationship between the priority area in the recognition image P0 and the position information of each object. Specifically, the priority determination unit 14 first determines whether each object is within the priority area.

[0042] In FIG. 4, it is assumed that four persons P1, P2, P3, and P4 are included in the recognition image P0. Persons P1 and P2 are located within the priority area, and persons P3 and P4 are located within the non-priority area. In this case, the priority of persons P1 and P2 is higher than that of persons P3 and P4.

[0043] In addition, the priority determination unit 14 can determine the priority based on the relationship between the boundary line between the priority area and the non-priority area and the position information of the object. More specifically, the priority determination unit 14 can determine the priority based on the distance between the boundary line between the priority area and the non-priority area and the object. For example, in the priority area, if the greater the distance from the boundary line, the higher the risk, the priority determination unit 14 may increase the priority of the object farther from the boundary line. The priority determination unit 14 can calculate the distance of each object from the boundary line. The priority determination unit 14 can determine the priority of a plurality of objects based on the information indicating whether each object is located within the priority area and the distance of each object from the boundary line. Conversely, in the priority area, if the closer the distance from the boundary line, the higher the risk, the priority determination unit 14 may increase the priority of the object closer to the boundary line.

[0044] In the example shown in FIG. 4, both the person P1 and the person P2 are located within the priority area, but the person P1 is farther from the boundary line than the person P2. That is, the person P1 is located at a position farther from the sliding area than the person P2. In this case, the priority of the person P1 is higher than that of the person P2.

[0045] Note that, as shown in FIG. 4, in the case of the viewing angle where the distant object is located above the recognition image P0, the distant object above the recognition image P0 appears small, and the near object below the recognition image P0 appears large. In this case, even if the distance between the distant object and the boundary line and the distance between the near object and the boundary line on the recognition image P0 are the same, the actual distance is such that the distance between the distant object and the boundary line is longer than the distance between the near object and the boundary line.

[0046] In this case, the priority determination unit 14 may calculate the actual distance from the boundary line of each object using an existing method. For example, the priority determination unit 14 can obtain the distance between each object and the boundary line based on an object whose size is known. As an example, the priority determination unit 14 can calculate the actual distance assuming the size of the person shown in the recognition image P0 as the standard height. Also, the priority determination unit 14 may calculate the actual distance using the viewing angle information and installation conditions of the imaging device 20. Further, a sensor capable of measuring the distance between each object and the boundary line may be used. The priority determination unit 14 may acquire distance information from the sensor and use it to determine the priority of each object.

[0047] Note that the priority determination unit 14 may set the priority of the object in the non-priority area to be higher for the object closer to the priority area, that is, the object with a shorter distance from the boundary line. For example, when there is no object in the priority area, the priority determination unit 14 may, for example, set the priority to be higher for the object closer to the priority area within the non-priority area. In the example shown in FIG. 4, the priority of P3 is higher than that of P4.

[0048] Furthermore, the priority determination unit 14 may determine the priority by considering not only the distance to the priority area within the non-priority area but also the motion vector calculated by the motion vector calculation unit 13. For example, within the non-priority area, an object that is moving rapidly towards the priority area has a high degree of danger and thus a higher priority. In this case, the object in the non-priority area may have a higher priority than the object in the priority area. The priority determination unit 14 may, for example, assign higher priorities in ascending order of the predicted arrival time of the object at the priority area based on the distance to the priority area and the moving direction and moving amount of the object over time indicated by the motion vector.

[0049] In addition, the priority determination unit 14 may further determine the priority based on information regarding the orientation of the object. Information regarding the orientation of the object may be generated, for example, by the recognition process of the image recognition device 30. Alternatively, information regarding the orientation of the object may be generated by the motion vector calculated by the motion vector calculation unit 13. For example, when the object existing in the non-priority area is a person, a person whose face is oriented towards the priority area has a high possibility of advancing from the non-priority area to the priority area. In this case, the priority determination unit 14 can acquire information regarding the orientation of the person's face and comprehensively determine the priority taking this information into account. When the object in the non-priority area has a high possibility of advancing to the priority area, the priority determination unit 14 can raise the priority of the person.

[0050] Note that the information regarding the orientation of the object generated by the recognition process of the image recognition device 30 may include information regarding the orientation of the face and its changes, the orientation of the body and its changes, the orientation of the line of sight and its changes, and the gait including features of the way a person walks when the object is a person. The priority determination unit 14 may determine the priority taking into account other factors that can determine that the object in the non-priority area has a high possibility of advancing to the priority area. Also, when the object in the priority area has a high possibility of advancing to the non-priority area, the priority determination unit 14 may lower the priority of the object.

[0051] Here, with reference to FIG. 5, an example of a method for determining the priority of a plurality of objects will be described. FIG. 5 is a diagram showing another example of the positional relationship between the priority area and the objects in the recognition image. In the example shown in FIG. 5, it is assumed that the recognition image P0 shows a ski field as in FIG. 4. In the example of FIG. 5, since the recognition image P0 is taken from directly above the shooting range, when the distance between the object above on the recognition image P0 and the boundary line is the same as the distance between the object below and the boundary line, it is considered that the actual distances are also the same. In the example shown in FIG. 5, as in FIG. 4, in the recognition image P0, the central portion is a non-priority area, and the left and right sides of the non-priority area are priority areas. Let the width of each of the left and right priority areas be H1, and the width of the non-priority area be H2.

[0052] In FIG. 5, it is assumed that the recognition image P0 contains four persons P1, P2, P3, and P4 as objects. Persons P1 and P2 are located within the priority area, and persons P3 and P4 are located within the non-priority area. Let the shortest distances from each of the persons P1, P2, P3, and P4 to the nearer one of the left and right boundary lines be h1, h2, h3, and h4, respectively. Note that h1 ≤ H1 and h2 ≤ H1. Also, h3 ≤ H2 / 2 and h4 ≤ H2 / 2. It is assumed that h4 > h1 > h2 > h3.

[0053] The priority determination unit 14 calculates the priority based on the position of each object and the distance to the boundary line, and determines the priority order according to the priority. For the objects within the priority area, the priority is obtained by the following formula (1). Priority (priority area) = K1 × (h / H1) ··· (1) For the objects within the non-priority area, the priority is obtained by the following formula (2). Priority (non-priority area) = K2 × (1 - (h / (H2 / 2)) ··· (2)

[0054] In addition, when the priority range is a value from 0 to 1, K1 in formula (1) and K2 in formula (2) can take any number from 0 to 1 such that K1 > K2. K1 and K2 can be appropriately selected considering differences in risk levels between the priority area and the non-priority area, etc.

[0055] - Example where an object exists within the priority area Since persons P3 and P4 exist within the non-priority area, there is no high need to improve the recognition rate for persons P3 and P4 in the recognition image. Therefore, in this example, it may be sufficient to determine only the priority order of persons P1 and P2. For example, K1 = 1 can be set in formula (1) and K2 = 0 can be set in formula (2). As a result, the priority of person P1 is h1 / H1, and the priority of person P2 is h2 / H2. Also, the priorities of persons P3 and P4 are 0. Since h1 > h2, the priority of person P1 > the priority of person P2, so the priority order is persons P1, P2 in that order. That is, among persons P1 and P2 within the priority area, person P1, who is farther from the boundary line, has a higher priority than person P2. Note that persons P3 and P4 are not included in the priority order. Note that even in an example where an object exists within the priority area, K2 can be set to a value other than 0 to include persons P3 and P4 in the priority order.

[0056] - Example where no object exists within the priority area In an example where persons P1 and P2 within the priority area do not exist, only the priority order of persons P3 and P4 within the non-priority area is determined. For example, K1 = 0 can be set in formula (1) and K2 = 1 can be set in formula (2). As a result, the priority of person P3 is (1 - (h3 / (H2 / 2)), and the priority of person P4 is (1 - (h4 / (H2 / 2)). Since h4 > h3, the priority of person P3 > the priority of person P4, so the priority order is persons P3, P4 in that order. That is, among persons P3 and P4 within the non-priority area, person P3, who is closer to the boundary line, has a higher priority than person P4.

[0057] In the above example, the priority of each object is determined according to the distance from the boundary line. However, as described above, other parameters such as the arrival time at the priority area considering the motion vector and the orientation of the object may be taken into account to determine the priority.

[0058] The parameter setting unit 15 determines the image quality adjustment parameters for the input image for which the recognition process is to be performed so that the recognition rate indicating the certainty of recognizing the object with the highest priority in the image recognition device 30 is improved.

[0059] Specifically, the parameter setting unit 15 first calculates a control recognition rate that serves as the basis for the feedback control of the image quality adjustment parameters in consideration of the priority. That is, the image quality adjustment parameters function as a control recognition rate calculation unit that calculates the control recognition rate using the recognition rates of the respective objects and considering the priority. As the method for calculating the control recognition rate, an optimal method can be used depending on the system, purpose of use, detection environment, etc. For example, as the method for calculating the control recognition rate, the following calculation methods (1) to (3) can be considered. (1) Use the recognition rate of the object with the highest priority. (2) Use the average value of the recognition rates of the objects whose priorities fall within, for example, the top 10% among the detected multiple objects. (3) Calculate the recognition rate considering the weight coefficient for each priority.

[0060] (3) In the method, for example, weight coefficients whose sum is 1 are assigned so that the coefficient with a larger value is from the object with the higher priority. The control recognition rate can be the sum of the products of the recognition rate of each object and the assigned weight coefficient. For example, in the example of FIG. 5, the recognition rate of the person P1 with the first priority is 0.8, the recognition rate of the person P2 with the second priority is 0.6, and the weight coefficients are 0.7 and 0.3, respectively. Here, it is assumed that the persons P3 and P4 are excluded from the target of priority determination. The control recognition rate is 0.8×0.7 + 0.6×0.3 = 0.74.

[0061] The parameter setting unit 15 determines the image quality adjustment parameters of the recognition signal processing unit 24 so that the recognition rate for control becomes a high value. Fig. 6 shows an example of the change in the recognition rate accompanying the change in the image quality adjustment parameters. In the example shown in Fig. 6, the image quality adjustment parameter is assumed to be the luminance gain (digital gain) in the recognition signal processing unit 24. As shown in Fig. 6, the recognition rate increases until the luminance gain is small and the image data of the recognition target becomes a dark value Y1 with respect to the recognition process, and then becomes almost constant between the value Y1, which does not significantly affect the recognition process, and the value Y2, and decreases when the image data is too bright with respect to the recognition process and is larger than the value Y2. That is, when the luminance gain is changed, the graph of the recognition rate has a mountain-shaped characteristic with a flat top.

[0062] As described above, the image recognition support device 10 stores and manages the recognition rate and the image quality adjustment history for each object in the storage unit 2. That is, the storage unit 2 can also be said to be a management unit that manages the change in the recognition rate of the object with a high priority when the image quality adjustment parameter of the recognition signal processing unit 24 is changed. The parameter setting unit 15 obtains the change in the recognition rate for control accompanying the change in the previous image quality adjustment parameter, and can determine a new image quality adjustment parameter so that the recognition rate for control is improved with reference to the change in the recognition rate for control.

[0063] Specifically, when the recognition rate for control is improving, the parameter setting unit 15 changes the new image quality adjustment parameter in the increasing direction when the image quality adjustment parameter was increased at the time of the previous adjustment of the image quality adjustment parameter, and in the decreasing direction when it was decreased. That is, when the recognition rate for control is improving, the parameter setting unit 15 sets the direction of change of the image quality adjustment parameter to the same direction as at the time of the previous adjustment.

[0064] Further, when the recognition rate for control is decreasing, if the image quality adjustment parameter was increased during the previous adjustment of the image quality adjustment parameter, the parameter setting unit 15 changes the new image quality adjustment parameter in the decreasing direction, and if it was decreased, in the increasing direction. That is, when the recognition rate for control is decreasing, the parameter setting unit 15 sets the direction of change of the image quality adjustment parameter to be opposite to that during the previous adjustment.

[0065] For example, in the region below the value Y1 shown in FIG. 5, when the luminance gain is increased, the recognition rate increases. In this case, by further increasing the luminance gain, it is possible to further improve the recognition rate. Also, in the region above the value Y2 shown in FIG. 5, when the luminance gain is decreased, the recognition rate increases. In this case, by further decreasing the luminance gain, the recognition rate can be further improved.

[0066] In this way, the parameter setting unit 15 can determine the image quality adjustment parameter so that the recognition rate becomes near the flat top shown in FIG. 5 by repeating the readjustment of the image quality adjustment parameter. The parameter setting unit 15 can set the image quality adjustment parameter determined for the recognition signal processing unit 24 via the IF unit 3. As a result, the recognition signal processing unit 24 can generate recognition image data in a state where the recognition rate of the recognition process by the image recognition device 30 is high.

[0067] Note that since the image recognition support device 10 performs feedback control of the image quality adjustment parameter based on the recognition rate for control, the image quality adjustment parameter always fluctuates near the top of the recognition rate. To avoid such fluctuations in the image quality adjustment parameter, the parameter setting unit 15 may perform control such as not changing the image quality adjustment parameter once the recognition rate for control reaches a value near the top of FIG. 5, unless the recognition rate for control falls below a predetermined threshold (hereinafter referred to as the readjustment threshold).

[0068] Here, referring to FIG. 7, an example of an image recognition support method according to an embodiment will be described. FIG. 7 is a flowchart showing the flow of the image recognition support process according to the embodiment. First, the image recognition support apparatus 10 performs initial setting of image adjustment parameters on the recognition signal processing unit 24 (S10). Note that the initial-set image adjustment parameters may be set in advance in the recognition signal processing unit 24. In this case, the step of S10 can be omitted.

[0069] Although not shown in FIG. 6, the recognition signal processing unit 24 generates recognition image data using the initial-set image quality adjustment parameters and outputs it to the image recognition apparatus 30. The image recognition apparatus 30 performs recognition processing on the input recognition image data and generates a recognition result.

[0070] Also, the image recognition support apparatus 10 acquires setting information regarding the priority area via the input apparatus 50 (S11). Note that if the installation location of the image recognition system 100 is determined and there is no need to change the priority area, the priority area may be set in advance. In this case, the step of S11 can be omitted.

[0071] The image recognition support apparatus 10 acquires the recognition result from the image recognition apparatus 30. Specifically, the image recognition support apparatus 10 acquires at least the recognition rate of each of a plurality of objects recognized in the recognition image and the position information of each object (S12). Next, the image recognition support apparatus 10 determines the priority order of each object based on the relationship between the position information of each object and the priority area (S13). The image recognition support apparatus 10, for example, determines whether each object is within the priority area based on the position information of each object, and calculates the distance between each object and the boundary line. Then, based on the determination result indicating whether each object is within the priority area and the distance between each object and the boundary line, the priority order of the plurality of objects can be determined.

[0072] Thereafter, the image recognition support device 10 calculates a control recognition rate considering the priority (S14). The control recognition rate can be calculated, for example, considering a weight coefficient for each priority. Then, the image recognition support device 10 compares the current control recognition rate with the previous control recognition rate (S15).

[0073] If the current control recognition rate has decreased compared to the previous control recognition rate, the process proceeds to step S16. In step S16, the image recognition support device 10 determines whether the current control recognition rate is a value near the peak. If the control recognition rate is near the peak (S16, YES), it is determined whether the control recognition rate is less than or equal to the readjustment threshold (S17). If the control recognition rate is not near the peak (S16, NO), or if the control recognition rate is near the peak but the control recognition rate is less than or equal to the readjustment threshold (S17, YES), the image quality adjustment parameter is changed in the direction opposite to the direction of change of the previous image quality adjustment parameter (S18).

[0074] Then, the image recognition support device 10 saves the current position of the object, the recognition rate, and the changed image quality adjustment parameter (S23). On the other hand, if the control recognition rate is greater than the readjustment threshold (S17, NO), the image recognition support device 10 saves the current position of the object, the recognition rate, and the previous image quality adjustment parameter without changing the image quality adjustment parameter (S23).

[0075] If the current control recognition rate has increased compared to the previous control recognition rate, the process proceeds to step S20. In step S20, the image recognition support device 10 determines whether the current control recognition rate is a value near the peak. If the control recognition rate is near the peak (S20, YES), it is determined whether the control recognition rate is less than or equal to the readjustment threshold (S21). If the control recognition rate is not near the peak (S20, NO), or if the control recognition rate is near the peak but the control recognition rate is less than or equal to the readjustment threshold (S21, YES), the image quality adjustment parameter is changed in the same direction as the direction of change of the previous image quality adjustment parameter (S22).

[0076] Then, the image recognition support device 10 saves the position of the current object, the recognition rate, and the image quality adjustment parameters after the change (S23). On the other hand, when the control recognition rate is greater than the readjustment threshold (S21, NO), the image recognition support device 10 saves the position of the current object, the recognition rate, and the image quality adjustment parameters before the change without changing the image quality adjustment parameters (S23).

[0077] Note that in S15, when the current control recognition rate has not changed from the previous control recognition rate, the process proceeds to step S23, and the position of the current object, the recognition rate, and the image quality adjustment parameters before the change are saved.

[0078] As described above, according to the embodiment, based on the position information of a plurality of objects in the recognition image using the recognition result by the image recognition device 30, the priority is determined from the relationship with the priority area. Then, by adjusting the image quality of the recognition image input to the image recognition device 30 so that the recognition rate of the object with a high priority becomes high, it is possible to support the improvement of the recognition accuracy in accordance with the detection purpose. For example, when a plurality of objects are included in the recognition image, it is possible to improve the recognition accuracy of the object with a high priority that is in a dangerous state among these objects.

[0079] Each functional block that performs various processes described in the drawings can be configured hardware-wise by a processor, a memory, and other circuits. Also, it is possible to realize the above-described processes by causing a processor to execute a program. Therefore, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and are not limited to any one of them.

[0080] The above-described program can be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include semiconductor memories (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM). Also, the program may be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable media can supply the program to the computer via wired communication paths such as electric wires and optical fibers, or wireless communication paths.

[0081] In the above embodiment, after the imaging data captured by the camera unit 21, the recognition signal processing unit 24 generates a recognition image with a high recognition rate by adjusting the image quality through software calculation, but it is not limited to this. For example, it is also possible to adjust the brightness of the recognition image by adjusting the amount of light incident on the imaging element using the mechanical shutter or iris diaphragm of the imaging device 20. Specifically, the imaging device 20 may control the mechanical shutter according to the exposure time per frame specified by the image quality adjustment parameter input from the image recognition support device 10 to adjust the charge accumulation time of the imaging element.

[0082] The content of the present disclosure can be used in various fields that utilize image recognition. Some or all of the above embodiments may be described as follows in the appended claims, but are not limited thereto.

[0083] (Appended Claim A1) A recognition result acquisition unit that acquires a recognition result obtained by performing recognition processing of a plurality of objects on an input image input from an imaging device including an image quality adjustment unit that adjusts the image quality using image quality adjustment parameters; A position information acquisition unit that acquires position information of the plurality of objects on the input image based on the recognition result; A priority determination unit that determines the priority order of the objects based on the relationship between a priority area to be preferentially recognized in the input image and the position information; An image quality adjustment parameter setting unit that determines image quality adjustment parameters for an input image on which recognition processing is performed so that the recognition rate indicating the certainty of the recognition result of an object with a high priority in the image recognition device is improved, and sets the determined image quality adjustment parameters in the image quality adjustment unit; Including An image recognition support device. (Appendix A2) The priority determination unit determines the priority order based on the distance between the boundary line between the priority area and the non-priority area and the object. The image recognition support device according to Appendix A1. (Appendix A3) The image recognition support device further includes a motion vector calculation unit that calculates a motion vector of the object, The priority determination unit determines the priority order further based on the motion vector. The image recognition support device according to Appendix A1 or A2. (Appendix A4) The priority determination unit determines the priority order further based on information regarding the orientation of the object. The image recognition support device according to any one of Appendices A1 to A3. (Appendix B1) A computer A process of acquiring a recognition result obtained by performing recognition processing of a plurality of objects on an input image input from an imaging device including an image quality adjustment unit that adjusts the image quality using image quality adjustment parameters; A process of acquiring position information of the plurality of objects on the input image based on the recognition result; A process of determining the priority order of the object based on the relationship between the priority area to be preferentially recognized in the input image and the position information; A process of determining the image quality adjustment parameters for the input image for performing the recognition process so that the recognition rate indicating the certainty of the recognition result of the object with a high priority order in the image recognition apparatus is improved, and setting the determined image quality adjustment parameters in the image quality adjustment unit; An image recognition support method for executing the above. (Appendix C1) A process of obtaining a recognition result obtained by performing a recognition process of a plurality of objects on an input image input from an imaging apparatus including an image quality adjustment unit that adjusts the image quality using image quality adjustment parameters; A process of obtaining the position information of the plurality of objects on the input image based on the recognition result; A process of determining the priority order of the object based on the relationship between the priority area to be preferentially recognized in the input image and the position information; A process of determining the image quality adjustment parameters for the input image for performing the recognition process so that the recognition rate indicating the certainty of the recognition result of the object with a high priority order in the image recognition apparatus is improved, and setting the determined image quality adjustment parameters in the image quality adjustment unit; An image recognition support program for causing a computer to execute the above.

[0084] Some or all of the elements described in Appendices A2 to A4 subordinate to Appendix A1 (image recognition support apparatus) may also be subordinate to Appendix B1 (image recognition support method) and Appendix C1 (image recognition support program) by the same subordinate relationship.

Explanation of Reference Numerals

[0085] 100 Image recognition system 1 Processing unit 2 Storage unit 3 IF unit 10 Image recognition support apparatus 11 Recognition result acquisition unit 12 Position information acquisition unit 13 Motion vector calculation unit 14 Priority Determination Unit 15 Parameter Setting Unit 16 Priority Area Setting Unit 20 Imaging Device 21 Camera Unit 22 Signal Processing Unit 23 Image Output Unit 24 Recognition Signal Processing Unit 30 Image Recognition Device 40 Display Device 50 Input Device P0 Recognition Image P1 - P3 People

Claims

1. A recognition result acquisition unit that acquires a recognition result obtained by performing recognition processing of a plurality of objects on an input image input from an imaging device including an image quality adjustment unit that adjusts the image quality using image quality adjustment parameters; A position information acquisition unit that acquires position information of the plurality of objects on the input image based on the recognition result; A priority determination unit that determines a priority order of the objects based on a relationship between a priority area to be preferentially recognized in the input image and the position information; An image quality adjustment parameter setting unit that determines image quality adjustment parameters for an input image on which recognition processing is performed so that a recognition rate indicating the reliability of a recognition result of an object with a high priority order in the image recognition device is improved, and sets the determined image quality adjustment parameters in the image quality adjustment unit; An image recognition support device, comprising:

2. The priority determination unit determines the priority order based on a distance between a boundary line between the priority area and a non-priority area and the object. The image recognition support device according to Claim 1.

3. The apparatus further includes a motion vector calculation unit that calculates a motion vector of the object, and The priority determination unit determines the priority order further based on the motion vector. The image recognition support device according to Claim 1.

4. The priority determination unit determines the priority order further based on information regarding the orientation of the object. The image recognition support device according to Claim 1.

5. A computer executes a process of acquiring a recognition result obtained by performing recognition processing of a plurality of objects on an input image input from an imaging device including an image quality adjustment unit that adjusts the image quality using image quality adjustment parameters; a process of acquiring position information of the plurality of objects on the input image based on the recognition result; a process of determining a priority order of the objects based on a relationship between a priority area to be preferentially recognized in the input image and the position information; a process of determining image quality adjustment parameters for an input image on which recognition processing is performed so that a recognition rate indicating the reliability of a recognition result of an object with a high priority order in the image recognition device is improved, and setting the determined image quality adjustment parameters in the image quality adjustment unit; An image recognition support method. ​

Images