Image recognition device, image recognition method and program

Abstract

To reduce a possibility of erroneous recognition of a position of a detected person.SOLUTION: An image recognition device 10 includes: an image acquisition part 12 configured to acquire a picked-up image; a person detection part 14 configured to detect a person included in the picked-up image acquired at the image acquisition part 12 to detect a moving destination or an orientation of the person in the picked-up image; a frame line determination part 16 configured to determine a location of a frame line with regard to the location of the person according to the moving destination or the orientation of the person detected at the person detection part 14; an image generation part 18 configured to generate a display image with the frame line at a location determined by the frame line determination part 16 overlapping the location of the person in the picked-up image; and a display control part 20 configured to display the picked-up image generated by the image generation part 18 on the display device 28.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image recognition device, an image recognition method, and a program. [Background technology]

[0002] There is known a technology for detecting a person included in a captured image, and generating and displaying an image in which a rectangular frame is superimposed on the detected person. The size of the rectangular frame displayed superimposed on the captured image may have an aspect ratio of 2:1 so as to surround the range of the detected person. Patent Document 1 also discloses a technology for displaying a frame of a standard size (e.g., equivalent to a height of 170 cm) based on the distance to the detected person. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-204374 A Summary of the Invention [Problem to be solved by the invention]

[0004] If the size of the frame line is set to a fixed value regardless of the size of the person, there is a possibility that attributes such as whether the person is an adult or a child may not be properly grasped. On the other hand, if the size of the frame line corresponds to the size of the person, the size of the frame line superimposed on a short person such as a child will be small. If the size of the frame line is small, when viewing a display of a captured image on which the frame line is superimposed, there is a possibility that the impression that the person is located farther away than the actual position will be given, and the position of a short person nearby may be erroneously grasped.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and has an object to provide a technique for reducing the possibility of erroneously determining the position of a detected person. [Means for solving the problem]

[0006] An image recognition device of one embodiment of the present invention includes an image acquisition unit that acquires a captured image, a person detection unit that detects a person included in the captured image acquired by the image acquisition unit and detects the direction of movement of the person or the orientation of the person in the captured image, a border determination unit that determines a position of a border relative to a position of the person based on the direction of movement of the person or the orientation of the person detected by the person detection unit, an image generation unit that generates a display image in which the border at the position determined by the border determination unit is superimposed on the position of the person in the captured image, and a display control unit that causes the display image generated by the image generation unit to be displayed on a display device.

[0007] Another aspect of the present invention is an image recognition method executed by an image recognition device, which includes the steps of acquiring a captured image, detecting a person included in the acquired captured image and detecting a moving direction or an orientation of the person in the captured image, determining a position of a frame line relative to a position of the person based on the detected moving direction or orientation of the person, generating a display image in which the frame line at the determined position is superimposed on the position of the person in the captured image, and displaying the generated display image on a display device. Effect of the Invention

[0008] According to the present invention, the possibility of erroneously determining the position of a detected person can be reduced. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram illustrating a functional configuration of an image recognition device according to a first embodiment. [Diagram 2] FIG. 4 is a diagram showing an example of a captured image. [Diagram 3] 10A and 10B are diagrams illustrating the detected position and detected size of a person. [Figure 4] FIG. 13 is a diagram showing an example of a display image on which a frame line is superimposed. [Diagram 5]4 is a flowchart showing the flow of an image recognition method according to the first embodiment. [Figure 6] 6(a) to (d) are diagrams showing other examples of frame line display. [Figure 7] FIG. 11 is a block diagram illustrating a functional configuration of an image recognition device according to a second embodiment. [Figure 8] 10 is a flowchart showing the flow of an image recognition method according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Specific numerical values ​​and the like shown in the embodiment are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the drawings, elements that are not directly related to the present invention are omitted.

[0011] [First embodiment] Before describing the first embodiment in detail, an overview will be provided. In the first embodiment, a person included in a captured image is detected, and a display image in which a frame surrounding the detected person is superimposed on the captured image is generated and displayed. In this embodiment, a frame of a size corresponding to the detected size of the detected person is superimposed, and when the height of the detected person is less than a predetermined value, such as less than 120 cm, a display image in which a frame of a size vertically larger than the detected size of the person is superimposed is generated. According to this embodiment, by superimposing a frame of a size larger than the detection size on a person whose height is less than the predetermined value, the possibility of misidentifying the detected person as being farther away than the actual position due to the small vertical size of the frame of the detected person is reduced, and the possibility of erroneously grasping the position of the detected person is reduced.

[0012] FIG. 1 is a block diagram showing a schematic functional configuration of an image recognition device 10 according to a first embodiment. The image recognition device 10 includes an image acquisition unit 12, a person detection unit 14, a frame determination unit 16, an image generation unit 18, a display control unit 20, and a storage unit 22. In this embodiment, the image recognition device 10 is mounted on a vehicle. The image recognition device 10 is, for example, a device mounted on a vehicle, and is configured by a control device such as a CPU (Central Processing Unit) provided on the vehicle. The image recognition device 10 may be configured using a navigation system provided on the vehicle. The image recognition device 10 may also be realized by a portable device such as a video camera device or a smartphone. The image recognition device 10 may include at least one of a camera 26 and a display device 28.

[0013] Each functional block shown in this embodiment can be realized in hardware terms by elements and mechanical devices such as a computer CPU and memory, and in software terms by a computer program, etc., but is depicted here as a functional block realized by the cooperation of these. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms by combining hardware and software.

[0014] The image acquisition unit 12 acquires an image captured by the camera 26. The camera 26 is mounted on a vehicle and captures an image of the surroundings of the vehicle. For example, the camera 26 captures an image in front of the vehicle. The camera 26 may capture an image of the rear of the vehicle or an image of the side of the vehicle. The camera 26 is configured to capture visible light. The camera 26 may be configured to capture color images of red, green, and blue, or may be configured to capture monochrome images of visible light.

[0015] The camera 26 may be configured to capture infrared rays. The camera 26 may be a so-called infrared thermography device, and may image the temperature distribution around the vehicle so that a heat source existing around the vehicle can be identified. The camera 26 may be configured to detect mid-infrared rays with a wavelength of about 2 μm to 5 μm, or may be configured to detect far-infrared rays with a wavelength of about 8 μm to 14 μm.

[0016] The captured images captured by the camera 26 and acquired by the image acquisition unit 12 are a moving image in which images are successively captured at, for example, 30 frames per second. The processing described below is performed on the moving image as needed.

[0017] FIG. 2 shows an example of a captured image 30 acquired by the image acquisition unit 12. In the example of FIG. 2, a plurality of persons 32a, 32b, 32c, and 32d are included in the captured image 30. The first person 32a and the second person 32b are located at a first distance L1 that is a relatively short distance from the vehicle. The third person 32c and the fourth person 34d are located at a second distance L2 that is a relatively long distance from the vehicle. The first distance L1 corresponds to, for example, a distance of about 15 m to 20 m from the vehicle. The second distance L2 corresponds to, for example, a distance of about 40 m to 50 m from the vehicle.

[0018] The first person 32a and the third person 32c are about 170 cm tall, and the second person 32b and the fourth person 32d are about 100 cm tall, i.e., so-called short people. In the illustrated example, the apparent size of the second person 32b, who is short, and the apparent size of the third person 32c, that is, the detected size in the captured image 30, are about the same. The detected size is the size on the image in the captured image.

[0019] The person detection unit 14 detects people included in the captured image acquired by the image acquisition unit 12. The person detection unit 14 searches for pedestrians, cyclists (people riding bicycles), and the like using a person recognition dictionary. The person detection unit 14 searches for people in the captured image using the person recognition dictionary, and calculates a person score indicating the possibility that a person exists in the searched section. For example, the person detection unit 14 detects a person as a person when the person score of the searched section is equal to or greater than a predetermined threshold. The person recognition dictionary used by the person detection unit 14 is generated by machine learning using a model that receives an image of a person as input and outputs a person score. A convolutional neural network (CNN) or the like can be used as a model used for machine learning. In the example of FIG. 2, the person detection unit 14 performs a process of detecting people in the captured image 30 acquired by the image acquisition unit 12, and detects multiple people 32a, 32b, 32c, and 32d.

[0020] The shape of the block used by the person detection unit 14 for search is determined in advance according to the person recognition dictionary to be used. In the person recognition dictionary according to this embodiment, the block to be searched is rectangular, and the ratio of the vertical and horizontal image sizes of the block is determined to be approximately 2:1. The shape of the block used for search corresponds to, for example, the vertical and horizontal image sizes of the learning image used in the machine learning for generating the person recognition dictionary.

[0021] The person detection unit 14 specifies the detection position and detection size of the detected person. The person detection unit 14 assigns a tag number to the detected person, and stores the detection position and detection size for each tag number in the storage unit 22. The detection position is the position coordinate of the section in which the person is detected in the captured image 30, and is determined, for example, by the position coordinate of the center of the bottom end of the section. The bottom end of the section in which the person is detected corresponds to the position of the person's feet, and can also be said to be the ground position where the person touches the ground. The ground position of the person can be used to estimate the distance to the person. The detection size is the size of the section in which the person is detected, and is determined, for example, by the vertical image size of the section in which the person is detected. The detection size can be used to estimate the height of the person.

[0022] FIG. 3 is a diagram showing a schematic diagram of the detection positions and detection sizes of the people 32a to 32d detected in the captured image 30 shown in FIG. 2. FIG. 3 shows the sections 34a, 34b, 34c, and 34d in which the people 32a to 32d are detected in the captured image 30 of FIG. 2, and the grounding positions 36a, 36b, 36c, and 36d of the people 32a to 32d. The person detection unit 14 stores at least the vertical sizes ha to hd of the sections 34a to 34d detected as the detection sizes of the people 32a to 32d in the storage unit 22. The vertical sizes ha to hd of the sections 34a to 34d correspond to the vertical image sizes of the people 32a to 32d in the captured image 30. The vertical sizes ha to hd of the sections 34a to 34d are determined by the vertical image sizes from the feet to the heads of the people 32a to 32d. The human detection unit 14 stores in the storage unit 22 the coordinates of ground contact positions 36a to 36d located in the center of the bottom ends of the sections 34a to 34d as the detected positions of the human beings 32a to 32d.

[0023] The frame line determination unit 16 determines whether or not to superimpose a frame line on the captured image 30 based on the detection result of the person detection unit 14, and if a frame line is to be superimposed, determines the position and size of the frame line to be superimposed. The image generation unit 18 generates a display image by superimposing the frame line of the position and size determined by the frame line determination unit 16 on the captured image 30. The display control unit 20 causes the display image generated by the image generation unit 18 to be displayed on the display device 28. The display device 28 is, for example, a display mounted on a vehicle.

[0024] The frame line determination unit 16 calculates the distance to the person detected by the person detection unit 14, and stores the calculated distance in the storage unit 22 for each tag number. The frame line determination unit 16 may calculate the distance to the person based on the vertical position coordinate of the grounding position of the detected person in the captured image. The frame line determination unit 16 may calculate the distance to the person using a table or formula indicating the correlation between the vertical position coordinate of the captured image and the distance. In this case, the distance to the person is the distance from the vehicle on which the camera 26 is mounted to the person.

[0025] The frame line determination unit 16 determines whether or not to superimpose a frame line based on the distance from the vehicle to the person. The frame line determination unit 16 determines to superimpose a frame line if the distance from the vehicle to the person is less than a threshold (e.g., 40 m), and not to superimpose a frame line if the distance from the vehicle to the person is equal to or greater than the threshold (e.g., 40 m). The frame line determination unit 16 may determine the color of the frame line to be superimposed according to the distance from the vehicle to the person. For example, if the distance to the person is less than a first threshold (e.g., 20 m), the frame line may be red, and if the distance to the person is equal to or greater than the first threshold (e.g., 20 m) and less than a second threshold (e.g., 40 m), the frame line may be yellow. Note that if the distance to the person is equal to or greater than the second threshold, no frame line may be used. The frame line determination unit 16 stores the determined necessity of a frame line and the color of the frame line for each tag number in the storage unit 22.

[0026] The frame line determination unit 16 calculates the height of the person detected by the person detection unit 14, and stores the calculated height for each tag number in the storage unit 22. The frame line determination unit 16 calculates the height of the person based on the grounding position of the detected person and the detected size in the vertical direction. The frame line determination unit 16 may calculate the height of the person using a table or a formula indicating the ratio between the detected size and the height, which is determined according to the position coordinate in the vertical direction of the captured image 30.

[0027] The frame line determination unit 16 determines the size of the frame line based on the height of the person whose distance is less than the threshold value. The frame line determination unit 16 determines the size of the frame line according to the height of the person. When the height of the person is equal to or greater than a predetermined value, such as 120 cm or more, the frame line determination unit 16 sets the vertical size of the frame line to a size equivalent to the detection size of the person. When the height of the person is less than a predetermined value, such as less than 120 cm, that is, when the person is short, the frame line determination unit 16 sets the vertical size of the frame line to a size larger than the detection size of the person. The vertical size of the frame line of a person whose height is less than the predetermined value is, for example, a large size corresponding to a height of 150 cm to 170 cm at the detection position of the person, and has a size larger than the detection size of the detected person.

[0028] The size of the frame for a person whose height is equal to or greater than a predetermined value varies depending on the detected size of the person, and can vary depending on the height of the person and the distance to the person. In other words, the size of the frame for a person whose height is equal to or greater than a predetermined value is proportional to the height of the person and inversely proportional to the distance to the person. If the distance to the person is the same, the size of the frame varies depending on the height of the person. For example, a frame superimposed on a person whose height is 180 cm and located at a distance of 20 m is vertically larger than a frame superimposed on a person whose height is 160 cm and located at a distance of 20 m. Also, if the heights of the people are the same, the size of the frame varies depending on the distance to the person. For example, a frame superimposed on a person whose height is 180 cm and located at a distance of 10 m is vertically larger than a frame superimposed on a person whose height is 180 cm and located at a distance of 20 m.

[0029] The size of the frame for a person whose height is less than a predetermined value does not necessarily follow the detected size of the person. For example, a frame of a fixed size may be set for a person whose height is less than a predetermined value. In this case, for example, when a person whose height is 100 cm and a person whose height is 90 cm are detected, a frame of a size equivalent to a height of 180 cm is set for both. Also, for a person whose height is less than a predetermined value, the frame size may be set so that the vertical size of the frame is a value obtained by multiplying the height of the detected person by a coefficient such as 1.5. In this case, for example, a frame of a size equivalent to a height of 150 cm is set for a person whose height is 100 cm, and a frame of a size equivalent to a height of 135 cm is set for a person whose height is 90 cm.

[0030] FIG. 4 is a diagram showing an example of a display image 40 on which frame lines 42 and 44 are superimposed. In FIG. 4, the height of the first person 32a in FIG. 2 is determined to be equal to or greater than a predetermined value based on the grounding position 36a and the vertical size Ha in the display image 40, so a frame line 42 of the detection size of the first person 32a is superimposed. That is, a frame line 42 with an aspect ratio of 2:1 is superimposed with the vertical detection size Ha of the first person 32a as the vertical size. The height of the second person 32b is determined to be less than a predetermined value based on the grounding position 36a and the vertical size Hb in the display image 40, so a frame line 44 of a size vertically larger than the detection size of the second person 32b is superimposed as a frame line of a size different from the detection size of the second person 32b. In FIG. 4, the frame lines 42 and 44 drawn on the display image 40 are shown in thick lines. The frame lines of the first section 34a and the second section 34b shown in dashed lines in FIG. 4 are not drawn on the display image 40. In FIG. 4, frame lines 42, 44 are superimposed only on the first person 32a and the second person 32b, who are located at a first distance L1, which is a close distance, and frame lines are not superimposed on the third person 32c and the fourth person 32d, who are located at a second distance L2, which is a long distance.

[0031] The frame 42 superimposed on the first person 32a whose height is equal to or greater than a predetermined value has a size corresponding to the detection size of the first person 32a, for example, the same size as the first section 34a in which the first person 32a is detected. The size of the frame 42 may be slightly different from the size of the first section 34a, for example, by about 5% to 10%. Therefore, the vertical size Ha of the frame 42 may be the same as the vertical size ha of the first section 34a, or may be slightly smaller or larger. The aspect ratio of the frame 42 is the same as the aspect ratio of the detection size of the first person 32a, which is 2:1. The frame 42 is superimposed based on the grounding position 36a of the first person 32a, and is superimposed so that the center of the lower end of the frame 42 coincides with the grounding position 36a.

[0032] The frame 44 superimposed on the second person 32b whose height is less than a predetermined value is larger than the detection size of the second person 32b. The vertical size Hb of the frame 44 is significantly larger than the vertical size hb of the second section 34b in which the second person 32b is detected, for example, by 10% or more. In the example of FIG. 4, the vertical size Hb of the frame 44 is about 1.5 times the vertical size hb of the second section 34b. The aspect ratio of the frame 44 is the same as the aspect ratio of the detection size of the second person 32b, which is 2:1. In other words, the aspect ratio of the frame 44 is the same as the aspect ratio of the frame 42. The frame 44 is superimposed based on the ground position 36b of the second person 32b, and is superimposed so that the center of the bottom end of the frame 44 coincides with the ground position 36b. As a result, a gap 46 exists between the head of the second person 32b and the top end of the frame 44. The vertical size of the gap 46 is, for example, 10% or more of the vertical size hb of the second section 34b, for example, about 20% to 50%.

[0033] According to the display image 40 of FIG. 4, a frame 44 larger than the actual height of the second person 32b is superimposed on the second person 32b. This makes the second person 32b look larger than when a frame of a size corresponding to the detected size of the second person 32b, that is, the size corresponding to the second section 34b shown by the dashed line, is superimposed. In addition, since the size of the frame 44 superimposed on the second person 32b is close to the size of the frame 42 superimposed on the first person 32a, it is possible to more clearly present that the first person 32a and the second person 32b are located at approximately the same distance. In addition, since the grounding positions 36a, 36b of the first person 32a and the second person 32b coincide with the bottom ends of the frame lines 42, 44, it is possible to more clearly present that the first person 32a and the second person 32b are located at the same distance from the bottom ends of the frame lines 42, 44. As a result, the possibility of the second person 32b appearing small being mistaken as if the second person 32b is located farther away than it actually is can be reduced. For example, it is possible to reduce the possibility of mistakingly recognizing that the second person 32b is present near the second distance L2 where the third person 32c, whose apparent size is the same as that of the second person 32b, is located.

[0034] Fig. 5 is a flowchart showing the flow of the image recognition method according to the first embodiment. When the image recognition device 10 is mounted on a vehicle, the process in Fig. 5 starts and ends when the vehicle is started and stopped, or when the engine or power source is started and stopped. Alternatively, the process may start and end when the user operates the vehicle.

[0035] First, when the process starts, the image acquisition unit 12 acquires the captured image 30 from the camera 26 (S10), and the person detection unit 14 starts detecting a person included in the captured image (S12). If a person is detected (Y in S12), the person detection unit 14 specifies the detected position and detected size of the person (S14). If the distance to the detected person is less than a threshold (Y in S16) and the height of the detected person is less than a predetermined value (Y in S18), the frame line determination unit 16 determines the size of the frame line to be superimposed on the detected position of the person to a size different from the detected size of the person, specifically, a size larger than the detected size of the person, and the image generation unit 18 generates a display image in which a frame line of the determined size is superimposed (S20). If the height of the detected person is not less than the predetermined value (N in S18), a display image in which a frame line of a size corresponding to the detected size of the person is superimposed on the detected position of the person is generated (S22). The generated display image is displayed on the display device 28 by the display control unit 20 (S24). If the distance to the person is not less than the threshold (N in S16), the processes in S18 to S22 are skipped and no frame is superimposed on the person. If no person is detected (N in S12), the processes in S14 to S22 are skipped and the captured image 30 without the frame is displayed as the display image (S24).

[0036] In the above flow, when multiple people are detected in the captured image 30, the process of S12 to S22 determines whether or not a frame line needs to be superimposed and the size of the frame line to be superimposed for each of the multiple detected people.

[0037] An aspect of the present embodiment may be a program. The program may be configured to cause a computer to realize a function of acquiring a captured image, a function of detecting a person included in the acquired captured image and specifying a detection position and a detection size of the person in the captured image, a function of calculating a height of the person from the detection position and detection size of the detected person, a function of determining a frame size different from the detection size based on the calculated height of the person, a function of generating a display image in which the frame size determined is superimposed on the detection position of the captured image, and a function of displaying the generated display image on a display device.

[0038] The present invention has been described above with reference to the above-mentioned embodiments. However, the present invention is not limited to the above-mentioned embodiments, and appropriate combinations or substitutions of the configurations shown in the embodiments are also included in the present invention.

[0039] In a modification of the first embodiment, the frame line for a person whose height is less than a predetermined value may be displayed in a manner different from that shown in Fig. 4. Figs. 6(a) to (d) are diagrams showing display examples of other frame lines 44a to 44d corresponding to the frame line 44 shown in Fig. 4. All of the people 32 shown in Fig. 6 are people whose height is less than a predetermined value.

[0040] The frame 44a shown in Fig. 6(a) is larger only in the vertical direction than the size of the section 34 in which the person 32 is detected, while the horizontal direction remains the same. Therefore, the aspect ratio of the frame 44a in Fig. 6(a) is larger than approximately 2:1, which is the aspect ratio of the detection size of the person 32. The vertical direction size of the frame 44a in Fig. 6(a) is, for example, more than twice the horizontal direction size, and is approximately 2.2 to 3.5 times.

[0041] The frame 44b shown in Fig. 6(b) is larger only in the horizontal direction than the size of the section 34 in which the person 32 is detected, while the vertical direction remains the same. Therefore, the aspect ratio of the frame 44b in Fig. 6(b) is smaller than the aspect ratio of the size of the section 34 in which the person 32 is detected, which is approximately 2:1. The horizontal size of the frame 44b in Fig. 6(b) may be smaller than the vertical size of the frame 44b, and may be, for example, approximately 0.6 to 1 times the vertical size.

[0042] A frame 44c shown in Fig. 6(c) has the same shape and size as the frame 44 in Fig. 4, but the position where the frame 44c is superimposed is different. The bottom end of the frame 44c in Fig. 6(c) is located below the grounding position 36 of the person 32. The center position of the frame 44c in Fig. 6(c) coincides with, for example, the center position of the section 34 in which the person 32 is detected. By shifting the frame 44c downward and superimposing it, it is possible to give the impression that the person 32 is closer, and the presence of a short person can be further emphasized.

[0043] The frame line 44d shown in FIG. 6(d) has the same shape and size as the frame line 44 in FIG. 4, but the position where the frame line 44d is superimposed is different, and the superimposed position is shifted in the left-right direction. The frame line 44d in FIG. 6(d) is superimposed with a shift to the movement direction or line of sight 38 of the detected person 32. In the example of FIG. 6(d), the movement direction or line of sight 38 of the person 32 is to the right, and the center of the frame line 44d is located to the right of the grounding position 36. By superimposing the frame line 44d with a shift to the movement direction or line of sight 38 of the person 32, the movement direction or line of sight 38 of the person 32 can be presented, and the behavior of a short person can be suggested. Note that if the movement direction or line of sight 38 of the person 32 is not left-right but downward or diagonal, the superimposed position of the frame line 44d may be shifted downward or diagonal.

[0044] The moving direction of the person 32 may be identified based on the transition of the person detection position for each frame of the captured image, which is a moving image, the direction of the person's hands and feet, etc. Furthermore, the gaze direction 38 of the person 32 may be identified based on the face direction of the person 32, and the face direction of the person 32 may be regarded as the gaze direction 38. The face direction of the person 32 is identified based on the person detection result by the person detection unit 14.

[0045] In the above embodiment, the bottom end of the detected section is set as the ground contact position. In another embodiment, the ground contact position may be detected based on the image content of the detected section. For example, when the feet of a person included in the detected section are hidden and cannot be seen, the height of the person may be estimated based on the position and size of the head of the person included in the detected section, and the ground contact position may be detected based on the estimated height.

[0046] In the above embodiment, the necessity of a border and the color of the border may be determined according to the distance to the person. In another embodiment, the necessity of a border and the color of the border may be determined based on the detected position of the person. For example, the border determination unit 16 may hold the vertical position coordinate of the captured image corresponding to the above-mentioned threshold distance (for example, 20 m or 40 m) as a threshold, and determine whether or not to superimpose the border and the color of the border based on the threshold position coordinate.

[0047] In the above embodiment, the size of the frame line for a person whose height is less than a predetermined value is changed to a size different from the detected size of the person by, for example, expanding the size vertically, and a frame line of a size different from the detected size of the person is superimposed. In another embodiment, a first frame line corresponding to the detected size of a person whose height is less than a predetermined value and a second frame line expanded vertically may be superimposed on the person whose height is less than a predetermined value. In this case, the frame line surrounding the detected person is a double frame line. Also, an outline drawn to trace the outline of a person whose height is less than a predetermined value within the vertically expanded frame line may be added to the frame line and superimposed.

[0048] In the above embodiment, a frame is not superimposed on a person whose distance is equal to or greater than a threshold. In another embodiment, a frame may be superimposed on a person whose distance is equal to or greater than a threshold. For example, if a person whose distance is less than the threshold is not detected, a frame may be superimposed on a person whose distance is equal to or greater than a threshold. A frame may be superimposed on a person whose distance is equal to or greater than a threshold, regardless of whether a person whose distance is less than the threshold is detected.

[0049] In the above embodiment, the size of the frame for a person whose height is less than a predetermined value is made different from the detection size of the person by enlarging it vertically, and a frame of a size different from the detection size of the person is superimposed. In another embodiment, the necessity of such processing may be determined according to the distance between a person whose height is less than a predetermined value and a person whose height is equal to or greater than the predetermined value. For example, when a person whose height is equal to or greater than the predetermined value exists in the vicinity of a person whose height is less than a predetermined value, for example, within a range equivalent to 2 m, the distance to the person whose height is less than the predetermined value is less likely to be misidentified due to the presence of a person whose height is equal to or greater than the predetermined value. For this reason, when there is no person whose height is equal to or greater than the predetermined value in the vicinity of a person whose height is less than a predetermined value, for example, within a predetermined range such as 2 m, the size of the frame for the person whose height is less than a predetermined value may be made different from the detection size of the person.

[0050] In the above embodiment, the frame line determination unit 16 calculates the distance to the detected person and calculates the height of the detected person. In another embodiment, instead of the frame line determination unit 16, the person detection unit 14 may calculate the distance to the detected person and calculate the height of the detected person. In this case, the person detection unit 14 may detect a person included in the captured image acquired by the image acquisition unit 12, identify the detection position and detection size of the person in the captured image, and calculate the height of the person from the detection position and detection size of the person. The frame line determination unit 16 may determine a frame size different from the detection size based on the height of the person calculated by the person detection unit 14. When the height of the person detected by the person detection unit 14 is less than a predetermined value, the frame line determination unit 16 may set the size of the frame to a size larger than the detection size at least in the vertical direction.

[0051] [Second embodiment] Next, a second embodiment of the present invention will be described with reference to the drawings. In the second embodiment, the height of a person is not calculated, and a frame size different from the detected size is determined based on a specified size according to the detected position of the person. The second embodiment will be described below, focusing on the differences from the first embodiment, and drawings or descriptions of the commonalities with the first embodiment will be omitted as appropriate.

[0052] 7 is a block diagram showing a schematic functional configuration of an image recognition device 10a according to the second embodiment. The image recognition device 10a includes an image acquisition unit 12, a person detection unit 14, a frame line determination unit 16a, an image generation unit 18, a display control unit 20, and a storage unit 22. In the second embodiment, the image acquisition unit 12, the person detection unit 14, the image generation unit 18, the display control unit 20, and the storage unit 22 are configured in the same manner as in the first embodiment.

[0053] The frame line determination unit 16a determines a frame line size different from the detection size based on the detection position of the person specified by the person detection unit 14. The frame line determination unit 16a differs from the first embodiment in that it does not determine the frame line size based on the height of the person, but determines the frame line size based on the detection position of the person and a specified size according to the detection position. Specifically, when the detection size of the person detected by the person detection unit 14a is less than the specified size according to the detection position, the frame line size is determined so that the size of the frame line is at least larger in the vertical direction than the detection size. The specified size according to the detection position is, for example, the same size as the detection size of a person with a height of 170 cm present at the detection position. The specified size may change according to the detection position, and for example, the specified size may be larger as the detection position in the captured image is located lower, and the specified size may be smaller as the detection position in the captured image is located higher. The specified size according to the detection position is stored in advance in the storage unit 22, for example. The specified size according to the detection position can be determined using a table or a formula indicating the correlation between the detection position and the specified size. The frame line detection unit 16a may be configured in the same manner as the frame line determination unit 16 according to the first embodiment, except for the process of determining the frame line size according to the person's height.

[0054] The processing of the frame determination unit 16a will be described with reference to FIG. 3 and FIG. 4. The person detection unit 14 detects the first person 32a, and specifies the detection size and detection position (i.e., the ground position 36a) of the section 34a in which the first person 32a is detected. The frame determination unit 16a compares the detection size of the first person 32a with a prescribed size according to the ground position 36a of the first person 32a. If the prescribed size corresponds to the detection size of a person with a height of 170 cm and the height of the first person 32a is 170 cm or more, the detection size of the first person 32a is equal to or larger than the prescribed size according to the ground position 36a. In this case, the frame determination unit 16a determines the detection size of the section 34a in which the first person 32a is detected as the size of the frame. As a result, for example, as shown in FIG. 4, a frame 42 of a size equivalent to the detection size of the first person 32a is superimposed and displayed.

[0055] The person detection unit 14 detects the second person 32b, and identifies the detection size and detection position (i.e., the ground position 36b) of the section 34b in which the second person 32b is detected. The frame determination unit 16a compares the detection size of the second person 32b with a specified size according to the ground position 36b of the second person 32b. If the specified size corresponds to the detection size of a person with a height of 170 cm and the height of the second person 32b is about 100 cm, the detection size of the second person 32b is less than the specified size according to the ground position 36b. In this case, the frame determination unit 16a determines the size of the frame to be at least vertically larger than the detection size of the section 34b in which the second person 32b is detected. The frame determination unit 16a may determine the size of the frame of the second person 32b to be the same as the specified size according to the ground position 36b. As a result, for example, as shown in FIG. 4, a frame 44 that is larger in size at least in the vertical direction than the detected size of the second person 32b is displayed in a superimposed manner.

[0056] FIG. 8 is a flowchart showing a flow of the image recognition method according to the second embodiment. The image acquisition unit 12 acquires a captured image 30 from the camera 26 (S50), and the person detection unit 14 detects a person included in the acquired captured image (S52). If a person is detected (Y in S52), the person detection unit 14 identifies the detection position and detection size of the person (S54). If the distance to the detected person is less than a threshold (Y in S56) and the detection size of the detected person is less than a specified size according to the detection position (Y in S58), the frame line determination unit 16a determines the size of the frame line to be superimposed on the detection position of the person to a size different from the detection size of the person, specifically, a size larger than the detection size of the person, and the image generation unit 18 generates a display image in which a frame line of the determined size is superimposed (S60). If the detection size of the detected person is not less than the specified size according to the detection position (N in S58), a display image in which a frame line of a size corresponding to the detection size of the person is superimposed on the detection position of the person is generated (S62). The generated display image is displayed on the display device 28 by the display control unit 20 (S64). If the distance to the person is not less than the threshold (N in S56), the processes of S58 to S62 are skipped and a frame line is not superimposed on the person. If no person is detected (N in S52), the processes of S54 to S62 are skipped and the captured image 30 without a frame line superimposed is displayed as the display image (S64).

[0057] The second embodiment can also achieve the same effects as the first embodiment. Note that the second embodiment may be modified from the first embodiment.

[0058] In the above-described embodiments, the distance to a person is calculated based on a captured image. In another embodiment, the distance to a person may be measured using a sensor other than the camera 26. For example, any distance measuring sensor such as an ultrasonic sensor, a radar sensor, or a LIDAR (Light Detection and Ranging) may be used.

[0059] In each of the above-described embodiments, the image recognition device 10 is mounted on a vehicle. In another embodiment, the installation location of the image recognition device 10 is not particularly limited, and the image recognition device 10 may be used for any purpose. [Industrial Applicability]

[0060] According to the present invention, the possibility of erroneously determining the position of a detected person can be reduced. [Explanation of symbols]

[0061] 10...image recognition device, 12...image acquisition unit, 14...person detection unit, 16, 16a...frame line determination unit, 18...image generation unit, 20...display control unit, 28...display device, 30...captured image, 32...person, 34...section, 36...ground position, 40...displayed image, 42...frame line, 44...frame line.

Claims

1. an image acquisition unit that acquires a captured image; a person detection unit that detects a person included in a captured image acquired by the image acquisition unit and detects a moving direction or a direction of the person in the captured image; a frame line determination unit that determines a position of a frame line relative to a position of the person based on a moving direction of the person or an orientation of the person detected by the person detection unit; an image generating unit that generates a display image in which the frame line at the position determined by the frame line determining unit is superimposed on the position of the person in the captured image; a display control unit that causes a display device to display the display image generated by the image generation unit; An image recognition device comprising:

2. the frame line determination unit determines a position of the frame line so as to be shifted in a direction of movement of the person detected by the person detection unit or in a direction of the orientation of the person. The image recognition device according to claim 1 .

3. the frame line determination unit determines a position of the frame line so as to include the person detected by the person detection unit and to be shifted in a direction of movement of the person detected by the person detection unit or a direction of the orientation of the person. The image recognition device according to claim 1 .

4. Obtaining a captured image; detecting a person included in the captured image, and detecting a moving direction or a direction of the person in the captured image; determining a position of a frame line relative to a position of the person based on the detected direction of movement of the person or an orientation of the person; generating a display image in which a frame line at the determined position is superimposed on the position of the person in the captured image; displaying the generated display image on a display device; The image recognition method is executed by an image recognition device.

Images