Object recognition control device and object recognition method

Abstract

To properly recognize a person and the like.SOLUTION: An object recognition control device 1A comprises: an image data acquisition unit 21 for acquiring image data captured by a camera 11 that captures surroundings around a vehicle; a recognition processing unit 22A for recognizing the acquired image data as a person or a bicycle that a person is riding if a score indicating likelihood of the acquired image data being the person or the bicycle that the person is riding is equal to or greater than a predetermined threshold value; a presentation processing unit 23 for presenting, to a driver of the vehicle, information about the recognized person or the bicycle that the person is riding; and a travel speed information acquisition unit 24A for determining a travel-able speed on a road on which the vehicle is traveling. Therein, when the travel-able speed acquired by the travel speed information acquisition unit 24A indicates that the travel-able speed is faster than a first predetermined speed, with respect to the image data acquired by the image data acquisition unit 21, the recognition processing unit 22A sets the predetermined threshold value relatively lower for a range that is far from the vehicle than for a range that is close to the vehicle.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an object recognition control device and an object recognition method. [Background technology]

[0002] In person recognition in a captured image, if a score indicating human-likeness is equal to or greater than a predetermined threshold as a result of recognition processing using person recognition processing, the person is determined to be a person.

[0003] For example, Patent Document 1 discloses a technique for determining that an object is a pedestrian based on a high score indicating pedestrian resemblance using a recognition dictionary for a captured image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-15029 Summary of the Invention [Problem to be solved by the invention]

[0005] The range in which a specific object such as a person must be recognized early varies depending on the environment in which the vehicle is traveling. If a specific object such as a person cannot be recognized in the range in which the specific object must be recognized early, this may lead to danger. Furthermore, presenting a recognition result that includes erroneous recognition to the driver in a range outside the range in which the specific object such as a person must be recognized early may impair the driver's judgment.

[0006] An object of the present invention is to provide an object recognition control device and an object recognition method that can appropriately recognize people and the like. [Means for solving the problem]

[0007] An object recognition control device according to one embodiment of the present invention comprises a video data acquisition unit that acquires video data captured by a camera that captures the area around the vehicle; a recognition processing unit that recognizes the video data acquired by the video data acquisition unit as a person or a bicycle ridden by a person if the score indicating the likelihood of the video data acquired by the video data acquisition unit being a person or a bicycle ridden by a person is equal to or greater than a predetermined threshold; a presentation processing unit that presents information about the person or bicycle recognized by the recognition processing unit to the driver of the vehicle; and a traveling speed information acquisition unit that determines the allowable traveling speed of the road on which the vehicle is traveling, wherein if the allowable traveling speed acquired by the traveling speed information acquisition unit indicates that the allowable traveling speed is faster than a first predetermined speed, the recognition processing unit lowers the predetermined threshold for a range far from the vehicle for the video data acquired by the video data acquisition unit below the predetermined threshold for a range near the vehicle.

[0008] An object recognition control device according to another aspect of the present invention comprises a video data acquisition unit that acquires video data captured by a camera that captures the area around the vehicle; a recognition processing unit that recognizes the video data acquired by the video data acquisition unit as a person or a bicycle ridden by a person if the score indicating the likelihood of the video data acquired by the video data acquisition unit being a person or a bicycle ridden by a person is equal to or greater than a predetermined threshold; a presentation processing unit that presents information about the person or bicycle recognized by the recognition processing unit to the driver of the vehicle; and a traveling speed information acquisition unit that determines the allowable traveling speed for the road on which the vehicle is traveling, wherein if the allowable traveling speed acquired by the traveling speed information acquisition unit indicates that the allowable traveling speed is slower than a second predetermined speed, the recognition processing unit lowers the predetermined threshold for a range corresponding to the vicinity of the vehicle for the video data acquired by the video data acquisition unit below the predetermined threshold for a range corresponding to the far side of the vehicle.

[0009] An object recognition method according to one aspect of the present invention includes a video data acquisition step of acquiring video data captured by a camera that captures the area around a vehicle; a recognition processing step of recognizing the video data acquired in the video data acquisition step as a person or a bicycle ridden by a person if the score indicating the likelihood of the video data acquired in the video data acquisition step being a person or a bicycle ridden by a person is equal to or greater than a predetermined threshold; a presentation processing step of presenting information about the person or bicycle recognized in the recognition processing step to a driver of the vehicle; and a traveling speed information acquisition step of determining the allowable traveling speed of the road on which the vehicle is traveling.In the recognition processing step, if the allowable traveling speed acquired in the traveling speed information acquisition step indicates that the allowable traveling speed is faster than a first predetermined speed, an object recognition control device executes processing for the video data acquired in the video data acquisition step to lower the predetermined threshold for a range far from the vehicle below the predetermined threshold for a range near the vehicle.

[0010] An object recognition method according to another aspect of the present invention includes a video data acquisition step of acquiring video data captured by a camera that captures the area around a vehicle; a recognition processing step of recognizing the video data acquired in the video data acquisition step as a person or a bicycle ridden by a person if the score indicating the likelihood of the video data acquired in the video data acquisition step being a person or a bicycle ridden by a person is equal to or greater than a predetermined threshold; a presentation processing step of presenting information about the person or bicycle recognized in the recognition processing step to the driver of the vehicle; and a traveling speed information acquisition step of determining the allowable traveling speed of the road on which the vehicle is traveling, wherein in the recognition processing step, if the allowable traveling speed acquired in the traveling speed information acquisition step indicates that the allowable traveling speed is slower than a second predetermined speed, the object recognition control device executes processing for the video data acquired in the video data acquisition step to lower the predetermined threshold for a range corresponding to the vicinity of the vehicle below the predetermined threshold for a range corresponding to the far side of the vehicle. [Effects of the Invention]

[0011] According to the present invention, people and the like can be appropriately recognized. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an object recognition device according to the first embodiment. [Figure 2A] FIG. 2A is a diagram for explaining the range corresponding to the far side of the vehicle. [Figure 2B] FIG. 2B is a diagram for explaining the range corresponding to the vicinity of the vehicle. [Figure 3] FIG. 3 is a flowchart showing an example of the flow of the person recognition process according to the first embodiment. [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of an object recognition device according to the second embodiment. [Figure 5] FIG. 5 is a flowchart showing an example of the flow of person recognition processing according to the second embodiment. [Figure 6] FIG. 6 is a block diagram showing an example of the configuration of an object recognition device according to the third embodiment. [Figure 7] FIG. 7 is a flowchart showing an example of the flow of person recognition processing according to the third embodiment. [Figure 8] FIG. 8 is a block diagram showing an example of the configuration of an object recognition device according to a modified example of the third embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of the flow of person recognition processing according to a modified example of the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to these embodiments, and when there are multiple embodiments, the present invention also includes configurations that combine the embodiments. Furthermore, in the following embodiments, the same components are designated by the same reference numerals, and redundant explanations will be omitted.

[0014] [First embodiment] An object recognition device according to a first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of an object recognition device according to a first embodiment.

[0015] 1, the object recognition device 1 includes an imaging unit 11, a dictionary data storage unit 12, a display unit 13, a CAN (Controller Area Network) interface unit 14, and a control unit (object recognition control device) 20. The object recognition device 1 may be installed in a vehicle, or may be a portable device that can be used in a vehicle. The object recognition device 1 may also be implemented as a function of a device with a safe driving support function that is pre-installed in a vehicle, a navigation device, a drive recorder, or the like.

[0016] The imaging unit 11 is mounted on the vehicle and positioned to capture images in the direction of travel of the vehicle. The imaging unit 11 is configured, for example, by a visible light camera or a far-infrared camera. The imaging unit 11 may also be configured, for example, by a combination of a visible light camera and a far-infrared camera. The imaging unit 11 outputs the captured image data to the image data acquisition unit 21.

[0017] The dictionary data storage unit 12 stores dictionary data for recognizing various objects from video data. The dictionary data storage unit 12 stores recognition dictionary data that can be used to identify objects included in the video data as people, for example, by machine learning video of a person's body viewed from various directions. The dictionary data storage unit 12 stores recognition dictionary data that can be used to identify objects included in the video data as bicycles ridden by people, for example, by machine learning video of a bicycle viewed from various directions. The dictionary data storage unit 12 may store recognition dictionary data that can be used to identify objects included in the video data as automobiles, motorcycles, etc., by machine learning video of a vehicle viewed from various directions. The dictionary data storage unit 12 is realized by a storage device, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory.

[0018] Display unit 13 displays various types of information. When a specific object is recognized, display unit 13 notifies the driver of the vehicle by displaying information about the specific object. Display unit 13 is, for example, a display including a liquid crystal display (LCD) or an organic electroluminescence (EL) display.

[0019] The CAN interface unit 14 is an interface that acquires various types of vehicle information via the CAN. The vehicle information includes status information related to the state of the vehicle. The status information includes, for example, information related to the vehicle's traveling speed and vehicle acceleration. The vehicle information may also include driving operation information related to the driving operation of the vehicle. The driving operation information includes, for example, operation information such as steering operation, braking operation, and accelerator operation.

[0020] The control unit 20 controls the operation of each unit of the object recognition device 1. The control unit 20 is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing a program stored in a storage unit (not shown) using a RAM or the like as a work area. Therefore, the control unit 20 causes the object recognition method to be performed by the object recognition device 1. The control unit 20 is also a computer that runs the program according to the present invention. The control unit 20 may be realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 20 may be realized by a combination of hardware and software.

[0021] The control unit 20 includes a video data acquisition unit 21, a recognition processing unit 22, a presentation processing unit 23, and a driving speed information acquisition unit 24. In Fig. 1, the video data acquisition unit 21, the recognition processing unit 22, the presentation processing unit 23, and the driving speed information acquisition unit 24 are shown as being connected via a bus BS.

[0022] The video data acquisition unit 21 acquires video data captured by the imaging unit 11. The video data acquisition unit 21 acquires, for example, video data output by the imaging unit 11, capturing video of the area ahead of the vehicle.

[0023] The recognition processing unit 22 performs object recognition processing on the video data acquired by the video data acquisition unit 21. The recognition processing unit 22 performs various types of object recognition processing using, for example, dictionary data stored in the dictionary data storage unit 12, and recognizes objects included in the video data.

[0024] The recognition processing unit 22 uses dictionary data for detecting people to perform person recognition processing on the video data to recognize people included in the video data. The recognition processing unit 22 uses dictionary data for detecting bicycles ridden by people to recognize bicycles ridden by people included in the video data.

[0025] For example, when a score indicating the likelihood of an object being a person or a bicycle ridden by a person is equal to or greater than a predetermined threshold as a result of object recognition processing performed on the video data acquired by the video data acquisition unit 21, the recognition processing unit 22 recognizes the object included in the video data as a person or a bicycle ridden by a person. In the following explanation, the recognition processing unit 22 will be described as recognizing a person as an example of a specific object, but a bicycle ridden by a person can also be recognized as an example of a specific object.

[0026] The recognition processing unit 22 calculates a score indicating the likelihood of a person through person recognition processing, and recognizes a person based on the calculated score. The maximum value of the score indicating the likelihood of a person is, for example, 1.0. For example, under normal circumstances, the recognition processing unit 22 determines that the recognized object is a person, which is a specific object, when the score is 0.9 or higher.

[0027] The recognition processing unit 22 changes the score threshold for determining that an object recognized in the range corresponding to the vicinity of the vehicle and the range corresponding to the distance from the vehicle is a person for the video data acquired by the video data acquisition unit 21, depending on the vehicle's traveling speed.

[0028] For example, when the vehicle's traveling speed is equal to or greater than a first predetermined speed, the recognition processing unit 22 lowers the score threshold for determining that an object is a person for an area in the video data that corresponds to the far side of the vehicle. For example, when traveling at a speed equal to or greater than the first predetermined speed continues for a predetermined period of time (e.g., 60 seconds or more), the recognition processing unit 22 may lower the score threshold for determining that an object is a person for an area in the video data that corresponds to the far side of the vehicle. For example, when the vehicle's traveling speed is equal to or greater than 60 km / h, the recognition processing unit 22 determines that a recognized object is a person if the score is 0.7 or greater for an area in the video data that corresponds to the far side of the vehicle. When the vehicle's traveling speed is equal to or greater than a first predetermined value, the recognition processing unit 22 performs recognition processing using the normal score threshold for an area in the video data that corresponds to the near side of the vehicle. In other words, when the vehicle's traveling speed is equal to or greater than the first predetermined speed, the recognition processing unit 22 relatively lowers the score threshold for an area in the video data that corresponds to the far side of the vehicle.

[0029] For example, when the vehicle's traveling speed is less than a second predetermined speed that is slower than the first predetermined speed, the recognition processing unit 22 lowers the score threshold for determining that an object is a person for an area in the video data that corresponds to the vicinity of the vehicle. For example, when traveling at a speed less than the second predetermined value continues for a predetermined period (e.g., 60 seconds or more), the recognition processing unit 22 may lower the score threshold for determining that an object is a person for an area in the video data that corresponds to the vicinity of the vehicle. For example, when the vehicle's traveling speed is less than 30 km / h, the recognition processing unit 22 determines that a recognized object is a person if the score for the area in the vicinity of the vehicle is 0.7 or higher. When the vehicle's traveling speed is less than the second predetermined speed, the recognition processing unit 22 performs recognition processing using the normal score for an area far from the vehicle. In other words, when the vehicle's traveling speed is less than the second predetermined speed, the recognition processing unit 22 relatively lowers the score threshold for the area in the video data that corresponds to the vicinity of the vehicle.

[0030] The range corresponding to the far side of the vehicle in the video data and the range corresponding to the near side of the vehicle in the video data will be described using Figures 2A and 2B. Figure 2A is a diagram for explaining the range corresponding to the far side of the vehicle in the video data. Figure 2B is a diagram for explaining the range corresponding to the near side of the vehicle in the video data.

[0031] 2A shows video data 30 captured in the direction in which the vehicle is traveling. The video data 30 includes, for example, a road 31 and a horizon 32. In the video data 30, for example, a range including an upper region 31A of the road 31 and the horizon 32 is a distant range 41 corresponding to the distant range of the vehicle. The upper region 31A of the road 31 is, for example, about 20% of the upper part of the road 31, but is not limited to this.

[0032] 2B shows video data 30 captured in the direction in which the vehicle is traveling. The video data 30 includes a road 31 and a horizon 32. In the video data 30, for example, a range including an upper region 31A of the road 31, a lower region 31C of the road 31, and a central region 31B of the road 31 that is off the horizon 32 is a nearby range 42 that corresponds to the range near the vehicle. The lower region 31C of the road 31 may also be included in nearby range 42.

[0033] The range corresponding to the distance from the vehicle in the video data is located near the center of the video data 30 in the vertical direction, as shown in Fig. 2A. Similarly, the range corresponding to the proximity of the vehicle in the video data is located below the range corresponding to the distance, as shown in Fig. 2B. These ranges also change depending on the vertical installation angle of the imaging unit 11 and the vertical imaging angle of view.

[0034] The far range 41 and the near range 42 may be set according to the distance from the horizon 32 in the video data 30. The far range 41 and the near range 42 may be set according to the distance from the bottom end of the video data 30. The far range 41 and the near range 42 may be set according to the size of an object included in the video data 30.

[0035] For a range corresponding to the distance from the vehicle in the video data, the recognition processing unit 22 may gradually change the score threshold for determining that a recognized object is a person according to the vehicle's traveling speed. For example, when the vehicle's traveling speed is 60 km / h or more and less than 70 km / h, the recognition processing unit 22 may determine that a recognized object is a person if the score is 0.7 or more. For example, when the vehicle's traveling speed is 70 km / h or more and less than 80 km / h, the recognition processing unit 22 may determine that a recognized object is a person if the score is 0.6 or more. For example, when the vehicle's traveling speed is 80 km / h or more and less than 90 km / h, the recognition processing unit 22 may determine that a recognized object is a person if the score is 0.5 or more.

[0036] The recognition processing unit 22 may linearly change the score threshold for determining that a recognized object is a person in accordance with the traveling speed of the vehicle. For example, the recognition processing unit 22 may change the score to 0.7 when the traveling speed of the vehicle reaches 60 km / h, and then linearly decrease the score threshold as the traveling speed of the vehicle increases.

[0037] In the present embodiment, the score value at which the recognition processing unit 22 determines that an object is a person is merely an example and is not limited to this. The score value at which the recognition processing unit 22 determines that an object is a person can be changed as desired depending on the design.

[0038] The presentation processing unit 23 presents information about the person or bicycle the person is riding that has been recognized by the recognition processing unit 22 to the driver, etc. The presentation processing unit 23 presents information about the person or bicycle the person is riding to the driver using display or audio on the display unit 13. Therefore, the presentation processing unit 23 functions as a display control unit when presenting information about the person or bicycle the person is riding to the driver by display on the display unit 13. When displaying information about the person or bicycle the person is riding on the display unit 13, the presentation processing unit 23 displays the video data acquired by the video data acquisition unit 21 on the display unit 13, and if a specific object is recognized, displays a frame that surrounds the recognized person or bicycle the person is riding, thereby allowing the driver to recognize the person or bicycle the person is riding.

[0039] The traveling speed information acquisition unit 24 acquires information relating to the traveling speed of the vehicle. For example, the traveling speed information acquisition unit 24 acquires information relating to the speed of the vehicle from the CAN via the CAN interface. [Person Recognition Processing] The person recognition processing according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the flow of the person recognition processing according to the first embodiment.

[0040] 3 starts, the control unit 20 starts image capture and person recognition processing (step S11). Specifically, the control unit 20 causes the imaging unit 11 to start capturing images, the image data acquisition unit 21 acquires the image data captured by the imaging unit 11, and the recognition processing unit 22 starts recognition processing for the image data acquired by the image data acquisition unit 21. Then, the process proceeds to step S12. The process of FIG. 3 starts under any condition. For example, this may be when the vehicle equipped with the object recognition device 1 becomes available, such as when the engine of the vehicle is started, or when the operation of the object recognition device 1 is started by a user operation. Furthermore, the process of step S11 may be executed on the condition that the vehicle is traveling.

[0041] When the process of step S11 is started, the control unit 20 determines whether the traveling speed of the vehicle is equal to or greater than a first predetermined speed (step S12). Specifically, the recognition processing unit 22 determines whether the traveling speed of the vehicle is equal to or greater than the first predetermined speed based on information related to the traveling speed of the vehicle acquired by the traveling speed information acquisition unit 24. If it is determined that the traveling speed is not equal to or greater than the first predetermined speed (step S12; No), the process proceeds to step S13. If it is determined that the traveling speed is equal to or greater than the first predetermined speed (step S12; Yes), the process proceeds to step S15.

[0042] If the determination in step S12 is No, the control unit 20 determines whether the traveling speed of the vehicle is less than a second predetermined speed (step S13). Specifically, the recognition processing unit 22 determines whether the traveling speed of the vehicle is less than a second predetermined speed, which is slower than the first predetermined speed, based on the information about the traveling speed of the vehicle acquired by the traveling speed information acquisition unit 24. If it is determined that the traveling speed is not less than the second predetermined speed (step S13; No), the process proceeds to step S14. If it is determined that the traveling speed is less than the second predetermined speed (step S13; Yes), the process proceeds to step S16.

[0043] If the determination in step S13 is No, the control unit 20 determines that the object is a person if the score indicating the human-likeness is equal to or greater than a first specified value in the human detection range (step S14). As a specific example, the recognition processing unit 22 determines that the object is a person if the score indicating the human-likeness is equal to or greater than 0.9. Then, the process proceeds to step S17.

[0044] If the determination in step S12 is Yes, the control unit 20 determines that the object is a person if the score indicating the likeliness of a person is equal to or greater than a second specified value that is smaller than the first specified value for the far range of the video data (step S15). As a specific example, the recognition processing unit 22 determines that the object is a person if the score indicating the likeliness of a person for the far range of the video data is equal to or greater than 0.7. Then, the process proceeds to step S17.

[0045] If the determination in step S13 is Yes, the control unit 20 determines that the object is a person if the score indicating the human-likeness is equal to or greater than a second specified value that is smaller than the first specified value for the vicinity range of the video data (step S16). As a specific example, the recognition processing unit 22 determines that the object is a person if the score indicating the human-likeness for the vicinity range of the video data is equal to or greater than 0.7. Then, the process proceeds to step S17.

[0046] The control unit 20 determines whether a person has been detected (step S17). Specifically, the recognition processing unit 22 determines whether a person has been recognized under the conditions set in any of steps S14 to S16. If it is determined that a person has been detected (step S17; Yes), the process proceeds to step S18. If it is determined that a person has not been detected (step S17; No), the process proceeds to step S19.

[0047] If the determination in step S17 is Yes, the control unit 20 displays a frame around the detected person (step S18). Specifically, the presentation processing unit 23 displays a frame around the detected person so that the driver or the like can recognize the detected person. Then, the process proceeds to step S19.

[0048] The control unit 20 determines whether or not to end the specific object recognition process (step S19). The control unit 20 determines to end the specific object recognition process when, for example, an operation to end the specific object recognition process or an operation to turn off the power of the object recognition device 1 is received. If it is determined to end the specific object recognition process (step S19; Yes), the process of FIG. 3 ends. If it is determined not to end the specific object recognition process (step S19; No), the process proceeds to step S12.

[0049] In the above description, it has been explained that a person is determined to be equal to or greater than the second specified value in both the cases where the determination in step S12 is Yes and where the determination in step S13 is Yes, but different thresholds may be used for the cases where the determination in step S12 is Yes and the cases where the determination in step S13 is Yes, as long as the threshold is smaller than the first specified value. In particular, in the recognition process for the far range of video data, the number of pixels constituting the object to be recognized is small, so the threshold for determining a person in the far range of video data may be set to be larger than the threshold for determining a person in the near range of video data.

[0050] In the first embodiment, a person included in video data is detected by changing a predetermined value of a score indicating a person-likeness between a far range and a near range of the vehicle based on the traveling speed of the vehicle. As a result, the first embodiment can appropriately recognize a person included in video data in the far range and the near range of the vehicle even when the traveling speed of the vehicle is faster than a first predetermined speed or slower than a second predetermined speed.

[0051] In addition, in the first embodiment, when the vehicle's traveling speed is faster than a first predetermined speed, the recognition process is performed by lowering the threshold value of the score indicating the likelihood of a person in the far range of the vehicle, or when the vehicle's traveling speed is slower than a second predetermined speed, the recognition process is performed by lowering the threshold value of the score indicating the likelihood of a person in the near range of the vehicle. Therefore, in the range where the score threshold is lowered, specific objects that are partially hidden behind other objects and may not be recognized as specific objects as specific objects with a normal score are also recognized as specific objects. This allows the driver of the vehicle to more appropriately know the presence of a specific object that is a person.

[0052] Furthermore, in the first embodiment, when the vehicle traveling speed is faster than a first predetermined speed, the recognition process is performed by lowering the threshold value of the score indicating the likelihood of a person in the far range of the vehicle, or when the vehicle traveling speed is slower than a second predetermined speed, the recognition process is performed by lowering the threshold value of the score indicating the likelihood of a person in the near range of the vehicle. Therefore, in the range where the score threshold is lowered, the number of erroneous recognitions in which an object that is not a specific object, such as a person, is determined to be a specific object increases. However, even if there is an erroneous recognition, the presence of a specific object is presented, so the driver of the vehicle can drive with a greater emphasis on safety.

[0053] [Second embodiment] An object recognition device according to the second embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the configuration of the object recognition device according to the second embodiment.

[0054] 4, the object recognition device 1A does not include a CAN interface unit 14, but instead includes a GNSS (Global Navigation Satellite System) receiving unit 15 and a map information storage unit 16. The object recognition device 1A also differs from the object recognition device 1 shown in FIG. 1 in that a control unit 20A includes a position information acquisition unit 25 and a map information acquisition unit 26.

[0055] The GNSS receiver 15 receives GNSS signals from GNSS satellites (not shown). The GNSS receiver 15 is realized by a GNSS receiver circuit, a GNSS receiver device, etc. The GNSS receiver 15 outputs the received GNSS signals to the position information acquirer 25.

[0056] The map information storage unit 16 stores various types of map information and is realized by, for example, a semiconductor memory element such as RAM or flash memory, or a storage device such as a hard disk, or further, an external server connected via a communication function (not shown).

[0057] The position information acquisition unit 25 acquires the GNSS signal from the GNSS receiver 15. The position information acquisition unit 25 calculates current position information of the current position of the vehicle based on the GNSS signal acquired from the GNSS receiver 15, for example, using a well-known method.

[0058] The map information acquisition unit 26 acquires map information from the map information storage unit 16. The map information acquisition unit 26 acquires, for example, location information corresponding to the current location information calculated by the location information acquisition unit 25 from the map information storage unit 16.

[0059] The recognition processing unit 22A changes the score threshold for determining that a recognized object is a person, depending on the allowable travel speed of the road on which the vehicle is traveling.

[0060] For example, when the allowable speed on the road on which the vehicle is traveling is equal to or greater than a first predetermined speed, the recognition processing unit 22A may lower the score threshold for determining that an object is a person for an area in the video data that corresponds to the far side of the vehicle. For example, when the vehicle has traveled for a predetermined period (e.g., 60 seconds or more) on a road on which the vehicle is travelable at a speed equal to or greater than the first predetermined speed, the recognition processing unit 22A may lower the score threshold for determining that an object is a person for an area in the video data that corresponds to the far side of the vehicle. For example, when the allowable speed on the road on which the vehicle is traveling is equal to or greater than 60 km / h, the recognition processing unit 22A may determine that a recognized object is a person for an area in the video data that corresponds to the far side of the vehicle if the score is 0.7 or greater. When the vehicle's possible traveling speed is equal to or greater than the first predetermined value, the recognition processing unit 22A performs recognition processing using the normal score for the range corresponding to the vicinity of the vehicle in the video data.

[0061] For example, when the allowable speed on the road on which the vehicle is traveling is less than a second predetermined speed that is slower than the first predetermined speed, the recognition processing unit 22A lowers the score threshold for determining that an object is a person for an area near the vehicle. For example, when the allowable speed on the road on which the vehicle is traveling is less than 30 km / h, the recognition processing unit 22A determines that a recognized object is a person if the score is 0.7 or higher for an area near the vehicle in the video data. When the vehicle's traveling speed is less than the second predetermined speed, the recognition processing unit 22A performs recognition processing using the normal score for an area far from the vehicle.

[0062] The traveling speed information acquisition unit 24A determines the allowable traveling speed of the road on which the vehicle is traveling, based on the current location information calculated by the location information acquisition unit 25 and the map information acquired by the map information acquisition unit 26. The traveling speed information acquisition unit 24A determines, for example, whether the road on which the vehicle is traveling is a road on which travel is permitted at 60 km / h or more, such as a motorway or expressway. The traveling speed information acquisition unit 24A determines, for example, whether the road on which the vehicle is traveling is a road in a residential or commercial area where the speed limit is less than 30 km / h. [Person Recognition Processing] The person recognition processing according to the second embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the flow of the person recognition processing according to the second embodiment.

[0063] The processes of step S21 and steps S24 to S29 shown in FIG. 5 are the same as the processes of step S11 and steps S14 to S19 shown in FIG. 3, respectively, and therefore will not be described further.

[0064] The control unit 20A determines whether or not the vehicle is traveling on a road on which it is possible to travel at a first predetermined speed or higher (step S22). Specifically, the traveling speed information acquisition unit 24A determines whether or not the road on which the vehicle is currently traveling is a road on which it is possible to travel at a first predetermined speed or higher, based on the current position information and map information. If it is determined that the road is not a road on which it is possible to travel at a first predetermined speed or higher (step S22; No), the process proceeds to step S23. If it is determined that the road is a road on which it is possible to travel at a first predetermined speed or higher (step S22; Yes), the process proceeds to step S24.

[0065] If the determination in step S22 is No, the control unit 20A determines whether or not the vehicle is traveling on a road where the vehicle travels at less than a second predetermined speed, which is slower than the first predetermined speed (step S23). Specifically, the traveling speed information acquisition unit 24A determines whether or not the road on which the vehicle is currently traveling is a road where the vehicle travels at less than the second predetermined speed, based on the current position information and map information. If it is determined that the road is not a road where the vehicle travels at less than the second predetermined speed (step S23; No), the process proceeds to step S25. If it is determined that the road is a road where the vehicle travels at less than the second predetermined speed (step S23; Yes), the process proceeds to step S26.

[0066] In the second embodiment, a person included in video data is detected by changing the specified value of the score indicating the likelihood of a person between the far range and the near range of the vehicle based on the allowable traveling speed of the road the vehicle is traveling on. As a result, the second embodiment can appropriately recognize a person included in video data even when the allowable traveling speed of the vehicle is faster than a first predetermined speed or slower than a second predetermined speed.

[0067] Furthermore, in the second embodiment, when the possible travel speed of the road on which the vehicle is traveling is equal to or greater than a first predetermined speed, the recognition process is performed by lowering the threshold value of the score indicating the likelihood of a person in the far range of the vehicle, or when the possible travel speed of the road on which the vehicle is traveling is less than a second predetermined speed, the recognition process is performed by lowering the threshold value of the score indicating the likelihood of a person in the near range of the vehicle. Therefore, in the range where the score threshold is lowered, specific objects that are partially hidden behind other objects and that may not be recognized as specific objects as specific objects with a normal score are also recognized as specific objects. This allows the driver of the vehicle to more appropriately know the presence of specific objects as specific objects.

[0068] Furthermore, in the second embodiment, when the possible travel speed of the road on which the vehicle is traveling is equal to or greater than a first predetermined speed, the recognition process is performed by lowering the threshold value of the score indicating the likelihood of a person in the far range of the vehicle, or when the possible travel speed of the road on which the vehicle is traveling is less than a second predetermined speed, the recognition process is performed by lowering the threshold value of the score indicating the likelihood of a person in the near range of the vehicle. Therefore, in the range where the score threshold is lowered, the number of erroneous recognitions in which an object that is not a specific object, such as a person, is determined to be a specific object increases. However, even if there is an erroneous recognition, the presence of a specific object is presented, which allows the driver of the vehicle to drive with a greater emphasis on safety.

[0069] [Third embodiment] An object recognition device according to the third embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing an example of the configuration of an object recognition device according to the third embodiment.

[0070] As shown in FIG. 6, an object recognition device 1B differs from the object recognition device 1 shown in FIG. 1 in that a control unit 20B includes a traveling direction change information acquisition unit 27 instead of the traveling speed information acquisition unit 24.

[0071] The traveling direction change information acquisition unit 27 acquires traveling direction change information related to a change in the traveling direction of the vehicle. The traveling direction change information acquisition unit 27 acquires operation information of a direction indicator (winker) from the CAN via the CAN interface unit 14, for example.

[0072] For example, while the operation information acquired by the traveling direction change information acquisition unit 27 indicates that a turn signal is operating, the recognition processing unit 22B lowers the score threshold for determining that an object is a person for an area near the vehicle in the video data. For example, while the turn signal is operating, the recognition processing unit 22B determines that a recognized object is a person if the score is 0.7 or higher for an area near the vehicle in the video data. While the turn signal is operating, the recognition processing unit 22B performs recognition processing using the normal score for an area far from the vehicle in the video data.

[0073] For example, while the operation information acquired by traveling direction change information acquisition unit 27 indicates that the turn indicator is operating, recognition processing unit 22B may lower the threshold of the score for determining that an object is a person for a range in the vicinity of the direction indicated by the turn indicator in the video data. For example, when the turn indicator is indicating the left direction, recognition processing unit 22B may lower the threshold of the score for determining that an object is a person for a range corresponding to the vicinity of the left direction in the video data. [Person Recognition Processing] The person recognition processing according to the third embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the flow of the person recognition processing according to the third embodiment.

[0074] The processes of step S31 and steps S33 to S37 shown in FIG. 7 are the same as the processes of step S11, step S14, and steps S16 to S19 shown in FIG. 3, respectively, and therefore will not be described.

[0075] The control unit 20B determines whether or not a direction indication operation has been performed using the direction indicators of the vehicle (step S32). Specifically, the traveling direction change information acquisition unit 27 determines whether or not a direction indication operation has been acquired. If it is determined that a direction indication operation has not been performed (step S32; No), the process proceeds to step S33. If it is determined that a direction indication operation has been performed (step S32; Yes), the process proceeds to step S34.

[0076] In the third embodiment, while the turn signal of the vehicle is operating, a specified value of the score indicating the likelihood of a person in the vicinity of the vehicle in the video data is lowered to detect a person in the video data, thereby enabling the third embodiment to appropriately recognize a person in the vicinity of the vehicle when the turn signal of the vehicle is operating. [Modification of the third embodiment] An object recognition device according to a modified example of the third embodiment will be described with reference to Fig. 8. Fig. 8 is a block diagram showing an example of the configuration of an object recognition device according to a modified example of the third embodiment.

[0077] As shown in FIG. 8, the object recognition device 1C differs from the object recognition device 1B shown in FIG. 6 in the operation of a traveling direction change information acquisition unit 27A of a control unit 20C.

[0078] The traveling direction change information acquisition unit 27A acquires information indicating whether the vehicle is planning to change its traveling direction. The traveling direction change information acquisition unit 27A acquires, for example, from the navigation device 17, position information indicating whether the vehicle is located a predetermined distance before a position where the vehicle will change its traveling direction, based on a preset route.

[0079] For example, during the period from a position a predetermined distance before the position where the vehicle changes direction to when the vehicle actually changes direction, the recognition processing unit 22C lowers the score threshold for determining that the object is a person in the range corresponding to the vicinity of the vehicle. For example, when the score is 0.7 or higher in the range near the vehicle, the recognition processing unit 22C determines that the recognized object is a person. During the period from a position a predetermined distance before the position where the vehicle changes direction to when the vehicle actually changes direction, the recognition processing unit 22C performs recognition processing using a normal score in the range corresponding to the distance away from the vehicle.

[0080] For example, during the period from a position a predetermined distance before the position where the vehicle changes direction to the actual change of direction of travel, the recognition processing unit 22C may lower the threshold of the score for determining that a person exists in a range near the direction in which the vehicle will change direction. For example, when there is a plan to change the direction of travel to the left, the recognition processing unit 22C may lower the threshold of the score for determining that a person exists in a range corresponding to the vicinity of the left direction in the video data. [Person Recognition Processing] The person recognition process according to the modified example of the third embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the flow of the person recognition process according to the modified example of the third embodiment.

[0081] The processing in step S41 and steps S43 to S47 are the same as the processing in step S31 and steps S33 to S37 shown in FIG. 7, respectively, and therefore will not be described here.

[0082] The control unit 20C determines whether or not the vehicle is planning to change its traveling direction (step S42). Specifically, the traveling direction change information acquisition unit 27A determines whether or not the vehicle is located a predetermined distance before the position where the vehicle will change its traveling direction, based on a route set in advance by the navigation device 17. If it is determined that the vehicle is not planning to change its traveling direction (step S42; No), the process proceeds to step S43. If it is determined that the vehicle is planning to change its traveling direction (step S42; Yes), the process proceeds to step S44.

[0083] In a modified example of the third embodiment, when the vehicle is scheduled to change direction, a specified value of the score indicating the likelihood of a person in the vicinity of the vehicle is lowered to detect a person included in the video data. This allows the third embodiment to appropriately recognize a person included in the video data in the vicinity of the vehicle when the vehicle is scheduled to change direction.

[0084] In addition, in the third embodiment, the recognition process is performed by lowering the threshold value of the score indicating the likelihood of a person in the vicinity of the vehicle based on a change in the vehicle's traveling direction, so that in the range where the threshold value of the score is lowered, specific objects that are partially hidden behind other objects, which may not be recognized as specific objects as people with a normal score, are also recognized as specific objects. This allows the driver of the vehicle to more appropriately know the presence of specific objects as people.

[0085] Furthermore, in the third embodiment, the recognition process is performed by lowering the threshold value of the score indicating the likelihood of a person in the vicinity of the vehicle based on a change in the vehicle's traveling direction, so that in the range where the threshold value of the score is lowered, the number of erroneous recognitions in which an object that is not a specific object, such as a person, is determined to be a specific object increases. However, even in the case of erroneous recognition, the presence of a specific object is presented, so that the driver of the vehicle can drive with a greater emphasis on safety.

[0086] Although the embodiments of the present invention have been described above, the present invention is not limited to the contents of these embodiments. In the embodiments, the recognition of a person or a bicycle ridden by a person has been described as an example, but the object to be recognized is not limited to the contents of the embodiments, and the present invention can be applied to the recognition of various objects. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments. [Explanation of symbols]

[0087] 1,1A,1B,1C Object Recognition Device 11 Imaging unit 12 Dictionary data storage unit 13 Display section 14 CAN interface section 15 GNSS receiver 16 Map information storage unit 17 Navigation devices 20, 20A, 20B, 20C control section 21 Video data acquisition unit 22, 22A, 22B, 22C Recognition processing section 23 Presentation processing unit 24. Travel speed information acquisition unit 25 Location information acquisition section 26 Map information acquisition unit 27, 27A Direction change information acquisition unit

Claims

1. a video data acquisition unit that acquires video data captured by a camera that captures the surroundings of the vehicle; a recognition processing unit that recognizes the video data acquired by the video data acquisition unit as a specific object when a score indicating a likelihood of the video data being a person or a bicycle ridden by a person is equal to or greater than a predetermined threshold; a presentation processing unit that presents information about the specific object recognized by the recognition processing unit to a driver of the vehicle; a traveling speed information acquisition unit that determines the allowable traveling speed of the road on which the vehicle is traveling; Equipped with When the possible traveling speed acquired by the traveling speed information acquisition unit indicates that the possible traveling speed is faster than a first predetermined speed, the recognition processing unit lowers the predetermined threshold for the range corresponding to the distance from the vehicle compared to the predetermined threshold for the range corresponding to the proximity of the vehicle, for the video data acquired by the video data acquisition unit, to an extent that the specific object partially hidden in the shadow of an object that would not be recognized as the specific object with a normal score can be recognized, or erroneous recognition of the specific object increases. Object recognition control device.

2. a video data acquisition unit that acquires video data captured by a camera that captures the surroundings of the vehicle; a recognition processing unit that recognizes the video data acquired by the video data acquisition unit as a specific object when a score indicating a likelihood of the video data being a person or a bicycle ridden by a person is equal to or greater than a predetermined threshold; a presentation processing unit that presents information about the person or the specific object recognized by the recognition processing unit to a driver of the vehicle; a traveling speed information acquisition unit that determines the allowable traveling speed of the road on which the vehicle is traveling; Equipped with When the possible traveling speed acquired by the traveling speed information acquisition unit indicates that the possible traveling speed is slower than a second predetermined speed, the recognition processing unit lowers the predetermined threshold for the range corresponding to the vicinity of the vehicle compared to the predetermined threshold for the range corresponding to the far side of the vehicle for the video data acquired by the video data acquisition unit to an extent that the specific object partially hidden in the shadow of an object that would not be recognized as the specific object with a normal score can be recognized, or erroneous recognition of the specific object increases. Object recognition control device.

3. a video data acquisition step of acquiring video data captured by a camera that captures the surroundings of the vehicle; a recognition processing step of recognizing the image data acquired in the image data acquisition step as a specific object when a score indicating a likelihood of the image data being a person or a bicycle ridden by a person is equal to or greater than a predetermined threshold; a presentation processing step of presenting information about the specific object recognized in the recognition processing step to a driver of the vehicle; a travel speed information acquisition step for determining an allowable travel speed for the road on which the vehicle is traveling; Including, In the recognition processing step, if the possible traveling speed acquired in the traveling speed information acquisition step indicates that the possible traveling speed is faster than a first predetermined speed, the predetermined threshold value for the range corresponding to the distance from the vehicle is lowered compared to the predetermined threshold value for the range corresponding to the proximity of the vehicle for the video data acquired in the video data acquisition step, to an extent that the specific object partially hidden in the shadow of an object that would not be recognized as the specific object with a normal score can be recognized, or that erroneous recognition of the specific object increases. An object recognition method executed by an object recognition control device.

4. a video data acquisition step of acquiring video data captured by a camera that captures the surroundings of the vehicle; a recognition processing step of recognizing the image data acquired in the image data acquisition step as a specific object when a score indicating a likelihood of the image data being a person or a bicycle ridden by a person is equal to or greater than a predetermined threshold; a presentation processing step of presenting information about the specific object recognized in the recognition processing step to a driver of the vehicle; a travel speed information acquisition step for determining an allowable travel speed for the road on which the vehicle is traveling; Including, In the recognition processing step, if the possible traveling speed acquired in the traveling speed information acquisition step indicates that the possible traveling speed is slower than a second predetermined speed, the predetermined threshold value for the range corresponding to the vicinity of the vehicle is lowered to a level lower than the predetermined threshold value for the range corresponding to the far side of the vehicle for the image data acquired in the image data acquisition step, such that the specific object partially hidden in the shadow of an object that would not be recognized as the specific object with a normal score can be recognized, or erroneous recognition of the specific object increases. An object recognition method executed by an object recognition control device.

Images