Image processing device, image processing method and non-transitory storage medium

Abstract

The image processing device includes: a recognition unit configured to recognize the category of an object within a predetermined distance relative to a vehicle based on a detection result of a sensor, the sensor including an image sensor, a decision unit configured to decide the number of images to be subjected to image processing per unit time based on the category of the object recognized by the recognition unit, an acquisition unit configured to acquire a plurality of images continuously shot with the object as a subject from the image sensor, and a calculation unit configured to select a target image from the plurality of images acquired by the acquisition unit based on the number of images to be subjected to image processing per unit time determined by the decision unit, and calculate information about the object based on the target image.

Description

Technical Field

[0001] The present disclosure relates to an image processing device, an image processing method, and a non-transitory storage medium. Background Art

[0002] Japanese Unexamined Patent Application Publication No. 2009-205386 (JP 2009-205386 A) discloses a device for setting the frame rate of a camera according to the driving state of a vehicle. The device increases the frame rate of an image sensor only when a detection signal from the sensor indicates that the vehicle may be in danger, and maintains the standard frame rate of the image sensor under normal conditions, thereby suppressing the amount of processing and storage for captured images. Summary of the invention

[0003] However, there are cases where the device described in JP 2009-205386 A cannot perform appropriate image processing on the subject. For example, when the subject is a pedestrian, image processing with high computational cost is required. When the detection signal from the sensor indicates that the vehicle may be in danger, the frame rate will increase, and the number of frames input per unit time will increase. Therefore, the image processing time allowed per frame is shortened, and thus image processing on pedestrians requiring high computational cost may not be performed. In addition, when the subject is a moving object (such as a vehicle) that moves faster than a pedestrian, the number of frames to be processed may be small. Therefore, depending on the moving speed of the subject, the position of the subject varies greatly from one frame to the next, which may affect differential detection. The present disclosure provides a technology that can perform appropriate image processing according to the subject.

[0004] According to the first aspect of the present disclosure, an image processing device includes: a recognition unit that recognizes the category of an object within a predetermined distance relative to a vehicle based on a detection result of a sensor, the sensor including an image sensor; a decision unit that determines the number of images to be subjected to image processing per unit time based on the category of the object recognized by the recognition unit; an acquisition unit that acquires, from the image sensor, a plurality of images obtained by continuously photographing the object as a subject; and a calculation unit that selects one or more target images from the plurality of images acquired by the acquisition unit based on the number of images to be subjected to image processing per unit time determined by the decision unit, and calculates information about the object based on the one or more target images.

[0005] In the above aspect, the number of images subjected to image processing per unit time (hereinafter, referred to as a processing rate) is determined based on the category of the object. Then, a target image is selected from a plurality of images acquired by the acquisition unit based on the processing rate. Then, information about the object is calculated based on the target image. Therefore, the image processing device can determine a processing rate suitable for each subject. Therefore, the image processing device can appropriately perform image processing according to the subject.

[0006] In the above aspect, the recognition unit may be configured to recognize the category of an object within a predetermined distance relative to the vehicle based on the detection result of the image sensor. In this case, the recognition of the object and the calculation of the information about the object may be performed by using a single image sensor.

[0007] In the above aspect, the determination unit may be configured to increase the number of images subjected to image processing per unit time determined by the decision unit when the relative speed between the vehicle and the object is equal to or higher than a threshold value. The greater the relative speed between the vehicle and the object, the greater the change in the position of the object between frames. Therefore, when the relative speed between the vehicle and the object is equal to or higher than the threshold value, the image processing device can suppress the change in the position of the object between frames by increasing the processing rate, thereby performing appropriate image processing.

[0008] In the above aspect, the decision unit may be configured to reduce the number of images subjected to image processing per unit time determined by the decision unit when the relative speed between the vehicle and the object is less than a threshold value. The smaller the relative speed between the vehicle and the object, the smaller the change in the position of the object between frames. Therefore, when the relative speed between the vehicle and the object is less than the threshold value, the image processing device can suppress the reduction in the image processing time allowed per frame by reducing the processing rate.

[0009] In the above aspect, the recognition unit may be configured to further recognize the state of the object, and the decision unit may be configured to decide the number of images subjected to image processing per unit time based on the state and category of the object recognized by the recognition unit. In this case, the image processing device can perform image processing in more detail not only according to the category of the object but also according to the state of the object.

[0010] In the above aspect, the decision unit may be configured to calculate a relative speed between the vehicle and the object based on a detection result of the sensor, and compare the relative speed calculated by the decision unit with a preset threshold value.

[0011] According to the second aspect of the present disclosure, the image processing method includes: identifying the category of an object within a predetermined distance relative to a vehicle based on a detection result of a sensor, the sensor including an image sensor; determining the number of images to be subjected to image processing per unit time based on the category of the object; acquiring from the image sensor a plurality of images obtained by continuously shooting with the object as a subject; selecting one or more target images from the plurality of images based on the number of images to be subjected to image processing per unit time; and calculating information about the object based on the one or more target images.

[0012] According to the third aspect of the present disclosure, a non-temporary storage medium stores instructions that can be executed by one or more processors and cause the one or more processors to perform functions including: identifying the category of an object within a predetermined distance relative to a vehicle based on a detection result of a sensor, the sensor including an image sensor; determining the number of images subjected to image processing per unit time based on the category of the object; acquiring from the image sensor a plurality of images obtained by continuously photographing the object as a subject; selecting one or more target images from the plurality of images based on the number of images subjected to image processing per unit time; and calculating information about the object based on the one or more target images.

[0013] According to the present disclosure, image processing can be appropriately performed according to the subject. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, in which like symbols represent the same elements, and in which:

[0015] Figure 1 is a functional block diagram of an example of a vehicle including an image processing device according to an embodiment;

[0016] Figure 2A The figure shows the detection results from the image sensor;

[0017] Figure 2B is a schematic diagram of the frame rate in the image sensor;

[0018] Figure 2C is a schematic diagram of a high processing rate in an image processing device;

[0019] Figure 2D is a schematic diagram of a low processing rate in an image processing device;

[0020] Figure 3 is a flowchart showing an example of the operation of the image processing apparatus according to the embodiment;

[0021] Figure 4is a flowchart showing an example of the operation of the image processing apparatus according to the modification; and

[0022] Figure 5 is a flowchart illustrating an example of the operation of the image processing apparatus according to the modification. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described below with reference to the accompanying drawings. Note that in the following description, the same or equivalent elements are denoted by the same symbols, and a repetitive description will not be repeated.

[0024] Configuration of vehicle and image processing device

[0025] Figure 1 is a functional block diagram of an example of a vehicle including an image processing device according to an embodiment. Figure 1 As shown in FIG. 1 , the image processing device 1 is mounted in a vehicle 2 such as a bus, a taxi, or a general passenger car. As an example, the vehicle 2 includes an external sensor 3, an internal sensor 4, an electronic control unit (ECU) 5, a human-machine interface (HMI) 6, and an actuator 7.

[0026] The external sensor 3 is a detector that detects information about the external environment of the vehicle 2. The external environment is the position of an object around the vehicle 2, the state of an object, etc. The detection result from the external sensor 3 includes the position, shape, color, etc. of an object ahead on the road on which the vehicle 2 is traveling. The object includes a vehicle, a pedestrian, a traffic light, a road surface marking, etc. An example of the external sensor 3 is a camera.

[0027] The camera is an imaging device that captures images of the external conditions of the vehicle 2. As an example, the camera is disposed behind the windshield of the vehicle 2. The camera acquires image information about the external conditions of the vehicle 2. The camera may be a monocular camera or a stereo camera. The stereo camera has two imaging units arranged to reproduce binocular parallax. The image information obtained by the stereo camera also includes information in the depth direction. The camera captures images at a predetermined frame rate and outputs a plurality of images obtained by continuously capturing a predetermined object as a subject. The predetermined frame rate is the number of images generated (output) per unit time. For example, a frame rate of 3fps means that the number of frames generated (output) per second is 3. The camera can output motion images as continuously captured images.

[0028] The external sensor 3 is not limited to a camera and may include a radar sensor or the like. The radar sensor is a detector that detects an object around the vehicle 2 by using radio waves (e.g., millimeter waves) or light. Examples of the radar sensor include millimeter wave radar and laser imaging detection and ranging (LIDAR). The radar sensor transmits radio waves or light to the surroundings of the vehicle 2 and detects the object by receiving the radio waves or light reflected by the object.

[0029] The internal sensor 4 is a detector that detects the running state of the vehicle 2. The internal sensor 4 may include a vehicle speed sensor, an acceleration sensor, and a yaw rate sensor.

[0030] ECU 5 controls the travel of vehicle 2. ECU 5 is an electronic control unit having a central processing unit (CPU), a read-only memory (ROM), a random access memory (RAM), a controller area network (CAN) communication circuit, etc. ECU 5 is connected to a network that uses a CAN communication circuit for communication, for example, and is connected to the above-mentioned components of vehicle 2 to be able to communicate with each other. For example, ECU 5 operates the CAN communication circuit to input / output data based on a signal output by the CPU, stores data in RAM, loads a program stored in ROM into RAM, and executes the program loaded into RAM, thereby realizing the functions described later. ECU 5 may be composed of a plurality of electronic control units.

[0031] The HMI 6 is an interface between the occupants (including the driver) of the vehicle 2 and the system implemented by the ECU 5. As an example, the HMI 6 includes a touch display capable of displaying information and accepting input of operations performed by the occupants, etc. The actuator 7 is a device that performs travel control of the vehicle 2. The actuator 7 includes at least an engine actuator, a brake actuator, and a steering actuator.

[0032] Functions of ECU

[0033] The image processing device 1 includes a recognition unit 10 , a decision unit 11 , an acquisition unit 12 , and a calculation unit 13 as a functional configuration.

[0034] The recognition unit 10 recognizes the category of the object around the vehicle based on the detection result from the external sensor 3. The external sensor 3 used in the recognition unit 10 may be an image sensor or a radar sensor. The recognition unit 10 recognizes the category of the object around the vehicle based on the detection result from the image sensor, and accordingly may share the sensor in the processing to be performed at a later stage. The term "surroundings of the vehicle" refers to within a range of a predetermined distance relative to the vehicle. The category of the object is a category pre-classified as an object of recognition, and is optionally classified into a stationary object, a vehicle, a pedestrian, etc. Figure 2A The diagram shows the detection results from the image sensor. Figure 2A The image of the area in front of the vehicle 2 is taken by the image sensor, and includes the vehicle 8 and the pedestrian 9. The recognition unit 10 performs object recognition based on a pattern matching technique or the like, and recognizes the categories of the vehicle 8 and the pedestrian 9. For example, the vehicle 8 is recognized as the category "vehicle", and the pedestrian 9 is recognized as the category "pedestrian".

[0035] The decision unit 11 decides the number of images subjected to image processing per unit time based on the category of the object recognized by the recognition unit 10. The number of images subjected to image processing per unit time is the processing rate of the calculation unit 13 operated at a later stage. For example, when the processing rate is 3fps, the calculation unit 13 performs image processing at 3 frames per second. The decision unit 11 decides the processing rate based on a predetermined relationship between the category of the object and the processing rate. For example, the predetermined relationship between the category of the object and the processing rate can be defined in the form of a table. As an example, the category "vehicle" is associated with a processing rate RA (e.g., 36fps), and the category "pedestrian" is associated with a processing rate RB (e.g., 9fps).

[0036] The reason why the processing rate of the category "vehicle" is higher than that of the category "pedestrian" is that unless the processing rate is faster, the position of the vehicle may not be accurately identified. Vehicles tend to move faster than pedestrians, and the position change per unit time of vehicles is greater than the position change per unit time of pedestrians. On the other hand, the reason why the processing rate of the category "pedestrian" is lower than that of the category "vehicle" is that there is no need to increase the processing rate. Pedestrians tend to move slower than vehicles, and the position change per unit time of pedestrians is less than the position change per unit time of vehicles. In addition, pedestrians are the highest priority objects among traffic participants, so the image processing device needs to strengthen recognition. Increasing the processing rate will reduce the processing time that can be spent on each image, and as a result, the recognition accuracy decreases. In this way, the processing rate of the category "vehicle" is set to be higher than the processing rate of the category "pedestrian". Note that the categories are not limited to vehicles and pedestrians, and can be appropriately classified into stationary objects and moving objects, etc.

[0037] The acquisition unit 12 acquires a plurality of images obtained by continuously photographing an object as a subject from the external sensor 3. The external sensor 3 as an acquisition source of the acquisition unit 12 is an image sensor. The acquisition unit 12 acquires images obtained by continuously photographing an object (a vehicle, a pedestrian, etc.) recognized by the recognition unit 10 as a subject. The plurality of images acquired by the acquisition unit 12 are images output at a frame rate of the image sensor. Figure 2B is a schematic diagram of the frame rate of the image sensor. Figure 2B In FIG. 1 , images Gn (where n is an integer) are shown in a time series with respect to the imaging instant. Figure 2B As shown in , the acquisition unit 12 acquires images Gn captured at predetermined intervals. The interval may be constant, but need not be constant. The acquisition unit 12 may acquire one image at a time from the image sensor. In addition, a plurality of images may be recorded in a buffer or the like from the image sensor, and the acquisition unit 12 may acquire a plurality of images from the buffer or the like multiple times or once.

[0038] The calculation unit 13 selects one or more target images from the multiple images acquired by the acquisition unit 12 based on the processing rate determined by the decision unit 11. The calculation unit 13 performs thinning on the multiple images acquired by the acquisition unit 12 according to the processing rate determined by the decision unit 11, and selects the remaining images as target images. For example, the processing rate RA of the category "vehicle" is faster than the processing rate RB of the category "pedestrian". That is, the processing rate RA of the category "vehicle" is a high processing rate, and the processing rate RB of the category "pedestrian" is a low processing rate. Figure 2C is a schematic diagram of a high processing rate in an image processing device, Figure 2D is a schematic diagram of a low processing rate in an image processing device. Figure 2C At the high processing rate shown in , the computing unit 13 performs thinning on the images G2, G3, G5, and G6, and selects the images G1, G4, and G7 as target images. Figure 2D At the low processing rate shown in , the computing unit 13 performs thinning on the images G2, G3, G4, G6, and G7, and selects the images G1 and G5 as target images. Figure 2C and Figure 2D As shown in , the number of images to be processed per second at a high processing rate is greater than the number of images to be processed per second at a low processing rate. Therefore, the image processing apparatus 1 changes the processing rate without changing the frame rate of the image sensor.

[0039] The calculation unit 13 calculates information about the object based on the target image. The information about the object is information about the object that can be obtained from a plurality of images obtained by continuously shooting with the object as the subject, and examples of the information include the speed or acceleration of the object, a classification result finer than a category, and the like. A classification result finer than a category is a category into which the above-mentioned category is further classified, and examples for vehicles include buses, trucks, taxis, and the like, and examples for pedestrians include adults, children, gender, hairstyles, and the like. Figure 2C As shown in , when the object (subject) is a vehicle and the target image is selected at a high processing rate, the calculation unit 13 acquires information about the vehicle 8 based on the image G1, the image G4, and the image G7. Figure 2D As shown in , when the object (subject) is a pedestrian and the target image is selected at a low processing rate, the calculation unit 13 acquires information on the pedestrian 9 based on the image G1 and the image G5 .

[0040] The ECU 5 may further include a vehicle control unit 14. For example, the vehicle control unit 14 controls the vehicle 2 by operating the actuator 7 based on the information about the object obtained by the image processing device 1. Alternatively, for example, the vehicle control unit 14 causes the HMI 6 to display information based on the information about the object obtained by the image processing device 1.

[0041] Operation of image processing device

[0042] Figure 3 is a flowchart showing an example of the operation of the image processing device according to the embodiment. For example, when a start button of an image processing function provided in the vehicle 2 is turned on, the image processing device 1 executes Figure 3 Note that in the following, the recognized objects will be described as pedestrians or vehicles.

[0043] like Figure 3 As shown in FIG. 1 , the recognition unit 10 of the image processing device 1 acquires the detection result of the sensor in step S10 . The recognition unit 10 acquires the detection result of the external sensor 3 .

[0044] Subsequently, in step S12, the recognition unit 10 recognizes the category of the object based on the detection result of the sensor acquired in step S10. When the category of the object recognized by the recognition unit 10 is a pedestrian, the decision unit 11 of the image processing device 1 determines in step S14 ( Figure 2C ) sets the processing rate to a low processing rate. When the category of the object recognized by the recognition unit 10 is a vehicle, the decision unit 11 sets the processing rate to a low processing rate in step S16 ( Figure 2D ) sets the processing rate to a high processing rate.

[0045] When step S14 or step S16 ends, the acquisition unit 12 of the image processing device 1 acquires a plurality of images obtained by continuously photographing the object as a subject from the image sensor in step S28 .

[0046] Subsequently, the calculation unit 13 of the image processing device 1 performs image processing in step S30. The calculation unit 13 selects a target image from the plurality of images acquired in step S28 based on the processing rate decided in step S14. The calculation unit 13 calculates information about the object based on the target image selected by the calculation unit 13.

[0047] Subsequently, the ECU 5 performs post-processing in step S32. For example, the vehicle control unit 14 of the ECU 5 operates the vehicle 2 as post-processing. Note that the image processing device 1 may store information about the object in the storage unit of the ECU 5 or display information about the object on the HMI 6 as post-processing. When step S32 ends, Figure 3 The flowchart shown in ends. Figure 3 When the flowchart shown in FIG. 1 ends, the process is executed again from step S10 until the end condition is satisfied. An example of the end condition is when the end button of the image processing function is turned on, etc. By executing Figure 3 As shown in the flowchart in , the processing rate varies depending on the type of the subject.

[0048] Summary of Examples

[0049] In the image processing device 1, a processing rate as the number of images subjected to image processing per second (an example of a unit time) is determined based on the category of the object. Then, based on the processing rate, a target image is selected from a plurality of images obtained by continuously photographing with the object as a subject. Then, information about the object is calculated based on the target image. Therefore, the image processing device 1 can determine a processing rate suitable for each subject. Therefore, the image processing device 1 can appropriately perform image processing according to the subject.

[0050] The recognition unit 10 of the image processing device 1 recognizes the type of the object around the vehicle based on the detection result of the image sensor. Therefore, the recognition unit 10 and the acquisition unit 12 can use a common sensor.

[0051] Variations

[0052] Although various types of exemplary embodiments have been described above, various types of omissions, substitutions, and changes may be made without being limited to the exemplary embodiments described above.

[0053] For example, the recognition unit 10 of the image processing device 1 may further recognize the state of the object. The state of the object is a state that changes in appearance. When the object is a pedestrian, examples of the state of the object include a state of waving, a state of not waving, a state of wearing a hat, a state of not wearing a hat, etc. When the object is a vehicle, examples of the state of the object include a lighting state of the brake lights (tail lights), a lighting state of the turn signal lights, etc. In the following, the case where the state of the object is the lighting state of the brake lights of the vehicle will be described as an example.

[0054] The decision unit 11 of the image processing device 1 decides a processing rate based on the category and state of the object recognized by the recognition unit 10. The decision unit 11 adjusts the processing rate determined according to the category according to the state of the object. For example, when the category is "vehicle", the decision unit 11 does not set the processing rate only to the high processing rate, but also sets the processing rate to the high processing rate or the medium processing rate. The medium processing rate is a processing rate higher than the low processing rate associated with the category "pedestrian", and is a processing rate lower than the high processing rate associated with the category "vehicle". As an example, the decision unit 11 sets the high processing rate when the recognition unit 10 recognizes that the brake lights of the vehicle 8 are lit, and sets the medium processing rate when the recognition unit 10 recognizes that the brake lights of the vehicle 8 are not lit. When the brake lights of the vehicle 8 are lit is a driving scene in which the speed difference between the vehicle 8 and the vehicle 2 may be larger than when the brake lights of the vehicle 8 are not lit. Therefore, the decision unit 11 sets the high processing rate when the brake lights of the vehicle 8 are turned on, and sets the medium processing rate when the brake lights of the vehicle 8 are not turned on, thereby being able to decide the appropriate processing rate in more detail. Note that the medium processing rate may be set in advance, or may be obtained by subtracting a predetermined processing rate from the high processing rate within a range that does not become the low processing rate.

[0055] Figure 4 1 is a flowchart showing an example of the operation of the image processing apparatus according to the modification. Figure 4 The flowchart shown in Figure 3 The same flowchart as shown in .

[0056] When the class of the object is identified as a vehicle in step S12, the decision unit 11 determines whether the brake lights are on in step S15. The decision unit 11 identifies whether the brake lights are on based on the detection result of the image sensor. When the decision unit 11 determines that the brake lights are on (step S15: Yes), the decision unit 11 sets the processing rate to a high processing rate in step S16. When the decision unit 11 determines that the brake lights are not on (step S15: No), the decision unit 11 sets the processing rate to a medium processing rate in step S17. All other processing is the same as Figure 3 By executing Figure 4 According to the flowchart shown in FIG. 1 , the image processing apparatus 1 can perform image processing not only according to the category of the object but also according to the state of the object.

[0057] As a further variation, when the relative speed between the vehicle 2 and the object is equal to or higher than a threshold, the decision unit 11 may also increase the processing rate. The decision unit 11 calculates the relative speed between the vehicle 2 and the object based on the detection result of the external sensor 3, and compares the relative speed calculated by the decision unit 11 with the threshold. The threshold is a pre-set value used to determine the increase or decrease in the processing rate. When the relative speed calculated by the decision unit 11 is equal to or higher than the threshold, the decision unit 11 determines that the position change of the object between frames is large and increases the processing rate. For example, when the relative speed between the vehicle 2 and the vehicle 8 is equal to or higher than the threshold, the decision unit 11 adds a predetermined increase rate (e.g., 2fps) to the processing rate RA of the category "vehicle". Note that the increase rate can be increased in proportion to the relative speed.

[0058] The decision unit 11 may also reduce the processing rate when the relative speed between the vehicle 2 and the object is less than a threshold value. When the relative speed calculated by the decision unit 11 is less than the threshold value, the decision unit 11 determines that the position of the object does not change much between frames and reduces the processing rate. For example, when the relative speed between the vehicle 2 and the vehicle 8 is less than the threshold value, the decision unit 11 subtracts a predetermined reduction rate (e.g., 2fps) from the processing rate RA of the category "vehicle". Note that the absolute value of the reduction rate may increase in proportion to the relative speed. Therefore, the decision unit 11 may determine the appropriate processing rate in more detail.

[0059] Figure 5 1 is a flowchart showing an example of the operation of the image processing apparatus according to the modification. Figure 5 The flowchart shown in Figure 3 The same flowchart as shown in .

[0060] In step S22, the decision unit 11 determines whether the relative speed between the vehicle 2 and the object is equal to or higher than a threshold value. When the relative speed is equal to or higher than the threshold value (step S22: Yes), the decision unit 11 increases the processing rate in step S24. When the relative speed is less than the threshold value (step S22: No), the decision unit 11 decreases the processing rate in step S26. Figure 5 According to the flowchart shown in , the image processing apparatus 1 can adjust the processing rate according to the relative speed.

[0061] In addition, Figure 3In step S12, the decision unit 11 of the image processing device 1 performs a process of determining the type of the object, but the type of the object is not limited to vehicles and pedestrians. Figure 3 In the example, when the category of the object is neither a vehicle nor a pedestrian, the processing rate may be set to a preset standard processing rate. For example, the standard processing rate may be set to be higher than the low processing rate and lower than the high processing rate.

Claims

1. An image processing device, characterized in that include: an identification unit configured to identify a category of an object within a predetermined distance relative to the vehicle based on a detection result of a sensor, the sensor including an image sensor; a decision unit configured to decide the number of images subjected to image processing per unit time based on the category of the object recognized by the recognition unit, the number of images subjected to image processing per unit time being smaller than the number of images subjected to image processing per unit time when the category of the object is a pedestrian; an acquisition unit configured to acquire, from the image sensor, a plurality of images obtained by continuously photographing the object as a subject at a predetermined frame rate; as well as A computing unit configured to selecting one or more target images from a plurality of images acquired by the acquisition unit without changing a predetermined frame rate of the image sensor based on the number of images subjected to the image processing per unit time determined by the determination unit, and Based on the one or more target images, information about the object is calculated.

2. The image processing device according to claim 1, characterized in that The recognition unit is configured to recognize a category of the object within a range of a predetermined distance from the vehicle based on a detection result of the image sensor.

3. The image processing device according to claim 1, characterized in that The decision unit is configured to increase the number of images subjected to the image processing per unit time decided by the decision unit when a relative speed between the vehicle and the object is equal to or higher than a threshold value.

4. The image processing device according to claim 1, characterized in that: The decision unit is configured to reduce the number of images subjected to the image processing per unit time determined by the decision unit when a relative speed between the vehicle and the object is less than a threshold value.

5. The image processing device according to any one of claims 1 to 4, characterized in that: The recognition unit is configured to further recognize the state of the object; and The decision unit is configured to decide the number of images subjected to the image processing per unit time based on the state and the category of the object recognized by the recognition unit.

6. The image processing device according to claim 3 or 4, characterized in that: The decision unit is configured to: Calculating a relative speed between the vehicle and the object based on the detection result of the sensor; and The relative speed calculated by the determination unit is compared with the preset threshold value.

7. The image processing device according to claim 5, characterized in that: The decision unit is configured as follows: The number of images subjected to the image processing per unit time is determined so that: in a case where the category of the object is a vehicle, the number of images subjected to the image processing per unit time when the state of the object is a state in which the brake lights of the vehicle are lit is greater than the number of images subjected to the image processing per unit time when the state of the object is a state in which the brake lights of the vehicle are not lit.

8. An image processing method, characterized in that include: identifying a category of an object within a predetermined distance relative to the vehicle based on a detection result of a sensor, the sensor comprising an image sensor; determining the number of images subjected to image processing per unit time based on the category of the object, the number of images subjected to image processing per unit time being smaller when the category of the object is a pedestrian than when the category of the object is a vehicle; acquiring, from the image sensor, a plurality of images obtained by continuously photographing the object as a subject at a predetermined frame rate; selecting one or more target images from the plurality of images based on the number of images subjected to the image processing per unit time without changing a predetermined frame rate of the image sensor; as well as Based on the one or more target images, information about the object is calculated.

9. A non-transitory storage medium storing instructions, wherein the instructions can be executed by one or more processors and cause the one or more processors to perform functions, characterized in that: The features include: identifying a category of an object within a predetermined distance relative to the vehicle based on a detection result of a sensor, the sensor comprising an image sensor; determining the number of images subjected to image processing per unit time based on the category of the object, the number of images subjected to image processing per unit time being smaller when the category of the object is a pedestrian than when the category of the object is a vehicle; acquiring, from the image sensor, a plurality of images obtained by continuously photographing the object as a subject at a predetermined frame rate; selecting one or more target images from the plurality of images based on the number of images subjected to the image processing per unit time without changing a predetermined frame rate of the image sensor; and Based on the one or more target images, information about the object is calculated.

Images