Program, image processing device, and image processing method
By dynamically adjusting IIR filter coefficients based on bounding box size, the image processing system addresses accuracy variations for both nearby and distant targets, ensuring stable and accurate vehicle tracking.
Patent Information
- Application Number
- JP2023084899
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2024-12-05
AI Technical Summary
Conventional image processing technologies using AI for vehicle tracking face issues with varying tracking accuracy due to fixed averaging filter coefficients, which are inadequate for both nearby and distant targets, leading to potential loss of tracking or response delays.
Adaptive adjustment of the IIR filter coefficients based on the size of the detected bounding box (Bbox) to stabilize tracking accuracy, increasing averaging strength for distant targets and reducing it for nearby targets.
Stabilizes tracking accuracy regardless of target distance by dynamically adjusting filter coefficients, ensuring smooth and accurate tracking of vehicles or other objects.
Smart Images

Figure 2024168332000001_ABST
Abstract
Description
[Technical field]
[0001] The disclosed embodiments relate to a program, an image processing device, and an image processing method. [Background technology]
[0002] Conventionally, there is known an image processing technology that tracks a preceding vehicle or the like as a target based on the result of image recognition using AI (Artificial Intelligence) on camera images taken from a vehicle.
[0003] In addition, in such image processing technology, a technology has been proposed in which filtering in the time direction using an averaging filter is performed in order to suppress the variation in the image recognition results in the time direction and smoothly track the target (see, for example, Patent Document 1). An infinite impulse response (IIR) filter or the like is used as the averaging filter. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2007-272735 A Summary of the Invention [Problem to be solved by the invention]
[0005] However, in the above-mentioned conventional technology, the coefficients of the averaging filter are fixed values, so there is a problem that the tracking accuracy varies depending on the distance between the target and the vehicle.
[0006] In AI image recognition, targets are detected as bounding boxes (hereinafter referred to as "Bboxes"), which are rectangular subregions. Distant targets are detected as Bboxes that are smaller in size than nearby targets.
[0007] Here, if the coefficients are designed in advance to match the interframe position and size variations of nearby Bboxes, the averaging strength of the distant Bboxes may be too weak to be corrected, making tracking impossible.On the other hand, if the coefficients are designed in advance to match the variations of distant Bboxes, the averaging strength of the nearby Bboxes may be too strong, causing a filter response delay and impairing tracking accuracy.
[0008] One aspect of the embodiment has been made in consideration of the above, and aims to provide a program, an image processing device, and an image processing method that can stabilize the tracking accuracy when tracking a target using image processing, regardless of whether the target is far away or close. [Means for solving the problem]
[0009] A program according to one aspect of the embodiment causes a computer to perform the following steps: detect a partial region in which a tracked object exists by image recognition of a camera image; apply a filter that performs a time-wise averaging process on the partial region; and, when applying the filter, change the coefficient of the filter depending on the size of the partial region after the previous averaging process. Effect of the Invention
[0010] According to one aspect of the embodiment, in tracking a target by image processing, it is possible to stabilize the tracking accuracy regardless of whether the target is far away or close. [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 is a schematic explanatory diagram of an image processing method according to the existing technology. [Diagram 2] FIG. 2 is a block diagram of an averaging process according to the existing technology. [Diagram 3] FIG. 3 is a block diagram of the averaging process according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of the image processing device according to the embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of the configuration of an image processing device according to a modified example. [Figure 6] FIG. 6 is a flowchart showing a processing procedure executed by the image processing device according to the embodiment. [Figure 7] FIG. 7 is an explanatory diagram of the Bbox size calculation process according to the embodiment. [Figure 8] FIG. 8 is an explanatory diagram of the adjustment coefficient calculation process according to the embodiment. [Figure 9] FIG. 9 is an explanatory diagram of the dynamic coefficient calculation process according to the embodiment. [Figure 10] FIG. 10 is a diagram showing an image of a case where the existing technology and this embodiment are applied to a nearby target. [Figure 11] FIG. 11 is a diagram showing an image of a case where the existing technology and this embodiment are applied to a distant target. [Figure 12] FIG. 12 is a diagram showing an image of tracking a distant target using existing technology. [Figure 13] FIG. 13 is a diagram showing an image of tracking a distant target in this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Hereinafter, embodiments of a program, an image processing device, and an image processing method disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.
[0013] In the following description, the image processing device 10 according to the embodiment is assumed to be an in-vehicle device mounted on a vehicle. The image processing device 10 is a device that is provided so as to be able to detect an object in an image by image recognition using AI for a camera image. In this embodiment, the object to be detected is assumed to be a vehicle traveling ahead of the vehicle.
[0014] Moreover, the image processing method according to the embodiment described below is executed by the controller 12 (see FIG. 4) included in the image processing device 10.
[0015] In the following, the term "predetermined" may be read as "predetermined", the term "target" may be read as "tracking object", and the term "tracking" may be read as "tracking".
[0016] In the following description, the nth frame (n is a natural number equal to or greater than 1) of the camera image corresponds to the "current" frame to be processed. Accordingly, the n-1th frame corresponds to the "previous" frame to be processed. Therefore, the n+1th frame corresponds to the "next" frame.
[0017] First, prior to describing the image processing method according to the embodiment, an overview of the image processing method according to the existing technology will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is an explanatory diagram of the overview of the image processing method according to the existing technology. Fig. 2 is a block diagram of averaging processing according to the existing technology.
[0018] As shown in FIG. 1, the most recent processing target, the nth frame (n is a natural number equal to or greater than 1) of the camera image 100 n From the front vehicle Bbox101 n Detected as this Bbox101 n There is an existing technology that tracks the vehicle distance D between frames and calculates the vehicle distance D successively.
[0019] In this existing technology, in order to reduce the variation in image recognition results in the time direction and to smoothly track the target, the detected Bbox101 n Then, an averaging process is performed using an IIR filter with the input Bbox(n) (step S2).
[0020] As shown in Fig. 2, the averaging process calculates the difference, which is the amount of change in the position and size of the input Bbox(n) relative to the output Bbox(n-1) after the averaging process of the previous frame (see "Calculate Difference" in the figure). The averaging process then multiplies this difference by a fixed coefficient to smooth it (see multiplier M1 in the figure).
[0021] The fixed coefficient is a strength coefficient that weights the strength of the averaging, and is a pre-designed fixed value. In addition, the averaging process adds the dulled difference to the output Bbox(n-1) (see adder A1 in the figure) and outputs it as the output Bbox(n).
[0022] Bbox102 shown in Figure 1 n-2 ,102 n-1 ,102 n is a trajectory of the bottom midpoint of this output Bbox(n) in the time direction (corresponding to the road contact point of the forward vehicle) from a bird's-eye view. The existing technology executes a viewpoint conversion process to convert the specific position of the output Bbox(n) to a bird's-eye view by perspective projection conversion (step S3). Then, the existing technology calculates the inter-vehicle distance D based on the specific position of the viewpoint-converted output Bbox(n).
[0023] In the existing technology, a tracking continuation determination process is executed prior to the averaging process (step S1). The tracking continuation determination process determines the degree of overlap between the input Bbox(n) and the output Bbox(n-1). The degree of overlap is IoU (Intersection over Union). The IoU is calculated by the Bbox prediction process (step S4).
[0024] In the tracking continuation determination process, if IoU is 0, it is determined that tracking cannot be continued, and so-called extrapolation processing is performed in which the predicted value of the input Bbox(n) calculated by the Bbox prediction processing is input to the averaging processing.
[0025] However, in the existing technology, as described above, the strength coefficient of the averaging process is set to a fixed value, which results in a problem that the tracking accuracy varies depending on the distance between the target and the vehicle V.
[0026] For example, distant targets are detected as Bboxes with smaller sizes than nearby targets.
[0027] Here, if the fixed coefficients are designed in advance to match the inter-frame variations of nearby Bboxes, the averaging strength of the distant Bboxes may be too weak to be corrected, making tracking impossible.On the other hand, if the fixed coefficients are designed in advance to match the variations of distant Bboxes, the averaging strength of the nearby Bboxes may be too strong, causing a response delay in the IIR filter, and the tracking accuracy may be impaired.
[0028] Therefore, in the image processing method according to the embodiment, when performing the averaging process in step S2, the strength coefficient of the averaging process is changed depending on the size of the output Bbox(n-1), which is the Bbox after the previous averaging process.
[0029] 3 is a block diagram of the averaging process according to the embodiment. In the image processing method according to the embodiment, when the Bbox averaging process is performed by the IIR filter, the strength of the averaging (i.e., the strength of the blurring) is dynamically changed according to the size of the output Bbox(n-1).
[0030] The closer a nearby target is to the vehicle V, the larger it is and the more stable the Bbox it is detected as. For this reason, it is considered that the influence of positional deviation on nearby targets is small. Therefore, when a relatively large Bbox is detected, the controller 12 changes the strength coefficient of the IIR filter so that the strength of averaging is reduced.
[0031] On the other hand, a distant target is detected as a smaller and more unstable Bbox the farther it is from the vehicle V. For this reason, it is considered that a distant target is significantly affected by positional deviation and size changes. Therefore, when a relatively small Bbox is detected, the controller 12 changes the strength coefficient of the IIR filter so that the strength of averaging is increased.
[0032] Specifically, as shown in FIG. 3, in the image processing method according to the embodiment, when performing the averaging process of step S2, a Bbox size calculation process (step S11), an adjustment coefficient calculation process (step S12), and a dynamic coefficient calculation process (step S13) are performed.
[0033] As mentioned above, when detecting a target, the closer the target is, the larger the Bbox is detected, and the farther away the target is, the smaller the Bbox is detected, so the distance of the target can be roughly determined by the size of the Bbox. Also, targets such as a vehicle ahead can usually be assumed not to move suddenly or change in size suddenly between frames.
[0034] Using such characteristics, the image processing method according to the embodiment changes the strength coefficient of the IIR filter linearly and dynamically from near to far depending on the size of the Bbox. By changing it linearly, it is possible to track the target smoothly and stably. Furthermore, by changing it dynamically, it is possible to suppress the variation of the Bbox regardless of the distance of the target. Furthermore, these make it possible to measure the vehicle distance D accurately.
[0035] In the Bbox size calculation process in step S11, the size of the Bbox is calculated in the image processing method according to this embodiment. Note that the Bbox size calculation process calculates the size of the input Bbox(n) for the first frame. Also, the Bbox size calculation process calculates the size of the output Bbox(n-1) for the second frame and onwards. Details of the Bbox size calculation process according to the embodiment will be described later with reference to FIG. 7.
[0036] In the adjustment coefficient calculation process in step S12, an adjustment coefficient indicating the position of the Bbox size within a predetermined size range is calculated. In other words, the adjustment coefficient calculation process calculates an adjustment coefficient indicating the ratio of the Bbox size to the predetermined size range. The adjustment coefficient is calculated so that the smaller the Bbox size is, the larger the value is, and the larger the Bbox size is, the smaller the value is. This makes it possible to calculate an adjustment coefficient according to the positioning of the Bbox size within the predetermined size range, and to calculate a dynamic coefficient suitable for the target Bbox size based on this adjustment coefficient. Details of the adjustment coefficient calculation process according to the embodiment will be described later with reference to FIG. 8.
[0037] In the dynamic coefficient calculation process in step S13, the strength coefficient of the IIR filter is calculated as a linear and dynamic dynamic coefficient based on the adjustment coefficient calculated in the adjustment coefficient calculation process. The strength coefficient is calculated so that the smaller the Bbox size is, the larger the value is, and the larger the Bbox size is, according to the adjustment coefficient. This makes it possible to set a dynamic coefficient suitable for the size of the target Bbox each time. In addition, the dynamic coefficient calculation process calculates the strength coefficient for each parameter related to the Bbox, for example, for each center position, width, and height of the Bbox, based on the adjustment coefficient. This makes it possible to set a dynamic coefficient according to each parameter constituting the target Bbox each time. Details of the dynamic coefficient calculation process according to the embodiment will be described later with reference to FIG. 9.
[0038] In the image processing method according to the embodiment, the averaging process in step S2 performs averaging using an IIR filter that receives as input the dynamic coefficients calculated by the dynamic coefficient calculation process, the input Bbox(n), and the output Bbox(n-1).
[0039] That is, the averaging process according to the embodiment calculates the difference between the output Bbox(n-1) after the averaging process of the previous frame and the input Bbox(n), and multiplies this difference by a dynamic coefficient instead of a fixed coefficient to dull the difference. Then, the averaging process adds the dulled difference to the output Bbox(n-1) and outputs it as the output Bbox(n). This makes it possible to execute a dynamic averaging process according to the size of the target Bbox. It also makes it possible to stably track the target regardless of whether the target is near or far away. The details of the averaging process according to the embodiment will be described later with reference to FIG. 6.
[0040] Thus, in the image processing method of the embodiment, when the controller 12 executes the averaging process of step S2, it changes the strength coefficient of the averaging process depending on the size of the output Bbox(n-1), which is the Bbox after the previous averaging process.
[0041] As a result, according to the image processing method of the embodiment, in tracking a target by image processing, it is possible to stabilize the tracking accuracy regardless of whether the target is far away or close.
[0042] An example of the configuration of the image processing device 10 to which the image processing method according to the above-described embodiment is applied will be described in more detail below.
[0043] Fig. 4 is a diagram showing an example of the configuration of an image processing device 10 according to an embodiment. As shown in Fig. 4, the image processing device 10 includes a storage unit 11 and a controller 12. In addition, a camera 3 and an external device 20 are connected to the image processing device 10.
[0044] The camera 3 is mounted on the vehicle V and is provided so as to be able to capture an image of the area in front of the vehicle V. Note that the camera 3 is not limited to the area in front of the vehicle V, and may be a 360-degree camera capable of capturing an image of the entire periphery of the vehicle V.
[0045] The external device 20 is various devices such as an ECU (Electronic Control Unit) that performs predetermined information processing based on the output from the image processing device 10. The external device 20 is a device that performs tailgating / tailgated determination of the vehicle V based on, for example, the inter-vehicle distance D output from the image processing device 10. The external device 20 is connected to the image processing device 10 via an in-vehicle network such as a CAN (Controller Area Network). The external device 20 may be wirelessly connected to the image processing device 10 via Bluetooth (registered trademark), Wi-Fi (registered trademark), UWB (Ultra Wide Band), or the like.
[0046] The image processing device 10 is, for example, a computer mounted on the vehicle V. The image processing device 10 executes at least the information processing of steps S11 to S13 and step S2 described with reference to FIG.
[0047] The storage unit 11 is realized by a storage device such as a RAM (Random Access Memory), a flash memory, etc. The storage unit 11 stores a program according to the embodiment executed by the controller 12. The storage unit 11 also stores various types of information used in the information processing executed by the controller 12.
[0048] The storage unit 11 stores, for example, an AI model for image recognition. The storage unit 11 also stores setting information for the adjustment coefficients described above and various types of threshold information used to calculate the dynamic coefficients.
[0049] The controller 12 corresponds to a so-called processor. The controller 12 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), or the like. The controller 12 reads a program according to the embodiment stored in the storage unit 11 and executes the program using the RAM as a working area. The controller 12 can also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).
[0050] The controller 12 executes information processing according to the processing procedure shown in the flowchart of Fig. 6. The explanation using Fig. 6 will be given later.
[0051] 4, the camera 3 and the image processing device 10 can be realized as a drive recorder 1. In this case, the camera 3 is realized by a camera unit mounted on the drive recorder 1. The image processing device 10 is realized by a microcomputer mounted on the drive recorder 1.
[0052] The configuration example shown in Fig. 4 is just one example, and other modified examples are also possible. Fig. 5 is a diagram showing a configuration example of an image processing device 10 according to a modified example. As shown in Fig. 5, the image processing device 10 can be realized as an image processing ECU 5. In this case, the camera 3 is realized by, for example, a camera unit mounted on the drive recorder 1. The camera 3 may also be realized by an in-vehicle camera unit other than the drive recorder 1.
[0053] 5, the camera 3 is connected to the image processing device 10 via an in-vehicle network such as a CAN. The camera 3 may be connected to the image processing device 10 wirelessly via Bluetooth, Wi-Fi, UWB, or the like.
[0054] Next, a procedure of information processing executed by the controller 12 of the image processing device 10 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the procedure of processing executed by the image processing device 10 according to the embodiment.
[0055] 6, the controller 12 acquires a camera image from the camera 3 (step S101). Next, the controller 12 detects a Bbox in which a target exists from the acquired camera image (step S102).
[0056] Next, the controller 12 executes the "tracking continuation determination process" described with reference to Fig. 1 (step S103). In the tracking continuation determination process, the controller 12 determines the IoU calculated in the Bbox prediction process in the above-mentioned step S4.
[0057] The controller 12 determines whether or not tracking can be continued based on the IoU (step S104). If the IoU is 0, the controller 12 determines that tracking cannot be continued (step S104, No), and executes the above-mentioned extrapolation process (step S105).
[0058] In the extrapolation process, the controller 12 inputs the predicted value of the input Bbox(n) calculated by the Bbox prediction process to the averaging process. If the IoU is other than 0, the controller 12 determines that tracking can be continued (Yes in step S104), and proceeds to step S106.
[0059] Then, the controller 12 executes the "Bbox size calculation process" (step S106) also shown in Fig. 3. Here, Fig. 7 is an explanatory diagram of the Bbox size calculation process according to the embodiment.
[0060] In the Bbox size calculation process, the controller 12 calculates the size of the Bbox by calculating the position P X ,P Y , width W and height H are calculated. X ,P Y, width W and height H are subject to filtering in the averaging process.
[0061] For the first frame, the controller 12 calculates the position X, Y, width W, and height H of the input Bbox(n). For the second frame and onward, the controller 12 calculates the position P of the output Bbox(n-1). X ,P Y , calculate the width W and height H.
[0062] Returning to the description of Fig. 6, the controller 12 then executes the "adjustment coefficient calculation process" (step S107) also shown in Fig. 3. Here, Fig. 8 is an explanatory diagram of the adjustment coefficient calculation process according to the embodiment.
[0063] In the adjustment coefficient calculation process, the controller 12 selects, for example, the smaller of the width W and height H calculated in step S106 as an index value indicating the size of the target Bbox. Then, the controller 12 calculates, as an adjustment coefficient, a ratio indicating to which position the index value corresponds within a predetermined range of Bbox sizes.
[0064] As shown in Fig. 8, the adjustment coefficient is defined in the range of 0 to 1.0. The adjustment coefficient is defined linearly within the range of the Bbox size defined by the minimum and maximum values, so that the smaller the Bbox size, the larger the value becomes. The formula is expressed as the following formulas (1) and (2).
[0065] (1) Bbox size = Min(Max(Bbox size, minimum), maximum) (2) Adjustment factor = 1.0 - (Bbox size - minimum) / (maximum - minimum)
[0066] 8 shows an example in which the adjustment coefficient is calculated as 0.9 when the Bbox size is 50, which is relatively small, within the range of Bbox sizes defined by minimum value = 40 and maximum value = 140. The example of setting the numerical values shown in FIG. 8 is merely one example.
[0067] Returning to the explanation of Fig. 6, the controller 12 then executes the "dynamic coefficient calculation process" also shown in Fig. 3 (step S108). Here, Fig. 9 is an explanatory diagram of the dynamic coefficient calculation process according to the embodiment.
[0068] In the dynamic coefficient calculation process, the controller 12 calculates the dynamic coefficient of the IIR filter in the averaging process by using the adjustment coefficient calculated in step S107. X ,P Y and the dynamic coefficient for each size (width W, height H).
[0069] The controller 12 is set to a position P X ,P Y The dynamic coefficients are calculated by the following formulas (3) to (5) within a range of a first threshold and a second threshold (first threshold>second threshold) that are preset for each of the size and the number of pixels. Note that the numerical example shown in FIG. 9 is merely an example.
[0070] (3) position P X Dynamic coefficient of = position P X The first threshold value of -(position P X First threshold - position P X (2nd threshold) * adjustment factor (4) position P Y Dynamic coefficient of = position P Y The first threshold value of -(position P Y First threshold - position P Y (2nd threshold) * adjustment factor (5) Dynamic size factor = 1st threshold size - (1st threshold size - 2nd threshold size) * adjustment factor
[0071] For example, the further away the target is, the smaller the Bbox size is, so the adjustment coefficient calculated by the adjustment coefficient calculation process is larger. And, the larger the adjustment coefficient is, the closer the dynamic coefficient calculated by the dynamic coefficient calculation process is to the second threshold, which is the smaller threshold, so the strength of averaging (i.e., the strength of dulling) is greater.
[0072] Returning to the description of FIG. 6, the controller 12 then executes "averaging processing" using an IIR filter (step S109). In step S109, the controller 12 calculates the position P X ,P Y The width W and height H are calculated using the following formulas (6) to (9), respectively.
[0073] (6) Position P of output Bbox(n) X = Position P of output Bbox(n-1) X + (position P of input Bbox(n) X - Position P of output Bbox(n-1) X )*Position P X Dynamic coefficient of (7) Position P of output Bbox(n) Y = Position P of output Bbox(n-1) Y + (position P of input Bbox(n) Y - Position P of output Bbox(n-1) Y )*Position P Y Dynamic coefficient of (8) Width W of output Bbox(n) = Width W of output Bbox(n-1) + (Width W of input Bbox(n) - Width W of output Bbox(n-1)) * Dynamic coefficient of size (9) Height H of output Bbox(n) = Height H of output Bbox(n-1) + (Height H of input Bbox(n) - Height H of output Bbox(n-1)) * Dynamic coefficient of size
[0074] Then, the controller 12 executes the "viewpoint conversion process" also shown in Fig. 1 (step S110). In the viewpoint conversion process, the controller 12 converts the specific position of the output Bbox(n) output from step S109 into a bird's-eye viewpoint by perspective projection conversion. Then, the controller 12 calculates the inter-vehicle distance D based on the specific position of the output Bbox(n) after the viewpoint conversion.
[0075] Then, the controller 12 outputs the inter-vehicle distance D calculated in step S110 to the external device 20 (step S111), and determines whether the process is finished (step S112). An event indicating the process is finished is, for example, the start-up of the vehicle V. An event indicating the process is finished may include a case where the extrapolation process is repeated a predetermined number of times or more.
[0076] If it is determined that the process has not ended (step S112, No), the controller 12 repeats the process from step S101. If it is determined that the process has ended (step S112, Yes), the controller 12 ends the process.
[0077] Next, an image of the case where the existing technology and this embodiment are applied to a nearby target and a distant target, respectively, will be described with reference to FIGS.
[0078] Fig. 10 is a diagram showing an image when the existing technology and this embodiment are applied to a nearby target. Fig. 11 is a diagram showing an image when the existing technology and this embodiment are applied to a distant target. Fig. 12 is a diagram showing an image when a distant target is tracked using the existing technology. Fig. 13 is a diagram showing an image when a distant target is tracked using this embodiment.
[0079] In addition, in FIG. 10 to FIG. 13, Bbox101 n corresponds to the input Bbox(n) detected in the nth frame (i.e., this time). n-1corresponds to the output Bbox(n-1) output after the averaging process in the n-1th frame (i.e., the previous frame).
[0080] Bbox102 n corresponds to the output Bbox(n) that is output after averaging processing in the nth frame. n+1 corresponds to the input Bbox(n) detected in the n+1th frame (that is, the next frame).
[0081] First, when the existing technology and the present embodiment are applied to a nearby target, the image recognition results by the AI are relatively stable in the vicinity, so there is almost no noticeable difference between using fixed coefficients and using dynamic coefficients. Therefore, as shown in FIG. 10, Bbox102 output after averaging processing in the nth frame n The Bbox 102 in the existing technology and this embodiment n-1 The position and size change in almost the same way.
[0082] In contrast, when applying existing technology to a distant target, for example, if the fixed coefficient is small, the strength of averaging is insufficient, and as shown in the left diagram of Figure 11, Bbox102 n-1 Bbox102 n The amount of change in position and size is large.
[0083] On the other hand, when this embodiment is applied to a distant target, Bbox102 n-1 The dynamic coefficient is adjusted to be larger depending on the size of Bbox102. n-1 Bbox102 n The amount of change in the position and size of the output Bbox(n) is smaller than when the existing technology is applied. This makes it possible to suppress abrupt changes in the time direction of the output Bbox(n). This also makes it possible to stabilize the measurement of the inter-vehicle distance D, for example.
[0084] In addition, when the existing technology is applied to a distant target, as shown in the left diagram of Figure 11, Bbox102 n-1 and Bbox101 n If there is no overlap between (i.e., IoU is 0), then the extrapolation process is performed as described above.
[0085] In addition, as shown in Figure 12, Bbox101 n+1 Again Bbox102 n-1 It is assumed that the target is detected in the vicinity of the target. In this case, extrapolation processing will be performed in the n+1th frame as well. If the extrapolation processing is repeated, there is a high possibility that the Bbox will deviate from the true target, and the extrapolation processing will not be able to fully correct the target, which may result in the target being completely lost.
[0086] In contrast, as shown in FIG. 13, when this embodiment is applied, Bbox101 is n+1 Again Bbox102 n-1 Even if it is detected nearby, Bbox102 n For Bbox101 n+1 The IoU of is highly likely to be other than 0. Therefore, when this embodiment is applied, it is possible to reduce the opportunities for extrapolation processing, and it is possible to stably track a target.
[0087] As described above, the program according to the embodiment detects a target (corresponding to an example of a "tracking object") in the Bbox 101 by image recognition of a camera image. n (An example of a "partial region"), Bbox101 n When applying the IIR filter, the Bbox102 after the previous averaging process is applied. n-1 The computer is caused to execute the above. In this way, the image processing device 10 changes the coefficients of the IIR filter according to the size of, for example, Bbox 102. n-1It is possible to track targets by increasing the averaging strength for distant targets and decreasing the averaging strength for nearby targets according to the size of the target. In other words, the image processing device 10 that executes the program according to the embodiment can stabilize the tracking accuracy in target tracking by image processing regardless of the distance of the target.
[0088] In the above embodiment, the target is a vehicle ahead, but the target is not limited to a vehicle. For example, the target may be a person or the like.
[0089] In the above-described embodiment, the image processing device 10 is an in-vehicle device, but the image processing device 10 is not limited to an in-vehicle device. For example, the image processing device 10 may be a computer such as a smartphone or a PC (Personal Computer) used by a user.
[0090] Accordingly, the program according to the embodiment is not limited to one that outputs the vehicle distance D as an output result, and may output, for example, the trajectory of the target as an output result. Similarly, the program according to the embodiment may be, for example, viewer software that tracks the target and successively displays the state of the target.
[0091] In the above embodiment, the index value indicating the size of the target Bbox is, for example, the smaller of the width W and height H, but the larger may be selected. Also, this index value may be the area of the target Bbox.
[0092] In addition, in the above-described embodiment, the first threshold and the second threshold when calculating the dynamic coefficient are common to the width W and the height H, but they may be separate for the width W and the height H.
[0093] Further advantages and modifications may readily occur to those skilled in the art. Thus, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof. [Explanation of symbols]
[0094] 1. Drive recorder 3. Camera 5 Image processing ECU 10 Image processing device 11 Storage section 12 Controller 20 External device D Following distance H Height P center position W width
Claims
1. Detecting a partial area in which a tracking object exists by image recognition of a camera image; applying a filter that performs averaging processing in a time direction to the partial region; changing a coefficient of the filter in accordance with a size of the partial region after the previous averaging process when applying the filter; A program that causes a computer to execute the following.
2. In the step of changing the coefficient, an intensity coefficient indicating an intensity of averaging in the averaging process is dynamically changed. The program according to claim 1.
3. In varying the coefficient, the intensity coefficient is varied linearly. The program according to claim 2.
4. In varying the coefficient, the intensity coefficient is varied so that the intensity becomes smaller as the size of the partial region becomes larger. The program according to claim 3.
5. In varying the coefficient, calculating an adjustment coefficient indicating a position of the size of the partial region within a predetermined size range, and calculating the strength coefficient based on the adjustment coefficient; The program according to claim 4.
6. In changing the coefficient, the intensity coefficient for each parameter related to the partial region is calculated based on the adjustment coefficient. The program according to claim 5.
7. The parameters include a center position, a width, and a height of the partial region. The program according to claim 6.
8. The filter comprises: executing the averaging process on the currently detected partial region by multiplying the intensity coefficient by the amount of change from the partial region after the previous averaging process and adding the result to the partial region after the previous averaging process; The program according to claim 6.
9. An image processing device comprising: The controller: Detects a partial area where a tracking target exists by image recognition on the camera image, A filter that performs averaging processing in the time direction is applied to the partial region; When applying the filter, a coefficient of the filter is changed according to a size of the partial region after the previous averaging process. Image processing device.
10. An image processing method executed by an image processing device, comprising: Detecting a partial region in which a tracking object exists by image recognition of a camera image; applying a filter for performing averaging processing in a time direction to the partial region; changing a coefficient of the filter in accordance with a size of the partial region after the previous averaging process when applying the filter; An image processing method comprising:
Citation Information
Patent Citations
Image processor and image processing method, program, and recording medium
JP2007272735A
