New object detection device and method

Abstract

A new object detection device comprises an image receiving circuit, a memory and a processor. The processor is configured to execute the following steps: performing an object recognition process on a first image of the plurality of images to define an initial object position from the first image; detecting the plurality of images to generate a plurality of optical flow diagrams corresponding to the plurality of images; and for the plurality of optical flow diagrams, executing the following steps: identifying a plurality of outlier regions outside the position of the initial object; generating at least one candidate region based on the plurality of image positions of the plurality of outlier regions; and identifying at least one new object corresponding to the plurality of images based on at least one candidate area corresponding to the plurality of second images. The invention further relates to a new object detection method.

Description

Technical Field

[0001] The present disclosure relates to an image processing technology, and more particularly to a new object detection device and method. Background Art

[0002] In current image processing technology, various machine learning techniques are often used to perform object recognition on images. Although this approach has high accuracy, when a large number of continuous images need to be processed for object recognition, it will impose a considerable burden on both hardware and software resources. In addition, this approach also requires a lot of time to perform object recognition on a large number of continuous images. Therefore, how to save computing resources and time while maintaining a certain recognition accuracy is a problem that technicians in this field are eager to solve. Summary of the invention

[0003] The present disclosure proposes a new object detection device, including an image receiving circuit, a memory and a processor. The image receiving circuit is used to receive a plurality of continuous images, wherein the plurality of images include a first image and a plurality of second images after the first image. The memory is used to store a plurality of instructions. The processor is used to process the plurality of instructions and execute the following steps: perform object recognition processing on the first image to define an initial object position from the first image; detect the plurality of images to generate a plurality of optical flow maps corresponding to the plurality of images, wherein each of the plurality of optical flow maps corresponds to one of the plurality of second images; and for each of the plurality of optical flow maps, execute the following steps: identify a plurality of outlier regions outside the initial object position; and generate at least one candidate region based on the plurality of image positions of the plurality of outlier regions; and identify at least one new object corresponding to the plurality of images based on the at least one candidate region corresponding to the plurality of second images.

[0004] The present disclosure also proposes a new object detection method, comprising: receiving a plurality of continuous images, wherein the plurality of images include a first image and a plurality of second images following the first image; performing object recognition processing on the first image to define an initial object position from the first image; detecting the plurality of images to generate a plurality of optical flow maps corresponding to the plurality of images, wherein each of the plurality of optical flow maps corresponds to one of the plurality of second images; and for each of the plurality of optical flow maps, performing the following steps: identifying a plurality of outlier regions outside the initial object position; and generating at least one candidate region based on the plurality of image positions of the plurality of outlier regions; and identifying at least one new object corresponding to the plurality of images based on the at least one candidate region corresponding to the plurality of second images. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 A schematic diagram showing a new object detection device in some embodiments of the present disclosure.

[0006] Figure 2 A flow chart of a method for detecting a new object in some embodiments of the present disclosure is shown.

[0007] Figure 3 A schematic diagram showing object frames of multiple objects in a first image in some embodiments of the present disclosure.

[0008] Figure 4 A schematic diagram showing multiple object masks in a second image in some embodiments of the present disclosure.

[0009] Figure 5 A schematic diagram showing multiple optical flow directions of an optical flow graph in some embodiments of the present disclosure.

[0010] Figure 6 A schematic diagram showing the horizontal positions and horizontal components of multiple optical flows in some embodiments of the present disclosure.

[0011] Fig. 7A A schematic diagram showing outlier regions of multiple outlier optical flows in some embodiments of the present disclosure.

[0012] Figure 7B A schematic diagram showing multiple candidate clusters in some embodiments of the present disclosure.

[0013] Figure 8 A schematic diagram showing outlier directions of multiple outlier optical flows in some embodiments of the present disclosure.

[0014] Fig. 9 A schematic diagram showing multiple outlier directions in some embodiments of the present disclosure.

[0015] Fig.10 A schematic diagram showing multiple image groups in some embodiments of the present disclosure.

[0016] Description of reference numerals:

[0017] 100: New object detection device

[0018] 110: Image receiving circuit

[0019] 120: Memory

[0020] 130: Processor

[0021] S210~S250: Steps

[0022] IMG1: First Image

[0023] OJ1~OJ3:Object frame

[0024] IMG2~IMG4: Second image

[0025] MK1~MK3: Object Mask

[0026] OF: Optical Flow Map

[0027] R1~R4: Area

[0028] DL1~DL2: dividing line

[0029] OFD1~OFD4, M_OFD: Optical flow direction

[0030] A~E: optical flow

[0031] d1~d5: horizontal component difference

[0032] HRL: Horizontal component regression line

[0033] BM: Bitmap

[0034] CL1~CL2: Candidate clusters

[0035] O_OFD1~O_OFD3: Outlier direction

[0036] IR: Object Recognition Processing

[0037] OFM: A method for identifying new objects based on optical flow

[0038] IMGG(1)~IMGG(N): Image group DETAILED DESCRIPTION

[0039] In view of the fact that the current general object recognition algorithms may consume too much computing resources due to the complexity of the image scenes and the excessive number of images. In addition, vehicles often need to detect vehicles or pedestrians that suddenly appear in front of the vehicle in real time while driving, which also requires a real-time and fast detection method. To solve the above problems, the present disclosure proposes a new object detection device and method, which performs object recognition only on the first image, and identifies the possible position of the new object from the optical flow map of the subsequent images, so as to further perform object recognition at this position. The above-mentioned technology of the present disclosure is specifically described by taking the following embodiments as an example.

[0040] Reference Figure 1 , Figure 1 FIG. 1 is a schematic diagram showing a new object detection device 100 in some embodiments of the present disclosure. Figure 1 As shown, the new object detection device 100 includes an image receiving circuit 110 , a memory 120 , and a processor 130 . The processor 130 is connected between the image receiving circuit 110 and the memory 120 .

[0041] In some embodiments, the new object detection device 100 may be a device having an image processor, or a device for providing image processing functions, such as a computer, a server, or a processing center, etc., which is not limited here.

[0042] In this embodiment, the image receiving circuit 110 is used to receive an image. In some embodiments, the image may be an image taken by a photographic device (not shown) disposed in front of any vehicle (e.g., a vehicle). In some embodiments, the image receiving circuit 110 may be implemented by a communication interface (e.g., a Wi-Fi communication interface) for communicating with other devices or systems.

[0043] Furthermore, the memory 120 is used to store a plurality of instructions, which are used to execute the detailed steps described in the subsequent paragraphs. In some embodiments, these instructions can be corresponding software or firmware instruction programs. In some embodiments, the memory 120 can be implemented by a storage unit, a flash memory, a read-only memory, a hard disk, or any equivalent storage component.

[0044] Furthermore, the processor 130 is used to process these instructions and execute the detailed steps described in the following subsequent paragraphs. In some embodiments, the processor 130 can be implemented by a processing unit, a central processing unit or a computing unit.

[0045] Refer to Figure 2 , Figure 2 A flowchart showing a method for detecting a new object in some embodiments of the present disclosure is shown. The method for detecting a new object in some embodiments of the present disclosure is applicable to an electronic device. The electronic device may be Figure 1 The newly-arrived object detection device 100, Figure 1 The components in the new object detection device 100 are used to perform steps S210 to S250 in the new object detection method. Figure 2 As shown, first, in step S210 , the image receiving circuit 110 receives a plurality of continuous images, wherein the plurality of images include a first image and a plurality of second images following the first image.

[0046] In some embodiments, the plurality of images are images of a plurality of frames taken continuously. For example, the first image may be an image of the first frame, and the plurality of second images may be images of the second to fifth frames. In another example, the first image may be an image of the sixth frame, and the plurality of second images may be images of the seventh to tenth frames.

[0047] Furthermore, in step S220 , the processor 130 performs object recognition processing on the first image to define an initial object position from the first image.

[0048] In some embodiments, the object recognition process may be any object recognition algorithm, such as a deep learning algorithm or a YOLO (you only look once) algorithm. In some embodiments, the processor 130 may perform object recognition processing on the first image to identify the position of a specific object (e.g., a vehicle) in the first image as the initial object position. In some embodiments, the initial object position may be a position corresponding to the center point of the specific object and its length and width. In some embodiments, the processor 130 may store the pixel coordinate of the center point of the specific object and the length and width of the specific object in the memory 120 as the initial object position. In other embodiments, the processor 130 may also store the pixel positions of the upper left end point and the lower right end point of the specific object in the memory 120 as the initial object position. In other embodiments, the processor 130 may also store the pixel positions of the upper right end point and the lower left end point of the specific object in the memory 120 as the initial object position. In other embodiments, the processor 130 may also store the pixel positions of the upper right end point and the lower left end point of the specific object in the memory 120 as the initial object position.

[0049] Although the initial object position of a specific object is used as an example here, in actual situations, when there are multiple objects in the first image, the same method can be used to generate multiple initial object positions from the first image. It is worth noting that object recognition processing usually requires a large amount of computing, but has extremely high accuracy. Therefore, in order to save computing resources and computing time, object recognition processing will only be performed on the first image. In addition, for the subsequent second image, based on the high-accuracy initial object position generated by the object recognition processing, optical flow is used to detect new objects (i.e., new objects appearing in the image).

[0050] In some embodiments, the processor 130 may perform object recognition processing on the first image to identify the bounding box of the specific object, and expand the size of the object box to generate an enlarged object box (for example, expand the size by 10%). Then, the processor 130 may generate an object mask of the same size as the enlarged object box, and cover the object mask on the initial object position on the plurality of second images. In other words, in subsequent steps, the optical flow will not be calculated for the pixels covered by the object mask. In other embodiments, the object mask may not be covered on the plurality of second images to calculate the optical flow maps corresponding to the second images.

[0051] The following is an actual example to illustrate the initial object position on the first image. Figure 3 , Figure 3 FIG. 1 is a schematic diagram showing object frames OJ1 to OJ3 of multiple objects in a first image IMG1 in some embodiments of the present disclosure. Figure 3 As shown, assuming that a vehicle is to be identified from the first image IMG1, object recognition processing can be performed on the first image IMG1 to identify multiple vehicles (i.e., objects) and object frames OJ1~OJ3 of these vehicles, and the positions of the center points of the object frames OJ1~OJ3 in the first image IMG1 are used as multiple initial object positions.

[0052] The following is an actual example to illustrate the object mask of the second image. Figure 4 , Figure 4 FIG. 4 is a schematic diagram showing a plurality of object masks MK1 to MK3 in a second image IMG2 in some embodiments of the present disclosure. Figure 3 as well as Figure 4 As shown, continued Figure 3 The example can be slightly expanded Figure 3 The object frames OJ1-OJ3 are sized to generate object masks MK1-MK3 respectively, and the object masks MK1-MK3 are covered at the pixel positions of the object frames OJ1-OJ3 on the second image IMG2 (i.e., the initial object positions mentioned above). For example, when the pixel position of the object frame OJ1 in the first image IMG1 is (123,456), the object mask MK1 can be covered at the pixel position (123,456) on the second image IMG2.

[0053] Furthermore, if Figure 2 As shown, in step S230, the processor 130 detects the plurality of images to generate a plurality of optical flow maps corresponding to the plurality of images. In some embodiments, each of the plurality of optical flow maps corresponds to one of the plurality of second images.

[0054] For example, assuming that the first image is an image of the first frame and the multiple second images are images of the second frame to the fifth frame, an optical flow map corresponding to the image of the second frame can be calculated between the image of the first frame and the image of the second frame, an optical flow map corresponding to the image of the third frame can be calculated between the image of the second frame and the image of the third frame, an optical flow map corresponding to the image of the fourth frame can be calculated between the image of the third frame and the image of the fourth frame, and an optical flow map corresponding to the image of the fifth frame can be calculated between the image of the fourth frame and the image of the fifth frame.

[0055] In some embodiments, when the processor 130 covers the object mask on the initial object position on the plurality of second images, the processor 130 may generate a plurality of optical flow maps from areas outside the area covered by the object mask on the plurality of second images. In other embodiments, the processor 130 may directly generate a plurality of optical flow maps from areas outside the area of ​​the initial object position on the plurality of second images. In other embodiments, the processor 130 may perform object tracking on the plurality of second images according to the initial object position to generate object tracking positions of the plurality of second images respectively. Then, the processor 130 may generate a plurality of optical flow maps from areas outside the area of ​​the object tracking positions of the plurality of second images respectively. In detail, the processor 130 may perform object tracking on the second images based on the specific object at the initial object position of the first image to generate the respective object tracking positions of the specific object in the second images. Then, the processor 130 generates a plurality of optical flow maps for areas outside the area of ​​the object tracking positions of the second images respectively.

[0056] Furthermore, in step S240 , the processor 130 performs the following steps for the plurality of optical flow maps: identifying a plurality of outlier regions outside the initial object position, and generating at least one candidate region based on a plurality of image positions of the plurality of outlier regions.

[0057] In some embodiments, the processor 130 may also cover the above-mentioned object mask on the initial object position of these optical flow maps after generating multiple optical flow maps to identify multiple outlier areas outside the area covered by the object mask of each of the multiple optical flow maps. In other embodiments, the processor 130 may also directly identify multiple outlier areas outside the initial object position of each of the multiple optical flow maps after generating multiple optical flow maps. In other embodiments, the processor 130 may also directly perform object tracking on multiple second images according to the initial object position after generating multiple optical flow maps to generate the object tracking position of each of the multiple second images. Then, the processor 130 may perform the following steps for each of the multiple optical flow maps: identify multiple outlier areas outside the multiple object tracking positions.

[0058] In some embodiments, the processor 130 may perform the following steps for each of the plurality of optical flow graphs: based on at least one optical flow direction component, perform a regression analysis algorithm and find at least one outlier to generate a plurality of outlier regions from the plurality of optical flows, and perform a clustering algorithm based on the plurality of outlier regions to generate at least one candidate region. In some embodiments, the at least one optical flow direction component may include a horizontal (i.e., X direction) direction component and a vertical (i.e., Y direction) direction component. It is worth noting that the recognition using the above-mentioned outlier optical flow is often prone to noise, and therefore, the above-mentioned clustering algorithm is used to prevent misjudgment of noise.

[0059] In some embodiments, the processor 130 may execute a regression analysis algorithm for each of the multiple optical flow maps, including the steps of: calculating a horizontal component regression line based on the horizontal components of the multiple optical flows, and calculating an average value of the multiple horizontal component differences between the horizontal components of the multiple optical flows and the horizontal component regression line, finding out a plurality of outlier optical flows that have a large difference from the average value from the multiple optical flows, and then executing a clustering algorithm on the outlier regions of the respective outlier optical flows to generate at least one candidate region.

[0060] In some embodiments, the processor 130 may calculate a horizontal component regression line from the horizontal components of the plurality of optical flows using any linear regression algorithm. In some embodiments, the horizontal component difference corresponding to the outlier optical flow may be greater than a preset component difference threshold. In other embodiments, the processor 130 may execute a regression analysis algorithm for each of the plurality of optical flow graphs, including: executing any outlier detection (e.g., a local outlier factor (LOF) algorithm or a random cut forest (RCF) algorithm, etc.) according to the horizontal components of the plurality of optical flows to select a plurality of outlier optical flows from the plurality of optical flows.

[0061] It is worth noting that although the horizontal component of the optical flow is used as an embodiment above, in fact, the same calculation as above is also performed based on the vertical component of the optical flow to select the outlier optical flow. Therefore, the vertical direction of the optical flow is not repeated. In other words, as long as the horizontal component or vertical direction of any optical flow is an outlier, this optical flow will be regarded as an outlier optical flow.

[0062] In some embodiments, the processor 130 may divide each of the multiple optical flow maps into multiple sub-optical flow maps, and generate at least one candidate area for each of the multiple sub-optical flow maps. For example, each optical flow map may be divided into four sub-optical flow maps (or other numbers of sub-optical flow maps), and then outlier optical flows are found for each of these sub-optical flow maps. In another example, each optical flow map may be divided into four sub-optical flow maps using the vanishing point of each optical flow map as the center point, for example, the area below the vanishing point includes the ground, the part above the vanishing point includes the sky, and the image of the area to the left of the vanishing point is moving to the left, and the image of the area to the right of the vanishing point is moving to the right. Furthermore, the method of finding outlier optical flows for each of these sub-optical flow maps can distinguish the optical flow properties of each area in advance, thereby improving the accuracy of finding outliers.

[0063] The following uses an actual optical flow graph to illustrate the direction of optical flow and the calculation of the partitions of the optical flow graph. Figure 5 , Figure 5 Schematic diagram showing multiple optical flow directions OFD1 to OFD4 of an optical flow graph OF in some embodiments of the present disclosure. Figure 5 As shown, the optical flow graph OF is an optical flow graph corresponding to the image in front of the vehicle taken when the vehicle is moving forward, and the optical flow graph OF can be divided into four regions R1 to R4 by dividing lines DL1 to DL2, and the regions R1 to R4 have optical flow directions OFD1 to OFD4 respectively. In other words, the direction of the optical flow in region R1 is all optical flow direction OFD1, the direction of the optical flow in region R2 is all optical flow direction OFD2, the direction of the optical flow in region R3 is all optical flow direction OFD3, and the direction of the optical flow in region R4 is all optical flow direction OFD4.

[0064] In detail, the direction of the horizontal component of the optical flow in region R1 is the direction of the horizontal component of the optical flow direction OFD1, and the direction of the vertical component of the optical flow in region R1 is the direction of the vertical component of the optical flow direction OFD1. The direction of the horizontal component of the optical flow in region R2 is the direction of the horizontal component of the optical flow direction OFD2, and the direction of the vertical component of the optical flow in region R2 is the direction of the vertical component of the optical flow direction OFD2. Similarly, the direction of the horizontal component and the direction of the vertical component of the optical flow can also be found from regions R3 to R4. Assuming that the horizontal component of the optical flow is positive to the right and negative to the left, the optical flow closer to the right of the optical flow map OF has a higher horizontal component, and the optical flow closer to the left of the optical flow map OF has a lower horizontal component. Assuming that the vertical component of the optical flow is positive downward and negative upward, the optical flow closer to the bottom of the optical flow map OF has a higher vertical component, and the optical flow closer to the top of the optical flow map OF has a lower vertical component. However, when a rear vehicle (ie, a new object) appears in front of the vehicle (eg, the rear vehicle intends to overtake), the above-mentioned outlier optical flow will appear in the optical flow map OF.

[0065] In addition, outlier optical flows in regions R1 to R4 can be found for regions R1 to R4 respectively. Further, when the rear vehicle overtakes, outlier optical flows often appear in regions R3 to R4, and only in a few cases will outlier optical flows appear in regions R1 to R2. For example, when the vehicle itself encounters an uphill or downhill slope in front, the vanishing point positions may be different, which may cause the area of ​​R1 to R4 to be different. At this time, it is possible that outlier optical flows temporarily appear in regions R1 to R2. In addition, when a vehicle turns out from the alley in front of the vehicle, a vehicle that was originally invisible in the distance becomes larger and visible due to approaching, or an object that was originally blocked by other objects (other vehicles, buildings) at any position of the picture and then appears in the picture, it is also possible that outlier optical flows temporarily appear in regions R1 to R2. It is worth noting that since outlier optical flows often appear in regions R3 to R4, outlier optical flows in regions R1 to R4 can also be found for regions R3 to R4 only.

[0066] For ease of explanation, the following uses a simple example of the horizontal component of optical flow to illustrate the outlier optical flow. Figure 6 , Figure 6 Schematic diagram showing the horizontal positions and horizontal components of multiple optical flows A to E in some embodiments of the present disclosure. Figure 6 As shown, from the horizontal positions of optical flows A to E, it can be known that optical flows A to E are optical flows arranged in order to the right of the optical flow graph, and each of them has a horizontal component. Then, a horizontal component regression line HRL can be calculated based on the horizontal components of optical flows A to E using a linear regression algorithm, and multiple horizontal component differences d1 to d5 between the horizontal direction components of optical flows A to E and the horizontal component regression line are calculated, wherein the horizontal component differences d1 and d3 to d5 are not greater than the preset component difference threshold, and the horizontal component difference d2 is greater than the preset component difference threshold. Since the horizontal component difference d2 is greater than the preset component difference threshold, optical flow B can be used as an outlier optical flow.

[0067] For ease of explanation, the following uses a simple example of outlier optical flow to illustrate the candidate region. Fig. 7A as well as Figure 7B , Fig. 7A A schematic diagram showing outlier regions of multiple outlier optical flows in some embodiments of the present disclosure, Figure 7B FIG. 4 is a schematic diagram showing a plurality of candidate clusters CL1 to CL2 in some embodiments of the present disclosure. Fig. 7A as well as Figure 7BAs shown, it is assumed that the optical flow map is converted into a bitmap BM, wherein the regions corresponding to the multiple image positions of the multiple outlier regions in the optical flow map in the bitmap BM are all 1. Then, a clustering algorithm can be performed on these outlier regions with bit values ​​of 1 to generate multiple candidate clusters CL1-CL2. In this way, the candidate regions corresponding to the candidate clusters CL1-CL2 can be marked on the second image.

[0068] The following is an actual example of candidate regions. Figure 8 , Figure 8 Schematic diagram showing outlier directions O_OFD1 of multiple outlier optical flows in some embodiments of the present disclosure. Figure 8 As shown, most of the optical flows in the second image IMG2 have an optical flow direction M_OFD, and the outlier optical flows in the second image IMG2 have an outlier direction O_OFD1. In addition, the clustering algorithm can be used to identify the outlier region of the outlier optical flow with the outlier direction O_OFD1 as a candidate region. Therefore, the image position of the candidate region in the second image IMG2 may have a newly appeared vehicle (i.e., a newly entered object).

[0069] Furthermore, if Figure 2 As shown, in step S250 , the processor 130 identifies at least one new object corresponding to the plurality of images based on at least one candidate region corresponding to the plurality of second images.

[0070] In some embodiments, the processor 130 may perform object recognition processing on at least one candidate region corresponding to the plurality of second images to recognize at least one new object corresponding to the plurality of images.

[0071] In some embodiments, the processor 130 may perform the following steps for the third image and the fourth image in the plurality of second images: Calculate the overlapping area based on the candidate area of ​​the third image and the candidate area of ​​the fourth image, wherein the third image is an adjacent image of the fourth image. Then, the processor 130 may identify at least one new object corresponding to the plurality of images based on the overlapping area. In detail, the overlap rate and the overlapping area between the candidate areas of two adjacent images in the plurality of second images may be calculated. Then, when the overlap rate is greater than a preset overlap threshold, object recognition processing may be performed on the overlapping area of ​​the image of the later frame of the two images to identify the new object. For example, as long as the overlap ratio between the candidate areas of the two adjacent images is higher than the overlap threshold, they may be determined to be the same candidate areas.

[0072] In some embodiments, the processor 130 may select a corresponding one of the at least one candidate region based on the number of occurrences of each corresponding one of the at least one candidate region, wherein the corresponding one of the at least one candidate region has a number of occurrences greater than a number threshold. Then, the processor 130 may identify at least one new object corresponding to the plurality of images based on the corresponding one of the at least one candidate region.

[0073] For example, assuming that the number of occurrences of a candidate region in the five optical flow maps corresponding to the five second images is 3, and the number threshold is 2, this candidate region can be used as the corresponding one of the at least one candidate region, and object recognition processing can be performed on it. In other words, as long as the number of occurrences of the candidate region is greater than 2, object recognition processing can be performed on the corresponding one of the at least one candidate region.

[0074] In other embodiments, the processor 130 may select a corresponding one of the at least one candidate area based on the appearance ratio of each corresponding one of the at least one candidate area, wherein the corresponding one of the at least one candidate area has an appearance ratio greater than an appearance ratio threshold. Then, the processor 130 may identify at least one new object corresponding to the plurality of images based on the corresponding one of the at least one candidate area.

[0075] For example, assuming that the appearance ratio of a candidate region in the five optical flow maps corresponding to the five second images is 60%, and the appearance ratio threshold is 50%, this candidate region can be used as the corresponding one in the at least one candidate region, and object recognition processing can be performed on it. In other words, as long as the appearance ratio of the candidate region is greater than 50%, object recognition processing can be performed on the corresponding one in the at least one candidate region.

[0076] In some embodiments, the processor 130 may select a corresponding one of the at least one candidate region based on the number of occurrences and the ratio of occurrences of each of the at least one candidate region, wherein the corresponding one of the at least one candidate region has a number of occurrences greater than a number threshold and a ratio of occurrence greater than a ratio threshold. Then, the processor 130 may identify at least one new object corresponding to the plurality of images based on the corresponding one of the at least one candidate region.

[0077] In some embodiments, the processor 130 may identify multiple valid optical flow maps from multiple optical flow maps, wherein the multiple valid optical flow maps include a corresponding one in at least one candidate region. Then, the processor 130 may select the last one of the multiple valid optical flow maps. Then, the processor 130 may perform object recognition processing on the second image corresponding to the last one of the multiple valid optical flow maps.

[0078] For example, assuming that the appearance ratio of a candidate area in the five optical flow maps corresponding to the five second images is 60% (i.e., the candidate area appears three times in the five optical flow maps), and the appearance ratio threshold is 50% and the quantity threshold is 2, the candidate area can be used as the counterpart of the at least one candidate area, and the object recognition processing can be performed on the second image corresponding to the valid optical flow map in which the candidate area appears last. In addition, the object recognition processing can also be performed directly on the area in the second image corresponding to the candidate area. In other words, as long as the appearance ratio of the candidate area is greater than 50% and the appearance quantity is greater than 2, the object recognition processing can be performed on the second image corresponding to the optical flow map in which the candidate area appears last.

[0079] In other words, once it is determined from the optical flow map that a new object exists in the second image, object recognition processing will be performed on the second image value where the new object may appear. In this way, there is no need to perform object recognition processing that consumes more computing resources on the image all the time, but the above-mentioned optical flow operation that consumes less resources is often used on the image. In this way, computing resources can be greatly reduced and a certain recognition accuracy can be maintained.

[0080] It is worth noting that since the size of the new object when it just appears in the second image may be too small, it is more difficult to identify the object. Therefore, the above-mentioned appearance ratio threshold and quantity threshold are set, and when the appearance ratio is greater than the ratio threshold and the appearance quantity is greater than the quantity threshold, it means that the new object has appeared for a long enough time and the size of the object in the second image is sufficient for object recognition processing.

[0081] The following is an actual example to illustrate the above-mentioned object recognition processing performed on the second image that last appears in the candidate area. Fig. 9 , Fig. 9 Schematic diagram showing multiple outlier directions O_OFD1 to O_FD3 in some embodiments of the present disclosure. Fig. 9 As shown, it is assumed that there are a plurality of continuous images, the plurality of images sequentially include a first image and three second images IMG2-IMG4, the second images IMG3-IMG4 are Figure 8 In the images after the second image IMG2 in the image, the optical flow maps of the second images IMG2 to IMG4 have outlier directions O_OFD1 to O_FD3. In these optical flow maps, the appearance ratio of the candidate regions with outlier directions O_OFD1 to O_FD3 is 100% and the appearance number is 3, wherein the appearance ratio is greater than the ratio threshold and the appearance number is greater than the number threshold. Therefore, object recognition processing can be performed on the region corresponding to the candidate region of the outlier direction O_FD3 in the second image IMG4 ranked last to identify the newly appeared vehicle (i.e., the newly entered object).

[0082] In some embodiments, the processor 130 may calculate an area ratio of each of at least one candidate region corresponding to the plurality of second images and the object frame corresponding to the first image to select a corresponding one of the at least one candidate region, wherein the corresponding one of the at least one candidate region has an area ratio greater than an area ratio threshold. Then, the processor 130 may identify at least one new object corresponding to the plurality of images based on the corresponding one of the at least one candidate region.

[0083] Specifically, the object frame of the object in the first image may be used as a reference (for example, Figure 3 The object frame OJ1 in the optical flow map is used as a reference), and an area ratio between the candidate region and the object frame is calculated. Once the area ratio of the candidate region in the optical flow map is greater than the area ratio threshold, object recognition processing can be performed on the second image corresponding to the candidate region of the optical flow map, or object recognition processing can be performed on the region in the second image corresponding to the candidate region of the optical flow map.

[0084] It is worth noting that although the above is described in an embodiment of an image group including a first image and a plurality of second images, in actual application, the above method can be applied to multiple image groups. The following will use an actual example to illustrate the situation where there are multiple image groups. Fig.10 , Fig.10 A schematic diagram showing a plurality of image groups IMGG(1) to IMGG(N) in some embodiments of the present disclosure is shown, where N can be any positive integer without any particular limitation. Fig.10 As shown, the first image of each of the image groups IMGG(1) to IMGG(N) is subjected to the object recognition processing IR, and the multiple second images of each of the image groups IMGG(1) to IMGG(N) are subjected to the method OFM for recognizing new objects based on optical flow. In addition, when it is determined that a new object is recognized in the multiple second images, the object recognition processing IR is also executed. In other words, the object recognition processing IR can be executed periodically on these continuous images, or when a new object is found to be possible.

[0085] In summary, the novel object detection device and method proposed in the present disclosure combines the traditional object recognition processing and the optical flow method to greatly improve the computing speed and save computing resources, and this approach can also maintain a certain degree of accuracy. In addition, this overcomes the problem of time-consuming and resource-consuming object recognition processing for a large number of images, and solves the problem of misjudgment of objects caused by noise in the optical flow method.

[0086] Although the present disclosure has been disclosed as above by way of embodiments, it is not intended to limit the present disclosure. Any person skilled in the art may make some changes and modifications without departing from the concept and scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be determined by the claims.

Claims

1. A new object detection device, comprising: An image receiving circuit for receiving a plurality of continuous images, wherein the images include a first image and a plurality of second images following the first image; a memory for storing a plurality of instructions; and A processor is used to process the instructions and execute the following steps: performing object recognition processing on the first image to define an initial object position from the first image; detecting the images to generate a plurality of optical flow maps corresponding to the images; and For these optical flow maps, perform the following steps: identifying a plurality of outlier regions outside of the initial object location; as well as Based on the multiple image positions of the outlier regions, generating at least one candidate region; At least one new object corresponding to the images is identified based on the at least one candidate region corresponding to the second images.

2. The new object detection device as claimed in claim 1, wherein the processor is further configured to: Performing object tracking on the second images according to the initial object position to generate object tracking positions of the second images respectively; and For each of the optical flow graphs, perform the following steps: The outlier regions outside the object tracking locations are identified.

3. The new object detection device as claimed in claim 1, wherein the processor is further configured to: For each of the optical flow maps, perform the following steps: According to at least one optical flow direction component, at least one outlier is found to generate the outlier regions from a plurality of optical flows, and a clustering algorithm is executed based on the outlier regions to generate the at least one candidate region.

4. The new object detection device as claimed in claim 1, wherein the processor is further configured to: For a third image and a fourth image among the second images, the following steps are performed: Calculating an overlapping area based on the candidate area of ​​the third image and the candidate area of ​​the fourth image, wherein the third image is an adjacent image of the fourth image; and Based on the overlapping area, the at least one new object corresponding to the images is identified.

5. The new object detection device as claimed in claim 1, wherein the processor is further configured to: Based on an occurrence number corresponding to each of the at least one candidate region, selecting a corresponding one of the at least one candidate region, wherein the corresponding one of the at least one candidate region has an occurrence number greater than a quantity threshold; and At least one new object corresponding to the images is identified based on the correspondence in the at least one candidate region.

6. The new object detection device as claimed in claim 5, wherein the processor is further configured to: Based on the occurrence quantity and an occurrence ratio corresponding to each of the at least one candidate region, selecting the corresponding one in the at least one candidate region, wherein the corresponding one in the at least one candidate region has an occurrence quantity greater than a quantity threshold and an occurrence ratio greater than an occurrence ratio threshold; and At least one new object corresponding to the images is identified based on the correspondence in the at least one candidate region.

7. The new object detection device as claimed in claim 5, wherein the processor is further configured to: Identifying a plurality of valid optical flow maps from the optical flow maps, wherein the valid optical flow maps include the corresponding one in the at least one candidate region; Selecting the last one of the valid optical flow maps; and An object recognition process is performed on the second image corresponding to the last one of the valid optical flow maps.

8. The new object detection device as claimed in claim 1, wherein the processor is further configured to: Based on an appearance ratio of each of the at least one candidate region, selecting the corresponding one in the at least one candidate region, wherein the corresponding one in the at least one candidate region has an appearance ratio greater than an appearance ratio threshold; and At least one new object corresponding to the images is identified based on the correspondence in the at least one candidate region.

9. The new object detection device as claimed in claim 1, wherein the processor is further configured to: Calculating an area ratio between each of the at least one candidate region corresponding to the second images and an object frame corresponding to the first image to select a corresponding one of the at least one candidate region, wherein the corresponding one of the at least one candidate region has an area ratio greater than an area ratio threshold; and At least one new object corresponding to the images is identified based on the correspondence in the at least one candidate region.

10. The new object detection device as claimed in claim 1, wherein the processor is further configured to: Each of the optical flow maps is divided into a plurality of sub-optical flow maps, and the at least one candidate region is generated for each of the sub-optical flow maps.

11. A method for detecting a new object, used in an electronic device, the method comprising: Receiving a plurality of continuous images, wherein the images include a first image and a plurality of second images following the first image; performing object recognition processing on the first image to define an initial object position from the first image; detecting the images to generate a plurality of optical flow maps corresponding to the images; and For these optical flow maps, perform the following steps: identifying a plurality of outlier regions outside of the initial object location; as well as Based on the multiple image positions of the outlier regions, generating at least one candidate region; At least one new object corresponding to the images is identified based on the at least one candidate region corresponding to the second images.

12. The method for detecting a newly-arrived object according to claim 11, further comprising: Performing object tracking on the second images according to the initial object position to generate object tracking positions of the second images respectively; as well as For each of the optical flow graphs, perform the following steps: The outlier regions outside the object tracking locations are identified.

13. The method for detecting a newly-arrived object according to claim 11, further comprising: For each of the optical flow maps, perform the following steps: According to at least one optical flow direction component, at least one outlier is found to generate the outlier regions from a plurality of optical flows, and a clustering algorithm is executed based on the outlier regions to generate the at least one candidate region.

14. The method for detecting a newly-arrived object according to claim 11, further comprising: For a third image and a fourth image among the second images, the following steps are performed: Calculating an overlapping area based on the candidate area of ​​the third image and the candidate area of ​​the fourth image, wherein the third image is an adjacent image of the fourth image; and Based on the overlapping area, the at least one new object corresponding to the images is identified.

15. The method for detecting a newly-arrived object according to claim 11, further comprising: Based on an occurrence number corresponding to each of the at least one candidate region, selecting a corresponding one of the at least one candidate region, wherein the corresponding one of the at least one candidate region has an occurrence number greater than a quantity threshold; as well as At least one new object corresponding to the images is identified based on the correspondence in the at least one candidate region.

16. The method for detecting new objects as claimed in claim 15, wherein the step of selecting the corresponding one of the at least one candidate area based on the number of occurrences of the corresponding one of the at least one candidate area comprises: Based on the occurrence quantity and an occurrence ratio corresponding to each of the at least one candidate region, selecting the corresponding one in the at least one candidate region, wherein the corresponding one in the at least one candidate region has an occurrence quantity greater than a quantity threshold and an occurrence ratio greater than an occurrence ratio threshold; as well as At least one new object corresponding to the images is identified based on the correspondence in the at least one candidate region.

17. The method for detecting a newly-arrived object according to claim 15, further comprising: Identifying a plurality of valid optical flow maps from the optical flow maps, wherein the valid optical flow maps include the corresponding one in the at least one candidate region; Selecting the last one of the valid optical flow maps; and An object recognition process is performed on the second image corresponding to the last one of the valid optical flow maps.

18. The method for detecting a newly-arrived object according to claim 11, further comprising: Based on an appearance ratio of each of the at least one candidate region, selecting the corresponding one in the at least one candidate region, wherein the corresponding one in the at least one candidate region has an appearance ratio greater than an appearance ratio threshold; as well as At least one new object corresponding to the images is identified based on the correspondence in the at least one candidate region.

19. The method for detecting a newly-arrived object according to claim 11, further comprising: Calculating an area ratio between each of the at least one candidate region corresponding to the second images and an object frame corresponding to the first image to select a corresponding one of the at least one candidate region, wherein the corresponding one of the at least one candidate region has an area ratio greater than an area ratio threshold; as well as At least one new object corresponding to the images is identified based on the correspondence in the at least one candidate region.

20. The method for detecting a new object according to claim 11, further comprising: Each of the optical flow maps is divided into a plurality of sub-optical flow maps, and the at least one candidate region is generated for each of the sub-optical flow maps.

Images