Object positioning system and method

Abstract

The present invention relates to an object positioning system, comprising an object detection module for detecting an object in a current frame by referring to the content in a reference frame, and an object positioning module for identifying the position of the object according to a graphic segment. The object detection module comprises a difference module for calculating the difference between the reference frame and the current frame at the same position to generate a difference frame, a selected block detection module for identifying a selected block of the difference frame according to a predetermined minimum difference value, and a graphic segment generation module for generating a graphic segment of the current frame.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition systems, and in particular to an object positioning system and method. Background Art

[0002] Image recognition technology for detecting objects has been widely used in various devices such as surveillance cameras, vehicle safety devices, and digital cameras. Such technology is expected to be widely used to identify suspicious persons, collect marketing information, or perform risk prediction by tracking a target and determining the target's behavior type.

[0003] Image recognition technologies such as mobile object localization and / or classification include various types of deep convolutional neural networks, but these neural networks require large amounts of computation and are usually processed by high-performance servers in remote data centers. These image recognition technologies are not optimized for IoT devices with low computing power.

[0004] Traditional methods for processing moving areas in video frames require a lot of computation. In order to reduce the impact of noise caused by changes in light and camera equipment, different images of the reference frame and the current frame can be processed. The differential image (the image processed by pixel difference technology) is processed to remove isolated fragments in the image difference to obtain a processed image, and the processed image is segmented into different segments corresponding to different moving targets. However, the amount of computation required by this method is extremely large, so a system and method with lower computational complexity is needed to perform image segmentation, positioning and classification. Summary of the invention

[0005] In view of the technical problems existing in the prior art, the present invention proposes an object positioning system, including an object detection module for detecting an object in a current frame with reference to the content in a reference frame, and an object positioning module for identifying the position of the object according to a graphic segment. The object detection module includes a difference module for calculating the difference between the reference frame and the current frame at the same position to generate a difference frame, a selected block detection module for identifying a selected block of the difference frame according to a predetermined minimum difference value, and a graphic segment generation module for generating a graphic segment of the current frame, wherein the graphic segment is a part of the object and is defined by interconnecting the selected block of the difference frame with adjacent selected blocks, and is used to assign a graphic segment label to each selected graphic segment in a raster scan, wherein the graphic segment label identifies the graphic segment, and each non-adjacent graphic segment has a different graphic segment label.

[0006] According to another aspect of the present invention, a method for locating an object in a current frame with reference to a reference frame in a video frame is provided, the method comprising detecting an object. Detecting an object comprises generating a difference frame equal to the absolute difference between the current frame and the reference frame, calculating the difference of the same position between the reference frame and the current frame to generate the difference frame, identifying a block in the difference frame having a difference value greater than a minimum difference value as a selected block, and generating a graphic segment of the current frame. The graphic segment is a part of the object and is defined by interconnecting a selected block of the difference frame with adjacent selected blocks, wherein a graphic segment label can identify the graphic segment, and each non-adjacent graphic segment has a different graphic segment label.

[0007] According to another aspect of the present invention, a method for generating selected graphic segments of an image by raster scanning is provided, comprising identifying pixels as selected pixels according to a first predetermined brightness value, generating selected graphic segments, and assigning a graphic segment label to each selected pixel of the selected graphic segment during raster scanning. The selected graphic segment is defined by selected pixels of the image, and adjacent selected pixels are connected to each other. The graphic segment label identifies the selected graphic segment, and each non-adjacent graphic segment label is different. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The preferred embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein:

[0009] Figure 1 is a schematic diagram of an object positioning system in an embodiment.

[0010] Figure 2 yes Figure 1 Schematic diagram of the object detection module of the object positioning system.

[0011] Figure 3 yes Figure 2 Schematic diagram of a pixel difference frame generated by a pixel difference module.

[0012] Figure 4 yes Figure 2 Schematic diagram of an index frame generated by a graphics segment generation module.

[0013] Figure 5 yes Figure 2 Schematic diagram of a label frame generated by the graphic segment generation module.

[0014] Figure 6 is a schematic diagram of a graphics segment label array and a graphics segment label block count array.

[0015] Figure 7 The figure is a schematic diagram of the application of the graphic segment generation method to an image with text.

[0016] Figure 8 is a schematic diagram of an indexed image produced by raster scanning.

[0017] Fig. 9 is a schematic diagram of a graphics segment label array and a graphics segment label pixel count array.

[0018] 100 Object Positioning System

[0019] 110 Object Detection Module

[0020] 120 Object Positioning Module

[0021] 130 Tracking Module

[0022] 111 Pixel Difference Module

[0023] 112 Selected block detection module

[0024] 113 Graphics segment generation module

[0025] 114 Candidate Region Module

[0026] 210 pixel difference frame

[0027] 211 Mobile Area

[0028] 212 Selected Block

[0029] 213 Graphics Segment

[0030] 214 Candidate Areas

[0031] Blocks 215 and 216

[0032] 310 index frame

[0033] 410 Tagged Frame

[0034] 710 background pixels

[0035] 720 selected pixels DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] In the following detailed description, reference may be made to the various specification drawings that are part of the present application and are used to illustrate specific embodiments of the present application. In the accompanying drawings, similar reference numerals describe substantially similar components in different figures. The various specific embodiments of the present application are described below in sufficient detail so that a person of ordinary skill in the art with relevant knowledge and skills in the art can implement the technical solutions of the present application. It should be understood that other embodiments may also be used or structural, logical or electrical changes may be made to the embodiments of the present application.

[0038] Figure 1 is a schematic diagram of an object positioning system 100 in an embodiment. The modules in the object positioning system 100 can be implemented by hardware or software. The object positioning system 100 includes an object detection module 110, an object positioning module 120 and a tracking module 130. The object detection module 110 detects a moving object in a current frame of a video frame sequence with reference to a reference frame (e.g., a reference frame). The object positioning module 120 can identify the position of a moving object. The tracking module 130 is used to track the position of a moving object. In addition, the object positioning module 120 may also include an object classification function for identifying the type of the object.

[0039] like Figure 1 As shown, the object positioning system 100 receives an input of a plurality of video frames (image frames). The plurality of video frames include a current frame and a reference frame before the current frame. In addition, additional parameters such as the size of the object and the moving speed of the object may also be input into the object positioning system 100. The number of frames between the reference frame and the current frame may be determined by the moving speed of the object and the frame rate of the camera device. The object positioning module 120 may process the current frame to obtain information about the position and type of the object.

[0040] Figure 2 is a schematic diagram of the object detection module 110 in the embodiment. Figure 3 2 is a schematic diagram of a pixel difference frame 210 in an embodiment. The object detection module 110 includes a pixel difference module 111, a selected block detection module 112, a graphic segment generation module 113 and a candidate region module 114. The pixel difference module 111 is used to calculate the difference in pixel values ​​of pixels at the same position between the reference frame and the current frame to generate a pixel difference frame 210. A preferred way to achieve low computational complexity is to use information on pixel brightness to calculate pixel differences, and other calculation methods are still within the scope of the present invention, and the present invention is not limited thereto. The selected block detection module 112 is used to identify a selected block 212 ( Figure 3 The selected block 212 has a predetermined minimum difference value in pixel values ​​at the same position between the detection frame and the reference frame. The graphic segment generation module 113 is used to generate a graphic segment 213 ( Figure 3The graphic segment 213 is defined by the selected block 212 of the current frame, and the selected block 212 of the current frame is connected to the adjacent selected block 212. The candidate region module 114 is used to generate a candidate region 214 including the graphic segment.

[0041] The object detection module 110 can locate the moving area 211 containing the moving object. The object detection module 110 can locate the moving area 211 by identifying the selected blocks 212 and generating a bounding box surrounding the selected blocks 212. Figure 3 As shown, the graphic segment 213 is within the moving area 211 . Figure 3 The blocks of the pixel difference frame 210 are segmented from the current frame. The size and shape of the blocks can be defined by the size of the smallest and largest objects in the current frame. For example, the blocks can be squares of size 8x8, 16x16, or 20x20. If the block size is 8x8, it contains 64 pixels. If the block size is 16x16, it contains 256 pixels. If the block size is 20x20, it contains 400 pixels. In addition, the candidate area 214 can be defined by the object detection module 110 based on the position information from the object detection module 110. The candidate area 214 includes all selected blocks of the moving area 211 or the graphic segment 213 of the object. Block 215 is a block with image noise caused by lighting or camera devices, and block 216 is a block with a pixel difference between the current frame and the reference frame below a predetermined minimum difference value. Block 215 and block 216 are not included in the movement area 211 and thus are not discussed in the present invention.

[0042] Figure 4 2 is a schematic diagram of an index frame 310 generated by the graphic segment generation module 113 through raster scanning and index assignment in an embodiment. The pixel difference frame 210 can be input into the graphic segment generation module 113. Each block of the pixel difference frame 210 is checked in the raster scan to mark the selected block. In an embodiment, the selected block 212 is defined as a block whose pixel difference at the same position between the current frame and the reference frame is greater than or equal to a predetermined minimum difference value. In addition, the neighboring blocks are defined as the left, upper, upper left and upper right blocks of a block. In the index frame 310, if the scanned block is a non-selected block, the block is assigned an index value of 0. Otherwise, if all the neighboring blocks of the block are non-selected blocks, the block is assigned an index value with the smallest unused value. If at least one neighboring block of the block is a selected block, the block is assigned an index value with the highest index value among the neighboring selected blocks.

[0043] Please refer to the following description Figure 5 and Figure 6 . Figure 54 is a schematic diagram of a label frame 410 according to an embodiment. The label frame 410 has a graphic segment label assigned to each block by the graphic segment generation module 113 . Figure 6 Schematic diagram of a graphics segment tag array and a graphics segment tag block count array in an embodiment. Figure 6 As shown, each graphics segment index is a pointer to a corresponding graphics segment label, and the graphics segment label can be used as a pointer to the corresponding graphics segment label block count array. During raster scanning, if the block being scanned is a selected block, the values ​​of the graphics segment labels of all adjacent selected blocks are changed to the same value as the index value of the selected block. For example, the graphics segment label value 3 is assigned to the selected blocks with indices 1, 2, and 3 in the graphics segment label array.

[0044] In addition, before raster scanning, the segment label block count array is initialized to 0. During raster scanning, when a segment label is assigned to a block to be scanned, its corresponding segment label block count is increased by 1. If the block to be scanned is a selected block, its corresponding segment label block count is increased by the sum of the segment label block count values ​​of the adjacent selected blocks, and then the segment label block count values ​​of the adjacent selected blocks are set to 0. Figure 6 As shown, in this embodiment, the graphic segment label of index 0 is 0, including 524 blocks. The graphic segment labels of indexes 1 and 2 are 3, and the graphic segment label of index 3 is also 3, including 52 blocks. The graphic segment label block count can be used in the subsequent image processing module. In addition, the size of the graphic segment can be used for subsequent processing, such as deleting small graphic segments caused by noise or graphic segments smaller than the size of the target object.

[0045] After image processing, a binary mask of the pixel difference frame 210 may be generated. For example, a mask value of 1 (corresponding to a segment label value of 3) indicates a selected block, while other blocks have a mask value of 0 (corresponding to a segment label value of 0). Similarly, if there is more than one segment in the pixel difference frame 210, this method allows the segment generation module 113 to assign different segment labels to non-intersecting segments. After the above image processing, the object detection module 110 outputs the binary mask to the candidate region module 114 and the tracking module 130, where further image processing is performed, such as object localization and classification and object tracking. The binary mask can be generated by a single raster scan, which shortens the image processing time. In one embodiment, the system can generate the binary mask before receiving the next frame. Image processing only on the candidate region can greatly reduce the amount of computation, so that the method can be implemented on devices with lower computing power. However, the method can also be applied to computationally intensive processing methods to obtain better localization and classification results, such as deep convolutional neural networks.

[0046] The same image segment generation method can also be applied to fixed images. Figure 7 7 is a schematic diagram of the method for generating a graphic segment in an embodiment applied to an image with text. After the graphic segment is generated, the pixels of the image can be distinguished into background pixels 710 and selected pixels (text) 720 based on the pixel values ​​(e.g., brightness) of the pixels. For example, if the brightness value of the pixel is lower than a predetermined brightness value, the pixel is distinguished as a background pixel 710.

[0047] Figure 8 is a schematic diagram of an index image generated by raster scanning in an embodiment. An index can be assigned to each pixel by a graphic segment generation method. Pixels with index values ​​3, 4, and 10 belong to the word "5". Thereafter, each pixel of the image is checked in the order of raster scanning to mark the selected pixels. In one embodiment, the selected pixels can be defined as pixels having a brightness value lower than a first predetermined value. In another embodiment, the selected pixels can be defined as pixels having a brightness value higher than a second predetermined value.

[0048] Neighboring pixels are defined as the pixels to the left, top, left-upper, and right-upper of a pixel. If the scanned pixel is a non-selected pixel, the pixel is assigned an index value of 0, otherwise, if all the neighboring pixels of the pixel are non-selected pixels, the pixel is assigned an index value with the smallest unused value. For a pixel that has at least one neighboring pixel as a selected pixel, the pixel is assigned an index value with the highest index value among the neighboring selected pixels.

[0049] Fig. 9 is a schematic diagram of a segment label array and a segment label pixel count array in an embodiment. Before raster scanning, the segment label pixel count array is initialized to 0. During raster scanning, when a segment label is assigned to a scanned pixel, its corresponding segment label pixel count may be increased by 1. If the scanned pixel is a selected pixel, its corresponding segment label pixel count will be increased by the sum of the segment label pixel count values ​​of the adjacent selected pixels, and then the segment label pixel count values ​​of the adjacent selected pixels will be set to 0. Fig. 9As shown, the graphic segment label of index 0 representing the background pixel is 0, and the graphic segment label pixel count of index 0 is 762. In one embodiment, the text "4" includes 103 pixels, the index is 1 and the graphic segment label is 1. The text "1" includes 50 pixels, the index is 6 and the graphic segment label is 6. The text "5" includes 134 pixels, the indexes are 3, 4 and 10 and the graphic segment label is 10. The text "2" includes 111 pixels, the index is 9 and the graphic segment label is 9. The graphic segment label pixel count can be applied to subsequent image processing. In addition, the size of the graphic segment can also be used for subsequent image processing, such as deleting small graphic segments caused by noise or graphic segments smaller than the target text size. Each graphic segment can be input to the object positioning module 120 and each is positioned as a text object such as "4", "1", "5" and "2".

[0050] In summary, the object positioning system using the graphic segment generation method of the embodiment of the present invention can effectively reduce the computational complexity of image processing. Therefore, the object positioning system and method can be applied to devices with low computing power, such as Internet of Things devices, to perform image recognition or other image processing.

[0051] The above embodiments are only used to illustrate the present invention, but not to limit the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the scope of the present invention. Therefore, all equivalent technical solutions should also fall within the scope of the present invention.

Claims

1. A system for locating an object in a current frame with reference to a reference frame in a series of image frames, comprising: An object detection module is used to detect an object in a current frame by referring to the content in a reference frame. The object detection module includes: a difference module for calculating the difference between the reference frame and the current frame at the same position to generate a difference frame; a selected block detection module, for identifying a selected block of the difference frame according to a predetermined minimum difference value; and A graphics segment generation module is used to: generating a graphic segment of the current frame, the graphic segment being a portion of the object and being defined by connecting the selected block of the difference frame with adjacent selected blocks, assigning an index value to each block of the current frame, wherein the index value assigned to each non-selected block of the difference frame is a predetermined index value, and the index value assigned to each selected block of the difference frame is equal to the smallest unused value or equal to the highest index value among adjacent selected blocks but not equal to the predetermined index value; and for assigning a graphics segment label to each selected graphics segment according to the index value of each block of the difference frame in raster scanning, wherein the graphics segment label identifies the graphics segment, adjacent selected blocks have the same graphics segment label, and each non-adjacent graphics segment has a different graphics segment label; and An object positioning module is used to identify the position of the object according to the graphic segment. 2 . The system of claim 1 , wherein the graphics segment generation module is further configured to calculate the number of selected blocks having the same graphics segment label.

3. The system of claim 2, wherein the graphics segment generation module is further configured to generate a two-bit mask for the difference frame. 4 . The system as claimed in claim 1 , wherein the object detection module further comprises a candidate region module, wherein the candidate region module is used to generate a candidate region surrounding the graphic segment to obtain information about the object.

5. The system of claim 1, further comprising: A tracking module is coupled to the object detection module to track the position of the object.

6. The system of claim 1, wherein the graphic segment is generated before the object detection module receives a next frame.

7. A method for locating an object in a current frame with reference to a reference frame in a series of image frames, the method comprising: Detecting an object, including: generating a difference frame equal to the absolute difference between the current frame and the reference frame; Calculating the difference of the same position between the reference frame and the current frame to generate a difference frame; identifying a block in the difference frame having a difference value greater than a minimum difference value as a selected block; and generating a graphic segment of the current frame, the graphic segment being a portion of the object and being defined by connecting the selected block of the difference frame with adjacent selected blocks, assigning an index value to each block of the current frame, wherein the index value assigned to each non-selected block of the difference frame is a predetermined index value, and the index value assigned to each selected block of the difference frame is equal to the smallest unused value or equal to the highest index value among adjacent selected blocks but not equal to the predetermined index value; and Assigning a graphics segment label to each selected graphics segment according to the index value of each block of the difference frame, wherein the graphics segment label identifies the graphics segment, adjacent selected blocks have the same graphics segment label, and each non-adjacent graphics segment has a different graphics segment label; and A position of the object is identified according to the graphic segment.

8. The method of claim 7, further comprising: Track the position of the object.

9. The method of claim 7, wherein detecting the object further comprises: Count the number of selected tiles that have the same graph segment label.

10. The method of claim 9, wherein detecting the object further comprises: generating a two-bit mask for the difference frame; and A candidate region including the graphic segment for obtaining information of the object is generated.

Images