Detection method and detection system

Abstract

This detection method is for detecting a moving object on the basis of image information and comprises: a step for extracting a feature point pixel from all pixels in the image information; and a step for detecting a moving object on the basis of information relating to data of the feature point pixel.

Description

Detection method and detection system

[0001] The present disclosure relates to detection methods and systems.

[0002] A system for detecting a moving object is known. In a conventional technique, a plurality of images are acquired by a sensor, and the moving object is detected by checking the object moving in the images (see, for example, Patent Document 1). The plurality of images can be acquired, for example, by a method described in Patent Document 2.

[0003] Patent Document 1: Japanese Patent Publication No. 2003-323625

[0004] However, in the conventional technology, multiple images are required, which may result in a large amount of information about the images acquired by the sensor being transmitted.

[0005] An object of the present disclosure is to provide a detection method and a detection system that can reduce the amount of image information transmitted.

[0006] A detection method according to the present disclosure is a detection method for detecting a moving object based on image information, and includes the steps of: extracting feature point pixels from all pixels in the image information; and detecting the moving object based on information on data of the feature point pixels.

[0007] The detection system according to the present disclosure is a detection system that detects a moving object based on image information, and includes an extraction unit that extracts feature point pixels from all pixels in the image information, and a detection unit that detects the moving object based on information on data of the feature point pixels.

[0008] According to the present disclosure, the amount of image information transmitted can be reduced.

[0009] FIG. 1 is a diagram illustrating a schematic configuration of a detection system according to an embodiment of the present disclosure. FIG. 2 is a diagram illustrating a noise removal parameter. FIG. 3 is a diagram illustrating a noise removal parameter. FIG. 4 is a diagram illustrating an example of text data. FIG. 5 is a diagram illustrating an example of an image captured by an imaging unit. FIG. 6 is a diagram illustrating an example of an image restored based on text data. FIG. 7 is a flowchart illustrating an example of a processing flow executed by each device in the present embodiment. FIG. 8 is a diagram illustrating an uneven distribution parameter. FIG. 9 is a diagram illustrating an example of text data.

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] The following describes in detail embodiments of the present disclosure with reference to the accompanying drawings. Fig. 1 is a diagram showing a schematic configuration of a detection system 1 according to an embodiment of the present disclosure.

[0011] As shown in Fig. 1, the detection system 1 is a system that detects moving objects from image information. Specifically, the detection system 1 is a system that can detect moving objects in a sky region, such as outer space, based on image information observed by a flying device (e.g., an artificial satellite) that can fly in the sky region. The moving object may be, for example, space debris, any object floating or flying in the sky region, etc. The detection system 1 includes a flying device 10 and a detection device 20.

[0012] The flight device 10 is, for example, an artificial satellite capable of flying in outer space, and is a device capable of transmitting image information to an external device. The flight device 10 has an imaging unit 11, a transmission unit 12, an extraction unit 13, and a recording unit 14.

[0013] The imaging unit 11 is, for example, a camera, and is capable of capturing an image of a specific observation area in outer space.

[0014] The transmitter 12 is a communication device capable of transmitting and receiving information to and from an external device (for example, the detection device 20 ), and is a device for transmitting information based on the image captured by the imaging unit 11 to the detection device 20 .

[0015] The extraction unit 13 includes a central processing unit (CPU), a read-only memory (ROM), a random access memory (RAM), and an input / output circuit (not shown). The extraction unit 13 extracts feature point pixels from image information captured by the imaging unit 11.

[0016] A feature point pixel is a pixel that indicates a characteristic part of an image, and may be, for example, a pixel in a part of the sky area that is mostly made up of relatively dark areas, such as a star or a moving object, that indicates characteristics different from those of the dark area.

[0017] Specifically, feature point pixels are pixels that satisfy, for example, brightness conditions and noise removal conditions. For example, if the target pixel is a moving object, the brightness value tends to be highest at the center of the moving object, and the brightness value tends to decrease as the pixel moves away from the center. The brightness condition is a condition for selecting pixels that meet this tendency as feature point pixels to be extracted.

[0018] Specifically, the condition for satisfying the brightness condition may be that the target pixel to be extracted as a feature point pixel is a pixel among a plurality of pixels included in the image, the brightness value of which is equal to or greater than a predetermined value, and the brightness value of which is brighter than a plurality of surrounding pixels located around the target pixel. The plurality of surrounding pixels may be, for example, eight pixels located around the target pixel.

[0019] The predetermined value can be set to any value, and may be set to a value (for example, 160) that is brighter than an area in the sky where no objects exist.

[0020] Therefore, by setting the above brightness conditions, it becomes easier to extract target pixels that make it easier to identify moving objects as feature point pixels.

[0021] For example, suppose there are nine pixels, each consisting of three pixels in the vertical and horizontal directions, as shown in Figure 2. Of the nine pixels, the target pixel is the pixel located in the middle, and the multiple surrounding pixels are the eight pixels located around the target pixel. The numbers shown inside each pixel in Figure 2 are the luminance values ​​of each pixel.

[0022] For example, in Fig. 2, the brightness value of the target pixel is 180, and the brightness values ​​of the surrounding pixels are 160, 155, 140, 150, 145, 135, 140, and 135. If the predetermined value is set to 160, the target pixel in Fig. 2 is equal to or greater than the predetermined value and is brighter than all of the surrounding pixels, so it satisfies the brightness condition described above.

[0023] The noise removal condition may be a condition for removing a target pixel as noise from the target of feature point pixels when the target pixel corresponds to noise contained in the image.

[0024] When noise is included in an image, the brightness value of the target pixel in the portion corresponding to the noise may be significantly higher than that of the surrounding pixels, as shown in Fig. 3. Specifically, in Fig. 3, the brightness value of the target pixel is 200, and the surrounding pixels are 1, 3, 4, 5, 3, 2, 3, and 3.

[0025] In this case, the target pixel in Figure 3 is a pixel that is equal to or greater than a predetermined value and is brighter than all of the surrounding pixels, so it satisfies the brightness condition described above. However, by setting noise removal conditions, it becomes possible to remove such target pixels as noise from the target feature point pixels.

[0026] For example, a pixel having a noise removal parameter equal to or greater than a predetermined threshold may be a condition for satisfying the noise removal condition.

[0027] The noise removal parameter is a parameter for removing pixels corresponding to noise in an image from the target of feature point pixels, and is a parameter calculated based on the pixel and a plurality of surrounding pixels.

[0028] For example, the noise removal parameter may be a value obtained by dividing the sum of the luminance value of the target pixel and the luminance values ​​of a plurality of surrounding pixels by the luminance value of the target pixel.

[0029] If the target pixel is noise, its luminance value will be significantly higher than those of the surrounding pixels, and the sum of the luminance values ​​of the target pixel and the surrounding pixels will be close to the luminance value of the target pixel. In other words, the value of the noise removal parameter will be close to 1.

[0030] Therefore, the predetermined threshold may be appropriately set to a value that can reliably remove, from the target feature point pixels, target pixels having a noise removal parameter value close to 1. For example, the predetermined threshold may be set to a value of 2 or greater (e.g., 2.5).

[0031] For example, the target pixel in Fig. 3 satisfies the brightness condition as described above, but the noise removal parameter is 1.12, which is the value obtained by dividing 224, the sum of the nine pixels, by 200. As a result, the noise removal parameter is less than the predetermined threshold, and therefore the target pixel in Fig. 3 can be removed from the targets for feature point pixel extraction.

[0032] 2, the noise removal parameter is equal to or greater than the predetermined threshold value, since it is 7.44, which is the sum of the nine pixels, 1340, divided by 180. Therefore, the target pixel in FIG. 2 is extracted as a feature point pixel by the extraction unit 13.

[0033] The recording unit 14 records in text data the data of the feature point pixels extracted by the extraction unit 13. The data of the feature point pixels may be, for example, the x coordinate of the feature point pixel in the image, the y coordinate of the feature point pixel in the image, the luminance value of the feature point pixel, or a noise removal parameter for the feature point pixel.

[0034] For example, Fig. 4 shows an example in which 18 feature point pixels are extracted from an image. The text data shown in Fig. 4 shows an example in which, from the left, the x coordinates of the feature point pixels, the y coordinates of the feature point pixels, the luminance values ​​of the feature point pixels, and the noise removal parameters of the feature point pixels are listed.

[0035] In other words, the text data only records data on feature-point pixels, and does not record data on pixels other than feature-point pixels, which makes it possible to reduce the amount of image information compared to text data that records all pixels in an image.

[0036] The transmitting unit 12 transmits the information of the text data recorded by the recording unit 14 to the detecting device 20 .

[0037] The image capturing unit 11 captures images at regular intervals, and the extracting unit 13 extracts feature point pixels for each of the images captured at different observation times. Detecting a moving object requires multiple (e.g., 10) consecutive images captured at relatively short intervals. Therefore, the regular interval can be set to any time suitable for detecting a moving object.

[0038] The recording unit 14 then records the data of the feature point pixels corresponding to each of the multiple images in the text data. Only the data of the feature point pixels corresponding to one image is recorded in the text data. In other words, multiple pieces of text data corresponding to each of the multiple images are generated.

[0039] The detection device 20 is a device that detects moving objects in a sky region based on image information, and may be, for example, a device such as a PC (Personal Computer). The detection device 20 includes a CPU, ROM, RAM, and an input / output circuit (not shown), and has a communication unit 21, a restoration unit 22, and a detection unit 23.

[0040] The communication unit 21 receives information of text data transmitted from the transmission unit 12 via, for example, a wired or wireless network.

[0041] The restoration unit 22 restores the image based on the information of the text data. Specifically, the restoration unit 22 restores the image by setting pixels corresponding to feature point pixels to a first luminance value and pixels that are not feature point pixels to a second luminance value different from the first luminance value.

[0042] The first luminance value may be the maximum luminance value, and the second luminance value may be the minimum luminance value.

[0043] Specifically, the restoration unit 22 restores the image by using a first brightness value for coordinates corresponding to the x and y coordinates recorded in the text data, and a second brightness value for coordinates corresponding to the x and y coordinates not recorded in the text data.

[0044] For example, suppose that the image shown in Fig. 5A is acquired by the imaging unit 11. If the circled portions in Fig. 5A are portions extracted as feature point pixels, the restored image restored by the restoration unit 22 will be, for example, like the image shown in Fig. 5B.

[0045] This makes it possible to obtain binarized image data, and also to obtain image data from which noise contained in the image has been removed.

[0046] The restoration unit 22 also generates restored images corresponding to the plurality of images observed at different times, i.e., the restoration unit 22 generates restored images for each of the plurality of text data corresponding to the plurality of images.

[0047] The detection unit 23 detects a moving object based on a plurality of restored images corresponding to the plurality of images, respectively. Specifically, the detection unit 23 may detect a moving object using the same method as the method described in Patent Document 1, for example.

[0048] Specifically, the detection unit 23 generates a median image of the multiple restored images and removes star images contained in the restored image. Then, the detection unit 23 generates an average image using the multiple median images. Furthermore, the detection unit 23 performs these processes for all possible velocity vectors of moving objects. Note that the detection method used by the detection unit 23 is a known technique, and therefore a detailed description thereof will be omitted. Other known methods may also be used to detect moving objects.

[0049] Next, an example of the operation of the detection system 1 will be described. Fig. 6 is a flowchart showing an example of the flow of processing executed by each device in this embodiment. From the left, an example of processing executed by the flight device 10 and an example of processing executed by the detection device 20 are shown.

[0050] 6, the flying device 10 acquires an image using the imaging unit 11 (step S101). After acquiring the image, the flying device 10 extracts feature point pixels from the image using the extraction unit 13 (step S102).

[0051] After extracting the feature pixels, the flying device 10 records the feature pixels as text data using the recording unit 14 (step S103). After recording the text data, the flying device 10 transmits the text data to the detection device 20 using the transmission unit 12 (step S104).

[0052] The processes of steps S101 to S104 are executed every time an image is acquired by the imaging unit 11. The process of step S104 may be a process of collectively transmitting a plurality of pieces of text data corresponding to a plurality of images observed at different times to the detection device 20.

[0053] Next, after receiving the text data, the detection device 20 restores the image based on the feature point pixels recorded in the text data using the restoration unit 22 (step S201). After restoring the image, the detection device 20 detects moving objects from the restored image using the detection unit 23 (step S202).

[0054] The process of step S201 may be executed as needed after a certain number of text data have been received.

[0055] After step S202, this control ends. Note that the process of the flowchart shown in FIG. 6 may be repeatedly executed after step S202.

[0056] According to this embodiment configured as described above, feature point pixels are extracted from all pixels in the image information, and a moving object is detected based on information in text data in which data on the feature point pixels is recorded.

[0057] This allows moving objects to be detected simply by having text data that records data on feature point pixels, thereby reducing the amount of image information transmitted compared to a configuration in which data including all pixels in the image information is transmitted.

[0058] Furthermore, a feature point pixel is a pixel whose luminance value is equal to or greater than a predetermined value and whose luminance value is brighter than the luminance values ​​of the surrounding pixels. Therefore, depending on the setting of the predetermined value, even if a moving object appears relatively dark in the image, it can be easily extracted as a feature point pixel.

[0059] In conventional technology, moving objects must be clearly visible in the image, but the size of the moving object that can be detected from the image depends on the capabilities of the imaging unit installed in the flying device. Therefore, for moving objects that may appear relatively dark in the image, a large optical system or a highly sensitive sensor imaging unit is required, which may increase costs.

[0060] In contrast, in the present embodiment, even moving objects that appear relatively dark in the image can be easily extracted as feature point pixels, so there is no need to use a large optical system or a highly sensitive sensor for the imaging unit as in the conventional technology, and costs can be kept low.

[0061] Furthermore, a feature point pixel is a pixel for which a noise removal parameter calculated based on the pixel and a plurality of surrounding pixels is equal to or greater than a predetermined threshold value.

[0062] Therefore, pixels that correspond to noise can be removed from the target of feature point pixels, making it easier to reliably extract portions that are likely to be moving objects as feature point pixels.

[0063] Furthermore, since the noise removal parameter is the sum of the luminance value of the target pixel and the luminance values ​​of multiple surrounding pixels divided by the luminance value of the target pixel, it is possible to reliably remove pixels that are likely to be noise.

[0064] Furthermore, when extracting feature point pixels, feature point pixels are extracted for each of a plurality of images observed at different times, making it easier to identify feature point pixels corresponding to moving objects.

[0065] Furthermore, based on the information in the text data, the image is restored by setting pixels corresponding to feature point pixels as a first luminance value and pixels that are not feature point pixels as a second luminance value, and moving objects are detected based on the multiple restored images. As a result, a restored image is generated in which the feature point pixel portions are emphasized more than other portions, so that even if a moving object appears dark in an image, the restored image can clearly show the moving object.

[0066] In addition, extraction of feature point pixels is performed by the flying device 10, extraction of moving objects is performed by the detection device 20, and text data recording the data of the feature point pixels is transmitted from the flying device 10 to the detection device 20.

[0067] For example, on the ground, there is a robust network, making it possible to transmit all pixel information in the image information to the detection device without any problems. However, when transmitting data from the flight device 10 to a detection device 20 installed in a different location from the flight device 10, there is no robust network like on the ground, and there is a certain limit to the amount of data communication. Therefore, the impact of large amounts of data (such as data transmission taking too long or data being unable to be transmitted) is likely to be significant.

[0068] In contrast, in this embodiment, text data recording data on feature point pixels is transmitted from the flight device 10 to the detection device 20, so only the necessary data needs to be transmitted. As a result, the impact of large amounts of data can be reduced, and the necessary image information can be transmitted smoothly.

[0069] Furthermore, since the flying device 10 is a device capable of flying in outer space, when the flying device 10 transmits images captured in outer space to the detection device 20, the problem of the impact of the large amount of data described above becomes more pronounced.

[0070] However, in this embodiment, as described above, it is only necessary to transmit the necessary data, which effectively solves the problems that arise when the flying device 10 transmits images captured in outer space to the detection device 20.

[0071] In the above embodiment, the data of the feature point pixels includes a noise reduction parameter, but the present disclosure is not limited to this, and the data may not include a noise reduction parameter, or may include a parameter other than the noise reduction parameter. For example, the data of the feature point pixels may include a distribution parameter.

[0072] The distribution parameter is a parameter that indicates the direction of distribution of the object's position in the image that is the target of the feature point pixel. For example, the distribution parameter is 1 if the object's distribution direction is to the upper left, 2 if the object's distribution direction is to the upper right, 3 if the object's distribution direction is to the lower right, and 4 if the object's distribution direction is to the lower left.

[0073] The bias direction of the object is the direction with the maximum value among the related parameters of each direction calculated based on the target pixel that is the target of the feature point pixel and eight surrounding pixels located around the target pixel.

[0074] The parameter related to the upper left direction is the sum of the luminance value of the target pixel and the luminance values ​​of the pixels above, to the left, and above the target pixel.

[0075] The relevant parameter in the upper right direction is the sum of the luminance value of the target pixel and the luminance values ​​of the pixels above, to the right, and above and to the right of the target pixel.

[0076] The related parameter in the lower right direction is the sum of the luminance value of the target pixel and the luminance values ​​of the pixels to the right, the lower right, and the lower side of the target pixel.

[0077] The related parameter in the lower left direction is the sum of the luminance value of the target pixel and the luminance values ​​of the pixels below, to the lower left, and to the left of the target pixel.

[0078] The deflection direction of the object is a value corresponding to the direction in which the relevant parameters for each direction are maximum.

[0079] For example, in Figure 7, the luminance value of the target pixel is 180, and the luminance values ​​of multiple surrounding pixels are 170, 160, 140, 170, 160, 140, 170, and 140. In this case, the related parameter in the upper left direction is 680. The related parameter in the upper right direction is 640. The related parameter in the lower right direction is 650. The related parameter in the lower left direction is 660.

[0080] In this case, the maximum value of the related parameters for each direction is the related parameter for the upper left direction, so the bias direction of the object is the upper left direction. Therefore, the bias parameter becomes 1, which corresponds to the upper left direction.

[0081] Furthermore, as shown in FIG. 8 , the text data may also include, in addition to the x-coordinate of the feature point pixel in the image, the y-coordinate of the feature point pixel in the image, the luminance value of the feature point pixel, and the noise removal parameter of the feature point pixel, a distribution parameter.

[0082] By providing such a maldistribution parameter, when binarizing image data, it is possible to select four pixels that have a higher probability of existence of a moving object, rather than one pixel. Since a moving object may not move exactly in units of one pixel between images, selecting the position of the moving object as four pixels, which has a range wider than one pixel, makes it easier to detect the moving object, and ultimately improves the detection accuracy of the moving object.

[0083] Furthermore, in the above embodiment, the luminance values ​​of feature point pixels, noise reduction parameters, uneven distribution parameters, etc. are recorded in the text data, but the present disclosure is not limited to this. For example, as long as the x-coordinates and y-coordinates of feature point pixels are recorded in the text data, the luminance values ​​of feature point pixels, noise reduction parameters, and uneven distribution parameters do not need to be recorded.

[0084] Furthermore, in the above embodiment, the flight device is capable of flying in outer space, but the present disclosure is not limited to this, and the flight device does not have to be capable of flying in outer space.

[0085] Furthermore, in the above embodiment, text data (information on feature point pixels) is transmitted from the flying device to the detection device, but the present disclosure is not limited to this. For example, text data may be transmitted from a mobile device other than the flying device to the detection device. Furthermore, if the detection device has an imaging unit or if the flying device (mobile device) has a detection unit, information on feature point pixels may not be transmitted.

[0086] Furthermore, in the above embodiment, information about feature point pixels is recorded as text data, but the present disclosure is not limited to this. For example, if the detection device is configured to have an imaging unit, the detection device may extract feature point pixels from acquired image information and detect a moving object without generating text data. Furthermore, if the flying device is configured to have a detection device, the flying device may detect a moving object without generating text data from information about feature point pixels. Furthermore, the flying device may be configured to convert information about feature point pixels into data other than text data and transmit it to the detection device.

[0087] Furthermore, the above-described embodiments are merely examples of specific embodiments for carrying out the present disclosure, and the technical scope of the present disclosure should not be interpreted as being limited by these embodiments. In other words, the present disclosure can be carried out in various forms without departing from its gist or main features.

[0088] The detection system of the present disclosure is useful as a detection method and detection system that can reduce the amount of image information transmitted.

[0089] REFERENCE SIGNS LIST 1 detection system 10 flying device 11 imaging unit 12 transmission unit 13 extraction unit 14 recording unit 20 detection device 21 communication unit 22 restoration unit 23 detection unit

Claims

1. A detection method for detecting a moving object based on image information, comprising the steps of: extracting feature point pixels from all pixels in the image information; and detecting the moving object based on information on the data of the feature point pixels.

2. The detection method according to claim 1, wherein the feature point pixel is a pixel that satisfies at least the following conditions: the luminance value of the target pixel to be extracted is equal to or greater than a predetermined value, and the luminance value is brighter than a plurality of surrounding pixels located around the target pixel.

3. The detection method according to claim 2, wherein the feature point pixel is a pixel that satisfies a condition that a noise removal parameter calculated based on the target pixel and the multiple surrounding pixels is equal to or greater than a predetermined threshold.

4. The detection method according to claim 3, wherein the noise removal parameter is a value obtained by dividing the sum of the luminance value of the target pixel and the luminance values ​​of the multiple surrounding pixels by the luminance value of the pixel.

5. The detection method according to claim 1, wherein the image information includes information on a plurality of images observed at different times, and in the step of extracting the feature point pixels, the feature point pixels are extracted for each of the plurality of images.

6. The detection method according to claim 5, wherein the step of detecting the moving object includes: a step of restoring an image by setting pixels corresponding to the feature point pixels to a first luminance value and pixels which are not the feature point pixels to a second luminance value different from the first luminance value; and a step of detecting the moving object based on a plurality of restored images corresponding to each of the plurality of images.

7. The detection method described in claim 1, wherein the step of extracting the feature point pixels is performed by a flying device equipped with an imaging unit capable of capturing images, the step of detecting the moving object is performed by a detection device installed at a location different from the flying device, and the method further includes a step of transmitting information on the data of the feature point pixels from the flying device to the detection device.

8. The detection method according to claim 7, wherein the flying device is capable of flying in outer space.

9. The detection method according to claim 1, further comprising a step of recording information on the data of the feature point pixels in text data, and in the step of detecting the moving object, the moving object is detected based on the text data.

10. A detection system for detecting a moving object based on image information, comprising: an extraction unit that extracts feature point pixels from all pixels in the image information; and a detection unit that detects the moving object based on information on the data of the feature point pixels.

Images