Object detection device and object detection method

Abstract

To enable highly accurate real-time processing using a discriminator generated by means of the machine learning technique to be performed even for large size observation data in an object detection device.SOLUTION: An object detection method comprises: generating a plurality of division images by acquiring the imaged observation data (S201, S202) in an object detection device which continuously acquires image data imaged by observing an object that relatively moves, and discriminates the acquired image data by using a discriminator; and performs rough ranking that indicates the priority sequence of performing object detection to each division image before inputting the division images to the discriminator, and selecting only the prescribed number of division images in descending order of rank to perform object detection (S203, S204). The ranking is performed by using a noise amount obtained from a score value of the discriminator in the previously acquired observation data, a maximum value (luminance) in the division image, and an average value or the like in the division image.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an object detection device and an object detection method for detecting a target object from observation data obtained at arbitrary intervals.

Background Art

[0002] For the recovery of a target object or the avoidance of an obstacle, research has been underway on a method of attaching a sensor to a moving object such as an automobile or a ship and detecting an object from observation data acquired by the sensor. For example, for the control of a moving object, the observation range of the sensor is set wide, and it is required to detect an object in real time from the observation data. Also, it is known that with a sensor using sound waves or radio waves, when the frequency is lowered, the observation distance increases, but the resolution per unit distance, that is, the resolution, decreases. In object detection in a moving object, there are cases where data with low resolution is handled.

[0003] In order to efficiently detect an object from a wide range of observation data, in Patent Document 1, object detection is performed from data obtained by observing the entire observation area at a sparse density, and re-observation is performed at a high density in a location where the reliability of object determination is low, and a technique for detecting an object from the data obtained by the re-observation is disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Object detection often utilizes classifiers generated by machine learning techniques because they offer good accuracy. Since the size of the data input to the classifier is often predetermined, detecting objects from a vast area of ​​observational data requires dividing the input data and performing detection in multiple steps. Furthermore, object detection using a classifier requires more processing time than conventional image processing methods. Therefore, if object detection is performed on images at their original resolution, the object detection process for the entire observation area may not be completed within the cycle time (until the next set of observational data is acquired). To address this, one approach is to compress the high-resolution image data to reduce its resolution, thereby decreasing the amount of images input to the classifier (the number of divided images) and speeding up the process. However, image data compression methods tend to significantly degrade object detection performance when the resolution of the observational data is low.

[0006] This invention has been made in view of the above background, and aims to provide an object detection device and object detection method that enable efficient, real-time object detection from large-scale (dimensional) observational data while using a classifier generated by machine learning. Another object of the present invention is to provide an object detection device and object detection method that enables real-time object detection by performing a rough classification of which segmented images to select after dividing large-capacity observation data, thereby narrowing down the number of segmented images to be input to a classifier generated by machine learning. [Means for solving the problem]

[0007] The following are some of the representative features of the invention disclosed in this application. According to one feature of the present invention, an object detection device has an analysis target acquisition unit that intermittently acquires image data obtained by imaging data of an object moving relatively using a sensor, an image division unit that divides the image data acquired by the analysis target acquisition unit into multiple divided images, and an inference unit that takes the selected divided images as input and determines whether or not an object to be detected is included in the divided images. The device is further equipped with a setting unit that selects a predetermined number of divided images from the divided images to be used for object detection processing. The setting unit sets the search range, the type of object detection classifier, and the parameters of the classifier so that the time required for object detection in the selected divided images is less than the target time. The above-mentioned image division unit, inference unit, and setting unit can be realized by a computer executing a specific program.

[0008] According to another feature of the present invention, the inference unit performs object detection processing on segmented images using a classifier according to parameters set in the setting unit. The setting unit also determines the priority of which segmented images should be subjected to object detection processing based on an index that combines one or more indicators, and selects a number of segmented images with high priority equal to the number of searches. The number of segmented images to search is set so that object detection processing for all selected segmented images is completed within the acquisition interval (cycle time) of the image data to be detected. However, instead of fixing the number of segmented images to search, if there is sufficient time after processing for the set number of segmented images is completed and before acquiring the image data of the next observation target, and further image processing can be performed on further segmented images, the system may be configured to perform object detection processing on further segmented images within the remaining time. [Effects of the Invention]

[0009] According to the present invention, in an object detection device, a classifier generated by machine learning techniques can be applied even to observation data of large size. Furthermore, real-time object detection processing using the classifier becomes possible while maintaining high accuracy in object detection. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic block diagram showing an example of the functional configuration of an object detection device 100 according to the first embodiment of the present invention. [Figure 2] This flowchart shows the processing procedure of the object detection device 100 according to the first embodiment of the present invention. [Figure 3] This diagram illustrates an example of dividing an observed image to be analyzed into multiple segmented images. [Figure 4] This diagram illustrates an example of selecting a target segment from a series of segmented images. [Figure 5] This diagram illustrates an example of an additional segmented image, specifically an image cut across multiple regions. [Figure 6] This diagram illustrates an example of calculating the coordinates of a detected object. [Figure 7] This diagram illustrates an example of an input / output interface. [Figure 8] This flowchart shows the processing procedure of an object detection device according to a second embodiment of the present invention. [Figure 9] This figure shows an example of an operating mode of a system using the object detection method of the present invention, where (a) shows the first operating mode and (b) shows the second operating mode. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will be described in detail below with reference to the drawings. Examples of the operation of the object detection system of the present invention include searching for artificial objects such as sunken ships using sound waves, and fish detection. In these object detection processes, sound waves are emitted into the water and the reflected sound waves are detected, the observed data of the detected sound waves is converted into an image, and the imaged observed data is input into a classifier generated by machine learning to detect objects. Locations where objects can be detected include underwater areas such as rivers, seas, lakes, and ponds, as well as places where water accumulates in artificial structures such as swimming pools. Furthermore, the object detection device of the present invention can be applied not only underwater but also in spaces or materials where sound propagates, such as on land, in the air, and underground. Moreover, as long as the input observed data can be converted into an image, the observed data is not limited to that observed with sound waves, but can also be obtained by a wide range of search methods such as radio waves, image capture by cameras, magnetic surveys, and electromagnetic induction surveys. Furthermore, the sensors that collect observational data may not only be directly attached to moving equipment (such as ships), but may also be attached to objects that move synchronously or asynchronously with the moving equipment. Moreover, the system can be applied even when the sensors are fixed in a fixed position in a river, such as for detecting fish in a river. [Examples]

[0012] The following describes embodiments of the present invention. The following embodiments assume the detection of target objects present in water, and detect objects present in water while a vessel is moving or stationary. Figure 1 is a schematic block diagram showing an example configuration of the object detection device 100 of this embodiment. The object detection device 100 can be realized, for example, by a high-performance information processing device or a personal computer. The object detection device 100 is composed of an analysis target acquisition unit 101, an image division unit 102, a setting unit 103, an inference unit 104, an input interface 105, an output interface 106, and a storage device 108, which are connected via a bus 107. The analysis target acquisition unit 101, the image division unit 102, the setting unit 103, and the inference unit 104 can realize the functions of each unit by having the computer's processor execute a specific program. In addition, the analysis target acquisition unit 101 and the image division unit 102 may be realized by dedicated hardware circuits or a combination of hardware circuits and software, rather than being processed by software on a processor. Bus 107 mediates the transmission of data handled by each processing unit connected to bus 107, as well as control information and analysis information.

[0013] The data acquisition unit 101 acquires the image to be analyzed from outside the information processing device 100. This image is, for example, image data obtained by converting observation data acquired from a sound wave sensor (not shown) into an image, and is assumed to be data that has been imaged before being input to the data acquisition unit 101. However, the location of the device (or processing unit) that images the observation data acquired from the sound wave sensor is arbitrary, and the information processing device 100 may be configured to directly input the observation data of the sound wave data and convert the observation data into image data within the information processing device 100.

[0014] The image segmentation unit 102 divides the image data acquired by the analysis target acquisition unit 101 into a set size, and generates a plurality of divided images that can be input to the inference unit 104. Also, when the size of the original observation data is too large and the number of divided images increases, or when the size of the divided images is different from the size of the images input to the classifier of the inference unit 104, it is also possible to configure it to perform operations of enlargement or reduction so as to match the size.

[0015] The setting unit 103 performs settings related to the conditions executed by the inference unit 104 based on the information input from the input device 110 via the input interface 105. The information input includes a target time, a change item, and a selection index. The target time may be input by the user from the input device 110 via the input interface 105, or a time determined in advance based on the acquisition cycle of the sensor may be automatically set. The designation of the change item is the number or ratio of the divided images to be searched, the designation of the classifier used for inference, the parameters at the time of inference, etc. Among these, at least one of them may be fixed, or any of them may be allowed to be changed. Furthermore, it may be something that specifies an item for which the amount of change is desired to be reduced. The divided images to be searched are selected based on the selection index. At this time, one index may be used, or a composite obtained by weighting and combining a plurality of indexes may be used. By the setting unit 103, the divided images to be searched, the classifier used for inference, and its parameters are determined.

[0016] The inference unit 104 performs an inference operation on the selected divided images using the classifier used for inference and its parameters determined by the setting unit 103. Thereby, it is possible to determine whether or not the target object is shown in each of the divided images. Note that the output result obtained by the inference unit 104 is the coordinate information of the detected object. Since the classifier outputs the coordinates of the detected object in the divided image, the inference unit 104 considers the coordinates of the origin of the divided image in the entire observation data and obtains the coordinates of the detected object.

[0017] The input interface 105 is connected to the input device 110 and receives the user's input operations performed using the input device 110. The input device 110 is configured using, for example, a mouse, a keyboard, or the like. When the user inputs various instruction operations and selection operations for the information processing device 100 using the input device 110, the content of the input operation is transmitted through the input interface 105 to each part (101 - 104) inside the information processing device 100. Thereby, each part (101 - 104) can perform processing according to the user's input operation. For example, in the setting unit 103, based on the user's input operation performed through the input interface 106, a divided image to be analyzed, an identifier to be used for inference, and the like can be acquired.

[0018] The output interface 106 is connected to the display device 111 and enables the output of characters, symbols, images, videos, etc. in color display to the display device 111. The displayed image data includes various types of information related to object detection, such as the observation data of the analysis target imaged and the coordinates of the detected object. The display device 111 is, for example, a display device connected to the object detection device 100. The output interface 106 may directly display the coordinates of the detected object, or may display it by superimposing a mark indicating the position of the object on the imaged observation data. Note that the display device 111 is not limited to only the device directly connected to the object detection device 100, and it may be configured to be displayed on the display screen of a personal computer, a tablet, a smartphone, etc. used by the user. In that case, the screen information may be transmitted from the output interface 106 to the device having the display screen by near - field wireless communication or via a network. Thus, the present invention does not only consist of the object detection device 100 as a stand - alone computer and its peripherals connected thereto by wire, but may also be configured as a system combined with the object detection device 100 and an external information device (such as the above - mentioned tablet, smartphone, or another computer) that can be connected via a wireless or wired network.

[0019] Next, the object detection procedure in the object detection device 100 of this embodiment will be explained using the flowchart in Figure 2. The flowchart shown in Figure 2 can be realized by a processor (not shown) included in the object detection device 100 executing a computer program.

[0020] First, the object acquisition unit 101 acquires an image to be analyzed (step S201). The acquired image is obtained by intermittently acquiring the observation data described above and creating an image, for example, image data of 100,000 pixels or more. The device that emits sound waves and the device that images reflected waves are provided separately from the object detection device 100. Here, "intermittently" refers to a case where the observation results of a specific point are acquired at predetermined time intervals (for example, every few seconds). For example, by sequentially sensing the entire observation area along a specific scan direction, it refers to a case where observation data for a single point is obtained at constant or irregular time intervals. In addition, the image acquired in step S201 may be image data selected from the input device 110 by user operation from among the images stored in the storage device 208.

[0021] Next, the image splitting unit 102 splits the target image acquired in step S201 (step S202). This image splitting process will be explained using Figure 3(a). Figure 3(a) is an example of image splitting performed in step S202. As shown in the left-hand figure, when the target image 301 is acquired as the image to be analyzed, the image splitting unit 102 performs a splitting process, and multiple (in this case, six) split images 302a to 302f are generated. The split images 302a to 302f shown in Figure 3(a) do not overlap in terms of region. The splitting method is to simply split the image at fixed intervals from the starting point of the observation data, and the splitting boundaries can be fixed. In the case of observing a moving object, the starting point of the split image may be shifted based on the amount of movement of the observed object relative to the sensor, in which case the object to be found can be continuously captured within the same split image. In this way, the image splitting unit 102 splits the image input via the input interface 105 in accordance with the magnitude of the input to the classifier in the inference unit 104. In this case, if some of the divided images extend beyond the target image, it is best to interpolate them with a background value (such as 0).

[0022] If the size of the divisions differs from the size of the classifier's input, it is also effective to enlarge or reduce the divided image 302a to suit the size of the classifier's input, as shown in Figure 3(b). In Figure 3(b), image 302a is reduced horizontally to generate reduced image 303a. For image reduction, known image reduction algorithms or image reduction programs can be used. Although Figure 3(b) describes the reduction process for only the divided image 302a, the remaining divided images 302b to 302f are similarly reduced to prepare reduced images 303b to 303f. Note that in Figure 3, the divided images 302a to 302f are not shown to overlap, but it is also possible to generate a larger number of divided images by performing image division that partially overlaps.

[0023] In this embodiment, since deep learning or the like is assumed as the discriminator using a machine learning method, square divided images such as 300×300 pixel images or 500×500 pixels, which are common in deep learning, are prepared as the size of the input to the discriminator. However, since the deformation processing of the divided image by the above image reduction can be applied, there is no limitation on the size and shape of the image to be divided. However, it is important that the magnification or reduction rate be aligned among a plurality of divided images, and it is important to unify the appearance for the discriminator. Preferably, it is recommended that the size of the divided image be the same resolution as that during the learning of machine learning.

[0024] Returning to the description of FIG. 2, the setting unit 103 selects a divided image to be inferred from the divided images generated in step S202, and selects a discriminator to be used for object detection and specifies parameters (step S203). The selection of the divided image is a process of narrowing down the number m (where m < n) of divided images to be searched using the discriminator from the divided observation data (total number n). Here, an example of the selection of the divided image to be inferred performed in step S203 will be described using FIG. 4.

[0025] Figure 4 shows an example where the observed data is divided into six segmented images, 401a to 401f, and three of these segmented images are selected. First, each segmented image is given a rough ranking based on one or more indicators. Examples of selection indicators include the maximum value (brightness) within the segmented image, the amount of noise calculated from the average value within the segmented image, and the classifier score value (likelihood of the target object) from the previously acquired data. A calculation formula using these selection indicators is set up, and points are assigned to each segmented image 401a to 401f as shown in Figure 4. For example, a segmented image with a large maximum brightness will be given a higher score as shown in Figure 4. Regarding the amount of noise, the score shown in Figure 4 may be higher or lower depending on the observed object. The calculation formula for this scoring can be set appropriately, taking into account the characteristics of the observation environment and the characteristics of the observed object. Furthermore, as an indicator based on the current situation, the scores may be corrected by doubling the value of the above indicator in segmented images where an object was detected at the most recent time, or for segmented images where a large amount of time has passed since the most recent selection, the scores may be corrected by increasing them by a fixed amount or a variable amount. By performing score correction in this way, segmented images of regions where the presence of an object has been continuously observed will be reliably assigned to a higher rank. In addition, it is possible to adjust the scores so that segmented images of regions that have been excluded from object detection processing using the classifier over multiple observation data will always be subjected to object detection processing at least once every few observations.

[0026] In the example in Figure 4, based on the index value, six segmented images 401a to 401f and their scores are shown numerically. Here, since the scores of segmented images 401f, 401a, and 401e are large in that order (74, 50, and 43), these three segmented images 401f, 401a, and 401e are selected and input to the inference unit 104. In the example in Figure 4, only the three with the largest scores are selected, but the number of segmented images to be selected is not limited to three. Also, depending on the selection index, the segmented images to be input to the inference unit 104 may be selected in order from the smallest score. Furthermore, if the above method is adopted, certain regions (corresponding to 401b, 401c, and 401d in Figure 4) will remain unselected among the observation data acquired at multiple intervals. Therefore, the scores of the segmented images in regions that have not been selected for a predetermined number of consecutive times are increased to avoid the occurrence of unselected segmented images that are not input to the inference unit 104 at all. Furthermore, a rough ranking based on scores can also be performed using external indicators obtained from external devices other than the object detection device 100.

[0027] In Figure 4, the observed data was divided into non-overlapping, equally sized segmented images. However, the data may be divided so that there is overlap between the segmented images, or furthermore, segmented images may be extracted again in a range shifted from the extraction boundary of the segmented images. Figure 5 shows an example of segmented images that have been re-extracted to span multiple regions as additional segmented images. Here, after segmented images 511 to 516 have been extracted, a new segmented image 517 has been extracted near coordinate 501 where the target object was detected most recently (T-1, if the current time is T). In other words, a total of seven segmented images 511 to 517 have been extracted from the observed data shown in Figure 5. Points are calculated for these segmented images 511 to 517 using the same method as described in Figure 4, and a predetermined number of segmented images with high points are selected and input to the inference unit 104. In this way, the extraction range of the segmented images is adjusted according to the most recent situation, and furthermore, segmented images with overlapping extraction ranges are prepared to prevent overlooking target objects. effect It can be effectively suppressed.

[0028] If the classifier and parameters to be used for object detection are determined, the time required for object detection per segmented image can be accurately estimated from prior experiments or user specifications via the input interface 105. In other words, since the target time and the number of segmented images to be selected are determined from the size of the input image data or from user specifications via the input interface 105, the time that can be spent on object detection processing for one segmented image can be calculated by dividing the target time by the number of selected segmented images. In addition, since the processing time and expected accuracy for each combination of classifier and parameters can be determined from prior experiments or user specifications via the input interface 105, the target processing time that can be allocated to processing by the inference unit 104 within one acquisition cycle can be calculated. By dividing the target processing time in the inference unit 104 by the time required for object detection per segmented image and rounding down to an integer, the number of segmented images for which object detection can be processed within the target time can be calculated. In step S203, the number of segmented images that can be processed within the target time calculated above is selected, starting with the segmented images with the highest ranking. Furthermore, the system may be configured to process only the segmented images whose ranking is within a predetermined number and which satisfy a score threshold specified in advance or by the user via the input interface 105 (for example, in Figure 4(a), the score is 50 or higher) using the inference unit 104.

[0029] Returning to the explanation of Figure 2, the inference unit 104 performs object detection processing based on the settings in step S203 (step S204). The inference unit 104 performs object detection processing for each segmented image selected by the setting unit 103 using the classifier and parameters selected by the setting unit 103. As the classifier here, various known object detection techniques can be applied, such as pre-trained models created using various machine learning techniques, including those using deep learning techniques such as YOLO (You Only Look Once) and SSD (Single Shot Multibox Detector), or those that do not rely on deep learning techniques such as SVM (Support Vector Machine) and RF (Random Forest), as well as image processing methods that make judgments based on the maximum value (brightness) or shape. If the computational complexity of the model decreases, the time required for object detection processing will decrease. Examples of parameters include those that affect the time of object detection processing, such as the number of past data used as input (number of recursions) and the number of weak classifiers, and the number of objects to detect (threshold). The number of recursives and weak classifiers represents the number of iterations of the process, so reducing them will decrease the time required for object detection.

[0030] The classifier included in the inference unit 104 outputs the coordinates (a,b) of the detected object. Since these coordinates (a,b) are within the segmented image, the coordinates of the detected object within the observed data can be determined by taking into account the coordinates in the observed data of the segmented image. Here, using Figure 6, we will explain an example of how to determine the coordinates of the detected object in the observed data, which is performed in step S204.

[0031] Figure 6 shows how to determine the coordinates of the detected object 601 relative to the origin 602 in the observation data. The classifier detects objects when the segmented image 603 is input. If the classifier detects object 601 at that time, it outputs the coordinates of object 601 relative to the origin 604 in the segmented image 603. Since the coordinates of the detected object in the observation data are the coordinates (a,b) relative to the origin 604 in the segmented image 603, the coordinates of object 601 in the observation data are obtained by adding the coordinates (x,y) of the origin 604 in the segmented image 603 relative to the origin 602, resulting in (a+x,b+y).

[0032] Returning to the explanation of Figure 2, the inference unit 104 outputs the coordinates of the detected object (step S205), which is the last step in the overall process after step S204. The coordinates obtained here may be output to the output device 16 in text format, transmitted to be displayed on the display screen of an external mechanism such as a control mechanism for a moving object, or presented to the user graphically by superimposing the position of the detected object onto the observation data displayed on the output device 16 by placing a mark on the observation data.

[0033] Figure 7 shows an example of the display screen 700 on the display device 111. The display screen 700 displays both the content input and set via the input interface 105 and the content outputted via the output interface 106. First, the input fields 711 to 733, which correspond to the settings target via the input interface 105, will be explained.

[0034] Input field 711 is the target time for performing object detection processing on the observed data. This field contains a value determined based on the sensor's acquisition cycle or a value specified by the user. The target time is, for example, a few seconds. If the user does not specify a time, it may be configured to automatically set to the same time as the sensor's acquisition interval or a shorter time. Input field 721 is for numerically inputting the size of the divided images 302a~302f (see Figure 3(a)) after dividing the acquired target image 301 (see Figure 3(a)). Here, for example, if the divided images are square, "300" meaning 300 x 300 pixels would be entered. Input field 722 is a setting related to the degree of overlap of the divided images; a numerical value is set to indicate whether or not to crop the image so that it overlaps with adjacent divided images. 、721、 7 22 This field contains either a value determined based on the settings during training or a value specified by the user.

[0035] In input fields 731-733, it is recommended to specify the indicators that you want to keep changing minimally in the setting unit 103. Input field 731 is where you enter the search number, which indicates the maximum number of objects to be detected within a single search range. In the classifier selection field 732, the user selects either the pre-trained classifiers A or B. In the object detection device 1 of Figure 7, the inference unit 104 is selected as either "Method A" (e.g., YOLO) or "Method B" (e.g., SVM). In the parameter selection field 733, the type of parameter and its numerical value are specified by entering them in either input field 733a or 733b. However, the search number and parameter values ​​can be either pre-set or entered and changed by the user.

[0036] Next, we will explain the observation image 740, which corresponds to the output interface, and the detected objects 741-743. The observation image 740 is an image of the observation data of sound waves that were emitted into the water and reflected, and this observation image 740 is displayed scrolling from bottom to top as the ship moves. For example, the center point of the observation image 740 corresponds to the installation position of the sound wave observation equipment, the upward direction corresponds to the front in the direction of travel, and the downward direction corresponds to the rear in the direction of travel. Here, the three objects 741-743 detected by the object detection processing by the classifier are displayed superimposed on the observation image 740 as markers (in this case, X marks) corresponding to the positions of objects 741-743. Note that in Figure 7, the case where three objects were detected is shown as an example, so only three markers 741-743 are displayed. However, within the range of the maximum number of searches set in the input field 731 (for example, 10), a number of markers corresponding to the number of detected objects will be displayed in the observation image 740.

[0037] As described above, in the object detection device 100 of this embodiment, the user can individually specify fixed and priority indicators used to narrow down the search range, such as selecting the search range based on observation data, selecting the object detection classifier, and adjusting parameters. This allows the user to specify weights and whether or not to use them, or to apply weights determined by an optimization method. Furthermore, the indicators used to narrow down the search range can be changed according to the state, such as when an object is detected or not. [Examples]

[0038] In Example 1, it was assumed that the time required for object detection processing by the classifier (processing time) was approximately constant for each segmented image (fixed time). This fixed time is set based on the expected value (average value) and maximum value of the processing time in prior experiments. However, if it is estimated that the observed image 740 does not contain values ​​that are likely to be the target object, and an object detection method is used that allows the classifier to cancel the object detection processing for each segmented image, the processing time may differ for each segmented image. In that case, if the processing time is treated as a fixed time, canceling the object detection processing for a segmented image midway will result in a buffer time before the next observed data is input after the processing of the set number of searched segmented images is completed, and this buffer time will not be effectively utilized. Therefore, in the second example, the buffer time that was not selected is not wasted, and the unselected segmented images are further input to the inference unit 104 for additional object detection processing.

[0039] Figure 8 is a flowchart showing the processing procedure of the object detection device 1 according to a second embodiment of the present invention. The hardware configuration of the object detection device 1 that performs the processing procedure shown in Figure 8 is the same as the configuration shown in Figure 2, and can be implemented by software with only minor changes to the program that enables the setting unit 103 and the program that enables the inference unit 104, which are executed by the CPU.

[0040] First, as in Example 1, the target image is acquired and divided in steps S201 and S202, and the divided images are ranked, and the classifier and parameters are selected in step S203. Next, in step S204A, the divided images are subjected to object detection processing one by one, starting with the highest priority divided images as shown in step S204 in Figure 2. However, if the system has multiple computing resources such as CPUs and GPUs that allow for parallel processing, it is also possible to perform object detection processing on the divided images as many times as there are parallel processes.

[0041] A process has been added to check the margin of the object detection process relative to the target time (step S801). Each time object detection is performed, the time required for object detection is measured for one segmented image, and the maximum value of this measurement, as well as the cumulative time since the start of object detection for the observed data, is recorded. If the sum of the cumulative time and the maximum time required for object detection does not exceed the target time, it is determined that there is sufficient processing time, and the next segmented image is subjected to object detection. On the other hand, if the sum of the cumulative time and the maximum time required for object detection exceeds the target time, it is determined that there is insufficient processing time, and the process proceeds to step S205. In this way, even if the processing time differs for each segmented image, object detection can be performed on as many segmented images as possible with a processing time close to the target time.

[0042] Step S205 is the same process as in Embodiment 1, and the coordinates may be output in character format and transmitted to an external mechanism such as a control mechanism for the moving object, or marks may be placed on the observed data and presented to the user graphically. In the second embodiment, even if the execution of inference takes longer than expected due to interrupt processing or some read / write processing in the object detection device 100 and the object detection processing for the planned number of divided regions cannot be completed within a predetermined time, the process can be moved on to object detection processing for the next observed data.

[0043] Figure 9 shows an example of an operating configuration of a system using the object detection method of the present invention. Figure 9(a) shows the first operational configuration. The first operational configuration is an example in which the equipment necessary for object detection processing is installed on a water-based moving object such as a ship, and the identification processing is completed as a local system. The sound wave observation device 901 transmits sound waves into the water and receives reflected signals, and also converts the received reflected signals into observation images. The detection device 902 performs detection processing using the observation images as input. The detection device 902 corresponds to the object detection device 100 in Figure 1. The detection results are displayed on the detection result display device (monitor) 903. The detection result display device 903 corresponds to the display device 111 in Figure 1.

[0044] Figure 9(b) is a block diagram illustrating the second operational configuration. The second operational configuration is an example in which the equipment necessary for object detection processing is installed over a network and identification processing is performed remotely. The local system, such as a ship, consists of an observation device 911, a detection result display device (monitor) 912, and a local control device. The observation image observed and converted by the observation device 911 is transmitted to the remote system via the local control device and the network 918. The remote system consists of an observation data storage device 914, a detection device 915, a detection result storage device, and a remote control device 917. The observation image transmitted via the network 918 is received by the remote control device 917 and stored in the observation data storage device. The stored observation image is processed by the detection device 915, and the results are stored in the detection result storage device 916. The detection results are also displayed on the detection result display device (monitor) 912 via the remote control device 917, the network 918, and the local control device 913.

[0045] Depending on the communication status of network 918, the observation data storage device 914 and the detection result storage device 916 may be installed on the local side, or they may not be installed at all. In addition to serving as a buffer for communication, each storage device can also be used to save results, making it possible to review them later.

[0046] Although the present invention has been described above based on embodiments, the present invention is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the invention. For example, the observation data to be imaged is not limited to sound wave observation values, but may also be image data acquired from a camera, radar images manipulated by radio waves, etc. [Explanation of Symbols]

[0047] 100 Object detection device 101 Analysis target acquisition unit 102 Image division section 103 Settings Section 104 Reasoning part 105 Input Interfaces 106 Output Interfaces 107 Bus 108 Storage device 301 (Target image where object detection occurs) 302a~302f Divided images 700 display screen

Claims

1. An analysis target acquisition unit intermittently acquires image data by visualizing data obtained from observing objects moving relatively using sensors, The image segmentation unit divides the image data acquired by the analysis target acquisition unit into multiple segmented images, An object detection device having an inference unit that inputs the selected segmented image and determines whether or not the object to be detected is included in the segmented image, The system selects a predetermined number of segmented images from the aforementioned segmented images to be used for object detection processing, and includes a setting unit that sets the search range, the type of object detection classifier, and the parameters of the classifier so that the time required for object detection in the selected segmented images is less than the target time. The system includes an input interface for setting the parameters used in the classifier, and an output interface for highlighting detected objects on the observation data acquired by the analysis target acquisition unit. The object detection device is characterized in that the inference unit sets the parameters by receiving a predetermined time required for object detection and a specification of an inference model to be used to narrow down the search range as the target of modification from the user via the input interface, and performs object detection processing of the divided images according to the parameters.

2. The object detection device according to claim 1, characterized in that the setting unit determines the priority of whether an object should be subjected to object detection processing based on one or more indicators combined for the divided images, and selects the number of divided images with the highest priority according to the number of searches.

3. The object detection device according to claim 2, characterized in that the setting unit fixes at least one of the search range, the type of classifier for object detection, and the parameters in the classifier, and changes the unfixed items so that the time required for object detection processing is less than the target time.

4. The object detection device according to claim 2, characterized in that the setting unit changes the parameters when an object is detected, when an object is not detected, or when specified by the user.

5. The object detection device according to claim 2, characterized in that the setting unit sets the parameters using an index determined by a prior experiment or an optimization method using data acquired to date.

6. The object detection device according to claim 2, wherein the setting unit transmits the number of division images with the highest priority to the inference unit, and when the processing of the inference unit is completed, if there is sufficient processing time relative to the target time, transmits the next highest priority division image after the completed division image to the inference unit.

7. An object detection method using an object detection device that continuously acquires image data obtained by imaging data of an object moving relatively using a sensor, divides the acquired image data to generate multiple divided images, and uses a classifier generated by a machine learning method to detect whether or not the object to be detected is included in the divided images, with the divided images as input, a) Acquire the image data and generate multiple divided images, b) Before inputting the data into the classifier, each of the divided images is scored using the score value of the classifier in the previously acquired observation data, the maximum value (luminance) in the divided image, the amount of noise calculated from the average value in the divided image, etc., thereby assigning a rough ranking to indicate the priority for object detection. c) Select a predetermined number of the aforementioned segmented images in descending order of rank, or in descending order, and perform object detection using the classifier. d) Display the coordinate data of the detected object via the output interface. e) Repeat the processes a) to d) above. A method for detecting an object characterized by the following features.

8. The aforementioned image data is acquired repeatedly in a fixed observation cycle, The object detection method according to claim 7, characterized in that the predetermined number of divided images is set to be less than the total number of divided images and within a range of a number that allows the object detection process to be completed using the classifier during the observation cycle.

9. The aforementioned image data is acquired repeatedly in a fixed observation cycle, The object detection method according to claim 7, characterized in that when the object detection processing of a set number of segmented images is completed during the observation cycle of the image data, if there is sufficient time remaining to perform object detection processing on the next segmented image, the object detection processing of the next higher-ranked segmented image is performed.

10. The object detection method according to claim 8, characterized in that the object detection device determines the number of searches using an index determined by an optimization method using prior experiments or data acquired to date.

11. The object detection method according to claim 7, characterized in that a correction process is performed to increase the number of points in the segmented image in which an object was detected at the most recent time.

Images