Classification device, classification system and classification method

Abstract

The classification device (100) includes a first target identification unit (113) that identifies a target, a feature position determination unit (114) that determines a feature position of the target, a first display processing unit (115) that generates first display control information for displaying the feature position of the target superimposed on a radar image, a feature position editing unit (116) that calculates feature specifications of the target using the edited feature position when an editing operation is performed on the feature position of the target superimposed on the radar image, a second target identification unit (117) that identifies, from known feature specifications for each target candidate, those that correspond to the feature specifications of the target calculated by the feature position editing unit (116) and selects a target candidate corresponding to the identified feature specifications, and a second display processing unit (118) that generates second display control information for displaying the selected target candidate.

Description

[Technical field]

[0001] The present disclosure relates to a classification device, a classification system and a classification method. [Background technology]

[0002] There is known a monitoring system that monitors the sea and coastline using still or moving ISAR images (hereinafter, referred to as ISAR images) generated by an inverse synthetic aperture radar (hereinafter, referred to as ISAR images). For example, in a radar device described in Patent Document 1, a display processing unit displays two measurement lines for measuring the overall length (LOA; Length Over All) of a target such as a ship on a display unit on which an ISAR image is displayed in response to a user's instruction. A brightness adjustment unit determines an area outside the two measurement lines in the ISAR image and increases the brightness of this area. The display processing unit updates the positions of the two measurement lines in response to a user's instruction. A LOA measurement unit calculates the distance between the two measurement lines and measures the overall length of the ship. An operator (user) can visually check the ISAR image to identify the type of target based on the LOA of the target in the ISAR image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-25972 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional techniques have a problem in that they are sometimes unable to accurately classify targets based on their characteristics in ISAR images. For example, the overall length of a target, which is a characteristic of the target, varies depending on the aspect angle of the target relative to the direction of the sensor's radio wave irradiation. For this reason, the conventional technology described in Patent Document 1 cannot estimate the true overall length of a target based on the aspect angle value in an ISAR image, and therefore may not be able to accurately classify the target.

[0005] The present disclosure is intended to solve the above-mentioned problems, and aims to provide a classification device that can improve the accuracy of target classification. [Means for solving the problem]

[0006] The classification device according to the present disclosure includes a first target identification unit that identifies a target from a radar image; a feature range calculation unit that calculates a feature range, which is the range of a feature portion of the target, based on aspect angle information of the target included in the radar image, target identification result information by the first target identification unit, and known feature specifications for each target candidate; a first display processing unit that determines a feature position, which is the position of the feature portion of the target, using the feature range calculated by the feature range calculation unit and generates first display control information for overlaying and displaying the determined feature position on the radar image; a feature position editing unit that, upon obtaining editing operation information indicating an editing operation on the feature position of the target in the radar image displayed on the display unit based on the first display control information, calculates the feature specifications of the target using the feature position of the target where the editing operation indicated by the editing operation information was performed; a second target identification unit that identifies known feature specifications for each target candidate that correspond to the feature specifications of the target calculated by the feature position editing unit and selects a target candidate corresponding to the identified feature specifications; and a second display processing unit that generates second display control information for displaying the target candidate selected by the second target identification unit. Effect of the Invention

[0007] According to the present disclosure, when the characteristic position of a target identified from a radar image is determined and editing operation information indicating an editing operation on the characteristic position of the target in the radar image displayed on the display unit is obtained, the characteristic parameters of the target are calculated using the characteristic position where the editing operation was performed, and the characteristic parameters corresponding to the calculated characteristic parameters of the target are identified from the known characteristic parameters of each target candidate, and the target candidate corresponding to the identified characteristic parameters is displayed. Even if the characteristic position of the target identified from the radar image and the characteristic position of the target in the radar image are far apart, the target is re-identified using the edited characteristic position of the target. This allows the classification device according to the present disclosure to improve the accuracy of target classification. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration example of a categorization system according to a first embodiment. [Diagram 2] 1 is a block diagram showing an example of the configuration of a classification device according to a first embodiment. [Diagram 3] FIG. 2 is a schematic diagram showing an overview of the relationship between a sensor, a target, a target direction, a radio wave irradiation direction, and an aspect angle. [Figure 4] 4A and 4B are schematic diagrams showing an example (1) of a learning model and a learning database. [Diagram 5] FIG. 2 is a schematic diagram showing an overview of a feature specification database. [Figure 6] FIG. 2 is a schematic diagram showing an overview of the relationship between a radar image, characteristic parameters, characteristic range, and radio wave irradiation direction. [Figure 7] FIG. 2 is a schematic diagram showing an overview of a process for superimposing a characteristic position of a target on a radar image. [Figure 8] 4 is a flowchart showing an operation of the radar device. [Figure 9] 4 is a flowchart showing a categorization method according to the first embodiment. [Figure 10] 10A and 10B are schematic diagrams showing an example (2) of a learning model and a learning database. [Figure 11]11A and 11B are block diagrams showing a hardware configuration for realizing the functions of the classification device according to the first embodiment. [Figure 12] FIG. 11 is a block diagram showing an example of the configuration of a classification device according to a second embodiment. [Figure 13] 13 is a flowchart showing a categorization method according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Embodiment 1 FIG. 1 is a block diagram showing a configuration example of a classification system 1 according to a first embodiment. In FIG. 1, the classification system 1 is a system for classifying targets existing in an external space, and includes a classification device 100 and a radar device 200. Targets to be classified are, for example, moving bodies such as ships or aircraft. The classification device 100 classifies targets from radar images generated by the radar device 200. The radar images are, for example, radar still images or moving image information obtained by ISAR or synthetic aperture radar (hereinafter abbreviated as SAR). The radar device 200 is a device that transmits radio waves to an external space and generates a radar image of a target using echoes of the radio waves received from a target existing in the external space.

[0010] Fig. 2 is a block diagram showing a configuration example of the classifying device 100 according to the first embodiment. In Fig. 2, the classifying device 100 includes a storage unit 101 in which a feature specification database is stored, a display unit 102, and an operation unit 103, and further includes a calculation unit not shown in Fig. 2. The storage unit 101 is a storage device included in a computer functioning as the classifying device 100, and includes storage such as a hard disk drive (HDD) or a solid state drive (SSD), or a memory 1004 shown in Fig. 11B, which will be described later. Note that the storage unit 101 may be provided outside the classifying device 100, as long as it is accessible from the classifying device 100.

[0011] The display unit 102 is a display device included in a computer that functions as the classification device 100. For example, the display unit 102 is an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display device. The operation unit 103 is an input device that accepts operations on the screen display of the display unit 102 included in the computer that functions as the classifying device 100. For example, the classifying device 100 is a touch panel that is integrated with the display screen of the display unit 102. The operation unit 103 may also be, for example, a mouse or a keyboard.

[0012] The calculation unit controls the overall operation of the classification device 100. The calculation unit includes an aspect angle calculation unit 111, a radar image input unit 112, a first target identification unit 113, a feature range calculation unit 114, a first display processing unit 115, a feature position editing unit 116, a second target identification unit 117, and a second display processing unit 118. The calculation unit executes a classification application to realize various functions of the aspect angle calculation unit 111, the radar image input unit 112, the first target identification unit 113, a feature range calculation unit 114, the first display processing unit 115, the feature position editing unit 116, the second target identification unit 117, and the second display processing unit 118.

[0013] The radar device 200 includes a transmission / reception unit 201 and a tracking processing unit 202. The transmission / reception unit 201 transmits and receives radio waves via a radar antenna. The tracking processing unit 202 performs processing for tracking a target moving in an external space.

[0014] FIG. 3 is a schematic diagram showing an overview of the relationship between the sensor 13, the target Ta, the direction D1 of the target Ta, the radio wave irradiation direction D2, and the aspect angle θ. In FIG. 3, the sensor 13 is a radar antenna that radiates radio waves to an external space and receives an echo of the target Ta from the external space. The sensor 13, which is a radar antenna, is included in the transmitting / receiving unit 201. The target Ta is, for example, a ship shown by a dashed line in FIG. 3. The arrow through which the bow of the target Ta passes is the direction D1 of the target Ta. The radio wave irradiation direction D2 is the direction of the radio waves radiated from the sensor 13. The aspect angle θ is the angle that the direction D1 of the target Ta makes with respect to the radio wave irradiation direction D2 (the direction of the sensor 13).

[0015] The transmitting / receiving unit 201 performs signal processing to generate a radar image of the target based on the echo of the radio wave of the target received by the sensor 13. In the following description, it is assumed that the radar image is an ISAR image. In addition, the radar device 200 constantly acquires real-time information on the radio wave irradiation direction D2 in order to control the radio wave irradiation direction D2 shown in FIG.

[0016] The aspect angle calculation unit 111 acquires information indicating the direction D1 of the target Ta from the tracking processing unit 202, acquires information indicating the radio wave irradiation direction D2 acquired from the transmitting / receiving unit 201, and calculates the aspect angle θ using the direction D1 of the target Ta and the radio wave irradiation direction D2. For example, when the target Ta is disposed as shown in Fig. 3, the aspect angle calculation unit 111 calculates the aspect angle θ according to the following formula (1). In the following formula (1), the unit of angle is "degrees". Aspect angle θ = 180 - radio wave irradiation direction D2 + target Ta direction D1 (1)

[0017] The radar image input unit 112 acquires a radar image from the transmitting / receiving unit 201 included in the radar device 200. The radar image acquired by the radar image input unit 112 is output to the first target identification unit 113.

[0018] The first target identification unit 113 identifies the target Ta from the radar image. For example, in the radar image, the luminance value on the image is determined according to the intensity of the radio wave received as a reflected wave. The first target identification unit 113 binarizes the image of the radar image and identifies the target by performing pattern recognition on the binarized image. The function of the aspect angle calculation unit 111 may be provided by the feature range calculation unit 114, and the function of the radar image input unit 112 may be provided by the first target identification unit 113. In this case, the classification device 100 does not need to include the aspect angle calculation unit 111 and the radar image input unit 112.

[0019] Furthermore, when a radar image is input, the first target identification unit 113 may identify the target using a first learning model that outputs target identification result information. Fig. 4A is a diagram showing an overview of the learning database 14 for generating the first learning model 15. Fig. 4B is a diagram showing an overview of the first learning model 15. As shown in Fig. 4A, the learning data registered in the learning database 14 includes a radar image E and annotation information F corresponding to this radar image E, and is stored in, for example, the storage unit 101. The annotation information F includes candidates for the target X included in the radar image X1. The first learning model 15 is generated by learning using learning data registered in the learning database . It should be noted that the radar image E and the corresponding annotation information F included in the learning data are not limited to those generated by the transmitting / receiving unit 201.

[0020] As shown in Fig. 4B, the first target identification unit 113 inputs the radar image E to the first learning model 15. As a result, the first learning model 15 infers the name or type of the target Ta in the radar image E, and outputs the inferred name or type of the target Ta as identification result information. The first target identification unit 113 uses the first learning model 15, which enables the classification device 100 to accurately classify the target Ta using the radar image E.

[0021] The feature range calculation unit 114 calculates a feature range, which is the range of the feature part of the target Ta, based on the aspect angle information of the target Ta included in the radar image E, the identification result information of the target Ta by the first target identification unit 113, and the known feature specifications for each target candidate. For example, the feature range calculation unit 114 receives the aspect angle θ and the identification result information of the target Ta, and calculates the feature range using the received information.

[0022] 5 is a schematic diagram showing an overview of the feature specification database, and shows the contents of the feature specification database stored in the storage unit 101. In the feature specification database, known feature specifications H are registered in association with annotation information G indicating the names or types of various target candidates. The feature specifications are parameters that represent characteristic parts of a target.

[0023] For example, if the target is a ship, the characteristic parameters include the ship's overall length (LOA), which is determined for each type of ship, the length between the bow and the mast, and the length between the bow and the container, etc. A single target candidate (e.g., target X) may be associated with multiple types of characteristic parameters (e.g., characteristic parameter X2, characteristic parameter X3, ...).

[0024] The feature range calculation unit 114 refers to the feature specification database and reads feature specifications corresponding to the target information indicated by the identification result information by the first target identification unit 113. For example, if the target information indicated by the identification result information is target X, the feature range calculation unit 114 reads feature specifications consisting of multiple types of feature specifications X2, feature specifications X3, ... corresponding to target X from the feature specification database.

[0025] The characteristic range calculation unit 114 converts the characteristic parameters read from the characteristic parameter database into a characteristic range. The characteristic range is information indicating the range of a characteristic part of a target, and is, for example, the slant range length of the target in a radar image when the target is a ship.

[0026] 6 is a schematic diagram showing an overview of the relationship between the radar image E, the characteristic parameter H, the characteristic range I, and the radio wave irradiation direction D2. For example, if the target Ta shown in the radar image E is a ship, and the characteristic parameter read from the characteristic parameter database is the length between the bow and the mast, the length between the bow and the mast is expressed as the characteristic range I, which is the slant range length in the radar image E.

[0027] Since the slant range axis is parallel to the radio wave irradiation direction D2, as shown in Fig. 6, there is a proportional relationship that depends on the aspect angle θ between the characteristic parameter H, which is the length between the bow and the mast, and the characteristic range I. When the aspect angle θ is defined by the relationship shown in Fig. 3, the characteristic range calculation unit 114 can calculate the characteristic range I by using the following formula (2). The characteristic range calculation unit 114 outputs the radar image E and the calculated characteristic range I to the first display processing unit 115. Feature range I(m) = Feature parameter H(m) × cos(aspect angle θ) (2)

[0028] The first display processing unit 115 determines a characteristic position PA of the target Ta using the characteristic range I calculated by the characteristic range calculation unit 114, and generates first display control information for displaying the determined characteristic position PA superimposed on the radar image E.

[0029] 6, the first display processing unit 115 converts the radar image E into an image in which one of the coordinate axes in the radar image E is a slant range axis parallel to the radio wave irradiation direction D2. The first display processing unit 115 uses the characteristic range I to generate first display control information for displaying the characteristic position PA of the target Ta superimposed on the radar image E. The characteristic position is the position of a characteristic part of the target Ta.

[0030] Fig. 7 is a schematic diagram showing an outline of the process of superimposing the characteristic position PA of the target Ta on the radar image E. In Fig. 7, the target Ta is a ship. The characteristic positions PA of the target Ta in the radar image E are a characteristic position J1 related to the bow, a characteristic position J2 related to the stern, and a characteristic position J3 related to the mast. The first display processing unit 115 calculates the center of gravity position G1 of the radar image E using pixel value information or complex signal information of the radar image E.

[0031] Of the characteristic positions PA of the target Ta, the characteristic position J1 for the bow is a position shifted in the negative direction of the slant range axis by half the value of the characteristic range I1 from the center of gravity position G1 in the characteristic range I1 calculated by the characteristic range calculation unit 114 based on the LOA of the target Ta. Meanwhile, the characteristic position J2 for the stern is a position shifted in the positive direction of the slant range axis by half the value of the characteristic range I1 from the center of gravity position G1 in the characteristic range I1. When the characteristic range I calculated according to the above formula (2) is a negative value, the positive and negative directions of the slant range axis are reversed.

[0032] Next, the first display processing unit 115 calculates, as a characteristic position J3 for the mast, a position shifted from the characteristic position J1 in the positive direction of the slant range axis by the length of the characteristic range I2 in the characteristic range I2 calculated by the characteristic range calculation unit 114 based on the length between the bow and the mast. When the characteristic range I2 calculated according to the above formula (2) is a negative value, the characteristic position J3 is a position shifted from the characteristic position J1 in the negative direction of the slant range axis by the length of the characteristic range I2.

[0033] The first display processing unit 115 converts characteristic specifications such as the length between the bow and the container into a characteristic range I, and sets a position shifted from the bow in the characteristic range I as a characteristic position PA. For example, as shown in FIG. 7, the first display processing unit 115 calculates a plurality of characteristic positions including a characteristic position J1, a characteristic position J2, and a characteristic position J3 as a characteristic position PA of the target Ta. The first display processing unit 115 generates first display control information for displaying the characteristic position PA of the target Ta superimposed on the radar image E, and outputs the first display control information to the display unit 102. The display unit 102 displays a screen as shown in FIG. 7 based on the first display control information.

[0034] The operator of the classification device 100 visually checks the radar image E displayed on the display unit 102, and compares the feature position PA of the target Ta displayed superimposed on the radar image E with the feature position PB of the actual target Ta captured in the radar image E. If the feature position PA and the feature position PB do not match, the operator uses the operation unit 103 to perform an editing operation for converting the feature position PA into a feature position PA1 that matches the feature position PB, and outputs editing operation information indicating this editing operation to the feature position editing unit 116. The feature position editing unit 116 generates a feature position PA1 obtained by performing the editing operation indicated by the editing operation information on the feature position PA of the target Ta.

[0035] Next, the characteristic position editing unit 116 uses the edited characteristic position PA1 of the target Ta to calculate a characteristic range R. An example will be given in which the target Ta is a ship, and the characteristic position PA1 includes a characteristic position J1 related to the bow, a characteristic position J2 related to the stern, and a characteristic position J3 related to the mast. In this case, the characteristic range R is, for example, the length from the characteristic position J1 related to the bow to another characteristic position, the characteristic position J2 related to the stern or the characteristic position J3 related to the mast.

[0036] Next, the feature position editing unit 116 converts the feature range R into a feature parameter CP. The feature parameter CP is a parameter of the length of the actual target Ta that can be estimated using the feature range R. For example, when the aspect angle θ is defined according to the relationship shown in Fig. 3, the feature position editing unit 116 calculates the feature parameter CP using the following formula (3). In the following formula (3), |cos(X)| is the absolute value of cos(X). Characteristic parameter CP(m) = characteristic range R(m) ÷ |cos(aspect angle θ)| (3)

[0037] The second target identification unit 117 identifies, from the known characteristic parameters for each target candidate stored in the characteristic parameter database, those corresponding to the characteristic parameter CP of the target Ta calculated by the characteristic position editing unit 116, and selects the target candidate corresponding to the identified characteristic parameter. For example, the second target identification unit 117 identifies the known characteristic parameters of each target candidate stored in the characteristic parameter database that are most highly correlated with the characteristic parameter CP of the target Ta calculated by the feature position editing unit 116, and selects the target candidate corresponding to the identified characteristic parameter.

[0038] Specifically, the second target identification unit 117 calculates a correlation function between the known feature parameter and the feature parameter CP to obtain a correlation coefficient indicating the relationship between the two, and selects the known feature parameter having the largest correlation coefficient with the feature parameter CP as the feature having the highest correlation. This enables the classification device 100 to select, from the known feature parameters, feature parameters that are close to the feature parameter CP of the target Ta obtained by editing the feature positions. The target candidate selected by the second target identifying unit 117 is output from the second target identifying unit 117 to the second display processing unit 118 as a re-identification result of the target Ta.

[0039] The second display processing unit 118 generates second display control information for displaying the target candidate selected by the second target identifying unit 117, and outputs the second display control information to the display unit . For example, the second display processing unit 118 generates second display control information for displaying the target candidate selected by the second target identifying unit 117 together with the radar image E. The display unit 102 displays the target candidates that are the re-identification results of the target Ta based on the second display control information. The operator can check the re-identification result of the target Ta from the radar image E by referring to the identification result displayed on the display unit 102.

[0040] When the operator visually checks the radar image E displayed on the display unit 102 based on the first display control information and determines that the difference between the characteristic position PA and the characteristic position PB is within an allowable range, the operator performs an operation indicating that re-identification is not necessary, for example, using the operation unit 103. Operation information indicating this operation is output from the operation unit 103 to the feature position editing unit 116. Based on this operation information, the feature position editing unit 116 notifies the second target identification unit 117 that re-identification of the target Ta is not necessary.

[0041] When the second display processing unit 118 receives a notification indicating that re-identification is not required, it generates second display control information for displaying the characteristic position PA of the target Ta as is on the radar image E, and outputs the second display control information to the display unit 102. Based on the second display control information, the display unit 102 displays the target Ta, which is the identification result, together with the radar image E. The operator can check the identification result of the target Ta from the radar image E by referring to the identification result displayed on the display unit 102.

[0042] In addition, in the above description, an operator visually checks the radar image E displayed on the display unit 102 based on the first display control information and then performs editing operations of the feature positions using the operation unit 103. However, the editing operations of the feature positions may be performed automatically. For example, the characteristic position editing unit 116 may compare the difference between the characteristic position PA and the characteristic position PB with an allowable threshold, and if the difference exceeds the allowable threshold, determine that the characteristic position PA is far from the characteristic position PB, and automatically convert (edit) the characteristic position PA to the characteristic position PA1.

[0043] In this case, the first display processing unit 115 may generate display control information for displaying, on the display unit 102, the editing operation of the feature position automatically performed by the feature position editing unit 116, and output the generated display control information to the display unit 102. In this case, the display unit 102 displays the process of editing the feature position PA to the feature position PA1 based on this display control information. By referring to the editing operation displayed on the display unit 102, the operator can visually confirm the process of the process of automatically editing the feature position PA to the feature position PA1.

[0044] Next, the operation of the radar device 200 will be described. FIG. 8 is a flowchart showing the operation of the radar device 200. The tracking processing unit 202 tracks a target Ta present in an external space, thereby calculating a direction D1 of the target Ta based on the traveling direction of the target Ta, and outputs the calculated direction D1 to the aspect angle calculation unit 111 (step ST1-1).

[0045] The transmitting / receiving unit 201 generates beam pointing information indicating the beam pointing of the radar signal transmitted to the external space, and outputs the beam pointing information to the aspect angle calculation unit 111 (step ST1-2). The beam pointing information is information regarding the beam pointing at the time of transmission, and indicates the radio wave irradiation direction D2 of the transmitted radio wave.

[0046] Furthermore, the transmitting / receiving unit 201 generates a radar image E by performing signal processing on the radar signal received from the external space, and outputs the radar image E to the radar image input unit 112 (step ST1-3). The order of steps ST1-1, ST1-2 and ST1-3 does not matter, and steps ST1-1, ST1-2 and ST1-3 may be performed simultaneously.

[0047] FIG. 9 is a flowchart showing the categorization method according to the first embodiment. The aspect angle calculation unit 111 calculates the aspect angle θ based on the information indicating the direction D1 of the target Ta acquired from the radar device 200 (step ST1A-1). For example, the aspect angle calculation unit 111 calculates the aspect angle θ according to the above formula (1). The radar image input unit 112 inputs the radar image E from the radar device 200 (step ST1A-2). The processes of steps ST1A-1 and ST1A-2 may be performed simultaneously in any order. These processes are pre-processing for the classification method according to the first embodiment, and it is assumed that these processes have already been performed in the classification method according to the first embodiment.

[0048] The first target identification unit 113 identifies the target Ta from the radar image E (step ST2A). Next, the feature range calculation unit 114 calculates a feature range I, which is the range of the feature part of the target Ta, based on the aspect angle information of the target Ta included in the radar image E, the identification result information of the target Ta by the first target identification unit 113, and the known feature specifications of each target candidate (step ST3A). The first display processing unit 115 determines the feature position PA of the target Ta using the feature range I calculated by the feature range calculation unit 114, and generates first display control information for displaying the determined feature position PA superimposed on the radar image E (step ST4A). The display unit 102 displays the radar image E together with the characteristic position PA of the target Ta based on the first display control information.

[0049] When the feature position editing unit 116 acquires editing operation information indicating an editing operation performed on the feature position PA of the target in the radar image E displayed on the display unit 102 based on the first display control information, the feature position editing unit 116 calculates the feature parameters CP of the target Ta using the feature position PA1 of the target for which the editing operation indicated by the editing operation information was performed (step ST5A). The second target identification unit 117 identifies the known characteristic parameters of each target candidate that correspond to the target characteristic parameters CP calculated by the characteristic position editing unit 116, and selects the target candidate that corresponds to the identified characteristic parameters (step ST6A). The second display processing unit 118 generates second display control information for displaying the target candidate selected by the second target identifying unit 117 (step ST7A). The display unit 102 displays the radar image E together with the re-identified target candidates based on the second display control information.

[0050] In step ST4A, the operator compares the characteristic position PA with the characteristic position PB in the radar image E. If the operator determines that the correspondence between the characteristic position PA and the characteristic position PB is valid, the operator can be convinced that the identification result obtained in step ST3A is not wrong. On the other hand, if the characteristic position PA and the characteristic position PB are far apart, the operator can determine that the identification result obtained in step ST3A may be wrong. In this case, the operator can be convinced of the re-identification result by editing the characteristic position PA and re-identifying the target using the edited characteristic position PA1. In addition to the sense of satisfaction, the classification accuracy of the classification device 100 is actually improved by increasing the amount of identification information input by the operator.

[0051] In addition, when the radar image E and the feature parameters CP of the target Ta calculated by the feature position editing unit 116 are input, the second target identification unit 117 may select a target candidate from among the target candidates corresponding to the known feature parameters using a second learning model that outputs a target candidate corresponding to the feature parameters CP of the target Ta.

[0052] Fig. 10A is a diagram showing an overview of learning data for generating a second learning model 15A. Fig. 10B is a diagram showing an overview of the second learning model 15A. In Fig. 10A, the learning data includes a radar image E, annotation information F corresponding to the radar image E, and feature parameters H corresponding to the annotation information F. The annotation information F includes candidates for the target X included in the radar image X1. The second learning model 15A is generated by learning using the learning data registered in the learning database 14A. The radar image E, the corresponding annotation information F, and the feature parameters H corresponding to the annotation information F, which are included in the learning data, are not limited to those generated by the transmitting / receiving unit 201.

[0053] As shown in Fig. 10B, the second target identification unit 117 inputs the radar image E and the feature specifications H1 to H4 to the second learning model 15A. As a result, the second learning model 15A infers target candidates corresponding to the radar image E and the feature specifications H1 to H4, and outputs the inferred target candidates as identification result information. In this manner, the second target identification unit 117 uses the second learning model 15A, so that the classification device 100 can accurately select target candidates corresponding to the radar image E and the feature specifications.

[0054] The second learning model 15A may be a machine learning model that outputs identification information of a target candidate including the name and type of the target candidate as annotation information when the radar image E and the characteristic position of the target Ta determined based on the annotation information F of the target Ta are input. The second target identification unit 117 uses the second learning model 15A, so that the classification device 100 can accurately select a target candidate including the name and type of the target candidate as annotation information.

[0055] Next, a hardware configuration for realizing the functions of the classifying device 100 will be described. Each function of the aspect angle calculation unit 111, radar image input unit 112, first target identification unit 113, feature range calculation unit 114, first display processing unit 115, feature position editing unit 116, second target identification unit 117, and second display processing unit 118 included in the classification device 100 is realized by a processing circuit. That is, the classification device 100 includes a processing circuit for executing each process from step ST1A-1 to step ST7A shown in Fig. 9. The processing circuit may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in a memory.

[0056] Fig. 11A is a block diagram showing a hardware configuration for realizing the functions of the classifying device 100. Fig. 11B is a block diagram showing a hardware configuration for executing software for realizing the functions of the classifying device 100. In Figs. 11A and 11B, the input interface 1000 is, for example, an interface for relaying aspect angle information and radar image E output from the radar device 200 to the classifying device 100, and editing operation information output from the operation unit 103 to the classifying device 100. The output interface 1001 is, for example, an interface for relaying identification result information output from the classifying device 100 to the display unit 102.

[0057] When the processing circuit is the dedicated hardware processing circuit 1002 shown in FIG. 11A, the processing circuit 1002 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof. Each function of the aspect angle calculation unit 111, radar image input unit 112, first target identification unit 113, feature range calculation unit 114, first display processing unit 115, feature position editing unit 116, second target identification unit 117 and second display processing unit 118 provided in the classification device 100 may be realized by a separate processing circuit, or these functions may be realized together by a single processing circuit.

[0058] When the processing circuit is the processor 1003 shown in Fig. 11B, the functions of the aspect angle calculation unit 111, the radar image input unit 112, the first target identification unit 113, the feature range calculation unit 114, the first display processing unit 115, the feature position editing unit 116, the second target identification unit 117, and the second display processing unit 118 included in the classification device 100 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 1004. The memory 1004 is, for example, the storage unit 101 shown in Fig. 2.

[0059] The processor 1003 reads out and executes a program of a classification application stored in the memory 1004, thereby realizing the functions of the aspect angle calculation unit 111, the radar image input unit 112, the first target identification unit 113, the feature range calculation unit 114, the first display processing unit 115, the feature position editing unit 116, the second target identification unit 117, and the second display processing unit 118 of the classification device 100. For example, the classification device 100 includes a memory 1004 for storing the above-mentioned program that, when executed by the processor 1003, results in the processing of steps ST1A-1 to ST7A shown in FIG. 9 being executed. This program causes a computer to execute the procedure or method of processing performed by the aspect angle calculation unit 111, the radar image input unit 112, the first target identification unit 113, the feature range calculation unit 114, the first display processing unit 115, the feature position editing unit 116, the second target identification unit 117, and the second display processing unit 118. The memory 1004 may be a computer-readable storage medium storing programs for causing a computer to function as an aspect angle calculation unit 111, a radar image input unit 112, a first target identification unit 113, a feature range calculation unit 114, a first display processing unit 115, a feature position editing unit 116, a second target identification unit 117 and a second display processing unit 118.

[0060] Memory 1004 may be, for example, a non-volatile or volatile semiconductor memory such as a Random Access Memory (RAM), a Read Only Memory (ROM), a flash memory, an Erasable Programmable Read Only Memory (EPROM), an Electrically-EPROM (EEPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD, etc.

[0061] A part of each function of the aspect angle calculation unit 111, the radar image input unit 112, the first target identification unit 113, the feature range calculation unit 114, the first display processing unit 115, the feature position editing unit 116, the second target identification unit 117, and the second display processing unit 118 included in the classification device 100 may be realized by dedicated hardware, and the remaining part may be realized by software or firmware. For example, the aspect angle calculation unit 111 and the radar image input unit 112 are realized by a processing circuit 1002 which is dedicated hardware, and the first target identification unit 113, the feature range calculation unit 114, the first display processing unit 115, the feature position editing unit 116, the second target identification unit 117, and the second display processing unit 118 are realized by a processor 1003 reading and executing a program stored in a memory 1004. Thus, the processing circuitry may implement the above-described functions through hardware, software, firmware or a combination of these.

[0062] As described above, the classification device 100 according to the first embodiment includes the first target identification unit 113 that identifies the target Ta from the radar image E, the feature range calculation unit 114 that calculates the feature range I of the target Ta based on the aspect angle information of the target Ta included in the radar image E, the identification result information of the target Ta by the first target identification unit 113, and known feature specifications for each target candidate, the first display processing unit 115 that determines the feature position PA of the target using the feature range I and generates first display control information for displaying the feature position PA superimposed on the radar image E, and the first display control unit 116 that calculates the feature range I of the target Ta based on the aspect angle information of the target Ta included in the radar image E, the identification result information of the target Ta by the first target identification unit 113, and known feature specifications for each target candidate. The classification device 100 includes a feature position editing unit 116 that, when acquiring editing operation information indicating an editing operation on a feature position PA of the target Ta in the radar image E displayed on the display unit 102 based on the feature position editing unit 116, calculates a feature parameter CP of the target Ta using the feature position PA1 of the target Ta where the editing operation has been performed, a second target identification unit 117 that identifies a feature parameter corresponding to the feature parameter CP from among known feature parameters for each target candidate and selects a target candidate corresponding to the identified feature parameter, and a second display processing unit 118 that generates second display control information for displaying the target candidate selected by the second target identification unit 117. Even if the feature position PA of the target Ta identified from the radar image E and the feature position PB of the target Ta are far apart, the target Ta is identified again using the feature position PA1 of the target Ta obtained by editing the feature position PA1, so that the classification device 100 can improve the classification accuracy of the target Ta.

[0063] In the classification device 100 according to the first embodiment, when a radar image E is input, the first target identification unit 113 identifies the target Ta using the first learning model 15 that outputs identification result information of the target Ta. The first target identification unit 113 uses the first learning model 15, so that the classification device 100 can accurately classify the target Ta using the radar image E.

[0064] In the classification device 100 according to the first embodiment, the second target identification unit 117 identifies the known characteristic parameters of each target candidate that are most highly correlated with the characteristic parameter CP of the target Ta calculated by the feature position editing unit 116, and selects the target candidate corresponding to the identified characteristic parameter. This enables the classification device 100 to select, from the known characteristic parameters, a characteristic parameter close to the characteristic parameter CP calculated based on the edited feature position PA1.

[0065] In the classification device 100 according to the first embodiment, when the radar image E and the feature parameters CP of the target Ta calculated by the feature position editing unit 116 are input, the second target identification unit 117 selects a target candidate using the second learning model 15A that outputs a target candidate corresponding to the feature parameters CP of the target Ta. By the second target identification unit 117 using the second learning model 15A, the classification device 100 can accurately select a target candidate corresponding to the feature parameters CP from the target candidates corresponding to the known feature parameters.

[0066] In the classification device 100 according to the first embodiment, the second learning model 15A outputs identification information of the target candidate including the name and type of the target candidate as annotation information when the radar image E and the characteristic position of the target Ta determined based on the annotation information F of the target Ta are input. The second target identification unit 117 uses the second learning model 15A, so that the classification device 100 can accurately select the target candidate including the name and type of the target candidate as annotation information.

[0067] The classification system 1 according to the first embodiment includes a radar device 200 that generates and outputs a radar image E including the searched target Ta, a display unit 102 that displays the radar image E based on first display control information and second display control information, an operation unit 103 that accepts an editing operation for a characteristic position PA of the target Ta in the radar image E displayed on the display unit 102, and a classifying device 100 that inputs the radar image E generated by the radar device 200 and the editing operation information. This makes it possible to provide a classifying system 1 with improved target classification accuracy.

[0068] The classification method according to the first embodiment includes step ST2A in which the first target identification unit 113 identifies the target Ta from the radar image E, step ST3A in which the feature range calculation unit 114 calculates a feature range I of the target Ta based on aspect angle information of the target Ta included in the radar image E, identification result information of the target Ta by the first target identification unit 113, and known feature specifications for each target candidate, step ST4A in which the first display processing unit 115 determines a feature position PA of the target Ta using the feature range I and generates first display control information for displaying the determined feature position PA superimposed on the radar image E, and step ST4A in which the feature position editing unit 116 generates a feature range I of the target Ta based on the first display control information. The method includes a step ST5A in which, when editing operation information indicating an editing operation on a feature position PA of the target in the radar image E displayed on the display unit 102 is acquired by the second target identification unit 117, the step ST5A calculates feature parameters CP of the target Ta using the feature position PA1 of the target Ta where the editing operation indicated by the editing operation information was performed, a step ST6A in which the second target identification unit 117 specifies, from among known feature parameters of each target candidate, those corresponding to the feature parameters CP of the target Ta calculated by the feature position editing unit 116 and selects a target candidate corresponding to the feature parameters CP, and a step ST7A in which the second display processing unit 118 generates second display control information for displaying the target candidate selected by the second target identification unit 117. The classification device 100 executes this method, thereby making it possible to improve the accuracy of classification of targets.

[0069] Embodiment 2 FIG. 12 is a block diagram showing a configuration example of a classification device 100A according to a second embodiment. In FIG. 12, the same components as those in FIG. 2 are denoted with the same reference numerals, and duplicated explanations are omitted. The classification device 100A is provided in place of the classification device 100 included in the classification system 1 shown in FIG. 1. Targets to be classified are, for example, moving bodies such as ships or aircraft, as in the case of the classification device 100. The classification device 100A classifies targets from radar images generated by the radar device 200. The radar images are, for example, radar still images or moving image information obtained by ISAR or SAR.

[0070] The classifying device 100A includes a storage unit 101 in which a feature specification database is stored, a display unit 102, and an operation unit 103, and further includes a calculation unit not shown in Fig. 12. The storage unit 101 is a storage device included in a computer that functions as the classifying device 100A, and includes a storage such as an HDD or SSD, or the memory 1004 shown in Fig. 11B. Note that the storage unit 101 may be provided outside the classifying device 100A as long as it is accessible from the classifying device 100A.

[0071] The calculation unit controls the overall operation of the classification device 100A. The calculation unit includes an aspect angle calculation unit 111, a radar image input unit 112, a first target identification unit 113, a feature range calculation unit 114, a first display processing unit 115, a feature position editing unit 116, a second target identification unit 117A, a second display processing unit 118, an annotation processing unit 119, and a learning model update unit 120. When the calculation unit executes the classification application, the functions of an aspect angle calculation unit 111, a radar image input unit 112, a first target identification unit 113, a feature range calculation unit 114, a first display processing unit 115, a feature position editing unit 116, a second target identification unit 117A, a second display processing unit 118, an annotation processing unit 119 and a learning model update unit 120 are realized.

[0072] The function of aspect angle calculation unit 111 may be provided by feature range calculation unit 114, and the function of radar image input unit 112 may be provided by first target identification unit 113. In this case, classification device 100A does not need to include aspect angle calculation unit 111 and radar image input unit 112.

[0073] The first target identification unit 113, when receiving the radar image E, identifies the target Ta by using the first learning model 15 that outputs identification result information of the target Ta. The first learning model 15 is updated by the learning model update unit 120.

[0074] When the radar image E and the characteristic parameters CP of the target Ta calculated by the characteristic position editing unit 116 are input, the second target identification unit 117A selects a target candidate from among target candidates corresponding to known characteristic parameters using the second learning model 15A that outputs a target candidate corresponding to the characteristic parameters CP of the target Ta. The second learning model 15A is updated by the learning model update unit 120.

[0075] The annotation processing unit 119 generates annotation information F that links the identification result of the target Ta by the second target identification unit 117A to the radar image E and the characteristic parameters CP of the target Ta calculated by the characteristic position editing unit 116. For example, when the target Ta is a ship, the characteristic parameters CP are the LOA of the ship, the length between the bow and the mast, or the length between the bow and the container, etc.

[0076] The learning model update unit 120 updates one or both of the first learning model 15 and the second learning model 15A using the annotation information F, the radar image E, and the feature specifications CP of the target Ta calculated by the feature position editing unit 116. For example, when updating the first learning model 15, the annotation processing unit 119 adds a radar image E and the corresponding annotation information F to the learning database 14 shown in FIG. 4A. Then, the learning model update unit 120 updates the first learning model 15 by re-learning the learning data including the added learning data. In this way, by updating the first learning model 15, the classification device 100A can identify targets using the first learning model 15 with improved inference accuracy.

[0077] Furthermore, when updating the second learning model 15A, the annotation processing unit 119 adds, to the learning database 14A shown in Fig. 10A, a radar image E, annotation information F corresponding to this radar image E, and feature specifications H corresponding to the annotation information F. Then, the learning model update unit 120 updates the second learning model 15A by re-learning the learning data including the added learning data. In this way, by updating the second learning model 15A, the classification device 100A can identify the target using the second learning model 15A with improved inference accuracy.

[0078] FIG. 13 is a flowchart showing the categorization method according to the second embodiment, and shows a case where the second learning model 15A is updated. The aspect angle calculation unit 111 calculates the aspect angle θ based on the information indicating the direction D1 of the target Ta acquired from the radar device 200 (step ST1B-1). For example, the aspect angle calculation unit 111 calculates the aspect angle θ according to the above formula (1). The radar image input unit 112 inputs the radar image E from the radar device 200 (step ST1B-2). The processes of steps ST1B-1 and ST1B-2 may be performed simultaneously in any order. These processes are pre-processing for the classification method according to the second embodiment, and it is assumed that these processes have already been performed in the classification method according to the second embodiment.

[0079] The first target identification unit 113 identifies the target Ta from the radar image E (step ST2B). Next, the feature range calculation unit 114 calculates a feature range I, which is the range of the feature part of the target Ta, based on the aspect angle information of the target Ta included in the radar image E, the identification result information of the target Ta by the first target identification unit 113, and the known feature specifications of each target candidate (step ST3B). The first display processing unit 115 determines the feature position PA of the target Ta using the feature range I calculated by the feature range calculation unit 114, and generates first display control information for displaying the determined feature position PA superimposed on the radar image E (step ST4B). The display unit 102 displays the radar image E together with the characteristic position PA of the target Ta based on the first display control information.

[0080] When the feature position editing unit 116 acquires editing operation information indicating an editing operation performed on the feature position PA of the target in the radar image E displayed on the display unit 102 based on the first display control information, the feature position editing unit 116 calculates the feature parameter CP of the target Ta using the feature position PA1 of the target for which the editing operation indicated by the editing operation information was performed (step ST5B). The second target identification unit 117A identifies the known characteristic parameters of the target corresponding to the characteristic parameters CP of the target calculated by the characteristic position editing unit 116 from the known characteristic parameters of each target candidate, and selects the target candidate corresponding to the identified characteristic parameters (step ST6B). Furthermore, the second target identification unit 117A outputs the target identification result, the radar image E, and the characteristic parameters CP to the annotation processing unit 119. The second display processing unit 118 generates second display control information for displaying the target candidate selected by the second target identifying unit 117A (step ST7B). The display unit 102 displays the radar image E together with the re-identified target candidates based on the second display control information.

[0081] The annotation processing unit 119 generates learning data consisting of the target identification result acquired from the second target identifying unit 117A, the radar image E, and the feature specifications CP, and adds the generated learning data to the learning database 14A (step ST8B). The learning model update unit 120 updates the second learning model 15A by re-learning the learning data including the learning data added by the annotation processing unit 119 from the learning database 14A (step ST9B). After that, the second target identification unit 117A executes the process of step ST6B using the updated second learning model 15A.

[0082] In step ST4B, the operator compares the characteristic position PA with the characteristic position PB in the radar image E. If the operator determines that the correspondence between the characteristic position PA and the characteristic position PB is valid, the operator can be convinced that the identification result obtained in step ST3B is not wrong. On the other hand, if the characteristic position PA and the characteristic position PB are far apart, the operator can determine that the identification result obtained in step ST3B may be wrong. In this case, the operator can be convinced of the re-identification result by editing the characteristic position PA and re-identifying the target using the edited characteristic position PA1. In addition to the sense of satisfaction, the classification accuracy of the classification device 100A is actually improved by increasing the amount of identification information input by the operator.

[0083] As described above, the classification device 100A according to the second embodiment includes the annotation processing unit 119 that generates annotation information linking the target identification result by the second target identification unit 117A to the radar image and the feature specifications of the target calculated by the feature position editing unit 116, and the learning model update unit 120 that updates one or both of the first learning model and the second learning model using the annotation information, the radar image, and the feature specifications of the target calculated by the feature position editing unit 116. In this way, by updating one or both of the first learning model 15 and the second learning model 15A, the classification device 100A can identify targets using one or both of the first learning model 15 and the second learning model 15A with improved inference accuracy.

[0084] It is possible to combine the embodiments, modify any of the components of the embodiments, or omit any of the components of the embodiments. [Industrial Applicability]

[0085] The classification device according to the present disclosure can be used, for example, for classification of ships or aircraft. [Explanation of symbols]

[0086] 1 classification system, 13 sensor, 14, 14A learning database, 15 first learning model, 15A second learning model, 100, 100A classification device, 101 memory unit, 102 display unit, 103 operation unit, 111 aspect angle calculation unit, 112 radar image input unit, 113 first target identification unit, 114 feature range calculation unit, 115 first display processing unit, 116 feature position editing unit, 117, 117A second target identification unit, 118 second display processing unit, 119 annotation processing unit, 120 learning model update unit, 200 radar device, 201 transmitting / receiving unit, 202 tracking processing unit, 1000 input interface, 1001 output interface, 1002 processing circuit, 1003 processor, 1004 memory.

Claims

1. a first target identification unit that identifies a target from a radar image; a feature range calculation unit that calculates a feature range, which is a range of a feature portion of the target, based on aspect angle information of the target included in the radar image, identification result information of the target by the first target identification unit, and known feature specifications for each target candidate; a first display processing unit that determines a feature position, which is a position of a feature part of the target, using the feature range calculated by the feature range calculation unit, and generates first display control information for displaying the determined feature position superimposed on the radar image; a feature position editing unit that, when acquiring editing operation information indicating an editing operation on a feature position of the target in the radar image displayed on a display unit based on the first display control information, calculates feature specifications of the target using the feature position of the target on which the editing operation indicated by the editing operation information has been performed; a second target identification unit that identifies a characteristic parameter corresponding to the characteristic parameter of the target calculated by the characteristic position editing unit from among known characteristic parameters of each of the target candidates, and selects the target candidate corresponding to the identified characteristic parameter; a second display processing unit that generates second display control information for displaying the target candidate selected by the second target identifying unit; A classification device characterized by:

2. The first target identification unit, when the radar image is input, identifies the target using a first learning model that outputs identification result information of the target.

2. The classification device according to claim 1 .

3. The second target identification unit identifies, from among known characteristic parameters for each of the target candidates, a characteristic parameter that is most highly correlated with the characteristic parameter of the target calculated by the characteristic position editing unit, and selects the target candidate corresponding to the identified characteristic parameter.

3. The classification device according to claim 2.

4. The second target identification unit, when the radar image and the feature specifications of the target calculated by the feature position editing unit are input, selects the target candidate using a second learning model that outputs the target candidate corresponding to the feature specifications of the target.

4. A classification device according to claim 3.

5. The second learning model outputs identification information of the target candidate including a name and a type of the target candidate as the annotation information when the radar image and a characteristic position of the target determined based on the annotation information of the target are input.

5. A classification device according to claim 4.

6. an annotation processing unit that generates the annotation information by linking the identification result of the target by the second target identification unit to the radar image and the feature specifications of the target calculated by the feature position editing unit; a learning model update unit that updates one or both of the first learning model and the second learning model by using the annotation information, the radar image, and the feature specifications of the target calculated by the feature position editing unit.

6. A classification device as claimed in claim 5.

7. a radar device that generates and outputs the radar image including the searched target; the display unit that displays the radar image based on the first display control information and the second display control information; an operation unit that receives an editing operation for a characteristic position of the target in the radar image displayed on the display unit; and a classification device according to any one of claims 1 to 6, which inputs the radar image generated by the radar device and the editing operation information. A classification system comprising:

8. A classification method using a classification device, comprising: A step of a first target identification unit identifying a target from a radar image; a feature range calculation unit calculating a feature range, which is a range of a feature portion of the target, based on aspect angle information of the target included in the radar image, identification result information of the target by the first target identification unit, and known feature specifications for each target candidate; a step of a first display processing unit determining a feature position, which is a position of a feature part of the target, by using the feature range calculated by the feature range calculation unit, and generating first display control information for displaying the determined feature position by superimposing it on the radar image; a step of, when a feature position editing unit acquires editing operation information indicating an editing operation performed on a feature position of the target in the radar image displayed on a display unit based on the first display control information, calculating feature specifications of the target using the feature position of the target on which the editing operation indicated by the editing operation information has been performed; A step in which a second target identification unit identifies a known characteristic parameter of each of the target candidates that corresponds to the characteristic parameter of the target calculated by the characteristic position editing unit, and selects the target candidate that corresponds to the identified characteristic parameter; A step of generating second display control information by a second display processing unit for displaying the target candidate selected by the second target identification unit. A classification method comprising:

Images