Automatic thunderlight guiding method with human-in-loop compatibility

Abstract

The invention discloses an automatic thunderlight guiding method with human-in-loop compatibility, and relates to the field of thunderlight guiding, and the method achieves the function that a system automatically uses radar detection information to guide photoelectric detection through a reinforcement learning algorithm, improves the effective utilization rate of equipment in the system, and achieves the efficient and accurate target tracking, searching or recognition. On the basis of an automatic thunderlight guiding method based on reinforcement learning, an automatic thunderlight guiding method with man-in-the-loop compatibility is derived, so that the state of radar photoelectric resources in an intelligent body can be influenced by manual operation, and the flexibility of the automatic thunderlight guiding method is improved.

Description

Technical Field

[0001] The invention relates to the field of lightning guidance, and in particular to an automatic lightning guidance method with man-in-the-loop compatibility. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0003] With the rapid development of science and technology, the drone industry has also made great strides. From small civilian drones used for entertainment travel photography, to commercial drones used for patrolling, forest protection, fire fighting and other purposes, to various types of military drones for militarized purposes, drones are gradually becoming closer to our lives. Whether in the field of military detection or in the field of civilian detection, facing low-altitude, small and highly maneuverable flying targets, the use of multi-sensor collaborative detection has become a mainstream trend, but for low-altitude flight regulators, more and more sensors are putting increasing pressure on regulators to operate. Therefore, the necessity of intelligent and automated equipment control methods is becoming increasingly high, and intelligent and automated equipment control technology also needs to be put on the agenda.

[0004] In a complex low-altitude control environment, it is an inevitable requirement to confirm the detection results of echo reflection detection equipment after detection. After confirming that the detection results of echo reflection detection equipment are targets that need to be handled, formulating relevant disposal measures for the identified targets can greatly improve the success rate of target disposal and reduce the cost of drone disposal.

[0005] The development of the current low-altitude economy is strongly supported by the country, and more and more people own drones, making the control of low-altitude environments more and more difficult. In the current low-altitude environment, drone control requires more and more drones to be detected, identified, and controlled. Therefore, most of the larger-scale drone control systems are distributed with multiple devices and multiple points. The more devices there are, the greater the labor cost of manually controlling the devices by common means. When the number of radar detection targets reaches a certain number, human labor can no longer use optoelectronic equipment to verify each target one by one. Summary of the invention

[0006] The purpose of the present invention is to provide a method for automatic lightning guidance with human-in-the-loop compatibility in response to the problems existing in the prior art, and to use a reinforcement learning algorithm to achieve the function of the system automatically using radar detection information to guide photoelectric detection, thereby improving the effective utilization rate of equipment in the system to achieve efficient and accurate target tracking, searching or identification; on the basis of the automatic lightning guidance method based on reinforcement learning, an automatic lightning guidance method with human-in-the-loop compatibility is derived, so that human operation can affect the state of radar photoelectric resources in the intelligent body, thereby improving the flexibility of the lightning guidance method.

[0007] The technical solution of the present invention is as follows:

[0008] A method for automatic lightning guidance with man-in-the-loop compatibility, comprising:

[0009] Step S1: Obtain a status list of photoelectric detection equipment and radar detection equipment;

[0010] Step S2: Obtain a radar detection target list;

[0011] Step S3: screening radar detection targets according to preset rules;

[0012] Step S4: Generate a lightning guidance strategy, i.e., a lightning guidance instruction, using a reinforcement learning agent according to environmental information, equipment status information, and radar detection target information;

[0013] Step S5: dispatching the photoelectric detection equipment to check the suspicious target detected by the radar detection equipment according to the lightning guidance instruction, completing the real-time guidance according to the real-time information of the target, and completing the identity confirmation of the radar detection target in combination with the image recognition technology;

[0014] Step S7: updating the recognition result of the target;

[0015] Step S8: Release the photoelectric detection device resources.

[0016] Furthermore, the step S1 comprises:

[0017] The command and control center obtains the status list of the photoelectric detection equipment and the radar detection equipment in an active or passive manner, including: the availability and occupancy status of the photoelectric detection equipment and the radar detection equipment.

[0018] Furthermore, the step S2 comprises:

[0019] Obtain radar detection target information uploaded by radar detection equipment at each point.

[0020] Furthermore, the step S3 comprises:

[0021] The radar detection targets with track points less than N and the radar detection targets whose identities have been confirmed are screened out, and the remaining radar detection targets are used as the screened radar detection target list.

[0022] Furthermore, a reward mechanism is provided in the reinforcement learning agent; the reward function constraint is:

[0023] (1) Number of targets detected by radar and number of targets with confirmed identities;

[0024] (2) Usage time for the same number of radar detected targets with confirmed identities;

[0025] (3) The average value of the difference between the identity confirmation time and the target appearance time.

[0026] Furthermore, the greater the number of radar detection targets and the greater the number of identity confirmations, the higher the efficiency of the overall system in confirming the identity of unknown radar detection targets, that is, the reward function 1 is:

[0027] R W a =100*N a

[0028] Among them, N a The number of targets detected by radar and the number of confirmed identities;

[0029] The shorter the usage time under the same number of radar detection target identity confirmations, the higher the detection efficiency of the photoelectric detection equipment for the detection targets within the detectable range, that is, the reward function 2 is:

[0030]

[0031] Among them, t is the time to complete the identification of all radar detection equipment, N b The number of targets detected by the radar;

[0032] The lower the average value of the difference between the identity confirmation time and the target appearance time, the more sensitive the photoelectric detection device is to the identity recognition of the radar detection target, that is, the reward function 3 is:

[0033]

[0034] Where n is the number of radar detection targets whose identities have been confirmed, t i is the time difference between the i-th target identity confirmation time and the target appearance time.

[0035] Furthermore, reward function 1, reward function 2, and reward function 3 are used for two types of training scenarios:

[0036] First, the radar detection target will fly out of the detection range of the optoelectronic device detection network. This type of training scenario uses two reward functions: reward function 1 and reward function 3.

[0037] Second, the radar detection target will not fly out of the detection range of the optoelectronic device detection network. This type of training scenario uses two reward functions: reward function 2 and reward function 3.

[0038] The overall reward function of the reinforcement learning agent is designed as:

[0039]

[0040] Furthermore, after screening the radar detection targets, a pruning optimization algorithm is executed, with whether the radar detection target is within the detection range of the photoelectric detection device and whether the radar detection target has been detected by the photoelectric detection device as constraints. If the selected maximum node meets the pruning conditions, it will be marked as disabled; once tree pruning occurs, the search process will restart from the root node.

[0041] Furthermore, it also includes a person in the loop, who can manually control the photoelectric detection device in the system, and the priority of manually controlling the photoelectric detection device is higher than the priority of controlling the photoelectric device under the automatic guidance of lightning;

[0042] Here’s how to do it:

[0043] (1) When the photoelectric detection device is not occupied by the automatic lightning guidance, the photoelectric detection device is occupied by manual control. The occupied photoelectric detection device will not be counted as a photoelectric detection device resource by the automatic lightning guidance algorithm, so the photoelectric detection device will not be occupied repeatedly;

[0044] (2) When the photoelectric detection device has been occupied by the automatic lightning guidance, the original automatic lightning guidance service is canceled, the occupation status of the photoelectric detection device is released, the status of "being photoelectrically detected" of the detected radar target is released, and the photoelectric detection device is re-occupied by manual control. The occupied photoelectric detection device will not be counted as a photoelectric detection device resource by the automatic lightning guidance algorithm, so the photoelectric detection device will not be occupied repeatedly.

[0045] Furthermore, the reinforcement learning agent introduces Monte Carlo tree search as a search tool.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. The present invention provides an automated method for target detection and target identity confirmation by coordinating an automated radar detection device with an optoelectronic detection device, thereby reducing the labor cost of drone monitoring systems.

[0048] 2. The present invention enables human operation to participate in the calculation of the intelligent body, making the coordination between the radar detection equipment and the photoelectric detection equipment more flexible and more in line with actual use requirements.

[0049] 3. The present invention significantly improves the equipment utilization rate of photoelectric detection equipment and radar detection equipment, and effectively improves the efficiency of drone monitoring systems in confirming the identity of drones. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The present invention is a flow chart of a method for automatic lightning guidance with man-in-the-loop compatibility. DETAILED DESCRIPTION

[0051] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0052] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0053] Embodiment 1

[0054] See also Figure 1 , a lightning automatic guidance method with man-in-the-loop compatibility, specifically comprising the following steps:

[0055] Step S1: Obtain a status list of photoelectric detection equipment and radar detection equipment;

[0056] Step S2: Obtain a radar detection target list;

[0057] Step S3: screening radar detection targets according to preset rules;

[0058] Step S4: Based on the environmental information, equipment status information, and radar detection target information, a reinforcement learning agent is used to generate a lightning guidance strategy, i.e., a lightning guidance instruction; at the same time, the optoelectronic equipment can also be directly occupied and guided by manually controlling the equipment;

[0059] Step S5: dispatching the photoelectric detection equipment to check the suspicious target detected by the radar detection equipment according to the lightning guidance instruction, completing the real-time guidance according to the real-time information of the target, and completing the identity confirmation of the radar detection target in combination with the image recognition technology;

[0060] Step S7: updating the recognition result of the target;

[0061] Step S8: releasing the photoelectric detection device resources;

[0062] It should be noted that in the actual automated drone identity verification process, multiple devices need to work together, and a lightning guidance strategy needs to be automatically generated. Then, based on the lightning guidance strategy, the photoelectric detection device is guided according to the radar detection target information and combined with the image recognition algorithm to identify the target identity. Given that general drone monitoring systems need to run uninterruptedly for a long time, this process needs to be repeated.

[0063] In this embodiment, specifically, step S1 includes:

[0064] The command and control center obtains the status list of the photoelectric detection equipment and the radar detection equipment in an active or passive manner, including: the availability and occupancy status of the photoelectric detection equipment and the radar detection equipment.

[0065] In this embodiment, specifically, step S2 includes:

[0066] Obtain radar detection target information uploaded by radar detection equipment at each point.

[0067] In this embodiment, specifically, step S3 includes:

[0068] The radar detection targets with track points less than N and the radar detection targets whose identities have been confirmed are screened out, and the remaining radar detection targets are used as the screened radar detection target list.

[0069] In this embodiment, it should be noted that in the implementation of the automatic guidance function of the lightning, there will be many radar detection targets that need to be verified. With the increase of radar detection targets, the state and action space will be significantly expanded, especially the action space will show exponential growth, which makes it difficult to process quickly and effectively using small-scale algorithms. Therefore, in order to find a solution, it is necessary to use intelligent search technology to improve search efficiency and accuracy. In this method, we introduce Monte Carlo Tree Search (MCTS) as an efficient search tool. Through intelligent search technology, it is possible to find a suitable scheduling solution more quickly to meet the actual needs of the use of the automatic guidance function of the lightning.

[0070] During the execution of the automatic guidance function of the lightning, there are several photoelectric detection devices in the system, and the photoelectric detection devices have several attributes, including the location information of the photoelectric detection devices, the occupancy status of the photoelectric detection devices, the current detection angle of the photoelectric detection devices, the working mode of the photoelectric detection devices, etc.; there are also several radar detection targets in the system, and the radar detection targets also have several attributes, including the location of the target, the flight speed of the target, whether the target identity is confirmed, whether there is a photoelectric detection device confirming the identity, etc., so the state S t It can be expressed as:

[0071] S t=[S1,S2,,S3,……,S i ,……,S n ]

[0072] During the execution of the automatic lightning guidance function, the intelligent agent can decide to take the following actions: occupy a certain photoelectric detection device to guide the device, and release the photoelectric detection device resources after the photoelectric detection device is used.

[0073] The reward in the RL problem determines the quality of the action, which affects its future state. The focus of the automatic guidance method of the lightning is to confirm the number of radar detection targets; the shorter the overall usage time is for the same number of radar detection targets, the better; the smaller the difference between the identity confirmation time and the target appearance time, the better. Since the reward determines how the state selects the following actions, in the context of the limited resource lightning automatic guidance problem, the reward function constraint is:

[0074] (1) Number of targets detected by radar and number of targets with confirmed identities;

[0075] (2) Usage time for the same number of radar detected targets with confirmed identities;

[0076] (3) the average of the differences between the identity confirmation time and the target appearance time;

[0077] The more radar detection targets the more identity confirmations are, the higher the efficiency of the overall system in confirming the identity of unknown radar detection targets. The reward function (i.e., reward function 1) is:

[0078] R W a =100*N a

[0079] Among them, N a The number of targets detected by radar and the number of targets with confirmed identities.

[0080] The shorter the usage time is when the number of radar detection target identity confirmations is the same, the higher the detection efficiency of the photoelectric detection equipment for the detection targets within the detectable range is. The reward function (i.e., reward function 2) is:

[0081]

[0082] Among them, t is the time to complete the identification of all radar detection equipment, N b is the number of targets detected by the radar.

[0083] The lower the average value of the difference between the identity confirmation time and the target appearance time, the more sensitive the photoelectric detection device is to the identity recognition of the radar detection target. The reward function (i.e., reward function 3) is:

[0084]

[0085] Where n is the number of radar detection targets whose identities have been confirmed, t i is the time difference between the i-th target identity confirmation time and the target appearance time.

[0086] The above three reward functions are used in two types of training scenarios:

[0087] First, the radar detection target will fly out of the detection range of the optoelectronic device detection network. This type of training scenario uses two reward functions: reward function 1 and reward function 3.

[0088] Second, the radar detection target will not fly out of the detection range of the optoelectronic device detection network. This type of training scenario uses two reward functions: reward function 2 and reward function 3.

[0089] Therefore, the total reward function for the agent is designed as:

[0090]

[0091] It should be noted that pruning optimization is a method based on branching and binding, which can improve the quality of action selection and increase the search speed. Tree pruning is performed in the selection step, with constraints such as whether the radar detection target is within the detection range of the photoelectric detection device and whether the radar detection target has been detected by the photoelectric detection device. If the selected maximum node meets the pruning conditions, it will be marked as disabled to prevent further exploration. Once tree pruning occurs, the search process will restart from the root node.

[0092] It should be noted that in the automatic lightning guidance method, the environment in which the intelligent agent is located includes the detection range of the photoelectric detection device, the rotation angular velocity of the photoelectric detection device, the time required for the photoelectric detection device to identify the target, etc.

[0093] In this embodiment, in response to the business needs of lightning guidance, during the continuous execution of the lightning automatic guidance function, the photoelectric detection equipment in the system can be manually controlled, and the priority of manually controlling the photoelectric detection equipment is higher than the priority of controlling the photoelectric equipment under the lightning automatic guidance;

[0094] Here’s how to do it:

[0095] (1) When the photoelectric detection device is not occupied by the automatic lightning guidance, the photoelectric detection device is occupied by manual control. The occupied photoelectric detection device will not be counted as a photoelectric detection device resource by the automatic lightning guidance algorithm, so the photoelectric detection device will not be occupied repeatedly;

[0096] (2) When the photoelectric detection device has been occupied by the automatic lightning guidance, the original automatic lightning guidance service is canceled, the occupation status of the photoelectric detection device is released, the status of "being photoelectrically detected" of the detected radar target is released, and the photoelectric detection device is re-occupied by manual control. The occupied photoelectric detection device will not be counted as a photoelectric detection device resource by the automatic lightning guidance algorithm, so the photoelectric detection device will not be occupied repeatedly.

[0097] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

[0098] This background section is provided to generally present the context of the invention, and the work of the presently named inventors, the work to the extent described in this background section, and aspects of the description in this section that did not constitute prior art at the time of application are neither explicitly nor implicitly admitted to be prior art to the present invention.

Claims

1. A method for automatic lightning guidance with man-in-the-loop compatibility, characterized in that: include: Step S1: Obtain a status list of photoelectric detection equipment and radar detection equipment; Step S2: Obtain a radar detection target list; Step S3: screening radar detection targets according to preset rules; Step S4: Generate a lightning guidance strategy, i.e., a lightning guidance instruction, using a reinforcement learning agent according to environmental information, equipment status information, and radar detection target information; Step S5: dispatching the photoelectric detection equipment to check the suspicious target detected by the radar detection equipment according to the lightning guidance instruction, completing the real-time guidance according to the real-time information of the target, and completing the identity confirmation of the radar detection target in combination with the image recognition technology; Step S7: updating the recognition result of the target; Step S8: Release the photoelectric detection device resources.

2. The method for automatic lightning guidance with human-in-the-loop compatibility according to claim 1, characterized in that: The step S1 comprises: The command and control center obtains the status list of the photoelectric detection equipment and the radar detection equipment in an active or passive manner, including: the availability and occupancy status of the photoelectric detection equipment and the radar detection equipment.

3. The method for automatic lightning guidance with human-in-the-loop compatibility according to claim 1, characterized in that: The step S2 comprises: Obtain radar detection target information uploaded by radar detection equipment at each point.

4. The method for automatic lightning guidance with human-in-the-loop compatibility according to claim 1, characterized in that: The step S3 comprises: The radar detection targets with track points less than N and the radar detection targets whose identities have been confirmed are screened out, and the remaining radar detection targets are used as the screened radar detection target list.

5. The method for automatic lightning guidance with human-in-the-loop compatibility according to claim 1, characterized in that: The reinforcement learning agent is provided with a reward mechanism; the reward function constraint is: (1) Number of targets detected by radar and number of targets with confirmed identities; (2) Usage time for the same number of radar detected targets with confirmed identities; (3) The average value of the difference between the identity confirmation time and the target appearance time.

6. The method for automatic lightning guidance with human-in-the-loop compatibility according to claim 5, characterized in that: The more radar detection targets the more identity confirmations are, the higher the overall system efficiency is in confirming the identities of unknown radar detection targets, that is, the reward function 1 is: RW a =100*N a Among them, N a The number of targets detected by radar and the number of confirmed identities; The shorter the usage time under the same number of radar detection target identity confirmations, the higher the detection efficiency of the photoelectric detection equipment for the detection targets within the detectable range, that is, the reward function 2 is: Among them, t is the time to complete the identification of all radar detection equipment, N b The number of targets detected by the radar; The lower the average value of the difference between the identity confirmation time and the target appearance time, the more sensitive the photoelectric detection device is to the identity recognition of the radar detection target, that is, the reward function 3 is: Where n is the number of radar detection targets whose identities have been confirmed, t i is the time difference between the i-th target identity confirmation time and the target appearance time.

7. The method for automatically guiding lightning with human-in-the-loop compatibility according to claim 6, characterized in that: Reward function 1, reward function 2, and reward function 3 are used for two types of training scenarios: First, the radar detection target will fly out of the detection range of the optoelectronic device detection network. This type of training scenario uses two reward functions: reward function 1 and reward function 3. Second, the radar detection target will not fly out of the detection range of the optoelectronic device detection network. This type of training scenario uses two reward functions: reward function 2 and reward function 3. The overall reward function of the reinforcement learning agent is designed as:

8. The method for automatic lightning guidance with human-in-the-loop compatibility according to claim 7, characterized in that: After screening the radar detection targets, a pruning optimization algorithm is executed, with whether the radar detection target is within the detection range of the photoelectric detection device and whether the radar detection target has been detected by the photoelectric detection device as constraints. If the selected maximum node meets the pruning conditions, it will be marked as disabled; once tree pruning occurs, the search process will restart from the root node.

9. The method for automatic lightning guidance with human-in-the-loop compatibility according to claim 8, characterized in that: It also includes a person in the loop, who can manually control the photoelectric detection equipment in the system, and the priority of manually controlling the photoelectric detection equipment is higher than the priority of controlling the photoelectric equipment under the automatic guidance of lightning; Here’s how to do it: (1) When the photoelectric detection device is not occupied by the automatic lightning guidance, the photoelectric detection device is occupied by manual control. The occupied photoelectric detection device will not be counted as a photoelectric detection device resource by the automatic lightning guidance algorithm, so the photoelectric detection device will not be occupied repeatedly; (2) When the photoelectric detection device has been occupied by the automatic lightning guidance, the original automatic lightning guidance service is canceled, the occupation status of the photoelectric detection device is released, the status of "being photoelectrically detected" of the detected radar target is released, and the photoelectric detection device is re-occupied by manual control. The occupied photoelectric detection device will not be counted as a photoelectric detection device resource by the automatic lightning guidance algorithm, so the photoelectric detection device will not be occupied repeatedly.

10. The method for automatic lightning guidance with human-in-the-loop compatibility according to claim 9, characterized in that: The reinforcement learning agent introduces Monte Carlo tree search as a search tool.

Images