A multi-sensor cooperative hit prediction method and system

Abstract

The application discloses a multi-sensor cooperative hit prediction method and system, the method comprises the following steps: selecting the corresponding millimeter wave detection range according to the field of view angle, resolution and blind area range of the visible light sensor; determining the opened sensor according to different detection ranges, and then measuring the distance between the target and the unmanned aerial vehicle; and generating the judgment result of the hit target and the hit position according to the information detected by the multiple millimeter wave sensors and a hit prediction judgment strategy algorithm. The application can continuously provide the unmanned aerial vehicle guidance hit information after the visible light sensor on the seeker enters the field of view blind area, and can more accurately calculate whether the unmanned aerial vehicle hits and the hit position according to the millimeter wave and visible light cooperative detection strategy.

Description

Technical Field

[0001] This application relates to the field of guided flight evaluation of unmanned aerial vehicles, specifically to the fields of natural language processing and deep learning, and particularly to a multi-sensor collaborative hit prediction method and system. Background Technology

[0002] Image guidance using visible light images is one of the commonly used guidance methods for unmanned aerial vehicles (UAVs). Images can provide rich information such as target texture, category, and scale. By utilizing the rich feature information in images, target recognition, position estimation, and guidance and control of UAVs for specific targets can be completed relatively quickly and accurately based on spatial location.

[0003] Single-sensor perception has certain limitations. For example, the strapdown image guidance method disclosed in patents CN113158753A and CN114199084A still leads to problems such as target locking errors due to the influence of obstacles and the small size of the target relative to the field of view. At the same time, as the target and the UAV continue to approach, the proportion of the target in the image will continue to increase until it fills the field of view of the visible light sensor, resulting in a blind spot for the sensor. Within the blind spot of the seeker, it is impossible to determine the relative positional change between the target and the UAV, which can easily cause the UAV to deviate from the predetermined target position. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, this invention provides a multi-sensor collaborative hit prediction method, in which the multi-sensor mainly involves the collaboration of visible light sensors and millimeter-wave sensors. This method can continuously provide guidance and hit information for the unmanned aerial vehicle (UAV) after the visible light sensor on the seeker enters the blind zone of the field of view. At the same time, it can also provide accurate distance information and relative orientation information between the target and the UAV, making the guidance and control of the UAV more precise. Based on the millimeter-wave and visible light collaborative detection strategy, the hit status and hit location of the UAV can be calculated relatively accurately.

[0005] In a first aspect, a multi-sensor collaborative hit prediction method is provided, the method comprising the following steps:

[0006] Step 1: Based on the field of view, resolution, and blind zone of the visible light sensor, select the corresponding millimeter-wave detection range. The blind zone of the visible light sensor is am. Distribute multiple millimeter-wave detectors at preset angles on a plane with the optical axis of the visible light sensor as the normal. The spacing angle between each millimeter-wave detector is related to the millimeter-wave detection range; that is, when the field of view of each millimeter-wave detector is β°, the number of millimeter-wave detectors deployed is... One, of which This indicates rounding up, with the detection field axes of each millimeter-wave sensor at intervals of [missing information]. Based on the target type and size identified by the visible light sensor, and after detecting the target distance when the millimeter-wave sensor is activated, the approximate size of the target is calculated based on the pre-calibrated image unit pixel length. The calculation formula for the calculated target size is as follows:

[0007]

[0008] Where d 标定数量 h represents the number of pixels of the calibrated object at the calibration distance during calibration. 标定距离 d represents the actual length of the calibrated object at the calibrated distance. 目标数量 h represents the number of pixels representing the side length of the outer rectangle of the target envelope during actual detection. 实际距离 The number of pixels representing the side length of the outer rectangle of the target envelope during actual detection; the tilt angle of the millimeter-wave antenna is adjusted based on the calculated target size;

[0009] Step 2: When the distance between the target and the UAV exceeds a preset range, the UAV activates its visible light sensor to detect the target; when the distance between the target and the UAV is less than the preset range, it activates its millimeter-wave sensor to detect the target. The condition for activating different sensors is determined by calculating the number of pixels n occupied by the selected target when the visible light sensor detects the target. target With total pixels n image proportional relationship When η > 85%, the millimeter-wave sensor is activated;

[0010] Step 3: Calculate the distance between the target and the UAV based on the radio waves fed back by the millimeter-wave sensor; after the millimeter-wave sensor emits a signal, it is detected by N targets each at a distance R. i The object m is reflected back, and after a period of time... The signal reaches the receiving antenna, where c represents the speed of light. The waveform S of the received signal is... r (t) can be represented as:

[0011]

[0012] Where t represents the time component, f c Let S(·) represent the center frequency of the transmitted millimeter-wave signal, S(·) represent the frequency modulation function of the millimeter-wave sensor, and θ0 represent the initial phase of the transmitted millimeter-wave signal. By performing a Fourier transform analysis on the received signal to determine its spectral characteristics, the center frequency f corresponding to each spectral peak in the received signal spectrum can be obtained. i Then the distance information R of each target i It can be represented as The echo with the largest amplitude among all components in the spectrum is selected as the primary target, and the relative distance between the current millimeter-wave detector and the target is taken. for Where f main This represents the center frequency of the signal with the largest amplitude among all spectral peaks.

[0013] Step 4: When the seeker enters the blind zone of the field of view, the system generates a judgment result on the hit target and the hit location based on the information detected by multiple millimeter-wave sensors and the hit prediction judgment strategy algorithm.

[0014] Step 5: Based on the judgment of the target hit and the hit location, adjust the guidance and control or feed back the information of the missed hit to the ground control terminal.

[0015] Optionally, the detection range of the millimeter wave is three times the blind zone of the visible light sensor; the millimeter wave detection field of view is 14°, and it is arranged circumferentially at uniform intervals around the visible light sensor. The millimeter wave detection field of view and the detection field of view of the visible light sensor overlap at the edges, and the overlap does not exceed 30%.

[0016] Optionally, the millimeter-wave detection field of view is arranged so as not to overlap with the detection field of view of the visible light sensor.

[0017] Optionally, the millimeter-wave sensor can be activated at a distance twice the detection range of the millimeter-wave sensor, or after a delay following the activation of the visible light sensor.

[0018] Optionally, the initial tilt angle of the millimeter-wave sensor's antenna to the axis of the unmanned aerial vehicle is 22°, and the antenna's axial range of motion is ±5°.

[0019] The hit prediction and judgment strategy algorithm in step 4 is as follows:

[0020] T = ∑wi(target)T(s)

[0021] Where wi(·) represents the decision weight of different decisions, and the change of weight is mainly determined by the target type; T(·) represents the result of whether the unmanned aerial vehicle guidance can hit the target, output by different sensors; si represents the type of sensor, including visible light sensors and millimeter wave sensors.

[0022] Optionally, the process for determining whether the visible light sensor outputs the unmanned aerial vehicle's guidance system can hit its target specifically includes:

[0023] The method uses deep learning to obtain the target category in the image information, and predicts the motion state of the UAV relative to the target based on the video information of the first 5 seconds. It also gives a prediction of whether the UAV can hit the target in the next 2 seconds based on guidance and control.

[0024] Optionally, the process for determining whether the millimeter-wave sensor outputs the guidance of the unmanned aerial vehicle (UAV) can hit the target specifically includes:

[0025] The strategy-based judgment method is used to predict whether the unmanned aerial vehicle can hit the target based on the pattern of the target appearing in the field of view of each millimeter-wave sensor.

[0026] Secondly, a multi-sensor collaborative hit prediction system, the system comprising;

[0027] The selection module is used to select the corresponding millimeter-wave detection range based on the field of view, resolution, and blind zone of the visible light sensor. The blind zone of the visible light sensor is am. Multiple millimeter-wave detectors are arranged at preset angles on a plane with the optical axis of the visible light sensor as the normal. The spacing angle between each millimeter-wave detector is related to the millimeter-wave detection range; that is, when the detection field of view of each millimeter-wave detector is β°, the number of millimeter-wave detectors arranged is... One, of which This indicates rounding up, with the detection field axes of each millimeter-wave sensor at intervals of [missing information]. Based on the target type and size identified by the visible light sensor, and after detecting the target distance when the millimeter-wave sensor is activated, the approximate target size is calculated based on the pre-calibrated image unit pixel length. The calculation formula for the calculated target size is as follows:

[0028]

[0029] Where d 标定数量 h represents the number of pixels of the calibrated object at the calibration distance during calibration. 标定距离 d represents the actual length of the calibrated object at the calibrated distance. 目标数量 h represents the number of pixels representing the side length of the outer rectangle of the target envelope during actual detection. 实际距离 The number of pixels representing the side length of the outer rectangle of the target envelope during actual detection; the tilt angle of the millimeter-wave antenna is adjusted based on the calculated target size;

[0030] The detection module is used to activate the visible light sensor to detect the target when the distance between the UAV and the target exceeds a preset range, and to activate the millimeter-wave sensor to detect the target when the distance between the UAV and the target is less than the preset range. The condition for activating different sensors is determined by calculating the number of pixels n occupied by the selected target when the visible light sensor detects the target. target With total pixels n image proportional relationship When η > 85%, the millimeter-wave sensor is activated;

[0031] The distance calculation module is used to calculate the distance between the target and the unmanned aerial vehicle (UAV) based on the radio waves fed back by the millimeter-wave sensor. When the millimeter-wave sensor emits a signal, it is received by N targets, each at a distance R. i The object m is reflected back, and after a period of time... The signal reaches the receiving antenna, where c represents the speed of light. The waveform S of the received signal is... r (t) can be represented as:

[0032]

[0033] Where t represents the time component, f c Let S(·) represent the center frequency of the transmitted millimeter-wave signal, S(·) represent the frequency modulation function of the millimeter-wave sensor, and θ0 represent the initial phase of the transmitted millimeter-wave signal. By performing a Fourier transform analysis on the received signal to determine its spectral characteristics, the center frequency f corresponding to each spectral peak in the received signal spectrum can be obtained. i Then the distance information R of each target i It can be represented as The echo with the largest amplitude among all components in the spectrum is selected as the primary target, and the relative distance between the current millimeter-wave detector and the target is taken. for Where f main This represents the center frequency of the signal with the largest amplitude among all spectral peaks.

[0034] The judgment module is used to generate a judgment result of the hit target and the hit location based on the information detected by multiple millimeter-wave sensors and the hit prediction judgment strategy algorithm when the seeker enters the blind zone of the field of view.

[0035] The feedback module is used to adjust the guidance control based on the judgment of the target hit and the hit location, or to feed back the information of non-direct hits to the ground control terminal.

[0036] Thirdly, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the multi-sensor collaborative hit prediction method described in any of the first aspects above.

[0037] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the multi-sensor collaborative hit prediction method described in any of the first aspects above.

[0038] Compared with the prior art, this application has at least the following beneficial effects:

[0039] This invention integrates information from visible light sensors and millimeter-wave sensors on unmanned aerial vehicles, enabling it to provide accurate target orientation and distance information outside the seeker's blind zone, and to provide information on target deviation from the detection field of view within the seeker's blind zone. Based on the millimeter-wave detection information and visible light sensor information, it can output an accurate judgment on whether the braking control can hit the target. Attached Figure Description

[0040] Figure 1 This is a flowchart of a multi-sensor collaborative hit prediction method according to this application;

[0041] Figure 2 The distribution map of the detection field of view in an example of a multi-sensor collaborative hit prediction method provided in this application embodiment;

[0042] Figure 3 A schematic diagram of the antenna distribution for a multi-sensor collaborative hit prediction method provided in an embodiment of this application;

[0043] Figure 4 A block diagram of the module architecture of a multi-sensor collaborative hit prediction system provided in one embodiment of this application;

[0044] Figure 5 This is a diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0046] In the description of this application, the terms “comprising,” “having,” and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are expressly listed, but may also include other steps or units that are not expressly listed but are inherent to these processes, methods, products, or devices, or steps or units added based on further optimizations of the inventive concept.

[0047] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that the following description provides detailed implementation methods and specific operating procedures, intended to provide a detailed explanation of this application, but the scope of protection of the present invention is not limited to these embodiments.

[0048] like Figure 1As shown, this invention provides a multi-sensor collaborative hit prediction method. This method integrates information from visible light sensors and millimeter-wave sensors on the seeker head. After the visible light sensor enters the blind zone of the field of view, the millimeter-wave sensor continuously provides guidance and hit information for the unmanned aerial vehicle (UAV). Simultaneously, it can also provide precise distance information and relative azimuth information between the target and the UAV, making the guidance and control of the UAV more accurate. Based on the millimeter-wave and visible light collaborative detection strategy, the hit status and hit location of the UAV can be calculated relatively accurately. The method includes the following steps:

[0049] Step 1: Select a suitable millimeter-wave detection range based on the field of view, resolution, and blind zone of the visible light sensor. Let the blind zone of the visible light sensor be am, meaning that when the distance between the target and the sensor is less than or equal to am, the target image will fill the entire field of view of the visible light sensor. At a distance am from the target, the envelope formed by multiple millimeter-wave detection ranges should be greater than the detection boundary of the visible light sensor at a distance am from the detector plane with its optical axis as the normal. Distribute multiple millimeter-wave detectors at a certain angle on a plane with the optical axis of the visible light sensor as the normal. The angle between each millimeter-wave detector should be related to the millimeter-wave detection range. That is, let the detection field of view of each millimeter-wave detector be β°, then the number of millimeter-wave detectors required is... One, of which This indicates rounding up, with the detection field axes of each millimeter-wave sensor at intervals of [missing information]. Based on the target type and size identified by the visible light sensor, and after detecting the target distance when the millimeter-wave sensor is activated, the approximate target size is calculated based on the pre-calibrated image unit pixel length. The calculation formula for the calculated target size is as follows:

[0050]

[0051] Where d 标定数量 h represents the number of pixels of the calibrated object at the calibration distance during calibration. 标定距离 d represents the actual length of the calibrated object at the calibrated distance. 目标数量 h represents the number of pixels representing the side length of the outer rectangle of the target envelope during actual detection. 实际距离 The number of pixels representing the side length of the outer rectangle of the target envelope during actual detection;

[0052] Based on the target size calculated above, the tilt angle of the millimeter-wave antenna is adjusted so that the range of the millimeter-wave antenna at a distance of M m from the target can just surround a square object with a width of W m.

[0053] In this embodiment, the visible light sensor and the unmanned aerial vehicle (UAV) are fixedly connected. The visible light sensor has a field of view of 34° and a resolution of 1080P. The initial tilt angle between the axis of the millimeter-wave field of view and the axis of the UAV is 22°, and its field of view is adjusted to 14°. Six millimeter-wave sensors are evenly arranged around the UAV. The detection range of the millimeter waves is three times the blind zone of the visible light sensor. The detection fields of view of the millimeter waves and the detection fields of view of the visible light sensors overlap to a certain extent at the edges, and the overlap should not exceed 30%. A schematic diagram of the detection field of view after the arrangement is shown below. Figure 2 As shown; the arrangement of the selected millimeter-wave antenna is as follows Figure 3 As shown;

[0054] In this embodiment, M is the distance of the blind zone of the visible light sensor, and W is the length of the range enclosed by the field of view of the millimeter-wave detector after the millimeter-wave antenna is adjusted.

[0055] Step 2: When the target is far from the UAV, the visible light sensor is mainly activated to detect the target. When the UAV enters a certain range, the millimeter-wave sensor is activated. The condition for activating different sensors is that when the visible light sensor detects the target, the pixel area n occupied by the selected target is calculated. target With total pixels n image proportional relationship When η > 85%, the millimeter-wave sensor is activated;

[0056] In this embodiment, the millimeter-wave sensor is activated at a distance twice the detection range of the millimeter-wave sensor. The determination condition is based on calculating the number of pixels n occupied by the target when the visible light sensor identifies the target. target With total pixels n image proportional relationship When η>85%, the millimeter-wave sensor is turned on. When the visible light sensor is turned on, the convolutional neural network is mainly used to identify and track the target. After the target identification is completed, the target type information is transmitted to the information processing unit.

[0057] Step 3: Calculate the distance between the target and the UAV based on the radio waves fed back by the millimeter-wave sensors. Assume that after a certain millimeter-wave sensor emits a signal, it is detected by N other sensors, each at a distance R. i The object m is reflected back, and after a period of time... The signal reaches the receiving antenna, where c represents the speed of light. The waveform S of the received signal is... r (t) can be represented as:

[0058]

[0059] Where t represents the time component, fc represents the center frequency of the transmitted millimeter-wave signal, S(·) represents the frequency modulation function of the millimeter-wave sensor, and θ0 represents the initial phase of the transmitted millimeter-wave signal. By performing a Fourier transform analysis on the received signal to determine its spectral characteristics, the center frequencies fi corresponding to each spectral peak in the received signal spectrum can be obtained. Then, the distance information Ri for each target can be expressed as... The echo with the largest amplitude among all components in the spectrum is selected as the primary target, and the relative distance between the current millimeter-wave detector and the target is taken. for Where f main This represents the center frequency of the signal with the largest amplitude among all spectral peaks.

[0060] Step 4: When the seeker enters the blind zone of the field of view, it generates an interpretation of whether the target can be hit and a judgment of the hit location based on the information detected by multiple millimeter-wave sensors and the hit prediction judgment strategy algorithm.

[0061] Specifically, the hit prediction algorithm is as follows:

[0062] T=∑w i (target)T(s i )

[0063] Where w i (·) represents the decision weight of different decisions, and the change of weight is mainly determined by the target type; T(·) represents the result of whether the UAV guidance can hit the target, and si represents the type of sensor, including visible light sensors and millimeter wave sensors.

[0064] In this embodiment of the application, the method for outputting whether the visible light sensor can guide the unmanned aerial vehicle to hit the target uses deep learning to obtain the target category in the image information, predicts the motion state of the unmanned aerial vehicle relative to the target based on the video information of the first 5 seconds, and gives a prediction of whether the unmanned aerial vehicle can hit the target according to the guidance control in the next 2 seconds.

[0065] The method for determining whether a UAV's guidance system can hit a target using millimeter-wave sensors employs a strategy-based judgment approach. It predicts whether the UAV's guidance and control system can hit the target based on the pattern of the target's appearance within the fields of view of each millimeter-wave sensor. If the target is small, and no signal is observed in multiple millimeter-wave sensor fields of view while the target pixels continuously fill the visible light sensor's field of view, then the UAV's guidance and control system is deemed capable of hitting the target. If the target appears in a particular millimeter-wave field of view, then the UAV's guidance and control system is deemed unable to hit the target, and the target's position is output based on the location of the millimeter-wave sensors. If the target is large, and no signal is observed in multiple millimeter-wave sensor fields of view while the target pixels continuously fill the visible light sensor's field of view, or if the target appears continuously in several surrounding millimeter-wave field of view, then the UAV's guidance and control system is deemed capable of hitting the target. Otherwise, the UAV's guidance and control system is deemed unable to hit the target.

[0066] Step 5: Based on the assessment of whether a hit is possible, adjust the guidance and control, or feed back the information of a missed hit to the ground control station to await further instructions and planning.

[0067] In one embodiment, such as Figure 4 As shown, a multi-sensor collaborative hit prediction system is provided, the system comprising:

[0068] The selection module is used to select the corresponding millimeter-wave detection range based on the field of view, resolution, and blind zone of the visible light sensor. The blind zone of the visible light sensor is am. Multiple millimeter-wave detectors are arranged at preset angles on a plane with the optical axis of the visible light sensor as the normal. The spacing angle between each millimeter-wave detector is related to the millimeter-wave detection range; that is, when the detection field of view of each millimeter-wave detector is β°, the number of millimeter-wave detectors arranged is... One, of which This indicates rounding up, with the detection field axes of each millimeter-wave sensor at intervals of [missing information]. Based on the target type and size identified by the visible light sensor, and after detecting the target distance when the millimeter-wave sensor is activated, the approximate target size is calculated based on the pre-calibrated image unit pixel length. The calculation formula for the calculated target size is as follows:

[0069]

[0070] Where d 标定数量 h represents the number of pixels of the calibrated object at the calibration distance during calibration. 标定距离 d represents the actual length of the calibrated object at the calibrated distance. 目标数量 h represents the number of pixels representing the side length of the outer rectangle of the target envelope during actual detection. 实际距离The number of pixels representing the side length of the outer rectangle of the target envelope during actual detection; the tilt angle of the millimeter-wave antenna is adjusted based on the calculated target size;

[0071] The detection module is used to activate the visible light sensor to detect the target when the distance between the UAV and the target exceeds a preset range, and to activate the millimeter-wave sensor to detect the target when the distance between the UAV and the target is less than the preset range. The condition for activating different sensors is determined by calculating the number of pixels n occupied by the selected target when the visible light sensor detects the target. target With total pixels n image proportional relationship When η > 85%, the millimeter-wave sensor is activated;

[0072] The distance calculation module is used to calculate the distance between the target and the unmanned aerial vehicle (UAV) based on the radio waves fed back by the millimeter-wave sensor. When the millimeter-wave sensor emits a signal, it is received by N targets, each at a distance R. i The object m is reflected back, and after a period of time... The signal reaches the receiving antenna, where c represents the speed of light. The waveform S of the received signal is... r (t) can be represented as:

[0073]

[0074] Where t represents the time component, f c Let S(·) represent the center frequency of the transmitted millimeter-wave signal, S(·) represent the frequency modulation function of the millimeter-wave sensor, and θ0 represent the initial phase of the transmitted millimeter-wave signal. By performing a Fourier transform analysis on the received signal to determine its spectral characteristics, the center frequency f corresponding to each spectral peak in the received signal spectrum can be obtained. i Then the distance information R of each target i It can be represented as The echo with the largest amplitude among all components in the spectrum is selected as the primary target, and the relative distance between the current millimeter-wave detector and the target is taken. for Where f main This represents the center frequency of the signal with the largest amplitude among all spectral peaks.

[0075] The judgment module is used to generate a judgment result of the hit target and the hit location based on the information detected by multiple millimeter-wave sensors and the hit prediction judgment strategy algorithm when the seeker enters the blind zone of the field of view.

[0076] The feedback module is used to adjust the guidance control based on the judgment of the target hit and the hit location, or to feed back the information of non-direct hits to the ground control terminal.

[0077] In one embodiment, an electronic device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the electronic device includes a processor, memory, and network interface connected via a system bus. The processor provides computing and control capabilities, and the network interface enables communication with external terminals via a network connection. The electronic device implements the aforementioned target motion prediction method by loading and running a computer program.

[0078] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0079] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A multi-sensor collaborative hit prediction method, characterized in that, The method includes the following steps: Step 1: Based on the field of view, resolution, and blind zone of the visible light sensor, select the corresponding millimeter-wave detection range. The blind zone of the visible light sensor is am. Distribute multiple millimeter-wave detectors at preset angles on a plane with the optical axis of the visible light sensor as the normal. The spacing angle between each millimeter-wave detector is related to the millimeter-wave detection range; that is, when the field of view of each millimeter-wave detector is β°, the number of millimeter-wave detectors deployed is... One, of which Indicates rounding up, the interval between the detection field axes of each millimeter-wave sensor is... Based on the target type and size identified by the visible light sensor, and after detecting the target distance when the millimeter-wave sensor is activated, the approximate size of the target is calculated based on the pre-calibrated image unit pixel length. The calculation formula for the calculated target size is as follows: ； Where d 标定数量 h represents the number of pixels of the calibrated object at the calibration distance during calibration. 标定距离 d represents the actual length of the calibrated object at the calibrated distance. 目标数量 h represents the number of pixels representing the side length of the outer rectangle of the target envelope during actual detection. 实际距离 This represents the actual length of the outer rectangle of the target envelope during actual detection; the tilt angle of the millimeter-wave antenna is adjusted based on the calculated target size. Step 2: When the distance between the target and the UAV exceeds a preset range, the UAV activates its visible light sensor to detect the target; when the distance between the target and the UAV is less than the preset range, it activates its millimeter-wave sensor to detect the target. The condition for activating different sensors is determined by calculating the number of pixels n occupied by the selected target when the visible light sensor detects the target. target With total pixels n image proportional relationship When η > 85%, the millimeter-wave sensor is activated; Step 3: Calculate the distance between the target and the UAV based on the radio waves fed back by the millimeter-wave sensor; after the millimeter-wave sensor emits a signal, it is detected by N targets each at a distance R. i The object is reflected back, and after a period of time The signal reaches the receiving antenna, where c represents the speed of light. The waveform S of the received signal is... r (t) is represented as: ； Where t represents the time component, f c S represents the center frequency of the transmitted millimeter-wave signal, and S(·) represents the frequency modulation function of the millimeter-wave sensor. The initial phase of the transmitted millimeter-wave signal is represented by the initial phase. A Fourier transform analysis of the received signal is performed to determine its spectral characteristics, yielding the center frequency f corresponding to each spectral peak in the received signal's spectrum. i Then the distance information R of each target i Represented as The echo with the largest amplitude among all components in the spectrum is selected as the main target, and the relative distance between the current millimeter-wave detector and the target is taken. for , where f main This represents the center frequency of the signal with the largest amplitude among all spectral peaks. Step 4: When the seeker enters the blind zone of the field of view, the system generates a judgment result on the hit target and the hit location based on the information detected by multiple millimeter-wave sensors and the hit prediction judgment strategy algorithm. Step 5: Based on the judgment of the target hit and the hit location, adjust the guidance and control or feed back the information of the missed hit to the ground control terminal.

2. The method according to claim 1, characterized in that, The detection range of millimeter waves is three times that of the blind zone of visible light sensors; the detection field of view of millimeter waves is 14°, and they are arranged circumferentially at uniform intervals around the visible light sensors. The detection field of view of millimeter waves and the detection field of view of visible light sensors overlap at the edges, and the overlap does not exceed 30%.

3. The method according to claim 1, characterized in that, The millimeter-wave detection field of view and the visible light sensor detection field of view are arranged without overlap.

4. The method according to claim 1, characterized in that, The millimeter-wave sensor is activated at a distance twice the detection range of the millimeter-wave sensor, or after a delay following the activation of the visible light sensor.

5. The method according to claim 1, characterized in that, The initial tilt angle of the millimeter-wave sensor's antenna to the axis of the unmanned aerial vehicle is 22°, and the antenna's axial range of motion is ±5°. The hit prediction and judgment strategy algorithm in step 4 is as follows: ； Where w i (·) represents the decision weight for different decisions, and the change in weight is mainly determined by the target type; T(·) represents the result of whether the UAV guidance can hit the target at different sensor outputs, s i The type of sensor is indicated, and the different sensors include visible light sensors and millimeter-wave sensors.

6. The method according to claim 1, characterized in that, The process for determining whether the visible light sensor outputs the guidance of the unmanned aerial vehicle (UAV) can hit the target specifically includes: The method uses deep learning to obtain the target category in the image information, and predicts the motion state of the UAV relative to the target based on the video information of the first 5 seconds. It also gives a prediction of whether the UAV can hit the target in the next 2 seconds based on guidance and control.

7. The method according to claim 1, characterized in that, The process for determining whether the millimeter-wave sensor outputs the guidance of the unmanned aerial vehicle (UAV) can hit the target specifically includes: The strategy-based judgment method is used to predict whether the unmanned aerial vehicle can hit the target based on the pattern of the target appearing in the field of view of each millimeter-wave sensor.

8. A multi-sensor collaborative hit prediction system, characterized in that, The system includes; The selection module is used to select the corresponding millimeter-wave detection range based on the field of view, resolution, and blind zone of the visible light sensor. The blind zone of the visible light sensor is am. Multiple millimeter-wave detectors are arranged at preset angles on a plane with the optical axis of the visible light sensor as the normal. The spacing angle between each millimeter-wave detector is related to the millimeter-wave detection range; that is, when the detection field of view of each millimeter-wave detector is β°, the number of millimeter-wave detectors arranged is... One, of which Indicates rounding up, the interval between the detection field axes of each millimeter-wave sensor is... Based on the target type and size identified by the visible light sensor, and after detecting the target distance when the millimeter-wave sensor is activated, the approximate size of the target is calculated based on the pre-calibrated image unit pixel length. The calculation formula for the calculated target size is as follows: ； Where d 标定数量 h represents the number of pixels of the calibrated object at the calibration distance during calibration. 标定距离 d represents the actual length of the calibrated object at the calibrated distance. 目标数量 h represents the number of pixels representing the side length of the outer rectangle of the target envelope during actual detection. 实际距离 This represents the actual length of the outer rectangle of the target envelope during actual detection; the tilt angle of the millimeter-wave antenna is adjusted based on the calculated target size. The detection module is used to activate the visible light sensor to detect the target when the distance between the UAV and the target exceeds a preset range, and to activate the millimeter-wave sensor to detect the target when the distance between the UAV and the target is less than the preset range. The condition for activating different sensors is determined by calculating the number of pixels n occupied by the selected target when the visible light sensor detects the target. target With total pixels n image proportional relationship When η > 85%, the millimeter-wave sensor is activated; The distance calculation module is used to calculate the distance between the target and the unmanned aerial vehicle (UAV) based on the radio waves fed back by the millimeter-wave sensor. When the millimeter-wave sensor emits a signal, it is received by N targets, each at a distance R. i The object is reflected back, and after a period of time The signal reaches the receiving antenna, where c represents the speed of light. The waveform S of the received signal is... r (t) is represented as: ； Where t represents the time component, f c S represents the center frequency of the transmitted millimeter-wave signal, and S(·) represents the frequency modulation function of the millimeter-wave sensor. The initial phase of the transmitted millimeter-wave signal is represented by the initial phase. A Fourier transform analysis of the received signal is performed to determine its spectral characteristics, yielding the center frequency f corresponding to each spectral peak in the received signal's spectrum. i Then the distance information R of each target i Represented as The echo with the largest amplitude among all components in the spectrum is selected as the main target, and the relative distance between the current millimeter-wave detector and the target is taken. for , where f main This represents the center frequency of the signal with the largest amplitude among all spectral peaks. The judgment module is used to generate a judgment result of the hit target and the hit location based on the information detected by multiple millimeter-wave sensors and the hit prediction judgment strategy algorithm when the seeker enters the blind zone of the field of view. The feedback module is used to adjust the guidance control based on the judgment of the target hit and the hit location, or to feed back the information of missed hits to the ground control terminal.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images