Method, device, equipment and medium for target recognition in farmland based on multi-source information fusion

Abstract

The present invention relates to a method, device, equipment, and medium for identifying targets in farmland using multi-source information fusion, and belongs to the technical field of farmland monitoring using multi-source information fusion. The method sets distance influence weights and environmental influence weights based on the characteristics of each sensor to characterize its recognition accuracy after being affected by distance and environment, and dynamically adjusts the weights under different distances and environments. The weights are used to weight the targets identified by each sensor to obtain a fusion result under multi-source data. While correcting the recognition results of different sensors using the matching weights of the targets identified by each sensor to ensure the accuracy of target recognition in multi-source information fusion, the method also avoids reprocessing the pixel or point cloud data of the target identified by each sensor during the fusion process, significantly improving the fusion processing speed. Ultimately, the method can significantly improve the target recognition speed under multi-source information fusion while ensuring recognition accuracy, thereby improving the agricultural protection effect.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-source information fusion target recognition, and in particular to a multi-source information fusion target recognition method, device, equipment and medium in farmland. Background Art

[0002] With the development of agricultural science and technology, smart agriculture has gradually become an important development direction of modern agriculture. In the sub-sectors of smart agricultural management, farmland safety monitoring has received more and more attention, especially in application scenarios such as scientific research experimental fields, high-value crop planting areas, and unmanned farms, which have put forward higher requirements for intrusion monitoring, theft prevention, and wild animal expulsion.

[0003] In farmland safety monitoring, multi-source sensors are often used to comprehensively identify intrusion targets in farmland to address the limitations of a single data source, which prevents all-weather and multi-scenario monitoring and leads to inaccurate monitoring. However, while multi-source sensors can accurately identify target type and location and perform targeted processing through complex target recognition and fusion algorithms, the complexity of multi-source data computation and fusion results in a slow output of intrusion targets, significantly compromising the real-time performance of target recognition. In agricultural protection scenarios, immediate alerts upon detection of intrusion targets and their swift removal by manual or drone-based means to prevent actual damage and further development are effective means of reducing farmland damage and safeguarding scientific research results. However, due to the low real-time performance of target recognition in current multi-source data fusion monitoring methods in agricultural protection scenarios, most cannot issue timely alerts before actual damage occurs, resulting in suboptimal agricultural protection effectiveness in practice. Therefore, improving the speed of target recognition using multi-source data is crucial for effectively ensuring crop safety and improving agricultural protection effectiveness.

[0004] In summary, the current farmland monitoring methods in agricultural insurance scenarios have the technical problem of slow target recognition speed under multi-source information fusion, resulting in unsatisfactory agricultural insurance effects. Summary of the Invention

[0005] In view of this, the embodiments of the present invention provide a method, device, equipment and medium for target identification in farmland using multi-source information fusion to solve the technical problem that the speed of target identification using multi-source information fusion in current agricultural insurance scenarios is slow, resulting in unsatisfactory agricultural insurance effects.

[0006] In a first aspect, a method for identifying targets in farmland using multi-source information fusion is provided, the method comprising:

[0007] Determine the targets identified by each sensor in the farmland area, and assign a distance impact weight and an environmental impact weight to each target identified by the current sensor;

[0008] Targets identified by different sensors and with position deviations less than a preset deviation threshold are classified as the same target to be confirmed; the product of the distance influence weight and the environmental influence weight of the target identified by a certain sensor under the current target to be confirmed is used as the effective weight of the certain sensor; the sum of the effective weights of all sensors under the current target to be confirmed is recorded as the comprehensive contribution value; when the comprehensive contribution value is greater than the preset contribution threshold, the current target to be confirmed is confirmed as a valid target;

[0009] The position of the effective target is determined and output, completing the target recognition in the farmland.

[0010] In a second aspect, a device for identifying targets in farmland using multi-source information fusion is provided, the device comprising:

[0011] The weight allocation module is used to determine the targets identified by each sensor in the farmland area and assign a distance impact weight and an environmental impact weight to each target identified by the current sensor;

[0012] The valid target confirmation module is used to classify targets identified by different sensors and whose position deviations are less than a preset deviation threshold as the same target to be confirmed; the product of the distance influence weight and the environmental influence weight of the target identified by a certain sensor under the current target to be confirmed is used as the effective weight of the certain sensor; the sum of the effective weights of all sensors under the current target to be confirmed is recorded as the comprehensive contribution value; and the current target to be confirmed is confirmed as a valid target when the comprehensive contribution value is greater than the preset contribution threshold;

[0013] The target recognition module is used to determine the position of the effective target and output it to complete the target recognition in the farmland.

[0014] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for identifying targets in farmland by fusion of multi-source information as described in the first aspect is implemented.

[0015] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for identifying targets in farmland by fusion of multi-source information as described in the first aspect.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] The present invention sets distance influence weights and environmental influence weights according to the characteristics of each sensor to characterize its recognition accuracy after being affected by distance and environment, and dynamically adjusts the weight size under different distances and environments. The weights are used to weight the targets recognized by each sensor to obtain the fusion results under multi-source data. While correcting the recognition results of different sensors with the matching weights of the targets recognized by each sensor to ensure the accuracy of target recognition in multi-source information fusion, it can also avoid reprocessing the pixels or point cloud data of the targets recognized by each sensor during the fusion process, significantly improving the fusion processing speed. Ultimately, the target recognition speed under multi-source information fusion can be significantly improved while ensuring recognition accuracy, thereby improving the agricultural protection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a schematic diagram of an application environment of a method for identifying targets in farmland using multi-source information fusion provided in the first embodiment of the present invention;

[0020] Figure 2 This is a flow chart of a method for identifying targets in farmland using multi-source information fusion, provided in the first embodiment of the present invention;

[0021] Figure 3 This is a flow chart of a method for identifying targets in farmland using multi-source information fusion under the condition of determining sensor types, provided by the second embodiment of the present invention;

[0022] Figure 4 This is a schematic structural diagram of a device for identifying targets in farmland using multi-source information fusion, provided in a third embodiment of the present invention;

[0023] Figure 5 This is a structural diagram of a computer device provided in Example 4 of the present invention. DETAILED DESCRIPTION

[0024] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0025] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0026] It will also be understood that the term "and / or" used in the present description and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0027] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0028] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0029] References to "one embodiment" or "some embodiments" in the present specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0030] Embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0031] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0032] It should be understood that the order of execution of the steps in the following embodiments does not necessarily mean the order in which they are executed. The order in which each process is executed should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0033] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0034] The first embodiment of the present invention provides a method for identifying targets in farmland by fusion of multi-source information, which can be applied in the following situations: Figure 1 In an application environment, the client communicates with the server, and the server communicates with the sensing device. The client includes but is not limited to PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud terminal devices, personal digital assistants (PDAs), and other terminal devices. The sensing device includes but is not limited to radars, multi-lens cameras, infrared cameras, spherical cameras, and other sensing devices. The server can be implemented as an independent server or a server cluster consisting of multiple servers. The above-mentioned multi-source information fusion target recognition method in farmland can be applied to the server in Figure 1. The computer device corresponding to the server is connected to the corresponding database, rule base, etc. to obtain the corresponding data in the database. The above-mentioned computer device can also be connected to the client to collect data and control instructions sent by the client user.

[0035] Example 1:

[0036] See also Figure 2 , is a flow chart of a method for identifying targets in farmland using multi-source information fusion provided in the first embodiment of the present invention, such as Figure 2 As shown, the method for identifying targets in farmland by fusion of multi-source information may include the following steps:

[0037] S101, determining the targets identified by each sensor in the farmland area, and assigning a distance impact weight and an environment impact weight to each target identified by the current sensor.

[0038] Specifically, multiple types of sensor devices are used to identify targets in the farmland area. Sensor devices may include radar, multi-lens cameras, infrared cameras, spherical cameras, etc. Each sensor device can identify targets in the farmland area and determine the target location using its own target location determination method.

[0039] Since different targets are at different distances from the sensor, and the degree of recognition error caused by different sensors due to recognition distance is also different, it is necessary to assign distance influence weights to different targets recognized by a single sensor to represent the accuracy differences of the single sensor when recognizing targets at different positions.

[0040] In addition, different sensors are affected differently by different types of environmental factors, and the same environmental factors have different degrees of impact on different types of sensors. Therefore, it is also necessary to assign respective environmental impact weights to the targets identified by different types of sensors to characterize the accuracy differences of a single sensor when identifying targets in different environments.

[0041] It is easy to understand that the specific values ​​of the distance influence weight and the environment influence weight assigned to each target identified by different sensors need to be set according to the distance of the identified target from the sensor and the type of sensor in combination with the sensor characteristics.

[0042] S102, targets identified by different sensors and with position deviations less than a preset deviation threshold are classified as the same target to be confirmed; the product of the distance influence weight and the environmental influence weight of the target identified by a certain sensor under the current target to be confirmed is used as the effective weight of the certain sensor, and the sum of the effective weights of all sensors under the current target to be confirmed is recorded as the comprehensive contribution value. When the comprehensive contribution value is greater than the preset contribution threshold, the current target to be confirmed is confirmed as a valid target.

[0043] Due to recognition errors, the positional results obtained by different sensors for the same object in a farmland area will not be exactly the same; there will typically be slight deviations, but it's easy to understand that these deviations are not significant. Therefore, this embodiment first classifies objects identified by different sensors with positional deviations less than a preset deviation threshold as the same pending object to facilitate subsequent analysis. The preset deviation threshold is determined based on the target recognition error or performance of each sensor. For example, a value of 1 meter can be used.

[0044] The same target to be confirmed actually corresponds to the same number of targets as the number of sensor types. Each of these targets is identified by a different type of sensor. As mentioned above, each sensor is affected by different distance and environmental influences when completing its corresponding target position identification. It is necessary to characterize the distance influence weight and environmental influence weight assigned to the target identified by each sensor.

[0045] Then, this embodiment uses the product of the distance influence weight and the environment influence weight of the target identified by a certain sensor under the current target to be confirmed as the effective weight of the certain sensor to comprehensively characterize the error influence of the distance and the environment on the target identified by the certain sensor, and records it as the effective weight of the certain sensor. In this way, the effective weight of each sensor under the current target to be confirmed can be determined in turn.

[0046] The effective weight of this type of sensor represents the accuracy of the sensor's recognition of the target after being affected by distance and environment when obtaining the target corresponding to the current target to be confirmed. The larger the effective weight, the less the sensor is affected by distance and environment when identifying the target, the smaller the corresponding recognition error, and the greater the recognition accuracy.

[0047] The sum of the effective weights of all sensors for the current target is recorded as the comprehensive contribution value. If this comprehensive contribution value is greater than a preset contribution threshold, it means that the target identified by all sensors can accurately represent the actual location of the object after fusion. This also means that the target identification results of each sensor corresponding to the current target are valid and the current target is considered a valid target. The preset contribution threshold can be determined through experimentation based on the actual sensor type used, and this embodiment does not specifically limit its value.

[0048] S103, determining the position of the effective target and outputting it, completing target recognition in the farmland.

[0049] After determining that the current target to be confirmed is a valid target, that is, after determining that the target results identified by each sensor corresponding to the current target to be confirmed are available, because the valid target is actually a collection of results of the target identified by each sensor rather than a target determination result, that is, it is not a single target, it is also necessary to fuse the targets identified by each sensor to determine the position of the valid target.

[0050] Specifically, the target position identified by each sensor under the effective target is weighted by the effective weight of the sensor to obtain a weighted value, and all the weighted values ​​are summed and divided by the comprehensive contribution value to obtain the position of the effective target and output it to complete target recognition in the farmland.

[0051] Example 2:

[0052] See also Figure 3 , is a flow chart of a method for identifying targets in farmland by multi-source information fusion after clarifying the type of sensors used, provided by the second embodiment of the present invention. Figure 3 As shown, the sensors specifically include radar, multi-camera and spherical camera.

[0053] In traditional modes, target recognition relies on radar or other single information input sources, which magnifies the recognition shortcomings of this type of sensor equipment. In actual application scenarios, the effect is very unsatisfactory. This embodiment simultaneously uses radar, multi-camera and spherical cameras to identify targets in farmland areas.

[0054] Radars offer all-weather capabilities, long-range target identification, high target accuracy, multi-target detection capabilities, and strong counting and ranging capabilities. However, radars have low-altitude blind spots, limited ability to detect stealth targets within those blind spots, and are susceptible to electronic jamming. Compared to infrared sensors or cameras, radars have limitations in target recognition.

[0055] Multi-cameras can simultaneously monitor multiple directions and angles, achieving all-around coverage and effectively reducing blind spots. They can simultaneously monitor multiple targets and areas from the same device, instantly identifying targets at different distances without zooming. However, multi-cameras have range limitations, limiting their accuracy in target recognition and tracking. In situations with a large number of targets, high-speed motion, or complex scenes, multi-cameras may result in misidentification or missed recognition, and their adaptability to the environment is poor.

[0056] Spherical cameras have the characteristics of fast operation speed, optical zoom, and precise positioning. They can flexibly adjust the direction and focal length of the camera to track and identify specific targets. They have the advantages of long tracking distance and can track moving targets. However, their optical zoom requires a reaction process or response time, so it will have a certain impact on recognition accuracy.

[0057] This embodiment first uses radar, multi-lens camera, and spherical camera to identify targets in the farmland area. The radar first uses an irregular boundary regularization method and combines it with irregular delineation on an electronic map to achieve physical boundary demarcation. Then, based on its own known position, it determines the relative position of the target from the radar by the target's distance from itself and its azimuth relative to itself. After radar relative coordinate conversion, the actual coordinate position of the target is determined. The multi-lens camera uses the camera azimuth and the identified image-based pixel coordinates to derive the actual geographic longitude and latitude coordinates corresponding to the target, i.e., the actual coordinate position, through a calibration algorithm. The spherical camera uses image recognition technology to automatically adjust the viewing angle through the PTZ function after tracking the target, so that the target always remains in the center of the field of view. The latitude and longitude of the target are calculated based on the horizontal and vertical rotation angles and zoom ratio of the dome camera to obtain the target's actual coordinate position.

[0058] When acquiring multi-source information, the target reporting sampling frequency of each sensor device is inconsistent, but the sampling interval of each sensor device is known. Therefore, the last target acquisition result of each device before the current moment can be used as the target acquisition result of each device at the current moment to solve the problem of inconsistent sampling frequency.

[0059] In addition, each sensor device has different recognition sensitivity and accuracy at different distance stages. In addition, under different environmental conditions, the recognition performance of each sensor device is also different. Therefore, it is necessary to set the distance influence weight and environmental influence weight for the above three sensor devices respectively to reflect the recognition accuracy of the device after considering the impact of distance and environment on each sensor device.

[0060] Specifically, regarding the distance impact weight distribution of radar, multi-camera and spherical camera:

[0061] Because farmland is generally large, radar and other sensor equipment are often mounted high on poles to fully identify the entire area. However, radars have blind spots. The distance between the blind spot boundary and the radar itself, or in other words, the size of the blind spot, increases with the height of the radar. Furthermore, the radar's recognition accuracy for targets closer than the blind spot boundary, or within the blind spot, decreases significantly, and accuracy decreases as the distance from the object decreases.

[0062] Therefore, the accuracy of the radar in identifying the target shows a changing pattern that increases as the distance of the target from the radar increases and becomes a fixed value after exceeding a first distance value. Correspondingly, the distance influence weight of the target identified by the radar is set to increase as the distance between the target and the radar increases, and becomes a first fixed value after the distance exceeds a first distance value. The first distance value is set in this embodiment based on the distance from the blind spot boundary of the radar to the radar. In this embodiment, for illustration, it is preferably set to 15m and the first fixed value to 1. That is, it is considered that after the distance of the object from the radar exceeds the first distance value, the radar's target identification accuracy is no longer affected by the distance.

[0063] A multi-camera, equipped with multiple cameras, can simultaneously monitor targets from multiple directions, angles, and distances, achieving full coverage of the monitored area (in this example, the farmland). This feature eliminates the need for lens zoom adjustment during monitoring, resulting in superior real-time performance. Distance only affects the clarity of the target image captured, and thus the accuracy of target recognition. It is easy to understand that as the distance of the target from the multi-camera increases, the accuracy of the multi-camera's target recognition decreases. Therefore, for the multi-camera, the distance influence weight of the target it recognizes decreases with increasing distance.

[0064] Spherical cameras feature fast speeds, optical zoom, and precise positioning. They can flexibly adjust the camera's direction and focal length to track and identify targets. However, it's precisely this method of tracking targets with a single camera through zooming that dictates that when identifying distant or nearer targets, it must zoom upward or downward from a single focal length. This zooming process requires a certain amount of response time, which can affect target recognition accuracy due to untimely image acquisition. This accuracy impact increases with the degree of zooming upward or downward from a single focal length. Therefore, the accuracy of a spherical camera's target recognition increases, then decreases, as the distance between the target and the camera increases. Correspondingly, the distance influence weight of the target identified by the spherical camera increases, then decreases, as the distance between the target and the camera increases.

[0065] To reflect the characteristic of the distance influence weight of targets detected by a spherical camera, which increases first and then decreases with increasing distance, at least three weighting stages are required, correspondingly requiring at least two distance values ​​to segment the overall distance. For ease of illustration, this embodiment uses the aforementioned first distance value and additionally sets a second distance value, preferably 200 meters. It should be understood that the specific values ​​of the first or second distance values ​​do not necessarily correspond to the locations where the trend of the distance influence weight of targets detected by the spherical camera changes; this embodiment is provided for illustrative purposes only.

[0066] Regarding the weight distribution of environmental impacts of radar, multi-camera and spherical camera:

[0067] In this embodiment, the environmental impact factors affecting the three sensor devices include rainfall, wind speed and visibility, and the corresponding environmental impact weight is set to the cumulative product of the rainfall impact weight, the wind speed impact weight and the visibility impact weight.

[0068] Regarding the rainfall impact weight, radar uses electromagnetic waves for detection. While rain doesn't shorten radar's effective detection range, the presence of rain does affect its detection performance to some extent. Spherical cameras and multi-lens cameras, because they use images for target recognition, are more significantly affected by rain than radar in terms of target recognition accuracy. Spherical cameras are more flexible than multi-lens cameras, so the impact is relatively minor. However, overall, the target recognition accuracy of all three sensor devices decreases with increasing rainfall. Therefore, the rainfall impact weight of targets identified by radar, spherical cameras, and multi-lens cameras decreases with increasing rainfall, and the rainfall impact weights of targets identified by radar, spherical cameras, and multi-lens cameras decrease in that order.

[0069] Regarding the wind speed impact weight, since spherical cameras and multi-lens cameras perform target recognition through images, the wind speed will not affect the target recognition accuracy of the two. The wind speed impact weight of the targets recognized by the spherical camera and the multi-lens camera is the second constant value. In this embodiment, the second constant value is preferably 1. When the radar recognizes targets in the farmland area, the swaying of plants such as seedlings and corn stalks will cause clutter interference to the radar, resulting in radar misjudgment. As the wind speed increases, the clutter interference increases and the radar's target recognition accuracy decreases. Therefore, the wind speed impact weight of the target recognized by the radar decreases as the wind speed increases.

[0070] Regarding the visibility impact weight, the above-mentioned radar uses electromagnetic waves to detect targets, while visible light sensor devices such as spherical cameras and multi-eye cameras use images to detect targets. Therefore, the accuracy of radar target recognition will not be affected by visibility. The visibility impact weight of the target recognized by the radar is a third constant. In this embodiment, the third constant is preferably 1. The accuracy of spherical cameras and multi-eye cameras in target recognition decreases as visibility decreases. Therefore, the visibility impact weight of the targets recognized by the spherical cameras and multi-eye cameras decreases as visibility decreases. However, since the spherical camera has infrared night vision capability, the visibility impact weight of the target recognized by the spherical camera is greater than the visibility impact weight of the target recognized by the multi-eye camera.

[0071] In this way, a distance impact weight table and an environmental impact weight table can be constructed for targets at different distances identified by different sensor devices in different environments, so as to calculate the comprehensive contribution value of the target to be confirmed, and based on the threshold judgment method, the preset contribution threshold is used to screen out valid targets with a comprehensive contribution value greater than the preset contribution value from the targets to be confirmed, and finally the position of the valid target is output to complete the farmland target recognition under multi-source information fusion.

[0072] For ease of explanation, the following assumes that when it rains, blows, or visibility decreases, the rainfall, wind speed, and visibility are all constant values, that is, it is assumed that the environmental influencing factors remain unchanged. Therefore, when environmental influencing factors exist, the rainfall influence weight, wind speed influence weight, and visibility influence weight corresponding to each sensor device are all constant values; at the same time, combined with the values ​​of the above-mentioned first distance value and the second distance value, the distance influence weight table and the environmental influence weight table at this time are given, as shown in Table 1 below, where R represents the rainfall influence weight, W represents the wind speed influence weight, and N represents the visibility influence weight.

[0073] Table 1 Distance impact weight table and environmental impact weight table when environmental impact factors are assumed to remain unchanged

[0074] Device Type Distance influence weight Environmental impact weight radar 200m+:1.0, 15-200m:1.0, <15m:0.5 R=0.8, W=0.5, N=1.0 Spherical camera 200m+:0.6, 15-200m:0.8, <15m:0.3 R=0.6, W=1.0, N=0.8 Multi-camera 200m+:0.1, 15-200m:0.4, <15m:1.0 R=0.5, W=1.0, N=0.3

[0075] It should also be noted that Table 1 above only shows the possible values ​​of the rainfall influence weight, wind speed influence weight, and visibility influence weight corresponding to the three sensor devices when it is raining, windy, or visibility is reduced. When it is not raining, windy, or visibility is not reduced, it is easy to understand that the rainfall influence weight, wind speed influence weight, and visibility influence weight corresponding to each device are all 1.

[0076] Then, use the comprehensive contribution value to confirm whether the target to be confirmed is a valid target, as shown below:

[0077] Scenario 1: During a clear day, the target is 150 meters away (medium distance). There is no rain or wind, and visibility is not reduced during the day. By looking up the distance impact weight table and the environmental impact weight table in Table 1, the effective weights corresponding to the target recognition of each device can be obtained as shown in Table 2.

[0078] Table 2 Effective weights corresponding to targets identified by each device in scenario 1

[0079] Device Type Distance influence weight Environmental impact weight Effective weight radar 1.0 1.0 1.0 × 1.0 = 1.0 Spherical camera 0.8 1.0 0.8 × 1.0 = 0.8 Multi-camera 0.4 1.0 0.5 × 1.0 = 0.4

[0080] The comprehensive contribution value is the sum of the effective weights corresponding to all sensor devices: 1.0 + 0.8 + 0.4 = 2.2;

[0081] In this case, the combined contribution values ​​of the radar and dome cameras are not significantly affected by distance and the surrounding environment, resulting in similar and high effective weights (contributions). The multi-camera, however, is significantly affected by distance, resulting in a moderate effective weight (contribution). However, due to the higher contributions of the radar and dome cameras, the overall combined contribution value is higher. This example assumes a preset contribution threshold of 0.7. This threshold can be determined through experimentation with specific scenarios, and this value is merely a preferred option. It can be seen that in Scenario 1, the combined contribution values ​​of the target (to be determined) identified by the three sensor devices are significantly greater than the preset contribution threshold, indicating that the sensor devices have a high degree of confidence in target identification and that the current target is a valid target.

[0082] Scenario 2: On a rainy and windy night, the target is 10 meters away (close distance). The environmental conditions are rainy, windy, and visibility is reduced at night. In this case, by looking up the distance impact weight table and the environmental impact weight table in Table 1, the effective weights corresponding to the target recognition of each device can be obtained as shown in Table 3 below.

[0083] Table 3 Effective weights corresponding to the targets identified by each device in scenario 2

[0084] Device Type Basic Weight Environmental attenuation coefficient Effective weight radar 0.5 0.8 (R) × 0.5 (W) × 1.0 (N) = 0.4 0.5 × 0.4 = 0.2 Spherical camera 0.3 0.6 (R) × 1.0 (W) × 0.8 (N) = 0.48 0.3 × 0.48 = 0.144 Multi-camera 1.0 0.5 (R) × 1.0 (W) × 0.3 (N) = 0.15 1.0 × 0.15 = 0.15

[0085] Total contribution value: 0.2 + 0.144 + 0.15 = 0.494;

[0086] At this time, the recognition accuracy of the two visible light sensing devices, the multi-view camera and the spherical camera, is significantly affected by rainfall and reduced visibility in the comprehensive contribution value. The spherical camera is also affected by distance, and the radar is affected by distance, rainfall, and strong wind interference. Therefore, the contribution value of each sensor has dropped significantly, and the comprehensive contribution value is lower than the preset contribution threshold, indicating that the sensor equipment's target recognition reliability is insufficient and the current target is invalid.

[0087] Scenario 3: At night, there is no wind, the target is 200 meters away (long distance), and the environment is rainless and windless with reduced visibility. In this case, by looking up the distance impact weight table and the environmental impact weight table in Table 1, the effective weights corresponding to the target recognition of each device can be obtained as shown in Table 4 below.

[0088] Table 4 Effective weights corresponding to the targets identified by each device in scenario 3

[0089] Device Type Basic Weight Environmental attenuation coefficient Effective weight radar 1.0 1.0 (N) = 1.0 1.0 × 1.0 = 1.0 Spherical camera 0.6 0.8 (N) = 0.8 0.6 × 0.8 = 0.48 Multi-camera 0.1 0.3 (N) = 0.3 0.1 × 0.3 = 0.03

[0090] Total contribution value: 1.0 + 0.48 + 0.03 = 1.51;

[0091] At this time, in the comprehensive contribution value, the radar is not significantly affected by the distance and environment and has a higher effective weight. The spherical camera and multi-camera are both affected by the distance and reduced visibility and have lower effective weights. However, because the radar makes a greater contribution, the final comprehensive contribution value is still greater than the preset contribution threshold, indicating that the sensor equipment has sufficient credibility for target recognition and the current target is a valid target.

[0092] After determining that the target to be confirmed is a valid target, the target position identified by each sensor device can be weighted according to the effective weight of the device, and then the final valid target position can be determined based on the weighted results of the target identification by each sensor to complete the target position fusion.

[0093] Specifically, the horizontal coordinates of the targets identified by the three sensor devices are weighted by the effective weights corresponding to each sensor, and then the sum is divided by the combined contribution value to obtain the horizontal coordinate of the effective target required for target position fusion. The vertical coordinate of the effective target is similarly determined and will not be further described. Of course, the target position identified by the sensor device with the largest effective weight that is greater than a preset weight threshold can also be directly selected as the effective target position, such as the target position identified by the radar in scenarios 1 and 3.

[0094] The present invention sets distance influence weights and environmental influence weights according to the characteristics of each sensor itself to characterize its recognition accuracy after being affected by distance and environment, and dynamically adjusts the weight size under different distances and environments. While ensuring the accuracy of the fusion result, the fusion target result can also be determined by weighting the recognition results of each sensor device by the weight, avoiding the reprocessing of the pixel or point cloud data of the target recognized by each sensor during the fusion process. The target fusion is completed on the basis of correcting the recognition results of different sensors with the matching weights of each sensor to obtain the final target position. The recognition accuracy of each device is characterized by the weight to ensure the accuracy of the fusion target position, and the fusion processing speed can be significantly improved, thereby improving the target recognition speed under multi-source information fusion and improving the agricultural protection effect.

[0095] Example 3:

[0096] Corresponding to the method of the above embodiment, Figure 4 This figure shows a block diagram of a device for identifying farmland targets using multi-source information fusion, according to a third embodiment of the present invention. This device is implemented in a computer device, which connects to a target database via a pre-defined application programming interface (API). When the target database is driven to execute tasks, corresponding task logs are generated, which can be collected via the API. For ease of illustration, only the portions relevant to this embodiment of the present invention are shown.

[0097] See also Figure 4 , the target recognition device comprises:

[0098] The weight allocation module 31 is used to determine the targets identified by each sensor in the farmland area and assign a distance impact weight and an environmental impact weight to each target identified by the current sensor;

[0099] The valid target confirmation module 32 is configured to classify targets identified by different sensors and having position deviations less than a preset deviation threshold as the same target to be confirmed; multiply the distance influence weight and the environmental influence weight of a target identified by a certain sensor under the current target to be confirmed as the effective weight of the certain sensor; record the sum of the effective weights of all sensors under the current target to be confirmed as the comprehensive contribution value; and confirm the current target to be confirmed as a valid target when the comprehensive contribution value is greater than a preset contribution threshold;

[0100] The target recognition module 33 is used to determine the position of the effective target and output it to complete the target recognition in the farmland.

[0101] Optionally, the weight distribution module 31 includes:

[0102] A distance influence weight limiting unit, used to limit the sensors to include radar, spherical camera and multi-camera;

[0103] The distance influence weight of the target identified by the radar increases as the distance between the target and the radar increases, and becomes a first constant value after the distance exceeds a first distance value;

[0104] The distance influence weight of the target recognized by the spherical camera first increases and then decreases as the distance between the target and the spherical camera increases;

[0105] The distance influence weight of the target recognized by the multi-camera decreases as the distance between the target and the multi-camera increases.

[0106] Optionally, the weight distribution module 31 further includes:

[0107] an environmental impact weight limiting unit, configured to limit the environmental impact weight to the cumulative product of the rainfall impact weight, the wind speed impact weight, and the visibility impact weight;

[0108] The weight of rainfall impact on the targets identified by the radar, spherical camera and multi-camera decreases as the rainfall increases;

[0109] The wind speed influence weight of the target identified by the spherical camera and the multi-camera is a second constant value, and the wind speed influence weight of the target identified by the radar decreases as the wind speed increases;

[0110] The visibility influence weight of the target identified by the radar is a third constant value, and the visibility influence weight of the target identified by the spherical camera and the multi-eye camera decreases as the visibility decreases.

[0111] Optionally, the environmental impact weight limiting unit is further configured to limit the rainfall impact weights of the targets identified by the radar, the spherical camera, and the multi-camera to decrease in sequence;

[0112] The visibility influence weight of the target recognized by the spherical camera is greater than the visibility influence weight of the target recognized by the multi-camera.

[0113] Optionally, the weight distribution module 31 further includes:

[0114] The value limiting unit is used to limit the first fixed value, the second fixed value and the third fixed value to all be 1.

[0115] Optionally, the value limiting unit is further used to limit the first distance value to be determined according to the distance from the blind spot boundary of the radar to the radar.

[0116] Optionally, the target recognition module 33 includes:

[0117] The effective target position determining unit is used to weight the target position identified by each sensor under the effective target with the effective weight of the sensor to obtain a weighted value, and to sum all the weighted values ​​and divide them by the comprehensive contribution value to obtain the position of the effective target.

[0118] It should be noted that the information interaction, execution process and other contents between the above modules are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0119] Example 4:

[0120] Figure 5 This is a schematic diagram of the structure of a computer device provided in the fourth embodiment of the present invention. Figure 5As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown), a memory, and a computer program stored in the memory and executable on at least one processor. When the processor executes the computer program, the steps in any of the above-mentioned embodiments of the method for identifying targets in farmland by fusion of multi-source information are implemented.

[0121] The computer device may include, but is not limited to, a processor and a memory. It will be understood by those skilled in the art that Figure 5 This is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0122] The processor may be a CPU, other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0123] Memory includes readable storage media, internal memory, and the like. Internal memory can be the internal memory of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage medium. The readable storage medium can be the computer device's hard drive. In other embodiments, it can also be an external storage device, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, or a flash memory card. Furthermore, memory can include both the computer device's internal storage unit and external storage devices. Memory is used to store the operating system, application programs, boot loaders, data, and other programs, such as the program code of computer programs. Memory can also be used to temporarily store data that has been output or is about to be output.

[0124] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the process steps in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. Computer-readable media can include at least: any entity or device capable of carrying computer program code, recording media, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunications signals, and software distribution media. Examples include USB flash drives, removable hard drives, magnetic disks, or optical disks. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunications signals.

[0125] The present invention may implement all or part of the processes in the above-mentioned method embodiments, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0126] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0127] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0128] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely illustrative. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0129] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0130] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for identifying targets in farmland by fusion of multi-source information, characterized in that: The method comprises: Determine the targets identified by each sensor in the farmland area, and assign a distance impact weight and an environmental impact weight to each target identified by the current sensor; Targets identified by different sensors and with position deviations less than a preset deviation threshold are classified as the same target to be confirmed; the product of the distance influence weight and the environmental influence weight of the target identified by a certain sensor under the current target to be confirmed is used as the effective weight of the certain sensor; the sum of the effective weights of all sensors under the current target to be confirmed is recorded as the comprehensive contribution value; when the comprehensive contribution value is greater than the preset contribution threshold, the current target to be confirmed is confirmed as a valid target; Determine the location of the effective target and output it to complete target identification in the farmland; The sensors include radar, spherical camera and multi-camera; The distance influence weight of the target identified by the radar increases as the distance between the target and the radar increases, and becomes a first constant value after the distance exceeds a first distance value; The distance influence weight of the target recognized by the spherical camera first increases and then decreases as the distance between the target and the spherical camera increases; The distance influence weight of the target identified by the multi-camera decreases as the distance between the target and the multi-camera increases; The environmental impact weight is the cumulative product of the rainfall impact weight, wind speed impact weight and visibility impact weight; The rainfall impact weight of the target identified by the radar, spherical camera and multi-camera decreases as the rainfall increases; The wind speed influence weight of the target identified by the spherical camera and the multi-camera is a second constant value, and the wind speed influence weight of the target identified by the radar decreases as the wind speed increases; The visibility influence weight of the target identified by the radar is a third fixed value, and the visibility influence weight of the target identified by the spherical camera and the multi-lens camera decreases as the visibility decreases; The first distance value is determined according to the distance from the blind zone boundary of the radar to the radar; Determining the position of the valid target includes: The target position identified by each sensor under the valid target is weighted by the effective weight of the sensor to obtain a weighted value, and the sum of all the weighted values ​​is divided by the comprehensive contribution value to obtain the position of the valid target.

2. The method for identifying targets in farmland using multi-source information fusion according to claim 1, characterized in that: The weights of the rainfall impact of the targets identified by the radar, spherical camera and multi-camera decrease in sequence; The visibility influence weight of the target recognized by the spherical camera is greater than the visibility influence weight of the target recognized by the multi-camera.

3. The method for identifying targets in farmland using multi-source information fusion according to claim 2, characterized in that: The first constant value, the second constant value, and the third constant value are all 1.

4. A device for identifying targets in farmland using multi-source information fusion, characterized in that: A method for identifying targets in farmland based on multi-source information fusion according to any one of claims 1 to 3, wherein the device comprises: The weight allocation module is used to determine the targets identified by each sensor in the farmland area and assign a distance impact weight and an environmental impact weight to each target identified by the current sensor; The valid target confirmation module is used to classify targets identified by different sensors and whose position deviations are less than a preset deviation threshold as the same target to be confirmed; the product of the distance influence weight and the environmental influence weight of the target identified by a certain sensor under the current target to be confirmed is used as the effective weight of the certain sensor; the sum of the effective weights of all sensors under the current target to be confirmed is recorded as the comprehensive contribution value; and the current target to be confirmed is confirmed as a valid target when the comprehensive contribution value is greater than the preset contribution threshold; The target recognition module is used to determine the position of the effective target and output it to complete the target recognition in the farmland.

5. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for identifying targets in farmland using multi-source information fusion as described in any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for identifying targets in farmland by fusion of multi-source information as described in any one of claims 1 to 3 is implemented.

Images