Multi-target moving object monitoring method and device based on multi-sonar

Abstract

The invention relates to the field of sonar image analysis, in particular to a multi-target moving object quantity monitoring method and device based on multiple sonars, sonar equipment comprises a first sonar and a second sonar, and a first sonar image corresponding to the first sonar and a second sonar image corresponding to the second sonar coincide with each other in a fixed detection area. According to the invention, a plurality of sonar devices are arranged in the breeding area to ensure that the detection range covers a large area and efficient monitoring is realized. The method is characterized in that a multi-sonar technology is fused with a target identification technology, a coordinate transformation algorithm and a trajectory analysis means, and the problems that a single sonar is limited in monitoring range and insufficient in precision are successfully solved. According to the method, the quantity and the activity state of the living beings in the breeding area can be accurately tracked and recorded in real time, and detailed data support is provided for breeding management.

Description

Technical Field

[0001] The present application relates to the field of sonar image analysis, and in particular to a multi-sonar-based multi-target moving object monitoring method and a multi-target moving object monitoring device. Background Art

[0002] As an advanced technology for detecting underwater objects through sound waves, sonar (SONAR) has played an important role in modern ocean exploration and ecological research. Sonar technology uses the propagation characteristics of sound waves in water to calculate the distance, direction and depth of underwater objects by measuring the time difference between the emission and reception of sound waves. This technology not only helps humans better understand the marine environment, but also brings endless application potential to the fields of science and industry.

[0003] At present, some studies and equipment have used sonar to monitor underwater targets. However, these studies and equipment are usually limited to the use of a single sonar device, which leads to limitations in the data collected. When detecting underwater targets, a single sonar device can often only monitor one or a few targets, and it is difficult to monitor multiple targets at the same time. Summary of the invention

[0004] In view of this, the purpose of this application is to provide a multi-sonar-based multi-target moving object monitoring method and device to solve the above-mentioned problems.

[0005] In order to solve the above technical problems, this application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a multi-target moving object monitoring method based on multi-sonar, the sonar device includes a first sonar and a second sonar, a first sonar image corresponding to the first sonar and a second sonar image corresponding to the second sonar overlap with each other in a fixed detection area, and the multi-target moving object monitoring method includes: step 1: acquiring a first sonar image, a second sonar image and fixed detection area deployment information, the fixed detection area deployment information including first sonar position information and second sonar position information; step 2: based on the first sonar image and the second sonar image, acquiring first target relative position information corresponding to the first sonar image and second target relative position information corresponding to the second sonar image; step 3: based on the first sonar position information and the second sonar position information, performing position conversion processing on the first target relative position information and the second target relative position information to obtain first target absolute position information and second target absolute position information; step 4: performing deduplication tracking processing on the first target absolute position information and the second target absolute position information to obtain biomass information of the multi-target moving objects.

[0007] Further, step 2 includes: step 2-0: based on a neural network model and / or an image processing algorithm, the first sonar image and the second sonar image are recognized and processed to determine multiple target moving objects in the image.

[0008] Furthermore, the fixed detection area deployment information includes: establishing a coordinate system corresponding to the fixed detection area with any position of the fixed detection area as the center, and determining the first sonar position information and the second sonar position information.

[0009] Further, step 2 specifically includes: step 2-1: according to the first sonar image and the second sonar image, obtaining the first sonar polar coordinate position information of the first multi-target moving object corresponding to the first sonar image and the second sonar polar coordinate position information of the second multi-target moving object corresponding to the second sonar image; step 2-2: performing three-dimensional coordinate conversion on the first sonar polar coordinate position information and the second sonar polar coordinate position information to determine the relative position information of the first target corresponding to the first sonar image and the relative position information of the second target corresponding to the second sonar image.

[0010] Further, step 3 specifically includes: step 3-1: determining rotation matrix information according to first sonar orientation information of the first sonar and second sonar orientation information of the second sonar, wherein the rotation matrix information includes a first rotation matrix corresponding to the first sonar image and a second rotation matrix corresponding to the second sonar image; step 3-2: performing position conversion processing on the first target relative position information and the second target relative position information according to the rotation matrix information, the first sonar position information and the second sonar position information to obtain the first target absolute position information and the second target absolute position information.

[0011] Further, step 3-2 specifically includes: step A: performing a rotation transformation on the first target relative position information and the second target relative position information according to the rotation matrix information to obtain the rotation position information; step B: performing a translation transformation on the rotation position information according to the first sonar position information and the second sonar position information to obtain the first target absolute position information and the second target absolute position information.

[0012] Furthermore, step 4 specifically includes: step 4-1: based on the target fusion algorithm, the absolute position information of the first target and the absolute position information of the second target are subjected to target fusion to screen out repeated multi-target moving objects; step 4-2: based on the multi-target tracking algorithm, the multi-target moving objects after target fusion are tracked to obtain the biomass information of the multi-target moving objects.

[0013] Furthermore, the biomass information includes trajectory information of multiple target moving objects, and the multi-target moving object monitoring method also includes: Step 5: Determine the activity frequency information and activity information of the multiple target moving objects based on the trajectory information of the multiple target moving objects; Step 6: Obtain the health status information and feeding requirement information of the multiple target moving objects based on the activity frequency information and activity information.

[0014] In a second aspect, the present application provides a target biomass monitoring device based on multiple sonars, the sonar device comprising a first sonar and a second sonar, the first sonar image corresponding to the first sonar and the second sonar image corresponding to the second sonar overlap each other in a fixed detection area, and the multi-target moving object monitoring device comprises: a first acquisition module, used to acquire a first sonar image, a second sonar image and fixed detection area deployment information, the fixed detection area deployment information comprising first sonar position information and second sonar position information; a second acquisition module, used to acquire, based on the first sonar image and the second sonar image, first target relative position information corresponding to the first sonar image and second target relative position information corresponding to the second sonar image; a conversion module, used to perform position conversion processing on the first target relative position information and the second target relative position information based on the first sonar position information and the second sonar position information, to obtain first target absolute position information and second target absolute position information; a processing module, used to perform deduplication tracking processing on the first target absolute position information and the second target absolute position information, to obtain biomass information of multiple target moving objects.

[0015] In a third aspect, the present application provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the multi-target mobile object monitoring method of the first aspect.

[0016] It can be seen from the above technical solution that the advantages and positive effects of the multi-sonar-based target biomass monitoring method and device proposed in this application are:

[0017] First, this application deploys multiple sonar devices in a fixed breeding area to ensure that the detection range of the sonar device covers a large area and achieves efficient monitoring of multiple target moving objects in the fixed detection area. Its innovation lies in the integration of multi-sonar technology with target recognition technology, coordinate transformation algorithm and trajectory analysis, successfully overcoming the problem of limited monitoring range and insufficient accuracy of a single sonar, and through this method, it can track and record the number and activity status of organisms in a fixed breeding area in real time and accurately, providing detailed data support for breeding management.

[0018] Secondly, this application also introduces an intelligent deduplication algorithm for multi-perspective intersection areas, which effectively improves the accuracy of the statistics on the number of mobile target organisms. Through in-depth analysis of the statistical results, the activity of the target organisms is accurately evaluated, providing a scientific basis for the formulation of intelligent feeding strategies. This innovative solution not only successfully obtains the biomass information of multiple target mobile objects in a fixed area, but also realizes the accurate monitoring and activity analysis of the target biomass, thereby improving the breeding efficiency of the target organisms.

[0019] In addition, the target fusion algorithm provided in the present application can be combined with the behavioral habits of the target moving object (for example, different fish like to be in the upper layer, middle layer, lower layer, etc.) to further improve the accuracy of the biological population estimation results of underwater organisms such as fish, and solve the problem of large difference between biomass quantity detection and actual quantity in the current existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above content of the present application and the following specific implementation methods will be better understood when read in conjunction with the accompanying drawings. It should be noted that the accompanying drawings are only examples of the technical solutions claimed for protection.

[0021] Figure 1 It is a schematic diagram of the sonar equipment of the present application being deployed in a breeding area;

[0022] Figure 2 is a flow chart of the method for multi-sonar-based target biomass monitoring of the present application;

[0023] Figure 3 is a schematic diagram of a sonar image of the present application;

[0024] Figure 4 is a schematic diagram of the sonar image recognition of fish schools in the present application;

[0025] Figure 5 It is a schematic diagram of the statistical results of fish school identification in this application.

[0026] The reference numerals are described as follows:

[0027] A first sonar device 10;

[0028] A first detection area 11;

[0029] A second sonar device 20;

[0030] A second detection area 21;

[0031] A third sonar device 30;

[0032] The third detection area 31;

[0033] a fourth sonar device 40;

[0034] Fourth detection area 41;

[0035] First detection overlap area 51;

[0036] The second detection overlap area 52;

[0037] The third detection overlap area 53;

[0038] Farming area 60. DETAILED DESCRIPTION

[0039] The detailed features and advantages of the present application are described in detail in the specific implementation manner below, and the content is sufficient to enable any technical personnel in this field to understand the technical content of the present application and implement it accordingly. According to the description, claims and drawings disclosed in this specification, the technical personnel in this field can easily understand the relevant purposes and advantages of the present application.

[0040] It should be noted that in this specification, similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0041] In the description of this embodiment, it should be noted that the terms "upper", "lower", "inner", "bottom", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the product is usually placed when in use. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on this application.

[0042] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0043] like Figure 1 As shown, the present application is applied to a fixed breeding area 60, which can be a seawater area or a freshwater area. Preferably, the breeding area 60 is an area where the breeding range can be determined. In the embodiment of the present application, the breeding area 60 is an underwater breeding box of a breeding ship, and a first sonar device 10, a second sonar device 20, a third sonar device 30 and a fourth sonar device 40 are arranged at multiple fixed positions in the underwater breeding box to cover the breeding area 60 as much as possible.

[0044] Among them, the first sonar device 10 corresponds to the first detection area 11, the second sonar device 20 corresponds to the second detection area 21, the third sonar device 30 corresponds to the third detection area 31, and the fourth sonar device 40 corresponds to the fourth detection area 41. Each detection area corresponds to the sonar image of its own sonar device, and the sonar devices transmit and receive sonar signals to obtain multiple moving objects in the sonar image.

[0045] There is a first detection overlap area 51 between the first detection area 11 and the second detection area 21, a second detection overlap area 52 between the second detection area 21 and the third detection area 31, and a third detection overlap area 53 between the third detection area 31 and the fourth detection area 41. In the perspectives of different sonar devices, the same moving object will be represented as different individuals in the detection overlap area, resulting in poor overall monitoring effect on the moving target. In addition, since a single moving object generates multiple data, it is easy to cause the processing system to process too much data, requiring the processing system to have better performance.

[0046] The deployment of multiple sonar devices can cover the monitoring area as much as possible, but multiple sonar devices will produce some overlapping detection areas, which will greatly affect the detection results of the target biomass. Therefore, this application combines multi-sonar technology with deduplication tracking technology to improve the monitoring effect of the target biomass.

[0047] Please refer to Figure 2 The present application provides a multi-target mobile object monitoring method based on multi-sonar, and the steps of the method are as follows:

[0048] Step 1: Acquire a first sonar image, a second sonar image, and fixed detection area deployment information, where the fixed detection area deployment information includes first sonar position information and second sonar position information.

[0049] Step 2: Based on the first sonar image and the second sonar image, obtain the relative position information of the first target corresponding to the first sonar image and the relative position information of the second target corresponding to the second sonar image.

[0050] Step 3: Based on the first sonar position information and the second sonar position information, position conversion processing is performed on the first target relative position information and the second target relative position information to obtain the first target absolute position information and the second target absolute position information.

[0051] Step 4: De-duplication tracking is performed on the absolute position information of the first target and the absolute position information of the second target to obtain biomass information of the multi-target moving objects.

[0052] In step 2, specific steps may include:

[0053] Step 2-0: Based on a neural network model and / or an image processing algorithm, the first sonar image and the second sonar image are recognized and processed to determine multiple moving objects in the image.

[0054] Specifically, the present application provides a neural network model based on the YOLO algorithm to perform target detection on sonar images to determine the target moving object, and its steps mainly include: 1. data set annotation; 2. network training; 3. neural network prediction of fish position; 4. target screening; 5. counting the number of fish.

[0055] For example, please refer to Figures 3 to 5 , by Figure 3 The data information in is processed by the YOLO algorithm, and the following is obtained: Figure 4 The multiple framed objects shown in the figure are each fish. After data statistical processing, the following is obtained: Figure 5 The fish population count results are shown.

[0056] Understandably, the advantage of this method is that it can maintain a high degree of accuracy and accurately estimate the distribution of fish schools even when the fish schools are dense and the data is huge. It has a powerful ability to accurately lock the specific location of each fish in the data image. In addition, when used with a GPU (graphics processing unit), the method has extremely fast calculation speed, which can greatly improve processing efficiency.

[0057] The present application can also perform target detection on sonar images based on counting networks to determine the target moving object, and the specific steps include: 1. Data set annotation; 2. Network training; 3. Counting network to estimate the number of fish schools; 4. Counting the number of fish schools; 5. Data export and analysis.

[0058] It can be understood that this method has shown relatively excellent performance in solving the occlusion problem of fish schools, and can more accurately identify and count partially occluded fish.

[0059] The present application can also perform target detection on the sonar image based on traditional image processing methods to determine the target moving object. The specific steps of this method are well known to those skilled in the art.

[0060] It can be understood that this method operates quickly, especially when the target density is low, and only requires the CPU to participate in the calculation without relying on the GPU, thereby reducing hardware costs and improving the popularity and convenience of calculations.

[0061] It should be noted that the target moving object in the present application can also be farmed organisms such as shrimps and crabs, and is not limited to fish.

[0062] Step 2-1: According to the first sonar image and the second sonar image, obtain the first sonar polar coordinate position information of the first multi-target moving object corresponding to the first sonar image and the second sonar polar coordinate position information of the second multi-target moving object corresponding to the second sonar image.

[0063] Step 2-2: Perform three-dimensional coordinate conversion on the first sonar polar coordinate position information and the second sonar polar coordinate position information to determine the first target relative position information corresponding to the first sonar image and the second target relative position information corresponding to the second sonar image.

[0064] This application takes the first sonar device 10 as an example to illustrate the acquisition of the absolute position information of the first target. The steps for acquiring the absolute position information of the first target and the absolute position information of the second target are the same.

[0065] The fixed detection area deployment information specifically includes: taking the geometric center position of the fixed breeding area 60 as the coordinate origin, establishing a three-dimensional rectangular coordinate system and a polar coordinate system, and the first sonar position information includes the three-dimensional coordinate position of the first sonar device 10 as P0 (x0, y0, z0), and the angle between the first sonar device 10 and the coordinate origin as θ0 (α0, β0, γ0).

[0066] The first sonar device 10 captures the underwater target by emitting sound waves, and preliminarily determines the position coordinate information of the target moving object in the sonar image.

[0067] The first sonar polar coordinate position information includes the polar coordinate position A0 (ρ0, θ0) of the mobile object centered at the first sonar device 10. The position of the first sonar device 10 is used as the three-dimensional coordinate origin. In the local rectangular coordinate system of the first sonar device 10, the polar coordinates of the mobile object are converted into three-dimensional coordinates to determine the first target relative position information of the mobile object, which is expressed as:

[0068] (x A ,y A ,z A )=(ρ0cosθ0,ρ0sinθ0,0)

[0069] It can be understood that, given that the scanning characteristics of the sonar equipment are mainly limited to the plane area, in the local rectangular coordinate system defined thereby, the vertical position (ie, the vertical coordinate) of the moving object is regarded as coinciding with the scanning plane, and therefore its value is set to 0.

[0070] In step 3, specific steps may include:

[0071] Step 3-1: Determine rotation matrix information according to first sonar orientation information of the first sonar and second sonar orientation information of the second sonar, wherein the rotation matrix information includes a first rotation matrix corresponding to the first sonar image and a second rotation matrix corresponding to the second sonar image.

[0072] Step 3-2: Perform position conversion processing on the first target relative position information and the second target relative position information according to the rotation matrix information, the first sonar position information and the second sonar position information to obtain the first target absolute position information and the second target absolute position information.

[0073] The specific steps of step 3-2 may include: step A: performing a rotation transformation on the first target relative position information and the second target relative position information according to the rotation matrix information to obtain the rotation position information. step B: performing a translation transformation on the rotation position information according to the first sonar position information and the second sonar position information to obtain the first target absolute position information and the second target absolute position information.

[0074] Specifically, the relative position information of the first target is the coordinates of the local rectangular coordinate system of the first sonar device 10, and the first target position information can be transformed by introducing a rotation matrix.

[0075] The angle between the first sonar device 10 and the coordinate origin is θ0 (α0, β0, γ0), which can be expressed as the orientation information of the first sonar device 10. The orientation angle of the first sonar device 10 relative to the coordinate origin of the fixed detection area is (α0, β0, γ0), which respectively represents the rotation angle around the x, y, and z axes. Therefore, the rotation matrix corresponding to the moving object detected by the first sonar device 10 is:

[0076] R=R z (γ0)·R y (β0)·R x (α0)

[0077] Among them, the rotation matrix of each axis is:

[0078]

[0079] The position coordinates of the target moving object in the coordinate system of the breeding area 60 are rotated and transformed to obtain the rotation position information of the relative position of the first target:

[0080]

[0081] Then, according to the previously obtained coordinate system position P0 (x0, y0, z0) of the first sonar device 10 in the breeding area 60, the rotation position information is translated to obtain the absolute position information of the first target:

[0082] (x,y,z)=(x′+x0,y′+y0,z′+z0)

[0083] Specifically, the first target absolute position information of the target moving object in the three-dimensional rectangular coordinate system of the fixed area is:

[0084]

[0085] After multiple sonar devices obtain the target absolute position information of the moving object, the minimum distance method can be used to find the best fitting point in the three-dimensional space to perform subsequent moving object activity analysis, moving object size calculation, and improve the recognition accuracy of the moving object.

[0086] Step 4 specifically includes: Step 4-1: Based on the target fusion algorithm, target fusion is performed on the absolute position information of the first target and the absolute position information of the second target to filter out repeated multi-target moving objects. Step 4-2: Based on the multi-target tracking algorithm, the multi-target moving objects after target fusion are tracked to obtain the biomass information of the multi-target moving objects.

[0087] It is understandable that the target moving objects displayed in the sonar images at different timestamps will change. For the cross-detection areas of multiple sonar devices, a target fusion algorithm, also known as a deduplication algorithm, can be used. The deduplication algorithm eliminates duplicate targets in the overlapping detection areas to ensure the accuracy of the target moving object counting.

[0088] The target fusion algorithm can combine the behavioral habits of the target moving object (for example, different fish like to stay in the upper layer, middle layer, lower layer, etc.) to further improve the accuracy of the biological population estimation results of underwater organisms such as fish, and solve the problem of large discrepancy between biomass quantity detection and actual quantity in the current existing technology.

[0089] Specifically, the target fusion algorithm may be a Bayesian filtering algorithm, which utilizes a probabilistic method to fuse data from different sonars to eliminate repeated counts of moving target objects.

[0090] The Bayesian filtering algorithm uses Bayes' theorem to continuously update the posterior distribution of the target state given the current observation.

[0091] The Bayes theorem formula is as follows:

[0092]

[0093] Among them, x t : target state (e.g. fish position, speed), z t : Observation data at the current moment (e.g., absolute position information of the target observed by sonar); p(x t |z 1:t ): posterior distribution of target state after given observation data; p(z t |x t ): Observation model, that is, the target is in state x tWhen the observation z is generated t The probability of p(x t |z 1:t-1 ): Predictive distribution, predicting the current state based on the state at the previous moment and the system model.

[0094] Bayesian filtering deduplication includes the following steps:

[0095] 1. Initialization:

[0096] Set the initial state distribution p(x0), which is usually assumed to be a uniform distribution or a Gaussian distribution.

[0097] 2. Prediction stage:

[0098] Using the system's motion model p(x t |x t-1 ), according to the state p(x t-1 |z 1:t-1 ), predict the prior distribution at the current moment: p(x t |z 1:t-1 )=∫p(x t |x t-1 )·p(x t-1 |z 1:t-1 )dx t-1

[0099] 3. Update phase:

[0100] Combined with the current observation z t And the predicted distribution, calculate the posterior distribution through Bayes theorem:

[0101]

[0102] Among them, p(z t )=∫p(z t |x t )·p(x t |z 1:t-1 )dx t is the normalization factor.

[0103] 4. Loop iteration:

[0104] The prediction and update steps are repeated to refine the estimate of the target state over time.

[0105] Step 3-2: Based on the multi-target tracking algorithm, the target moving object after deduplication processing is tracked to obtain the biomass information of the target moving object.

[0106] Specifically, through multi-target trajectory tracking, the motion trajectory of each target is analyzed in real time to ensure the effectiveness of biological activity analysis. The multi-target tracking algorithm can also be a Bayesian filtering algorithm, which achieves fish target deduplication and tracking in the following ways:

[0107] 1. Status definition:

[0108] State variable x t =[x,y,v x ,v y ], including the fish's position and speed.

[0109] 2. Observation model:

[0110] Sonar sensors provide target location observation data t = [x, y], establish p(z t |x t ).

[0111] 3. Prediction stage:

[0112] According to the motion model of the fish (such as uniform motion or acceleration motion model), the position of the fish is predicted.

[0113] 4. Update phase:

[0114] According to the observation data of different sonar sensors t Update the target state and fuse the information from different sensors.

[0115] 5. Multi-target tracking:

[0116] The Bayesian filter is extended to the multi-target environment, and the observation data of multiple fish are processed by the joint probabilistic data association (JPDA) or multi-hypothesis tracking (MHT) algorithm to obtain the trajectory information of multiple moving objects.

[0117] In the embodiment of the present application, the biomass information includes the trajectory information of the multi-target moving objects obtained after the Bayesian algorithm is processed, and the multi-target moving object monitoring method may further include:

[0118] Step 5: Determine the activity frequency information and activity degree information of the multi-target moving objects according to the trajectory information of the multi-target moving objects.

[0119] Step 6: Based on the activity frequency information and activity level information, obtain the health status information and feeding demand information of the multiple target mobile objects.

[0120] Specifically, based on multi-target trajectory information, the activity frequency and activity of the target moving object are calculated, and its health status and feeding needs are judged by analyzing the movement trend of the target moving object. In combination with the pre-trained breeding model, intelligent feeding suggestions are provided.

[0121] It can be understood that the breeding model can be loaded and deployed from the cloud, or it can be directly built and implemented locally. The specific construction method of the breeding model is well known to those skilled in the art and will not be repeated here.

[0122] In addition, the present application can also periodically select a single target moving object, apply the size measurement function, and use advanced image processing algorithms to accurately calculate the length, width and other key size information of the organism, and estimate its weight accordingly. By integrating the data of all individual organisms, the dynamic changes of the overall biomass in the breeding area 60 can be tracked and monitored in real time.

[0123] The present invention innovatively combines multiple sonar devices with advanced target recognition technology, coordinate transformation algorithms and trajectory analysis methods, effectively overcoming the problems of limited monitoring range and insufficient accuracy of a single sonar.

[0124] At the same time, the method can monitor the number of organisms and the state of biological activity in the breeding area 60 in real time and accurately. By adopting the intelligent deduplication algorithm of multi-view cross-region, the problem of real-time monitoring of the number, size, weight and activity of organisms in aquaculture is further solved, and the accuracy of biological quantity statistics is improved.

[0125] Moreover, based on these detailed statistical results, the system can deeply analyze the biological activity, thus providing a scientific basis for the formulation of intelligent feeding strategies.

[0126] Based on the same inventive concept above, the present application also provides a target moving object quantity monitoring device based on multi-sonar, and the target moving object quantity monitoring device includes: a first acquisition module, a second acquisition module, a conversion module and a processing module.

[0127] The first acquisition module is used to acquire the first sonar image, the second sonar image and the fixed detection area deployment information, and the fixed detection area deployment information includes the first sonar position information and the second sonar position information.

[0128] The second acquisition module is used to acquire first target relative position information corresponding to the first sonar image and second target relative position information corresponding to the second sonar image based on the first sonar image and the second sonar image.

[0129] The conversion module is used to perform position conversion processing on the first target relative position information and the second target relative position information based on the first sonar position information and the second sonar position information to obtain the first target absolute position information and the second target absolute position information.

[0130] The processing module is used to perform deduplication tracking processing on the absolute position information of the first target and the absolute position information of the second target to obtain the biomass information of the multi-target moving objects.

[0131] It should be noted that the systems, devices or modules described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0132] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0133] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0134] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0135] The terms and expressions used herein are for descriptive purposes only and the present application should not be limited to these terms and expressions. The use of these terms and expressions does not mean to exclude any equivalent features of the illustrations and descriptions (or parts thereof), and it should be recognized that various modifications that may exist should also be included in the scope of the claims. Other modifications, variations and substitutions may also exist. Accordingly, the claims should be deemed to cover all such equivalents.

[0136] Similarly, it should be pointed out that although the present application has been described with reference to the current specific embodiments, ordinary technicians in this technical field should realize that the above embodiments are only used to illustrate the present application, and various equivalent changes or substitutions may be made without departing from the spirit of the invention. Therefore, as long as the changes and modifications to the above embodiments are within the essential spirit of the present application, they will fall within the scope of the claims of the present application.

Claims

1. A multi-target moving object monitoring method based on multi-sonar, characterized in that: The sonar device includes a first sonar and a second sonar, a first sonar image corresponding to the first sonar and a second sonar image corresponding to the second sonar overlap each other in a fixed detection area, and the multi-target moving object monitoring method includes: Step 1: Acquire the first sonar image, the second sonar image, and the fixed detection area deployment information, wherein the fixed detection area deployment information includes first sonar position information and second sonar position information; Step 2: Based on the first sonar image and the second sonar image, obtaining relative position information of a first target corresponding to the first sonar image and relative position information of a second target corresponding to the second sonar image; Step 3: Based on the first sonar position information and the second sonar position information, position conversion processing is performed on the first target relative position information and the second target relative position information to obtain the first target absolute position information and the second target absolute position information; Step 4: Deduplication tracking is performed on the first target absolute position information and the second target absolute position information to obtain biomass information of the multiple target moving objects.

2. The multi-target moving object monitoring method according to claim 1, characterized in that: The step 2 comprises: Step 2-0: Based on a neural network model and / or an image processing algorithm, the first sonar image and the second sonar image are recognized and processed to determine the multiple moving objects in the image.

3. The multi-target moving object monitoring method according to claim 1, characterized in that: The fixed detection area deployment information includes: Taking any position of the fixed detection area as the center, a coordinate system corresponding to the fixed detection area is established to determine the first sonar position information and the second sonar position information.

4. The multi-target moving object monitoring method according to claim 1, characterized in that: The step 2 specifically includes: Step 2-1: acquiring first sonar polar coordinate position information of a first multi-target moving object corresponding to the first sonar image and second sonar polar coordinate position information of a second multi-target moving object corresponding to the second sonar image according to the first sonar image and the second sonar polar coordinate position information; Step 2-2: performing three-dimensional coordinate conversion on the first sonar polar coordinate position information and the second sonar polar coordinate position information to determine the first target relative position information corresponding to the first sonar image and the second target relative position information corresponding to the second sonar image.

5. The multi-target moving object monitoring method according to claim 1, characterized in that: The step 3 specifically includes: Step 3-1: determining rotation matrix information according to first sonar orientation information of the first sonar and second sonar orientation information of the second sonar, wherein the rotation matrix information includes a first rotation matrix corresponding to the first sonar image and a second rotation matrix corresponding to the second sonar image; Step 3-2: performing position conversion processing on the first target relative position information and the second target relative position information according to the rotation matrix information, the first sonar position information and the second sonar position information to obtain the first target absolute position information and the second target absolute position information.

6. The multi-target moving object monitoring method according to claim 5, characterized in that: The step 3-2 specifically includes: Step A: performing a rotation transformation on the first target relative position information and the second target relative position information according to the rotation matrix information to obtain rotation position information; Step B: performing a translation transformation on the rotation position information according to the first sonar position information and the second sonar position information to obtain the first target absolute position information and the second target absolute position information.

7. The multi-target moving object monitoring method according to claim 1, characterized in that: The step 4 specifically includes: Step 4-1: Based on a target fusion algorithm, target fusion is performed on the first target absolute position information and the second target absolute position information to filter out repeated multi-target moving objects; Step 4-2: Based on a multi-target tracking algorithm, the multi-target moving objects after target fusion are tracked to obtain biomass information of the multi-target moving objects.

8. The multi-target moving object monitoring method according to claim 1, characterized in that: The biomass information includes the trajectory information of the multi-target moving objects, and the multi-target moving object monitoring method further includes: Step 5: Determine the activity frequency information and activity degree information of the multi-target moving objects according to the trajectory information of the multi-target moving objects; Step 6: Obtain health status information and feeding requirement information of the multi-target mobile objects based on the activity frequency information and the activity degree information.

9. A multi-target moving object monitoring device based on multi-sonar, characterized in that: The sonar device includes a first sonar and a second sonar, a first sonar image corresponding to the first sonar and a second sonar image corresponding to the second sonar overlap each other in a fixed detection area, and the multi-target mobile object monitoring device includes: A first acquisition module is used to acquire the first sonar image, the second sonar image and the fixed detection area deployment information, wherein the fixed detection area deployment information includes first sonar position information and second sonar position information; a second acquisition module, configured to acquire, based on the first sonar image and the second sonar image, relative position information of a first target corresponding to the first sonar image and relative position information of a second target corresponding to the second sonar image; a conversion module, configured to perform position conversion processing on the first target relative position information and the second target relative position information based on the first sonar position information and the second sonar position information, so as to obtain first target absolute position information and second target absolute position information; The processing module is used to perform deduplication tracking processing on the first target absolute position information and the second target absolute position information to obtain the biomass information of the multi-target moving objects.

10. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the multi-target moving object monitoring method according to claim 1 are implemented.

Images