Multi-target identification method and system based on visual technology

Abstract

The invention relates to the technical field of image recognition, in particular to a multi-target recognition method and system based on a visual technology, and the method comprises the steps: obtaining the close degree of a sheep flock in a connected domain according to the shape of the connected domain and a suspected sheep flock distribution value; obtaining a false detection degree value of each target frame in the connected domain according to the edge distribution condition of the connected domain in the target frame and other target frames, and the inaccuracy and the target probability of only sheep segmentation; according to the close degree of the sheep flock in the connected domain and the false detection degree value of the target frame, obtaining the comprehensive sheep identification accuracy of the connected domain; obtaining an optimal step parameter of a convolutional neural network for performing multi-target recognition on the sheep flock in the pasture based on the comprehensive sheep recognition accuracy; and performing multi-target sheep identification on the remote sensing image of the sheep flock in the pasture based on the convolutional neural network with the optimal step length. According to the method, the accuracy of identifying the multi-target sheep in the sheep flock remote sensing image is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to a multi-target recognition method and system based on vision technology. Background Art

[0002] With the continuous progress of computer vision technology, object detection technology based on deep learning has been widely applied in multiple fields, such as video surveillance, autonomous driving, intelligent agriculture, etc. Especially in automated ranch management, accurately identifying and classifying each sheep in a flock is of great significance. For example, in Inner Mongolia, there are herdsmen using drones for grazing and sharing this process on a live streaming platform with netizens. This new grazing method not only improves efficiency but also attracts the interest of netizens; however, in the actual live streaming application process, since the sheep in the flock are usually close to each other and of different sizes, it is easy to cause inaccurate object recognition, which may result in the loss of individual sheep, thus bringing economic losses to the herdsmen. In a dense grazing scenario, the flock often presents the following characteristics: the individual space distribution of sheep is dense, and the overlap rate of sheep targets is high; the scale of sheep changes significantly due to age and posture differences; the pixel ratio of small sheep targets is low when photographed from a long distance; therefore, when performing multi-target recognition on the flock, it may cause multiple sheep to be misrecognized as one sheep, thereby affecting the accuracy and reliability of the recognition result. Summary of the Invention

[0003] The present invention provides a multi-target recognition method and system based on vision technology to solve the existing problem that when performing multi-target recognition on a flock of sheep, multiple sheep may be misrecognized as one sheep due to the dense individual space distribution of sheep and the high overlap rate of sheep targets.

[0004] The multi-target recognition method and system based on vision technology of the present invention adopt the following technical solutions: The present invention proposes a multi-target recognition method based on vision technology, which includes the following steps: Obtain the remote sensing image of the flock in the ranch; Obtain several connected regions in the remote sensing image of the flock in the ranch; according to the internal distribution of the connected regions, obtain the suspected flock distribution value of each connected region; according to the shape of the connected region and the suspected flock distribution value, obtain the closeness degree of the flock within the connected region; Perform multi-target recognition on the connected components to obtain several target bounding boxes and corresponding target probabilities within the connected components; according to the difference situation before and after the opening operation on the connected components, obtain the inaccuracy of sheep segmentation for each target bounding box within the connected components; according to the edge distribution of the connected components inside between the target bounding box and other target bounding boxes, as well as the inaccuracy of sheep segmentation and the target probability, obtain the misdetection degree value for each target bounding box within the connected components; according to the closeness degree of the sheep flock within the connected components and the misdetection degree value of the target bounding box, obtain the comprehensive sheep recognition accuracy of the connected components; based on the comprehensive sheep recognition accuracy, obtain the optimal step size parameter of the convolutional neural network for multi-target recognition of the sheep flock in the pasture. Use the convolutional neural network based on the optimal step size to perform multi-target single-sheep recognition on the remote sensing image of the sheep flock in the pasture.

[0005] Preferably, the specific method for obtaining the suspected sheep flock distribution value of each connected component according to the internal distribution situation of the connected component is as follows: Perform an erosion operation on the th connected component to obtain that the th connected component is divided into multiple connected components after erosion; and take the number of connected components that the th connected component is divided into after erosion as the number of suspected sheep within the th connected component; take the product of the normalized value of the number of suspected sheep within the th connected component and the area of the th connected component as the suspected sheep flock distribution value within the th connected component.

[0006] Preferably, the specific method for obtaining the closeness degree of the sheep flock within the connected component according to the shape of the connected component and the suspected sheep flock distribution value is as follows: According to the shape of the connected component, obtain the gap saliency between the sheep within the connected component; Take the product of the inverse normalization value of the gap saliency between the sheep within the th connected component and the suspected sheep flock distribution value within the th connected component as the closeness degree of the sheep flock within the th connected component.

[0007] Preferably, the specific method for obtaining the gap saliency between the sheep within the connected component according to the shape of the connected component is as follows: The difference between the convex hull area and the area of the th connected component is used as the blank area area of the th connected component; take the ratio of the blank area area of the th connected component to the convex hull area as the gap saliency between the sheep within the th connected component.

[0008] Preferably, the specific method for obtaining the inaccuracy of sheep segmentation for each target box in the connected component based on the difference before and after the opening operation is as follows: Use the opening operation on the connected component within the th target box in the th connected component to perform erosion and dilation operations, and obtain the connected component within the th target box after the opening operation; Denote the absolute value of the difference between the area of the connected component within the th target box and the area of the connected component within the th target box after the opening operation as the target difference; Perform erosion operation on the connected component within the th target box in the th connected component. Among the connected components obtained after erosion, denote the Euclidean distance between the center position of the connected component with the largest area and the center position of the th target box as the target distance; Denote the product of the target difference and the target distance as the inaccuracy of sheep segmentation for the th target box in the

[0009] Preferably, the specific method for obtaining the false detection degree value for each target box in the connected component based on the edge distribution of the connected component inside between the target box and other target boxes, as well as the inaccuracy of sheep segmentation and the target probability, is as follows: For any two target boxes in the th connected component, select the target box with a larger target probability as the standard target box, and the other as the comparison target box; Use the image pyramid technology to scale the comparison target box until it is the same size as the standard target box; Obtain the final matching similarity between the standard target box and the comparison target box; Obtain the final matching similarity between the th target box in the th connected component and each target box in all connected components. Denote the inverse proportional normalization value of the maximum value of the final matching similarity and the inaccuracy of sheep segmentation for the th target box in the th connected component as the false detection degree value for the th target box in the th connected component.

[0010] Preferably, the specific method for obtaining the final matching similarity between the standard target box and the comparison target box is as follows: Select vertically within the standard target box. Starting from the highest pixel point on the edge position of the connected region inside the standard target box, obtain the chain code sequence of the standard target box in a clockwise direction; select vertically within the comparison target box. Starting from the highest pixel point on the edge position of the connected region inside the comparison target box, obtain the chain code sequence of the comparison target box in a clockwise direction; take the inverse proportional normalization value of the DTW value between the chain code sequence of the standard target box and the chain code sequence of the comparison target box as the matching similarity between the standard target box and the comparison target box. Preset a rotation parameter , and rotate the comparison target box clockwise by each time, and obtain the matching similarity between the standard target box and the comparison target box after each rotation; among the matching similarities between the standard target box and the comparison target box corresponding to all rotations, take the maximum value of the matching similarity as the final matching similarity between the standard target box and the comparison target box.

[0011] Preferably, the specific method for obtaining the comprehensive sheep recognition accuracy of the connected region according to the closeness of sheep in the connected region and the misdetection degree value of the target box is as follows: Take the absolute value of the difference between the area of the -th connected region and the sum of the areas of the connected regions within all target boxes in the -th connected region, and denote it as the first difference; take the ratio of the average value of the misdetection degree values of all target boxes in the -th connected region to the closeness of sheep in the -th connected region, and denote it as the first ratio; take the inverse proportional normalization value of the product of the first difference and the first ratio as the comprehensive sheep recognition accuracy of the -th connected region.

[0012] Preferably, the specific method for obtaining the optimal step size parameter of the convolutional neural network for multi-target recognition of the sheep flock in the pasture based on the comprehensive sheep recognition accuracy is as follows: For a convolutional neural network with any step size parameter, when using the convolutional neural network with any step size parameter to perform multi-target recognition on the remote sensing image of the sheep flock in the pasture, take the average value of the comprehensive sheep recognition accuracies of all connected regions in the remote sensing image of the sheep flock in the pasture as the preference degree of the convolutional neural network with any step size parameter; Preset an initial maximum step size parameter and a minimum step size parameter , gradually reduce the step size parameter of the convolutional neural network from to , and obtain the preference degree of the convolutional neural network with each step size parameter; take the step size parameter with the maximum preference degree as the optimal step size parameter of the convolutional neural network for multi-target recognition of the sheep flock in the pasture.

[0013] The present invention also provides a multi - target recognition system based on vision technology, including a memory and a processor. The processor executes the computer program stored in the memory to implement the steps of the above - mentioned multi - target recognition method based on vision technology.

[0014] The beneficial effects of the technical solution of the present invention are as follows: According to the shape of the connected domain and the suspected sheep flock distribution value, the closeness degree of the sheep flock within the connected domain is obtained; multi - target recognition is performed on the connected domain to obtain several target boxes within the connected domain and the corresponding target probabilities; according to the difference situation before and after the opening operation of the connected domain, the inaccuracy of single - sheep segmentation for each target box within the connected domain is obtained; according to the edge distribution of the connected domain inside between the target box and other target boxes, as well as the inaccuracy of single - sheep segmentation and the target probability, the misdetection degree value of each target box within the connected domain is obtained; according to the closeness degree of the sheep flock within the connected domain and the misdetection degree value of the target box, the comprehensive sheep recognition accuracy of the connected domain is obtained; based on the comprehensive sheep recognition accuracy, the optimal step - size parameter of the convolutional neural network for multi - target recognition of the sheep flock in the pasture is obtained; the convolutional neural network based on the optimal step - size is used to recognize multi - target single sheep in the remote - sensing image of the sheep flock in the pasture; in this way, through the adaptive step - size parameter strategy, the convolutional kernel can scan the image more precisely, and by analyzing the closeness information of the sheep flock within the target box, the model can more accurately determine the position and size of the target box, improving the detection accuracy; moreover, the present invention can adapt to sheep flocks with different densities, and still maintain a high detection accuracy under complex backgrounds and different lighting conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0016] Figure 1 It is a flowchart of the steps of a multi - target recognition method based on vision technology of the present invention; Figure 2 It is a flowchart of the characteristic relationship of a multi - target recognition method based on vision technology of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0017] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following will, in conjunction with the accompanying drawings and preferred embodiments, elaborate in detail on a multi-target recognition method and system based on vision technology proposed according to the present invention, its specific implementation manners, structures, features and effects. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0019] The following will specifically describe the specific solutions of a multi-target recognition method and system based on vision technology provided by the present invention in conjunction with the accompanying drawings.

[0020] Please refer to Figure 1 , which shows a step flow chart of a multi-target recognition method based on vision technology provided by an embodiment of the present invention. The method includes the following steps: Step S001: Obtain the remote sensing image of the sheep flock in the pasture.

[0021] In a specific implementation manner of the embodiment of the present invention, the method for obtaining the remote sensing image of the sheep flock in the pasture is as follows: Select a DJI Phantom 4 RTK drone equipped with a high-definition RGB camera, and plan the flight route through remote sensing software such as DJI Terra to ensure coverage of the entire activity area of the sheep flock; set the flight altitude of the drone at 50 meters, and ensure regular shooting during flight. The captured images should have a certain degree of overlap for later stitching; use remote sensing image processing software (such as Pix4D) to stitch the collected images to generate the remote sensing image of the sheep flock in the pasture.

[0022] Thus, the remote sensing image of the sheep flock in the pasture is obtained through the above method.

[0023] Step S002: Obtain several connected components in the remote sensing image of the sheep flock in the pasture; according to the distribution of the interior of the connected components, obtain the suspected sheep flock distribution value of each connected component; according to the shape of the connected component and the suspected sheep flock distribution value, obtain the degree of closeness of the sheep flock within the connected component.

[0024] It should be noted that in the scenario of herders' live grazing, the sheep flock often presents the following characteristics: the sheep flock is closely distributed, forming a dense area, and the interval between target sheep in the dense area of the sheep flock is small, which affects accurate identification and classification. When segmenting the remote sensing image of the sheep flock in the pasture to obtain multiple connected regions, due to the close distribution of the sheep flock, there are multiple sheep in a single connected region. That is, the greater the degree of closeness of the sheep flock within the connected region, the greater the possibility of having multiple sheep in a single connected region, and thus the lower the accuracy of sheep identification.

[0025] Preferably, in some implementation manners of the embodiments of the present invention, since there is an obvious color distinction between sheep and the pasture, that is, sheep are white and the pasture grass is green, the Otsu threshold segmentation algorithm can be used to segment the remote sensing image of the sheep flock in the pasture to separate the sheep and the grass; and obtain several connected regions in the remote sensing image of the sheep flock in the pasture. The specific process is as follows: Use the Otsu threshold segmentation algorithm to perform threshold segmentation on the remote sensing image of the sheep flock in the pasture to obtain a binary image of the remote sensing image of the sheep flock in the pasture; obtain the gradient image of the binary image of the remote sensing image of the sheep flock in the pasture, and perform the watershed segmentation algorithm on the gradient image to obtain multiple regions of the binary image of the remote sensing image of the sheep flock in the pasture, and regard each region as a connected region, thereby obtaining several connected regions in the remote sensing image of the sheep flock in the pasture.

[0026] Among them, in the process of performing threshold segmentation on the remote sensing image of the sheep flock in the pasture, the gray value of the pixel points greater than the Otsu threshold is recorded as 1, and the pixel points less than the Otsu threshold are recorded as 0; then the area of the sheep will be white, while the grass area will be black; the Otsu threshold segmentation algorithm and the watershed segmentation algorithm are existing technologies, and will not be elaborated here in this embodiment.

[0027] Preferably, in some implementation manners of the embodiments of the present invention, since the larger the connected region, the more sheep there are in the connected region, but because sheep have different sizes, there are multiple connected and spaced regions within the connected region. Therefore, erosion operations can be used to distinguish these regions. The more the number of these regions, the more the number of suspected sheep in the connected region; according to the distribution within the connected region, obtain the suspected sheep flock distribution value of each connected region. The specific process is as follows: Perform an erosion operation on the th connected region to obtain that the th connected region is divided into multiple connected regions after erosion; and take the number of connected regions that the th connected region is divided into after erosion as the number of suspected sheep in the th connected region; take the product of the normalized value of the number of suspected sheep in the th connected region and the area of the th connected region as the The suspected sheep flock distribution value within a connected region.

[0028] Among them, the erosion operation is a prior art, and it will not be elaborated here in this embodiment.

[0029] Preferably, in some implementation manners of the embodiment of the present invention, for any connected region in the remote sensing image of the sheep flock in the pasture, when multiple sheep form a connected region, there must be gaps between the sheep. When the ratio between the area of the blank region inside the connected region and the area of the convex hull of the connected region is smaller, it indicates that the gaps between the sheep flocks are less significant, which means that the sheep flock distribution within the connected region is dense; according to the shape of the connected region and the suspected sheep flock distribution value, the closeness degree of the sheep flock within the connected region is obtained. The specific process is as follows: The difference between the area of the convex hull of the th connected region and the area is used as the area of the blank region of the th connected region; the ratio of the area of the blank region of the th connected region to the area of the convex hull is used as the gap significance degree between the sheep flocks within the th connected region; The product of the inverse normalization value of the gap significance degree between the sheep flocks within the th connected region and the suspected sheep flock distribution value within the th connected region is used as the closeness degree of the sheep flock within the Specific formula: In the formula, represents the closeness degree of the sheep flock within the th connected region; represents the suspected sheep flock distribution value within the th connected region; represents the gap significance degree between the sheep flocks within the th connected region; represents the exponential function with the natural constant as the base.

[0030] Among them, obtaining the area of the convex hull of the connected region is a prior art, and it will not be elaborated here in this embodiment.

[0031] Thus far, the closeness degree of the sheep flock within the connected region is obtained through the above method.

[0032] Step S003: Perform multi-object recognition on the connected components to obtain several target bounding boxes and corresponding target probabilities within the connected components; according to the difference situation before and after the opening operation on the connected components, obtain the inaccuracy of single-sheep segmentation for each target bounding box within the connected components; according to the edge distribution of the connected components inside between the target bounding box and other target bounding boxes, as well as the inaccuracy of single-sheep segmentation and the target probability, obtain the misdetection degree value for each target bounding box within the connected components; according to the closeness degree of the sheep flock within the connected components and the misdetection degree value of the target bounding box, obtain the comprehensive sheep recognition accuracy of the connected components; based on the comprehensive sheep recognition accuracy, obtain the optimal step size parameter of the convolutional neural network for multi-object recognition of the sheep flock in the pasture.

[0033] It should be noted that when herdsmen show the sheep flock they are grazing to netizens, they mostly drive the sheep flock to a fixed area for the convenience of netizens to observe the activities among the sheep flock; this will lead to a relatively large number of close contacts between the sheep flock. However, the erosion operation cannot effectively segment the connected components corresponding to each sheep. Therefore, a neural network can be used to further perform multi-object recognition on the sheep flock. The traditional method for recognizing and classifying each sheep in the sheep flock mostly uses a convolutional neural network with a fixed step size for target localization. However, due to the dense distribution of the sheep flock, during the process of using the target detection model to perform single-sheep target recognition, when there are multiple sheep within a connected component, the target bounding box will be segmented at this time. However, during the segmentation process, due to the close contact of the sheep flock, it will lead to a situation where not only a complete single sheep exists within a target bounding box but also incomplete parts of other sheep exist, and there is no complete single sheep, only incomplete parts of multiple sheep. These all belong to misdetected target bounding boxes; therefore, it is necessary to analyze and iterate the situation of misdetected target bounding boxes generated by the convolutional neural network with different step sizes, evaluate the single-sheep detection accuracy of the convolutional neural network at each step size, and then obtain the convolutional neural network with the optimal step size parameter.

[0034] Preferably, in some implementation manners of the embodiments of the present invention, perform multi-object recognition on the connected components to obtain several target bounding boxes and corresponding target probabilities within the connected components. The specific process is as follows: Preset an initial maximum step size parameter and a minimum step size parameter , input the remote sensing image of the sheep flock in the pasture into the target detection model that combines the Region Proposal Network (RPN) and the Convolutional Neural Network (CNN) with the initial maximum step size parameter being , and obtain several target bounding boxes and corresponding target probabilities; Among them, the target detection model that combines the Region Proposal Network (RPN) and the Convolutional Neural Network (CNN) is a prior art, and this embodiment will not elaborate too much here.

[0035] It should be noted that when there are multiple sheep in the connected component, during the process of identifying and segmenting using the target box, due to the close proximity of the sheep flock, there will be not only a complete single sheep but also incomplete parts of other sheep within one target box. Therefore, it is necessary to analyze the target boxes within the connected component to determine whether this situation exists. When the target box is a normal target box, first, its target probability is relatively large. Second, when compared with other normal target boxes, the edge contours of the sheep within the two boxes are similar. Therefore, the matching similarity between the two can be obtained using chain codes. However, there is a situation where the directions of the sheep heads are inconsistent, which will lead to inconsistent chain code directions. Therefore, it is necessary to make the directions of the two sheep heads consistent. After obtaining the matching similarity between the two, analyze the matching similarity between the target box and other target boxes to obtain the misdetection degree value of the target box. Furthermore, comprehensively analyze to obtain the accuracy of sheep recognition in the connected component.

[0036] Preferably, in some implementation manners of the embodiments of the present invention, due to the close proximity of the sheep flock, there will be not only a complete single sheep but also incomplete parts of other sheep within one target box. Therefore, when performing erosion and dilation operations on the connected component within the target box using morphological opening, the difference in the area of the connected component before and after the morphological opening is relatively large. And after the connected component within the target box is eroded, among the multiple connected components obtained after erosion, the central position of the largest connected component must have a large difference from the central position of the target box. When the target box is a normal target box, there is only one sheep within its box, and the central position of its largest connected component must be close to the central position of the target box. According to the difference situation before and after the morphological opening of the connected component, obtain the inaccuracy of sheep segmentation for each target box within the connected component. The specific method is as follows: Use morphological opening to perform erosion and dilation operations on the connected component within the th connected component within the th target box, and obtain the connected component within the th target box after the morphological opening; Denote the absolute value of the difference between the area of the connected component within the th target box and the area of the connected component within the th target box after the morphological opening as the target difference; When eroding the connected component within the th connected component within the th target box, among the multiple connected components obtained after erosion, denote the Euclidean distance between the central position of the largest connected component and the central position of the th target box as the target distance; Denote the product of the target difference and the target distance as the inaccuracy of sheep segmentation for the th connected component within the th target box; In the formula, Indicates the inaccuracy of sheep segmentation of the target box; Indicates the Euclidean distance between the center position of the target box and its eroded center position; Indicates the absolute value of the difference between the area of the connected domain within the target box before performing the opening operation and the area of the connected domain within the target box after performing the opening operation.

[0037] Among them, the opening operation is a prior art, and it will not be elaborated too much here in this embodiment.

[0038] Preferably, in some implementation manners of the embodiments of the present invention, when the target box is a normal target box, firstly, its target probability is relatively large; secondly, when compared with other normal target boxes, the edge contours of the sheep within the two boxes are similar. Therefore, the matching similarity between the two can be obtained by using chain codes; however, there is a situation where the sheep head directions are inconsistent, which will lead to inconsistent chain code directions. Therefore, it is necessary to make the sheep head directions of the two consistent. After obtaining the matching similarity between the two, analyze the matching similarity between the target box and other target boxes to obtain the misdetection degree value of the target box; according to the shape distribution of the connected domains inside the target box and other target boxes, as well as the target probability, obtain the misdetection degree value of each target box within the connected domain; the specific method is as follows: For any two target boxes in the th connected domain, select the target box with a larger target probability as the standard target box, and the other as the comparison target box; use the image pyramid technology to scale the comparison target box until it is the same size as the standard target box; In the standard target box, select a vertical direction, starting from the highest pixel point on the edge position of the connected domain inside the standard target box, and obtain the chain code sequence of the standard target box in a clockwise direction; in the comparison target box, select a vertical direction, starting from the highest pixel point on the edge position of the connected domain inside the standard target box, and obtain the chain code sequence of the comparison target box in a clockwise direction; use the inverse proportional normalization value of the DTW value between the chain code sequence of the standard target box and the chain code sequence of the comparison target box as the matching similarity between the standard target box and the comparison target box; Among them, the chain code sequence of the target box is the chain code sequence of the edge of the connected domain within the target box, and obtaining the chain code sequence of the connected domain edge and the image pyramid technology are prior arts, and they will not be elaborated too much here in this embodiment.

[0039] Preset a rotation parameter , rotate the comparison target box clockwise by each time, and obtain the matching similarity between the standard target box and the comparison target box after each rotation; among the matching similarities between the standard target box and the comparison target box corresponding to all rotations, use the maximum value of the matching similarity as the final matching similarity between the standard target box and the comparison target box; Obtain the The final matching similarity between the th target box in a connected component and each target box in all connected components, the inverse proportional normalization value of the maximum value of the final matching similarity is multiplied by the single-sheep segmentation inaccuracy of the th target box in the th connected component, and the product is used as the misdetection degree value of the th target box in the th connected component; The specific formula is: In the formula, represents the misdetection degree value of the th target box in the th connected component; represents the single-sheep segmentation inaccuracy of the th target box in the th connected component; represents the number of all target boxes in all connected components; represents the th target box and the th target box in the th connected component; represents the maximum value function; represents the exponential function with the natural constant as the base.

[0040] Among them, during the rotation process, the highest pixel point at the edge position of the connected component in the vertical direction is still selected as the starting point of the chain code; the greater the final matching similarity between the th target box and other target boxes, the greater the probability that the th target box is a normal target box; on the contrary, the greater the probability that the th target box is a misdetected target box, the greater the misdetection degree value of the th target box.

[0041] Preferably, in some implementation manners of the embodiments of the present invention, since the closer the sheep in the connected component are, the easier it should be to segment, but at this time, a large segmentation error appears, indicating that the convolutional neural network under this step size parameter has a lower recognition accuracy for the sheep in the connected component; and a high sheep overlap rate will lead to duplication between target boxes in the connected component. The larger the duplicate area, the worse the separation effect of the sheep in the connected component, and when the misdetection degree value of the target box in the connected component is larger, and the closer the sheep in each connected component are, the more inaccurate the output target box result is, then the multi-target recognition is less suitable, that is, the comprehensive sheep recognition accuracy of the connected component is lower; the specific method for obtaining the comprehensive sheep recognition accuracy of the connected component according to the closeness of the sheep in the connected component and the misdetection degree value of the target box is: The absolute value of the difference between the area of the th connected component and the cumulative sum of the areas of the connected components within all target bounding boxes in the th connected component is denoted as the first difference; the ratio of the mean misdetection degree value of all target bounding boxes in the th connected component to the degree of sheep flocks being close together in the th connected component is denoted as the first ratio; the inverse proportional normalization value of the product of the first difference and the first ratio is used as the comprehensive sheep recognition accuracy of the th connected component; The specific formula is: In the formula, represents the comprehensive sheep recognition accuracy of the th connected component; represents the area of the th connected component; represents the area of the connected component in the th target bounding box within the th connected component; represents the number of all target bounding boxes within the th connected component; represents the mean misdetection degree value of all target bounding boxes within the th connected component; represents the degree of sheep flocks being close together within the th connected component; represents the exponential function with the natural constant as the base.

[0042] It should be noted that for any convolutional neural network with a step size parameter, the greater the comprehensive sheep recognition accuracy of the connected components in the remote sensing image of the sheep flock in the pasture, the higher the preference degree of the convolutional neural network with this step size parameter; therefore, the optimal step size parameter of the convolutional neural network for multi-target recognition of the sheep flock in the pasture is obtained based on the comprehensive sheep recognition accuracy.

[0043] Preferably, in some implementation manners of the embodiments of the present invention, the specific method for obtaining the optimal step size parameter of the convolutional neural network for multi-target recognition of the sheep flock in the pasture is: For any convolutional neural network with a step size parameter, when using the convolutional neural network with this step size parameter to perform multi-target recognition on the remote sensing image of the sheep flock in the pasture, the mean value of the comprehensive sheep recognition accuracy of all connected components in the remote sensing image of the sheep flock in the pasture is used as the preference degree of the convolutional neural network with this step size parameter; The step size parameter of the convolutional neural network is gradually decreased from to , obtain the preference degree of the convolutional neural network for each step size parameter; take the step size parameter with the maximum preference degree as the optimal step size parameter of the convolutional neural network for multi-target recognition of the sheep flock in the pasture.

[0044] Thus, the optimal step size parameter of the convolutional neural network for multi-target recognition of the sheep flock in the pasture is obtained through the above method.

[0045] Step S004: Based on the convolutional neural network with the optimal step size, perform multi-target sheep recognition on the remote sensing image of the sheep flock in the pasture.

[0046] It should be noted that by analyzing the close information of the sheep flock in the live grazing scenario, the model can more accurately determine the updated position of the target box and improve the detection accuracy; this embodiment can adapt to sheep flocks with different densities in the live grazing scenario, and still maintain a high detection accuracy under complex backgrounds and different lighting conditions.

[0047] In a specific implementation manner of the embodiment of the present invention, the optimal step size parameter of the remote sensing image of the sheep flock in the pasture is input into the object detection model that combines the Region Proposal Network (RPN) and the Convolutional Neural Network (CNN) to obtain a number of target boxes, and each target box corresponds to each sheep in the sheep flock in the pasture; The specific method for obtaining the dataset of the neural network is as follows: Collect a large number of remote sensing images of the sheep flock in the pasture, and manually mark the position of each sheep in each remote sensing image of the sheep flock in the pasture with a bounding box. This marking result is recorded as the label of each remote sensing image of the sheep flock in the pasture; collect a large number of remote sensing images of the sheep flock in the pasture and their corresponding labels to form a dataset; use this dataset to train the neural network, and the loss function used in the training process is the mean square error loss function; the specific training process is well-known content of the neural network, and the specific training process is not described in detail in this solution.

[0048] Please refer to Figure 2 , which shows a feature relationship flowchart of a multi-target recognition method based on vision technology; Through the above steps, a multi-target recognition method based on vision technology is completed.

[0049] Another embodiment of the present invention provides a multi-target recognition system based on vision technology. The system includes a memory and a processor. When the processor executes the computer program stored in the memory, it executes the above method steps S001 to S004.

[0050] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application, and should all be included within the protection scope of the present application.

Claims

1. A multi-object recognition method based on vision technology, characterized in that The method includes the following steps: Obtain the remote sensing image of the sheep flock in the pasture; Obtain several connected components in the remote sensing image of the sheep flock in the pasture; according to the distribution of the interior of the connected components, obtain the suspected sheep flock distribution value of each connected component; according to the shape of the connected component and the suspected sheep flock distribution value, obtain the closeness degree of the sheep flock within the connected component; Perform multi-object recognition on the connected components to obtain several target boxes within the connected components and the corresponding target probabilities; according to the difference situation before and after performing the opening operation on the connected components, obtain the single-sheep segmentation inaccuracy of each target box within the connected components; according to the edge distribution of the connected components inside between the target box and other target boxes, as well as the single-sheep segmentation inaccuracy and the target probability, obtain the false detection degree value of each target box within the connected components; according to the closeness degree of the sheep flock within the connected component and the false detection degree value of the target box, obtain the comprehensive sheep recognition accuracy of the connected component; based on the comprehensive sheep recognition accuracy, obtain the optimal step size parameter of the convolutional neural network for multi-object recognition of the sheep flock in the pasture; Perform multi-object single-sheep recognition on the remote sensing image of the sheep flock in the pasture based on the convolutional neural network with the optimal step size.

2. The multi-object recognition method based on vision technology according to claim 1, wherein The specific method for obtaining the suspected sheep flock distribution value of each connected component according to the distribution of the interior of the connected component is as follows: Perform an erosion operation on the th connected component, and after erosion, the th connected component is divided into multiple connected components; and take the number of connected components that the th connected component is divided into after erosion as the number of suspected sheep within the th connected component; take the product of the normalized value of the number of suspected sheep within the th connected component and the area of the th connected component as the suspected sheep flock distribution value within the th connected component.

3. The multi-object recognition method based on vision technology according to claim 1, characterized in that, The specific method for obtaining the closeness degree of the sheep flock within the connected component according to the shape of the connected component and the suspected sheep flock distribution value is as follows: According to the shape of the connected component, obtain the gap significance between the sheep flocks within the connected component; Multiply the inverse normalization value of the gap saliency between the sheep flocks in the th connected region by the suspected sheep flock distribution value in the th connected region, and use the product as the closeness degree of the sheep flocks in the th connected region.

4. The multi-object recognition method based on vision technology according to claim 3, characterized in that The specific method for obtaining the gap significance between the sheep flocks within the connected component according to the shape of the connected component is as follows: The difference between the convex hull area and the area of the th connected component is used as the blank area of the th connected component; the ratio of the blank area of the th connected component to the convex hull area is used as the gap saliency between flocks within the th connected component.

5. The multi-object recognition method based on vision technology according to claim 1, wherein The specific method for obtaining the single-sheep segmentation inaccuracy of each target box within the connected component according to the difference situation before and after performing the opening operation on the connected component is as follows: Perform erosion and dilation operations on the connected components within the th connected component in the th target bounding box using opening operation, and obtain the connected components within the th target bounding box after opening operation; Denote the absolute value of the difference between the area of the connected components within the th target bounding box and the area of the connected components within the th target bounding box after opening operation as the target difference; When performing an erosion operation on the connected component within the th connected component in the th target bounding box, among the connected components obtained after erosion and divided into multiple connected components, the Euclidean distance between the center position of the largest connected component and the center position of the th target bounding box is denoted as the target distance; the product of the target difference and the target distance is used as the inaccuracy of sheep segmentation for the th connected component in the th target bounding box.

6. The multi-object recognition method based on vision technology according to claim 1, characterized in that The specific method for obtaining the false detection degree value of each target box within the connected component according to the edge distribution of the connected components inside between the target box and other target boxes, as well as the single-sheep segmentation inaccuracy and the target probability is as follows: For any two target bounding boxes in the th connected component, select the target bounding box with a higher target probability as the standard target bounding box, and the other as the comparison target bounding box; Use the image pyramid technique to scale the comparison target box until it is the same size as the standard target box; Obtain the final matching similarity between the standard target box and the comparison target box; Obtain the th final matching similarity between the th target bounding box in the th connected component and each target bounding box in all connected components, and multiply the inverse normalization value of the maximum value of the final matching similarity by the sheep segmentation inaccuracy of the th target bounding box in the th connected component as the false detection degree value of the th target bounding box in the th connected component.

7. The multi-object recognition method based on vision technology according to claim 6, characterized in that The specific method for obtaining the final matching similarity between the standard target box and the comparison target box is as follows: Select vertically within the standard target box, with the highest pixel point on the edge of the connected component inside the standard target box as the starting point, and obtain the chain code sequence of the standard target box in a clockwise direction; select vertically within the comparison target box, with the highest pixel point on the edge of the connected component inside the comparison target box as the starting point, and obtain the chain code sequence of the comparison target box in a clockwise direction; take the inverse proportional normalization value of the DTW value between the chain code sequence of the standard target box and the chain code sequence of the comparison target box as the matching similarity between the standard target box and the comparison target box; Preset a rotation parameter , rotate the comparison target box clockwise by each time, and obtain the matching similarity between the standard target box and the comparison target box after each rotation; among the matching similarities between the standard target box and the comparison target box corresponding to all rotations, take the maximum value of the matching similarity as the final matching similarity between the standard target box and the comparison target box.

8. The multi-object recognition method based on vision technology according to claim 1, characterized in that The specific method for obtaining the comprehensive sheep recognition accuracy of the connected component according to the closeness degree of the sheep flock within the connected component and the false detection degree value of the target box is as follows: Take the absolute value of the difference between the area of the th connected component and the cumulative sum of the areas of the connected components within all target bounding boxes in the th connected component, and denote it as the first difference; take the ratio of the mean of the misdetection degree values of all target bounding boxes in the th connected component to the sheep flock closeness degree in the th connected component, and denote it as the first ratio; The inverse normalization value of the product of the first difference and the first ratio is used as the comprehensive sheep recognition accuracy of the th connected component.

9. The multi-object recognition method based on vision technology according to claim 1, characterized in that, The specific method for obtaining the optimal step size parameter of the convolutional neural network for multi-object recognition of the sheep flock in the pasture based on the comprehensive sheep recognition accuracy is as follows: For any convolutional neural network with a step parameter, when using the convolutional neural network with any step parameter to perform multi-target recognition on the remote sensing image of the sheep flock in the pasture, the mean value of the comprehensive sheep recognition accuracy of all connected components in the remote sensing image of the sheep flock in the pasture is used as the preference degree of the convolutional neural network with any step parameter; Preset an initial maximum step size parameter and a minimum step size parameter , and gradually reduce the step size parameter of the convolutional neural network from to , and obtain the preference degree of the convolutional neural network for each step size parameter; use the step size parameter with the maximum preference degree as the optimal step size parameter of the convolutional neural network for multi-object recognition of the sheep flock in the pasture.

10. A multi-object recognition system based on vision technology, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the computer program is executed by a processor, it implements the steps of a multi-target recognition method based on vision technology according to any one of claims 1-9.

Images