Image recognition method and system based on biological asset shadow zone inventory count

Abstract

The application provides an image recognition method and system based on biological asset shadow area inventory counting, comprising the following steps: image segmentation learning step: collecting a breeding scene image to be analyzed, and performing segmentation learning on the image to be analyzed to determine the context of the image; image training learning step: performing image training learning from the image to be analyzed with the determined context, and judging whether there is a shadow, if not, a first result is obtained; edge device supplementary training step: if it is judged that there is a shadow in the image training learning step, supplementary lighting is performed, a supplementary photographed image to be analyzed is obtained, image segmentation learning and training learning are performed, and a second result is obtained. Through the image AI recognition algorithm, the counting interference caused by the shadow and other scenes is solved based on the convolutional network neural learning of image segmentation and the edge side terminal assistance, and the deviation problem of large biological assets in complex scene (shadow) counting inventory is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular, to an image recognition method and system based on biological asset shadow area inventory counting, and more particularly, to an image recognition method and edge side device process based on biological asset shadow area inventory counting. BACKGROUND

[0002] With the promotion and support of agricultural finance development, the support for agriculture and animal husbandry in the fields of logistics, cold chain, finance, etc. is also increasing. Biological assets with high economic value such as cattle, sheep, horses, etc. will also have financial attributes in addition to consumption attributes. How to efficiently and credibly digitize biological assets through technical means has become a problem for the development of agricultural finance. The breeding industry has always been faced with the problem of financing difficulty caused by the difficulty of "living asset" value identification, mortgage, and supervision, which has largely restricted the scale development of breeders and breeding enterprises.

[0003] The Chinese patent document with publication number CN112330107A discloses a breeding biological asset management system and method based on a blockchain. The system includes a blockchain platform, a procurement module, a breeding module, a transportation module, a slaughter module, a sales module, an insurance module, and a financing module. The method includes: procuring breeding objects and raw materials, saving raw material information on the blockchain platform; breeding the breeding objects, saving breeding data and culling data on the blockchain platform; transporting the breeding objects, and saving vehicle information and transportation information on the blockchain platform; slaughtering the breeding objects after quarantine registration, and saving quarantine data and slaughter data on the blockchain platform; selling the breeding objects, querying the whole cycle information through the label on the package; providing insurance services for breeders; and providing financing services for breeders, slaughter enterprises, and sales companies.

[0004] For the related technology in the above, the inventors believe that shadow and other scenarios will cause counting interference, and large biological assets will produce deviation in counting and inventory in complex scenarios (shadows). SUMMARY

[0005] In view of the defects in the prior art, the purpose of the present application is to provide an image recognition method and system based on biological asset shadow area inventory counting.

[0006] According to the image recognition method based on biological asset shadow area inventory counting provided by the present application, the following steps are included:

[0007] Image segmentation learning step: collect the to-be-analyzed image of the breeding scene, and perform segmentation learning on the to-be-analyzed image to determine the context of the to-be-analyzed image;

[0008] Image training learning step: image training learning is performed from the image to be analyzed in which the context is determined, and it is determined whether there is a shadow, and if there is no shadow, a first result is obtained;

[0009] Edge device supplementary training step: if it is determined that there is a shadow in the image training learning step, supplementary light shooting is performed, a picture to be analyzed after shooting is obtained, image segmentation learning and training learning are performed, and a second result is obtained.

[0010] Preferably, the image segmentation learning step includes the following steps:

[0011] Step S1: deploying a camera facing a breeding scene in a livestock large-scale breeding department;

[0012] Step S2: acquiring a picture to be analyzed through the camera, and transmitting the picture to a processing server;

[0013] Step S3: the processing server performs image segmentation on the picture to be analyzed using a multi-task convolutional neural network, takes the color spots of the segmented image as input, inputs the color spots to the convolutional neural network, and the convolutional neural network labels the pixels;

[0014] Step S4: the picture after pixel labeling is divided into a*b rectangular blocks, the RGB values of all pixel points in n corner rectangular blocks are extracted, the number of pixel points with the same RGB value in all pixel points in the n corner rectangular blocks is counted, and the RGB value corresponding to the predetermined number is determined as a threshold, wherein a represents the long side and b represents the wide side;

[0015] Step S5: calculate the RGB mean value of the remaining a*b-n rectangular blocks, and keep the RGB mean value greater than or equal to the threshold, to obtain C rectangular blocks corresponding to C mean values, form the RGB values of the pixel points of the C rectangular blocks into input data, and input the input data to the convolutional layer to classify each pixel, and determine the context of the picture to be analyzed.

[0016] Preferably, the image training learning step includes the following steps:

[0017] Step S6: convert each pixel data from two dimensions to one dimension, calculate the covariance matrix of the converted one-dimensional channel, calculate the eigenvector p and the eigenvalue λ of the covariance matrix, convert according to the formula, and complete the color random jitter judgment;

[0018] Step S7: color gamut adjustment, color judgment, and shadow judgment model generation;

[0019] Step S8: if the shadow judgment model determines that there is no shadow, the sliding window determines whether there is a living being, the network randomly extracts a frame from the original picture, performs positive and negative sample learning, enters the output network, and obtains a first result.

[0020] Preferably, the edge device supplementary training step comprises the following steps:

[0021] Step S9: providing an edge device;

[0022] Step S10: performing semantic analysis by a processor in the edge device, if the shadow judgment model judges that there is a shadow, taking a picture by the light compensation lamp, continuing steps S2-S6, judging whether there is a living being, randomly cutting a frame from the picture after light compensation by the network, performing positive and negative sample learning, entering an output network, and obtaining a second result.

[0023] The image recognition method based on biological asset shadow area inventory counting, characterized in that the edge device supplementary training step further comprises the following steps:

[0024] Step S11: comparing the first result and the second result, performing convolutional network learning, and subsequently using an average value in T moments;

[0025] Step S12: determining the sum of the number of living beings in a*b according to the average value.

[0026] According to the image recognition system based on biological asset shadow area inventory counting provided by the application, the following modules are included:

[0027] An image segmentation learning module: collecting an image to be analyzed of a breeding scene, and performing segmentation learning on the image to be analyzed to determine the context of the image to be analyzed;

[0028] An image training learning module: performing image training learning from the image to be analyzed with the determined context, and judging whether there is a shadow, and obtaining a first result if there is no shadow;

[0029] An edge device supplementary training module: if it is judged that there is a shadow in the image training learning module, taking a picture by light compensation, obtaining an analyzed picture after supplementary shooting, performing image segmentation learning and training learning, and obtaining a second result.

[0030] Preferably, the image segmentation learning module comprises the following modules:

[0031] Module M1: deploying a camera facing a breeding scene in a large-scale livestock breeding department;

[0032] Module M2: collecting an analyzed picture by the camera, and transmitting the analyzed picture to a processing server;

[0033] Module M3: the processing server performs image segmentation on the analyzed picture by using a multi-task convolutional neural network, takes the color spots of the segmented image as input, inputs the color spots to the convolutional neural network, and the convolutional neural network labels the pixels;

[0034] Module M4: the picture after pixel marking is divided into a*b rectangular blocks, the RGB values of all pixel points in the n corner rectangular blocks are extracted, the number of pixel points with the same RGB value in the RGB values of all pixel points in the n corner rectangular blocks is counted, and the RGB value corresponding to the predetermined number is determined as a threshold, wherein a represents the long side, and b represents the width;

[0035] Module M5: the RGB mean values of the remaining a*b-n rectangular blocks are calculated, the RGB mean values greater than or equal to the threshold are reserved to obtain C rectangular blocks corresponding to C mean values, the RGB values of the pixel points of the C rectangular blocks are formed into input data, and are transmitted to a convolution layer to classify each pixel and determine the context of the image to be analyzed.

[0036] Preferably, the image training learning module comprises the following modules:

[0037] Module M6: each pixel data is converted from two dimensions to one dimension, a covariance matrix is calculated for the converted one-dimensional channel, an eigenvector p and an eigenvalue λ of the covariance matrix are calculated, conversion is performed according to a formula, and color random jittering is determined.

[0038] Module M7: color gamut adjustment, a shadow judgment model is generated after color judgment;

[0039] Module M8: if the shadow judgment model judges that there is no shadow, a sliding window is used to judge whether there is a living being, a network randomly extracts a frame from an original picture, positive and negative sample learning is performed, an output network is entered, and a first result is obtained.

[0040] Preferably, the edge device supplementary training module comprises the following modules:

[0041] Module M9: an edge device is provided;

[0042] Module M10: semantic analysis is performed by a processor in the edge device, if the shadow judgment model judges that there is a shadow after module M7, light compensation is performed by a light compensation lamp, modules M2-M6 are continuously performed, a sliding window is used to judge whether there is a living being, a network randomly extracts a frame from the picture after light compensation, positive and negative sample learning is performed, an output network is entered, and a second result is obtained.

[0043] Preferably, the edge device supplementary training module further comprises the following modules:

[0044] Module M11: the first result and the second result are compared, convolution network learning is performed, and a mean value is calculated in T moments in the future;

[0045] Module M12: the number of living beings in a*b is summed according to the mean value.

[0046] Compared with the prior art, the present application has the following beneficial effects:

[0047] 1. The present application solves the deviation problem of large biological asset counting in complex scenes (shadows) and the like by using image AI (Artificial Intelligence) recognition algorithm, counting interference caused by scenes such as shadows, and edge side terminal assistance based on image segmentation convolution network neural learning. BRIEF DESCRIPTION OF DRAWINGS

[0048] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments with reference to the attached drawings:

[0049] Figure 1 Flowchart for beef cattle counting image learning based on shadows;

[0050] Figure 2 Resnet (Residual Network) network structure diagram. DETAILED DESCRIPTION

[0051] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of changes and improvements can be made. These are within the scope of the present application.

[0052] The present application discloses a kind of image recognition method and edge side device flow based on biological asset shadow area inventory counting, as shown in Figure 1 And Figure 2 It includes the following steps:

[0053] Image segmentation learning process, i.e. image segmentation learning step: collect the image to be analyzed of breeding scene, and carry out segmentation learning to the image to be analyzed, determine the context of the image to be analyzed. Specifically, it includes the following steps:

[0054] Step 1: deploy a camera facing the breeding scene in the large-scale livestock breeding.

[0055] Step 2: collect the picture to be analyzed by the camera, and transmit the picture to the processing server.

[0056] Step 3: the processing server uses MTCNN (Multi-task convolutional neural network) to perform image segmentation on the picture, takes the patch (color spot, patch) of the image as input, inputs to the convolutional neural network Refine-net (a multi-path reinforcement network), and the convolutional neural network labels the pixels. CNN represents convolutional neural network.

[0057] Step 4: The picture is divided into a*b rectangular blocks, the RGB values of all pixel points in the four corner rectangular blocks are extracted, the number of pixel points with the same RGB value in all pixel points in the four corner rectangular blocks is counted, and the RGB value corresponding to the maximum number is determined as the threshold value.

[0058] Step 5: Calculate the RGB mean value (compared with the threshold value) of the remaining a*b-4 rectangular blocks, retain all mean values greater than or equal to the first threshold value, obtain C rectangular blocks corresponding to C mean values, and form the input data Refine-net of the pixel points of the C rectangular blocks, and pass it to the convolution layer to classify each pixel (classify the pixel by softmax) to determine the context of the image, including the location of the target.

[0059] Image training and learning process, i.e. image training and learning step: image training and learning is performed from the context-determined image to be analyzed, and it is judged whether there is a shadow. If there is no shadow, a first result is obtained. Specifically, the following steps are included:

[0060] Step 6: The pixel data of each channel is first converted from two-dimensional to one-dimensional (for example, 256*256*3 to 65536*3), then the covariance matrix (3*3) of the three channels of the picture (65536*3) is calculated, then the eigenvector p and eigenvalue λ of the covariance matrix are calculated, and finally the conversion is completed according to the formula, and the color random jitter judgment is completed.

[0061] Step 7: Color gamut adjustment, the shadow judgment model is generated after the above color judgment.

[0062] Step 8: Resnet preprocessing is color gamut adjustment, then the window judgment is performed to determine whether there is beef cattle, the network randomly extracts the frame from the above data, performs positive and negative sample learning, and enters the Output Network (output network).

[0063] Edge device supplementary training step: if it is judged that there is a shadow in the image training and learning step, supplementary lighting is performed, a supplementary photographed image to be analyzed is obtained, image segmentation learning and training are performed, and a second result is obtained. Specifically, the following steps are included:

[0064] Step 9: An edge device is provided, and the terminal device comprises a processor, a memory, a communication unit, a camera, a light supplementing lamp and a bus (a data acquisition protocol of Internet of Things).

[0065] Step 10: The edge device processor performs first semantic analysis, and after the shadow is judged in step 7, the light is photographed, and steps 2-6 are continued.

[0066] Step 11: The results of step 8 and step 10 are compared, and more convolutional network learning is performed (subsequently, mean is calculated at T time to enhance robustness).

[0067] Step S12: Sum the number of organisms in a*b.

[0068] Figure 1 In the RPN layer, the region that may be the target is extracted. Figure 2 In the RPN layer, the region that may be the target is extracted.

[0069] The application also provides an image recognition system based on biological disability shadow area inventory counting, which can be realized by executing the process steps of the image recognition method based on biological disability shadow area inventory counting, i.e., the image recognition method based on biological disability shadow area inventory counting can be understood by those skilled in the art as the preferred embodiment of the image recognition system based on biological disability shadow area inventory counting.

[0070] The application also discloses an image recognition system based on biological disability shadow area inventory counting, which comprises the following modules:

[0071] An image segmentation learning module: collects an image to be analyzed in a breeding scene and performs segmentation learning on the image to be analyzed to determine the context of the image to be analyzed.

[0072] An image training learning module: performs image training learning from the image to be analyzed with the determined context, and judges whether there is a shadow, and if there is no shadow, a first result is obtained.

[0073] An edge device supplementary training module: if it is judged that there is a shadow in the image training learning module, supplementary lighting is performed, a picture to be analyzed after supplementary shooting is obtained, image segmentation learning and training learning are performed, and a second result is obtained.

[0074] The image segmentation learning module comprises the following modules:

[0075] Module M1: deploying a camera for a breeding scene in a large-scale livestock breeding department;

[0076] Module M2: collecting an image to be analyzed by the camera, and transmitting the image to be analyzed to a processing server;

[0077] Module M3: the processing server performs image segmentation on the image to be analyzed using a multi-task convolutional neural network, takes the color spots of the segmented image as input, inputs the color spots to the convolutional neural network, and the convolutional neural network labels the pixels;

[0078] Module M4: the picture after pixel marking is divided into a*b rectangular blocks, the RGB values of all pixel points in n corner rectangular blocks are extracted, the number of pixel points with the same RGB value in all pixel points in n corner rectangular blocks is counted, and the RGB value corresponding to the predetermined number is determined as the threshold, wherein a represents the long side and b represents the width;

[0079] Module M5: the RGB mean values of the remaining a*b-n rectangular blocks are calculated, the RGB mean values greater than or equal to the threshold are reserved to obtain C rectangular blocks corresponding to C mean values, the RGB values of the pixel points of the C rectangular blocks are formed into input data, and each pixel is classified by a convolution layer to determine the context of the image to be analyzed.

[0080] The image training learning module comprises the following modules:

[0081] Module M6: each pixel data is converted from two dimensions to one dimension, the covariance matrix of the converted one-dimensional channel is calculated, the eigenvector p and the eigenvalue λ of the covariance matrix are calculated, the conversion is performed according to the formula, and the color random jittering is determined;

[0082] Module M7: color gamut adjustment, a shadow judgment model is generated after color judgment;

[0083] Module M8: if the shadow judgment model judges that there is no shadow, the sliding window judges whether there is a living being, the network randomly extracts a frame from the original picture, learns positive and negative samples, enters an output network, and obtains a first result.

[0084] The edge device supplementary training module comprises the following modules:

[0085] Module M9: an edge device is provided;

[0086] Module M10: semantic analysis is performed by a processor in the edge device, if the shadow judgment model judges that there is a shadow after module M7, the shadow is photographed by a supplementary light, module M2-module M6 is continued, the sliding window judges whether there is a living being, the network randomly extracts a frame from the picture after the supplementary light, learns positive and negative samples, enters an output network, and obtains a second result.

[0087] Module M11: the first result and the second result are compared, a convolution network is learned, and a mean value in T moments is calculated subsequently;

[0088] Module M12: the number of living beings in a*b is summed according to the mean value.

[0089] The application relates to the field of artificial intelligence, in particular to an image recognition method and device for on-site AI in a biological asset vertical field; a deep learning method and an edge-side AI processor device and process for effectively counting a general biological asset represented by beef cattle in a compound scene such as a shadow area; in the method, the principle of light compensation is used for the above-mentioned scene, and mean light compensation is performed on A*B large grid area pixels of an edge-side processor, so that the accuracy of counting operation is realized; on the process, a general processor is provided, and light compensation is performed on the RGB value of a pixel point and threshold comparison is performed.

[0090] Those skilled in the art know that, in addition to implementing the system provided by the application and each device, module and unit thereof in a pure computer readable program code manner, the system provided by the application and each device, module and unit thereof can also be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps to realize the same functions. Therefore, the system provided by the application and each device, module and unit thereof can be considered as a hardware component, and the devices, modules and units included therein for realizing various functions can also be considered as structures in the hardware component; the devices, modules and units for realizing various functions can also be considered as both software modules realizing methods and structures in the hardware component.

[0091] The specific embodiments of the application are described above. It should be understood that the application is not limited to the specific embodiments described above, and various changes or modifications can be made by those skilled in the art within the scope of the claims, which does not affect the essential content of the application. The embodiments of the application and the features in the embodiments can be arbitrarily combined with each other without conflict.

Claims

1. An image recognition method based on biological asset shadow zone inventory count, characterized by, Comprising the following steps: The image segmentation learning step: collecting the image to be analyzed of the breeding scene, and performing segmentation learning on the image to be analyzed to determine the context of the image to be analyzed; The image training learning step: performing image training learning from the image to be analyzed with the determined context, and judging whether there is a shadow, if there is no shadow, a first result is obtained; The edge device supplementary training step: if it is judged that there is a shadow in the image training learning step, supplementary lighting is performed, a supplementary photographed image to be analyzed is obtained, image segmentation learning and training learning are performed, and a second result is obtained; The image segmentation learning step comprises the following steps: Step S1: deploying a camera facing the breeding scene in the livestock large-scale breeding department; Step S2: collecting the image to be analyzed by the camera, and transmitting the image to be analyzed to the processing server; Step S3: the processing server performs image segmentation on the image to be analyzed using a multi-task convolutional neural network, inputs the color spots of the segmented image into the convolutional neural network, and the convolutional neural network labels the pixels; Step S4: the image after pixel labeling is divided into a*b rectangular blocks, the RGB values of all pixel points in n corner rectangular blocks are extracted, the number of pixel points with the same RGB value in all pixel points in the n corner rectangular blocks is counted, and the RGB value corresponding to the predetermined number is determined as the threshold value, wherein a represents the long side and b represents the wide side; Step S5: calculate the RGB mean value of the remaining a*b-n rectangular blocks, and retain the RGB mean value greater than or equal to the threshold value to obtain C rectangular blocks corresponding to C mean values, and form the RGB values of the pixel points of the C rectangular blocks into input data, and input them to the convolutional layer to classify each pixel, and determine the context of the image to be analyzed.

2. The image recognition method based on bioasset shadow zone inventory count of claim 1, wherein, The image training learning step comprises the following steps: Step S6: convert each pixel data from two dimensions to one dimension, calculate the covariance matrix of the converted one-dimensional channel, calculate the eigenvector p and the eigenvalue λ of the covariance matrix, convert according to the formula, and complete the color random jitter judgment; Step S7: color gamut adjustment, generate a shadow judgment model after color judgment; Step S8: if the shadow judgment model judges that there is no shadow, the sliding window judges whether there is a living being, the network randomly extracts the frame from the original image, learns the positive and negative samples, enters the output network, and obtains a first result.

3. The image recognition method for biological asset shadow zone inventory count based on claim 2, characterized in that, The edge device supplementary training step comprises the following steps: Step S9: providing an edge device; Step S10: performing semantic analysis by the processor in the edge device, if the shadow judgment model judges that there is a shadow in step S7, performing supplementary lighting by the supplementary lighting lamp, continuing steps S2-S6, and the sliding window judges whether there is a living being, the network randomly extracts the frame from the image after supplementary lighting, learns the positive and negative samples, enters the output network, and obtains a second result.

4. The image recognition method for biological asset shadow zone inventory count based on claim 3, characterized in that, The edge device supplementary training step further comprises the following steps: Step S11: comparing the first result and the second result, performing convolutional network learning, and subsequently calculating the average value within T time; Step S12: determining the sum of the number of living beings within a*b according to the average value.

5. An image recognition system for biological asset shadow zone inventory count based on, Comprising the following modules: The image segmentation learning module collects an image to be analyzed of a breeding scene and performs segmentation learning on the image to be analyzed to determine a context of the image to be analyzed. The image training learning module performs image training learning from the image to be analyzed with the determined context and judges whether there is a shadow, and if there is no shadow, a first result is obtained. The edge device supplementary training module performs supplementary lighting shooting if it is judged that there is a shadow in the image training learning module, obtains an analyzed image after the supplementary shooting, performs image segmentation learning and training learning, and obtains a second result. The image segmentation learning module includes the following modules: Module M1: deploying a camera for a breeding scene in a large-scale livestock breeding department; Module M2: collecting an analyzed image through the camera and transmitting the analyzed image to a processing server; Module M3: the processing server performs image segmentation on the analyzed image using a multi-task convolutional neural network, takes the color spots of the segmented image as input, inputs the color spots into the convolutional neural network, and the convolutional neural network labels the pixels; Module M4: the image after the pixel labeling is divided into a*b rectangular blocks, the RGB values of all pixel points in n corner rectangular blocks are extracted, the number of pixel points with the same RGB value in all pixel points in the n corner rectangular blocks is counted, and the RGB value corresponding to the predetermined number is determined as a threshold, wherein a represents the long side and b represents the wide side; Module M5: calculating the RGB mean value of the remaining a*b-n rectangular blocks, retaining the RGB mean value greater than or equal to the threshold, obtaining C rectangular blocks corresponding to the C mean values, forming input data of the RGB values of the pixel points of the C rectangular blocks, and inputting the input data to a convolutional layer to classify each pixel and determine the context of the image to be analyzed.

6. The image recognition system for biological asset shadow zone inventory count based on claim 5, characterized in that, The image training learning module includes the following modules: Module M6: converting each pixel data from two dimensions to one dimension, calculating the covariance matrix of the converted one-dimensional channel, calculating the eigenvector p and the eigenvalue λ of the covariance matrix, converting according to the formula, and completing color random jitter judgment; Module M7: color gamut adjustment, generating a shadow judgment model after color judgment; Module M8: if the shadow judgment model judges that there is no shadow, the sliding window judges whether there is a living being, the network randomly extracts a frame from the original image, performs positive and negative sample learning, enters an output network, and obtains a first result.

7. The image recognition system for biological asset shadow zone inventory count based on claim 6, wherein, The edge device supplementary training module includes the following modules: Module M9: providing an edge device; Module M10: performing semantic analysis through a processor in the edge device, if the shadow judgment model judges that there is a shadow after the module M7, performing supplementary lighting shooting through a supplementary lighting lamp, continuing the module M2 to the module M6, the sliding window judges whether there is a living being, the network randomly extracts a frame from the image after the supplementary lighting, performs positive and negative sample learning, enters an output network, and obtains a second result.

8. The image recognition system for biological asset shadow zone inventory count based on claim 7, characterized in that, The edge device supplementary training module further includes the following modules: Module M11: comparing the first result and the second result, performing convolutional network learning, and subsequently calculating an average value within T moments; Module M12: determining the number of living beings within a*b according to the average value and summing the number.

Images