Intelligent pet machine feeding image visual recognition method and system

Abstract

The application discloses a kind of intelligent pet machine's feeding image visual recognition method and system, it is related to feeding identification technical field, including: obtaining multiple feeding quantity, and installing identification camera in pet machine;Using feeding analysis method to obtain full grain identification area and residual grain identification area;When pet machine works, obtain user feeding quantity, and based on real-time image control pet machine to feed and replenish;The application is used to solve the problem in the feeding identification method of existing pet intelligent machine, when eating pet has eating preference to food in grain bowl, such as only eating middle area grain or only eating one side grain, only through timing feeding or radar induction feeding mode can cause overfeeding or small amount feeding, thereby affecting the eating demand of pet.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of feeding recognition, in particular to a feeding image visual recognition method and system of an intelligent pet machine. BACKGROUND

[0002] An intelligent pet machine refers to a pet care and management tool that combines artificial intelligence technology and intelligent device functions. They help pet owners better care for and manage pets through various ways, enhancing the interaction and emotional connection between pets and owners. The feeding methods of intelligent pet machines usually include timed feeding, gravity feeding, metering feeding, and radar sensing feeding.

[0003] The existing feeding recognition method for intelligent pet machines usually improves the feeding method based on the existing feeding method. For example, based on the existing timed feeding or gravity feeding, the outlet of the intelligent pet machine is added, and each outlet is controlled to meet different feeding needs, or the feeding is performed by sensing whether the pet is close. This feeding method is relatively basic and can only feed through existing settings. When the pet has a feeding preference for the food in the food bowl, such as only eating the middle area of the food or only eating the food on one side, the timed feeding or radar sensing feeding method will cause overfeeding or underfeeding, affecting the pet's feeding needs. For example, in the patent application with the publication number CN117409447A, a pet recognition feeding system and method are disclosed, which identifies the pet through image recognition and feeds quantitatively according to the preset feeding instructions. Other improvements in the feeding recognition of intelligent pet machines usually analyze the pet's feeding amount, but still cannot solve the problem of overfeeding or underfeeding when the pet has a feeding preference for the food in the food bowl, such as only eating the middle area of the food or only eating the food on one side, affecting the pet's feeding needs. Therefore, it is necessary to improve the existing feeding recognition method for pet intelligent machines. SUMMARY

[0004] The present application aims to at least partially solve one of the technical problems in the prior art by proposing a feeding image visual recognition method and system of an intelligent pet machine, which solves the problem of overfeeding or underfeeding when the pet has a feeding preference for the food in the food bowl, such as only eating the middle area of the food or only eating the food on one side, affecting the pet's feeding needs.

[0005] To achieve the above object, in a first aspect, the application provides a feeding image visual recognition method of an intelligent pet machine, comprising the following steps:

[0006] Based on the feeding setting of the pet machine, a plurality of feeding amounts of the pet machine are obtained, and a recognition camera is installed in the pet machine; the recognition camera comprises a front camera, a left camera and a right camera, wherein the front camera is used for photographing the top of the grain bowl, and the left camera and the right camera are used for photographing the right side and the left side of the inner wall of the grain bowl on the left side and the right side of the grain outlet respectively.

[0007] Based on the images photographed by the recognition camera, a feeding analysis is performed on each feeding amount using a feeding analysis method, and based on the analysis result, a full grain recognition area and a residual grain recognition area corresponding to each feeding amount are obtained.

[0008] When the pet machine is working, the user-set feeding amount is recorded as a user feeding amount, the full grain recognition area and the residual grain recognition area corresponding to the user feeding amount are obtained, and based on the real-time images photographed by the recognition camera, the pet machine is controlled to feed and supplement.

[0009] Further, based on the feeding setting of the pet machine, a plurality of feeding amounts of the pet machine are obtained, and a recognition camera is installed in the pet machine, comprising:

[0010] The weights of the plurality of feeding amounts in the feeding setting of the pet machine are recorded as pet feeding amounts CT1 to CTn respectively. n wherein n is the number of feeding amounts in the feeding setting of the pet machine; three-dimensional size data of the pet machine is obtained.

[0011] A camera above the grain bowl is installed outside the pet machine, recorded as a front camera, and the orientation of the front camera is adjusted to the center of the inner wall of the grain bowl.

[0012] Further, the installation of the recognition camera in the pet machine further comprises:

[0013] A space coordinate system with a unit of cm is established, recorded as a model building coordinate system, a model corresponding to the pet machine is built in the model building coordinate system based on the three-dimensional size data of the pet machine, and recorded as a pet machine model; the grain outlet and the grain bowl in the pet machine model are recorded as a virtual grain outlet and a virtual grain bowl respectively.

[0014] Based on the left view and the right view of the pet machine model, the positions closest to the virtual grain outlet and meeting the diagonal photographing in the left side surface and the right side surface of the pet machine model are obtained respectively, and recorded as a left mounting point and a right mounting point respectively.

[0015] Further, the diagonal shooting is that, for the left surface of the pet machine, when the line between any one point and the rightmost point in the inner wall of the virtual food bowl does not coincide with the pet machine model in any length, the point is recorded as a point that meets the diagonal shooting; for the right surface of the pet machine, when the line between any one point and the leftmost point in the inner wall of the virtual food bowl does not coincide with the pet machine model in any length, the point is recorded as a point that meets the diagonal shooting;

[0016] The camera is installed in the pet machine based on the positions of the left mounting point and the right mounting point, and is recorded as a left camera and a right camera respectively; the shooting center of the left camera is adjusted to the rightmost side of the inner wall of the food bowl, and the shooting center of the right camera is adjusted to the leftmost side of the inner wall of the food bowl.

[0017] Further, the feeding analysis method includes empty volume analysis method and feeding analysis method, and the empty volume analysis method includes:

[0018] The food bowl of the pet machine is emptied, and the top camera, the left camera and the right camera are used for shooting respectively, and the obtained images are recorded as top panoramic image, left panoramic image and right panoramic image; the centers of the top panoramic image, the left panoramic image and the right panoramic image are recorded as the panoramic centers corresponding to the top panoramic image, the left panoramic image and the right panoramic image;

[0019] Based on image recognition, the images in the top panoramic image, the left panoramic image and the right panoramic image except the inner wall of the food bowl are removed, and the obtained images after removal are recorded as top food image, left food image and right food image; the areas of the top food image, the left food image and the right food image in the top panoramic image, the left panoramic image and the right panoramic image are recorded as top recognition area, left recognition area and right recognition area respectively.

[0020] Further, the empty volume analysis method further includes:

[0021] For any one of the top food image, the left food image and the right food image: based on image ranging, the distances between all points in the image and the camera are obtained, a coordinate system with the unit of cm is established, and is recorded as a ranging analysis coordinate system; the distance corresponding to the panoramic center obtained based on image ranging is recorded as L;

[0022] The origin of the ranging analysis coordinate system and the point (L, 0, 0) are recorded as the point corresponding to the camera and the point corresponding to the panoramic center of the image respectively; based on the positional relationship between the panoramic center and the points in the image except the panoramic center and the positional relationship between the camera and the points in the image except the panoramic center, the points corresponding to all points in the image are obtained in the ranging analysis coordinate system, and the curved surface formed by all points is recorded as an image curved surface; the points in the edge of the image curved surface are connected with the origin in turn, and the obtained closed area is recorded as a feeding relationship area;

[0023] Obtain the feeding relationship region corresponding to the top grain image, the left grain image and the right grain image, and record it as the empty grain relationship region corresponding to the top grain image, the left grain image and the right grain image.

[0024] Further, the feeding analysis method comprises:

[0025] For any pet feeding amount CT, the grain with a weight of the pet feeding amount CT is put into the bowl, the recognition camera is used for shooting, and the feeding relationship region corresponding to the top grain image, the left grain image and the right grain image at this time is obtained based on the empty amount analysis method, the top recognition region, the left recognition region and the right recognition region, and is recorded as the full grain relationship region corresponding to the top grain image, the left grain image and the right grain image.

[0026] Further, the feeding analysis method further comprises:

[0027] The pet eats the grain in the bowl, and after eating once, the recognition camera is used for shooting, and the feeding relationship region corresponding to the top grain image, the left grain image and the right grain image at this time is obtained based on the empty amount analysis method, the top recognition region, the left recognition region and the right recognition region, and is recorded as the residual grain relationship region corresponding to the top grain image, the left grain image and the right grain image, wherein the residual grain relationship region can be obtained by the pet eating the bowl containing the grain with the pet feeding amount CT for multiple times, and the residual grain relationship region is obtained after eating once, and the average value of the feeding relationship region obtained multiple times is recorded as the residual grain relationship region corresponding to the top grain image, the left grain image and the right grain image.

[0028] For any one of the top grain image, the left grain image and the right grain image, the region in the full grain relationship region and the empty grain relationship region which does not coincide is recorded as the full grain recognition region; the region in the residual grain relationship region and the empty grain relationship region which does not coincide is recorded as the residual grain recognition region.

[0029] Obtain the full grain recognition region and the residual grain recognition region corresponding to all pet feeding amounts CT.

[0030] Further, the control of the pet machine based on the real-time image shot by the recognition camera to feed and supplement the grain comprises:

[0031] When the pet machine is working, after the user feeding amount of grain is discharged from the discharge port, the feeding relationship region corresponding to the top grain image, the left grain image and the right grain image is obtained after the pet eats once based on image recognition, and is recorded as the eating relationship region corresponding to the top grain image, the left grain image and the right grain image.

[0032] For any one of the top grain image, left grain image and right grain image, the non-overlapping area of the eating relationship area and the empty grain relationship area corresponding to the image is recorded as an eating identification area; when the volume of the eating identification area is less than or equal to the full grain identification area and greater than or equal to the residual grain identification area, the image is recorded as a grain supplement image; when the eating identification area is less than the residual grain identification area, the image is recorded as a grain throwing image;

[0033] When the number of grain supplement images in the top grain image, the left grain image and the right grain image is greater than or equal to 2, the grain is put into the grain bowl until the grain in the grain bowl is the grain of the user's throwing amount; when the number of grain throwing images in the top grain image, the left grain image and the right grain image is greater than or equal to 2, the grain of the user's throwing amount is put into the grain bowl; in other cases, no grain is put.

[0034] In a second aspect, the application also provides a feeding image visual identification system of an intelligent pet machine, comprising a feeding camera installation module, a pet feeding analysis module and an identification feeding module;

[0035] The feeding camera installation module is used to obtain a plurality of feeding amounts of the pet machine based on the feeding setting of the pet machine, and install an identification camera in the pet machine; the identification camera comprises a front camera, a left camera and a right camera, wherein the front camera is used to take a picture of the top of the grain bowl, and the left camera and the right camera are used to take pictures of the right side and the left side of the inner wall of the grain bowl respectively on the left side and the right side of the grain outlet;

[0036] The pet feeding analysis module is used to analyze the feeding of each feeding amount based on the images taken by the identification camera using the feeding analysis method, and obtain the pet habit parameters corresponding to each feeding amount based on the analysis result; analyze the pet habit parameters, and obtain the pet feeding time corresponding to each feeding amount based on the analysis result;

[0037] The identification feeding module is used to record the user's feeding amount as the user's throwing amount when the pet machine is working, obtain the pet habit parameters and the pet feeding time corresponding to the user's throwing amount, and control the pet machine to feed and supplement based on the real-time images taken by the identification camera.

[0038] The beneficial effects of the present application are as follows: firstly, based on the feeding setting of the pet machine, multiple feeding amounts of the pet machine are obtained, and a recognition camera is installed in the pet machine; then, based on the image captured by the recognition camera, a feeding analysis is performed on each feeding amount using a feeding analysis method, and based on the analysis result, a full food recognition area and a residual food recognition area corresponding to each feeding amount are obtained. The advantages of this are that, by installing the recognition camera and obtaining the full food recognition area and the residual food recognition area through the recognition camera, the pet's eating preferences under different feeding amounts and the image information in the food bowl after the pet eats under different feeding amounts can be obtained, which helps to determine the eating situation of the pet when feeding the pet subsequently, so as to timely feed and replenish the food bowl to meet the eating needs of the pet.

[0039] The present application also records the feeding amount set by the user as the user feeding amount when the pet machine is working, obtains the full food recognition area and the residual food recognition area corresponding to the user feeding amount, and controls the pet machine to feed and replenish based on the real-time image captured by the recognition camera. The advantages of this are that, by feeding and replenishing based on the full food recognition area and the residual food recognition area, the pet's eating preferences can be determined, and when there is little food in the food bowl or it is not conducive to the pet's eating, the food bowl can be timely replenished to avoid the problems of overfeeding or underfeeding caused by conventional feeding methods. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 It is a principle block diagram of the system of the present application;

[0041] Figure 2 It is a step flowchart of the method of the present application;

[0042] Figure 3 It is a position relationship diagram of the virtual food outlet and the virtual food bowl of the present application;

[0043] Figure 4 It is a schematic diagram of the top food image of the present application;

[0044] Figure 5 It is a structural schematic diagram of the electronic device of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0046] Embodiment 1, please refer to Figure 1As shown, the application provides a feeding image visual recognition system of an intelligent pet machine, which comprises a feeding camera installation module, a pet feeding analysis module and a recognition feeding module;

[0047] The feeding camera installation module is used to obtain a plurality of feeding amounts of the pet machine based on the feeding setting of the pet machine, and install a recognition camera in the pet machine; the recognition camera comprises a front camera, a left camera and a right camera, wherein the front camera is used to take a picture of the top of the grain bowl, and the left camera and the right camera are used to take pictures of the right side and the left side of the inner wall of the grain bowl respectively at the left side and the right side of the grain outlet;

[0048] The feeding camera installation module comprises a camera installation unit, which is configured with a camera installation strategy, and the camera installation strategy comprises:

[0049] The weights corresponding to the plurality of feeding amounts in the feeding setting of the pet machine are respectively denoted as pet feeding amounts CT1 to CTn. n Wherein, n is the number of feeding amounts in the feeding setting of the pet machine; three-dimensional size data of the pet machine is obtained;

[0050] In the specific implementation process, the pet feeding amounts CT can be obtained according to the setting of the pet machine or the setting of the user, for example, in one data processing, the pet feeding amounts CT set by the user are 50g, 100g and 200g respectively, then the value of n can be 3, and the pet feeding amounts CT1 to CT3 are 50g, 100g and 200g in turn;

[0051] A camera above the grain bowl is installed outside the pet machine, denoted as a front camera, and the orientation of the front camera is adjusted to the center of the inner wall of the grain bowl;

[0052] A space coordinate system with a unit of cm is established, denoted as a model building coordinate system, a model corresponding to the pet machine is built in the model building coordinate system based on the three-dimensional size data of the pet machine, and denoted as a pet machine model; the grain outlet and the grain bowl in the pet machine model are denoted as a virtual grain outlet and a virtual grain bowl respectively;

[0053] Based on the left view and the right view of the pet machine model, the positions closest to the virtual grain outlet and meeting the diagonal shooting in the left side surface and the right side surface of the pet machine model are obtained respectively, and denoted as a left installation point and a right installation point respectively;

[0054] In the specific implementation process, for example, in one data processing, the positional relationship of the virtual grain outlet and the virtual grain bowl obtained is as follows Figure 3As shown in the figure, CLK is a virtual outlet, and LW is a virtual bowl. Through analysis, it can be found that the positions of the centers of the circles from left to right are a left mounting point, a mounting point of the top camera, and a right mounting point. By obtaining the left mounting point and the right mounting point that satisfy the diagonal shooting, it can be ensured that the images obtained during subsequent shooting and image analysis can analyze all areas in the bowl, thereby making the analysis of the eating habits of the pet more accurate and effective, and further improving the accuracy of feeding and replenishment in the bowl.

[0055] Diagonal shooting: for the left surface of the pet machine, when the line connecting any point and the rightmost point on the inner wall of the virtual bowl does not coincide with the pet machine model at any length, the point is recorded as a point that satisfies the diagonal shooting; for the right surface of the pet machine, when the line connecting any point and the leftmost point on the inner wall of the virtual bowl does not coincide with the pet machine model at any length, the point is recorded as a point that satisfies the diagonal shooting.

[0056] Based on the positions of the left mounting point and the right mounting point, the camera is installed in the pet machine and is recorded as a left camera and a right camera, respectively. The shooting center of the left camera is adjusted to the rightmost side of the inner wall of the bowl, and the shooting center of the right camera is adjusted to the leftmost side of the inner wall of the bowl.

[0057] The pet feeding analysis module is used to analyze each feeding amount using the feeding analysis method based on the images captured by the recognition camera, and obtain the pet habit parameters corresponding to each feeding amount based on the analysis result. The pet habit parameters are analyzed, and the pet feeding time corresponding to each feeding amount is obtained based on the analysis result.

[0058] The feeding analysis method includes empty volume analysis and feeding analysis. The empty volume analysis includes: emptying the bowl of the pet machine, and using the top camera, the left camera, and the right camera to capture images, respectively, and recording the obtained images as a top panoramic image, a left panoramic image, and a right panoramic image. The centers of the top panoramic image, the left panoramic image, and the right panoramic image are recorded as the panoramic centers corresponding to the top panoramic image, the left panoramic image, and the right panoramic image.

[0059] In the specific implementation process, for example, during one data processing, the obtained top panoramic image is as shown in Figure 4 The figure shows that TX1 is the top panoramic image, QZ is the center point of the top panoramic image, and TX2 is the top grain image corresponding to the top panoramic image after image recognition processing. By obtaining the top grain image, the left grain image, and the right grain image, the grain in the bowl can be analyzed from three directions, thereby obtaining the characteristics of the empty bowl state in the empty volume analysis and the characteristics of the full bowl and the remaining grain in the bowl after being eaten by the pet in the feeding analysis when the different pet feeding amounts CT, which helps to control the actual feeding of the pet machine during subsequent analysis.

[0060] Based on image recognition, the images in the top-up panoramic image, the left panoramic image and the right panoramic image except for the inner wall of the grain bowl are removed, and the images obtained after removal are recorded as the top-up grain image, the left grain image and the right grain image respectively. The regions of the top-up grain image, the left grain image and the right grain image in the top-up panoramic image, the left panoramic image and the right panoramic image are recorded as the top-up recognition region, the left recognition region and the right recognition region respectively.

[0061] For any one of the top-up grain image, the left grain image and the right grain image: based on image ranging, the distances between all points in the image and the camera are obtained, a coordinate system with the unit of cm is established, and is recorded as the ranging analysis coordinate system; the distance corresponding to the panoramic center obtained based on image ranging is recorded as L;

[0062] The origin of the ranging analysis coordinate system and the point (L, 0, 0) are recorded as the point corresponding to the camera and the point corresponding to the panoramic center of the image respectively. Based on the positional relationship between the panoramic center and the points in the image except for the panoramic center and the positional relationship between the camera and the points in the image except for the panoramic center, the points corresponding to all points in the image are obtained in the ranging analysis coordinate system, and the curved surface formed by all points is recorded as the image curved surface. The points in the edge of the image curved surface are sequentially connected with the origin, and the closed region obtained is recorded as the feeding relationship region.

[0063] In the specific implementation process, by obtaining the image curved surface, the positional relationship between the orientation of the camera and the grain in the grain bowl can be obtained, and by obtaining the feeding relationship region, the spatial features formed by the orientation of the camera and the grain in the grain bowl can be obtained, which is helpful for analyzing the feeding situation of the pet in the grain bowl based on the feeding relationship region in subsequent analysis.

[0064] The feeding relationship regions corresponding to the top-up grain image, the left grain image and the right grain image are obtained, and are recorded as the empty grain relationship regions corresponding to the top-up grain image, the left grain image and the right grain image.

[0065] The feeding analysis method includes: for any one pet feeding amount CT, feeding grain with a weight of the pet feeding amount CT into the grain bowl, using the recognition camera to take pictures, and based on the empty amount analysis method, the top-up recognition region, the left recognition region and the right recognition region, the feeding relationship regions corresponding to the top-up grain image, the left grain image and the right grain image at this time are obtained, and are recorded as the full grain relationship regions corresponding to the top-up grain image, the left grain image and the right grain image.

[0066] The pet eats the food in the food bowl, and after one feeding, the recognition camera is used for shooting, and based on the empty volume analysis method, the uppermost recognition area, the left recognition area and the right recognition area, the feeding relationship area corresponding to the uppermost food image, the left food image and the right food image is obtained at this time, and is recorded as the residual food relationship area corresponding to the uppermost food image, the left food image and the right food image, wherein when obtaining the residual food relationship area, the pet can feed the food bowl containing the food amount of the pet feeding amount CT multiple times, and the residual food relationship area is obtained after one feeding, and the average value of the feeding relationship area obtained multiple times is recorded as the residual food relationship area corresponding to the uppermost food image, the left food image and the right food image;

[0067] For any one of the uppermost food image, the left food image and the right food image, the area in the full food relationship area corresponding to the image and the empty food relationship area which is not coincident is recorded as the full food recognition area; the area in the residual food relationship area corresponding to the image and the empty food relationship area which is not coincident is recorded as the residual food recognition area;

[0068] The full food recognition area and the residual food recognition area corresponding to all pet feeding amounts CT are obtained.

[0069] The feeding recognition module is used to record the user feeding amount set by the user as the user feeding amount when the pet machine is working, obtain the pet habit parameters and the pet feeding time corresponding to the user feeding amount, and control the pet machine to feed and supplement based on the real-time image shot by the recognition camera; the feeding recognition module includes a feeding recognition unit, and the feeding recognition unit is configured with a feeding recognition strategy, and the feeding recognition strategy includes:

[0070] When the pet machine is working, after the user feeding amount of food is discharged from the discharge port, the feeding relationship area corresponding to the uppermost food image, the left food image and the right food image is obtained after the pet feeds once based on image recognition, and is recorded as the feeding relationship area corresponding to the uppermost food image, the left food image and the right food image;

[0071] For any one of the uppermost food image, the left food image and the right food image, the area in the feeding relationship area corresponding to the image and the empty food relationship area which is not coincident is recorded as the feeding recognition area; when the volume of the feeding recognition area is less than or equal to the full food recognition area and greater than or equal to the residual food recognition area, the image is recorded as a supplement food image; when the feeding recognition area is smaller than the residual food recognition area, the image is recorded as a feeding food image;

[0072] In the specific implementation process, when the food recognition area is less than or equal to the full food recognition area and greater than or equal to the residual food recognition area, it indicates that the pet has not completed a complete meal at this time, and the food in the food bowl is not full but more, so the food in the food bowl can be supplemented; when the food recognition area is less than the residual food recognition area, it indicates that the pet has completed a complete meal at this time, and the food in the food bowl is not full but less, and the food in the food bowl should be fed once to meet the pet's feeding demand;

[0073] When the number of the food supplement images in the front-up food image, the left food image and the right food image is greater than or equal to 2, the food is fed into the food bowl until the food in the food bowl is the user's feeding amount; when the number of the food feeding images in the front-up food image, the left food image and the right food image is greater than or equal to 2, the user's feeding amount of food is fed into the food bowl; and other cases do not feed the food.

[0074] In the specific implementation process, by feeding and supplementing based on the full food recognition area and the residual food recognition area, the food bowl can be supplemented in time when the food in the food bowl is less or not conducive to the pet's feeding on the premise of determining the pet's feeding preference, so as to avoid the problems of overfeeding or small amount of feeding caused by the conventional feeding method.

[0075] Embodiment 2, please refer to Figure 2 As shown in the figure, the application also provides a feeding image visual recognition method of an intelligent pet machine, which comprises the following steps:

[0076] Step S1, based on the feeding setting of the pet machine, a plurality of feeding amounts of the pet machine are obtained, and a recognition camera is installed in the pet machine; the recognition camera comprises a front camera, a left camera and a right camera, wherein the front camera is used for photographing the front of the food bowl, and the left camera and the right camera are used for photographing the right side and the left side of the inner wall of the food bowl on the left side and the right side of the food outlet respectively;

[0077] Step S1 comprises: step S101, the weights corresponding to the plurality of feeding amounts in the feeding setting of the pet machine are respectively recorded as pet feeding amount CT1 to pet feeding amount CTn; and step S102, the three-dimensional size data of the pet machine is obtained. n Wherein, n is the number of the feeding amount in the feeding setting of the pet machine; the three-dimensional size data of the pet machine is obtained.

[0078] Step S102, a camera above the food bowl is installed outside the pet machine, which is recorded as a front-up camera, and the orientation of the front-up camera is adjusted to the center of the inner wall of the food bowl;

[0079] Step S103, a space coordinate system with a unit of cm is established, which is recorded as a model building coordinate system, a model corresponding to the pet machine is built in the model building coordinate system based on the three-dimensional size data of the pet machine, and is recorded as a pet machine model; the food outlet and the food bowl in the pet machine model are recorded as a virtual food outlet and a virtual food bowl respectively.

[0080] Step S104, based on the left view and the right view of the pet machine model, the positions closest to the virtual feeding bowl in the left surface and the right surface of the pet machine model are obtained respectively, and are recorded as the left mounting point and the right mounting point respectively;

[0081] Step S105, the diagonal shooting is: step S1051, for the left surface of the pet machine, when the line of any one point and the rightmost point in the inner wall of the virtual feeding bowl does not coincide with the pet machine model in any length, the point is recorded as the point that satisfies the diagonal shooting; for the right surface of the pet machine, when the line of any one point and the leftmost point in the inner wall of the virtual feeding bowl does not coincide with the pet machine model in any length, the point is recorded as the point that satisfies the diagonal shooting;

[0082] Step S1052, based on the positions of the left mounting point and the right mounting point, the camera is installed in the pet machine, and is recorded as the left camera and the right camera respectively; the shooting center of the left camera is adjusted to the rightmost side of the inner wall of the feeding bowl, and the shooting center of the right camera is adjusted to the leftmost side of the inner wall of the feeding bowl.

[0083] Step S2, based on the image captured by the recognition camera, the feeding analysis method is used to analyze each feeding amount, and the full feeding recognition area and the residual food recognition area corresponding to each feeding amount are obtained based on the analysis result;

[0084] The feeding analysis method includes empty amount analysis method and feeding analysis method, and the empty amount analysis method includes: step S2011, empty the feeding bowl of the pet machine, and use the top camera, the left camera and the right camera to capture images respectively, and record the obtained images as top panoramic image, left panoramic image and right panoramic image; the center of the top panoramic image, the left panoramic image and the right panoramic image is recorded as the panoramic center corresponding to the top panoramic image, the left panoramic image and the right panoramic image;

[0085] Step S2012, based on image recognition, the images in the top panoramic image, the left panoramic image and the right panoramic image except the inner wall of the feeding bowl are removed, and the images obtained after removal are recorded as top grain image, left grain image and right grain image respectively, and the areas of the top grain image, the left grain image and the right grain image in the top panoramic image, the left panoramic image and the right panoramic image are recorded as the top recognition area, the left recognition area and the right recognition area respectively;

[0086] Step S2013, for any one of the top grain image, the left grain image and the right grain image: based on image ranging, the distance between all points in the image and the camera is obtained, a space coordinate system with the unit of cm of the coordinate axis is established, and is recorded as the ranging analysis coordinate system; the distance corresponding to the panoramic center obtained based on image ranging is recorded as L;

[0087] Step S2014, the origin of the coordinate system of the ranging analysis coordinate system and the point (L, 0, 0) are respectively recorded as the point corresponding to the camera and the point corresponding to the panoramic center of the image; based on the positional relationship between the panoramic center and the points in the image other than the panoramic center and the positional relationship between the camera and the points in the image other than the panoramic center, the points corresponding to all the points in the image are obtained in the ranging analysis coordinate system, and the curved surface formed by all the points is recorded as the image curved surface; the points in the edge of the image curved surface are sequentially connected with the origin of the coordinate, and the closed area obtained is recorded as the feeding relationship area;

[0088] Step S2015, the feeding relationship areas corresponding to the top grain image, the left grain image and the right grain image are obtained, and are recorded as the empty grain relationship areas corresponding to the top grain image, the left grain image and the right grain image.

[0089] The feeding analysis method includes: step S2021, for any one pet feeding amount CT, feeding the grain with a weight of the pet feeding amount CT into the bowl, using the recognition camera to take pictures, and based on the empty amount analysis method, the top recognition area, the left recognition area and the right recognition area, the feeding relationship areas corresponding to the top grain image, the left grain image and the right grain image at this time are obtained, and are recorded as the full grain relationship areas corresponding to the top grain image, the left grain image and the right grain image;

[0090] Step S2022, the pet eats the grain in the bowl, and after eating once, the recognition camera is used to take pictures, and based on the empty amount analysis method, the top recognition area, the left recognition area and the right recognition area, the feeding relationship areas corresponding to the top grain image, the left grain image and the right grain image at this time are obtained, and are recorded as the residual grain relationship areas corresponding to the top grain image, the left grain image and the right grain image, wherein the residual grain relationship areas can be obtained by the pet eating the grain bowl containing the grain with the pet feeding amount CT for multiple times, and the residual grain relationship areas are obtained after eating once, and the average value of the feeding relationship areas obtained multiple times is recorded as the residual grain relationship areas corresponding to the top grain image, the left grain image and the right grain image;

[0091] Step S2023, for any one of the top grain image, the left grain image and the right grain image, the non-overlapping area between the full grain relationship area and the empty grain relationship area corresponding to the image is recorded as the full grain recognition area; the non-overlapping area between the residual grain relationship area and the empty grain relationship area corresponding to the image is recorded as the residual grain recognition area;

[0092] Step S2024, the full grain recognition area and the residual grain recognition area corresponding to all the pet feeding amounts CT are obtained.

[0093] Step S3, when the pet machine is working, record the user set feeding amount as user feeding amount, obtain the full grain identification area and the residual grain identification area corresponding to the user feeding amount, and control the pet machine to feed and supplement based on the real-time image captured by the identification camera;

[0094] Step S3 includes: step S301, when the pet machine is working, after the user feeding amount of grain is discharged from the discharge port, based on image recognition after the pet eats once, obtain the feeding relationship area corresponding to the top grain image, the left grain image and the right grain image, and record it as the eating relationship area corresponding to the top grain image, the left grain image and the right grain image;

[0095] Step S302, for any one of the top grain image, the left grain image and the right grain image, record the non-overlapping area between the eating relationship area corresponding to the image and the empty grain relationship area as the eating identification area; when the volume of the eating identification area is less than or equal to the full grain identification area and greater than or equal to the residual grain identification area, record the image as a grain supplement image; when the eating identification area is less than the residual grain identification area, record the image as a grain feeding image;

[0096] Step S303, when the number of grain supplement images in the top grain image, the left grain image and the right grain image is greater than or equal to 2, feed the grain to the grain bowl until the grain in the grain bowl is the user feeding amount of grain; when the number of grain feeding images in the top grain image, the left grain image and the right grain image is greater than or equal to 2, feed the user feeding amount of grain to the grain bowl; in other cases, do not feed the grain.

[0097] Embodiment 3, please refer to Figure 5 , Figure 5 An example of a structural diagram of an electronic device is shown, which can include: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus. The memory stores computer readable instructions, and the processor can call the instructions in the memory, when the computer readable instructions are executed by the processor, run the steps in a kind of intelligent pet machine feeding image visual identification method, to realize the following functions: first, based on the feeding setting of pet machine, obtain multiple feeding amounts of pet machine, and install identification camera in pet machine;Then based on the image captured by the identification camera, use feeding analysis method to analyze feeding for each feeding amount, and based on the analysis result, obtain the full grain identification area and the residual grain identification area corresponding to each feeding amount;Finally, when the pet machine is working, record the user set feeding amount as user feeding amount, obtain the full grain identification area and the residual grain identification area corresponding to the user feeding amount, and control the pet machine to feed and supplement based on the real-time image captured by the identification camera.

[0098] In addition, the logic instructions in the memory described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0099] In embodiment 4, the present application also provides a computer readable storage medium, and the present application provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the image visual recognition method for feeding of the intelligent pet machine are run to realize the following functions: first, based on the feeding setting of the pet machine, a plurality of feeding amounts of the pet machine are obtained, and a recognition camera is installed in the pet machine; then, using the feeding analysis method based on the images captured by the recognition camera, feeding analysis is performed on each feeding amount, and based on the analysis result, a full grain recognition area and a residual grain recognition area corresponding to each feeding amount are obtained; finally, when the pet machine is working, the user set feeding amount is recorded as a user feeding amount, the full grain recognition area and the residual grain recognition area corresponding to the user feeding amount are obtained, and based on the real-time images captured by the recognition camera, the pet machine is controlled to feed and supplement.

[0100] Through the description of the above embodiments, the embodiments of the present application can be provided as a method, a system or a computer program product. Based on such understanding, the above technical solutions essentially or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments.

[0101] In the embodiments provided in this application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of systems, modules and units can be electrical, mechanical or other forms.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A feeding image visual recognition method for an intelligent pet machine, characterized in that: The steps include: Based on the feeding setting of the pet machine, multiple feeding amounts of the pet machine are obtained, and an identification camera is installed in the pet machine; the identification camera includes an upper camera, a left camera and a right camera, wherein the upper camera is used to shoot the top of the grain bowl, and the left camera and the right camera are used to shoot the right and left sides of the inner wall of the grain bowl on the left and right sides of the grain outlet respectively; Based on the images captured by the recognition camera, a feeding analysis method is used to analyze the feeding amount for each feeding amount, and based on the analysis results, the full grain identification area and residual grain identification area corresponding to each feeding amount are obtained; When the pet machine is working, the feeding amount set by the user is recorded as the user feeding amount, the full grain recognition area and the residual grain recognition area corresponding to the user feeding amount are obtained, and the pet machine is controlled to feed and refill based on the real-time image captured by the recognition camera; Feeding analysis method includes empty quantity analysis method and feeding analysis method. Empty quantity analysis method includes: Empty the food bowl of the pet machine and use the upper camera, left camera and right camera to take pictures respectively, and record the obtained images as the upper panoramic image, left panoramic image and right panoramic image; record the centers of the upper panoramic image, left panoramic image and right panoramic image as the panoramic centers corresponding to the upper panoramic image, left panoramic image and right panoramic image; Based on image recognition, the images other than the inner wall of the grain bowl are eliminated from the upper panoramic image, the left panoramic image, and the right panoramic image, and the images obtained after the elimination are recorded as the upper grain image, the left grain image, and the right grain image, respectively. The areas of the upper grain image, the left grain image, and the right grain image in the upper panoramic image, the left panoramic image, and the right panoramic image, respectively, are recorded as the upper recognition area, the left recognition area, and the right recognition area; Void analysis also includes: For any of the top grain image, left grain image, and right grain image: obtain the distance between all points in the image and the camera based on image ranging, establish a spatial coordinate system with coordinate axes in units of cm, and record it as the ranging analysis coordinate system; record the distance corresponding to the panoramic center obtained based on image ranging as L; The coordinate origin of the distance measurement analysis coordinate system and the point (L, 0, 0) are recorded as the point corresponding to the camera and the point corresponding to the panoramic center of the image, respectively. Based on the positional relationship between the panoramic center and points in the image other than the panoramic center, and the positional relationship between the camera and points in the image other than the panoramic center, the points corresponding to all points in the image are obtained in the distance measurement analysis coordinate system, and the surface formed by all points is recorded as the image surface. The points on the edge of the image surface are sequentially connected to the coordinate origin, and the resulting closed area is recorded as the feeding relationship area. Obtain feeding relationship areas corresponding to the front grain loading image, the left grain image, and the right grain image, and record them as empty grain relationship areas corresponding to the front grain loading image, the left grain image, and the right grain image; Feed analysis methods include: For any pet feeding amount CT, feed equal to the pet feeding amount CT is put into the food bowl, and the recognition camera is used to shoot the image. Based on the empty amount analysis method, the upper recognition area, the left recognition area, and the right recognition area, the feeding relationship areas corresponding to the upper grain image, the left grain image, and the right grain image are obtained, and recorded as the full grain relationship areas corresponding to the upper grain image, the left grain image, and the right grain image; Feed analysis also includes: The pet eats food from the food bowl and uses a recognition camera to photograph the food after eating once. Based on the empty amount analysis method, the upper recognition area, the left recognition area, and the right recognition area, the feeding relationship areas corresponding to the upper food image, the left food image, and the right food image are obtained, and recorded as the residual food relationship areas corresponding to the upper food image, the left food image, and the right food image. When obtaining the residual food relationship areas, the pet can eat from the food bowl with a food amount equal to the pet feeding amount CT multiple times, and obtain the residual food relationship areas after eating once. The average value of the feeding relationship areas obtained multiple times is recorded as the residual food relationship area corresponding to the upper food image, the left food image, and the right food image. For any one of the grain loading image, the left grain image, and the right grain image, the area corresponding to the image where the full grain relationship area and the empty grain relationship area do not overlap is recorded as the full grain recognition area; the area corresponding to the image where the residual grain relationship area and the empty grain relationship area do not overlap is recorded as the residual grain recognition area; Obtain the full food identification area and residual food identification area corresponding to the feed amount CT of all pets.

2. The method for visually recognizing feeding images of an intelligent pet machine according to claim 1, characterized in that: Acquiring multiple feeding amounts of the pet machine based on the feeding settings of the pet machine and installing a recognition camera in the pet machine include: The weights corresponding to the multiple feeding amounts in the feeding setting of the pet machine are recorded as pet feeding amount CT1 to pet feeding amount CT n , where n is the number of feeding amounts set in the pet machine; obtaining the three-dimensional size data of the pet machine; Install a camera directly above the food bowl on the outside of the pet machine, which is called the top camera. Adjust the direction of the top camera to the center of the inner wall of the food bowl.

3. The method for visually recognizing feeding images of an intelligent pet machine according to claim 2, characterized in that: Installing an identification camera in a pet machine also includes: Establish a spatial coordinate system with coordinate axes in centimeters, and record it as the model building coordinate system. Build a model of the pet machine within the model building coordinate system based on the 3D dimension data of the pet machine, and record it as the pet machine model. Record the food outlet and food bowl in the pet machine model as the virtual food outlet and virtual food bowl, respectively. Based on the left view and right view of the pet machine model, the positions on the left surface and right surface of the pet machine model that are closest to the virtual food outlet and meet the requirements of diagonal shooting are obtained respectively, and are recorded as the left installation point and the right installation point respectively.

4. The method for visually recognizing feeding images of an intelligent pet machine according to claim 3, characterized in that: Diagonal shooting means: for the left surface of the pet machine, when the line connecting any point with the rightmost point on the inner wall of the virtual food bowl does not overlap with the pet machine model at any length, the point is recorded as a point that meets the requirements for diagonal shooting; for the right surface of the pet machine, when the line connecting any point with the leftmost point on the inner wall of the virtual food bowl does not overlap with the pet machine model at any length, the point is recorded as a point that meets the requirements for diagonal shooting; Install cameras in the pet machine based on the positions of the left installation point and the right installation point, and record them as the left camera and the right camera respectively; adjust the shooting center of the left camera to the rightmost side of the inner wall of the food bowl, and adjust the shooting center of the right camera to the leftmost side of the inner wall of the food bowl.

5. The method for visually recognizing feeding images of an intelligent pet machine according to claim 4, characterized in that: The real-time image captured by the recognition camera controls the pet machine to feed and refill the feed, including: When the pet machine is working, after the user's input amount of food is released from the discharge port, based on image recognition, after the pet eats once, the feeding relationship area corresponding to the front food image, the left food image, and the right food image is obtained and recorded as the feeding relationship area corresponding to the front food image, the left food image, and the right food image; For any of the images of the front grain loading image, the left grain image, and the right grain image, the area of ​​the image corresponding to the feeding relationship area that does not overlap with the empty grain relationship area is recorded as the feeding recognition area; when the volume of the feeding recognition area is less than or equal to the full grain recognition area and greater than or equal to the residual grain recognition area, the image is recorded as the feeding image; when the feeding recognition area is smaller than the residual grain recognition area, the image is recorded as the feeding image; When the grain replenishment image in the positive grain loading image, the left grain image and the right grain image is greater than or equal to 2, grain is added to the grain bowl until the grain in the grain bowl reaches the amount of grain added by the user; when the grain feeding image in the positive grain loading image, the left grain image and the right grain image is greater than or equal to 2, the grain feeding amount added by the user is added to the grain bowl; in other cases, grain is not added.

6. A feeding image visual recognition system for an intelligent pet machine, used to implement the feeding image visual recognition method for an intelligent pet machine according to any one of claims 1 to 5, characterized in that: It includes feeding camera installation module, pet feeding analysis module and feeding identification module; The feeding camera installation module is used to obtain multiple feeding amounts of the pet machine based on the feeding settings of the pet machine, and install an identification camera in the pet machine; The recognition camera includes an upper camera, a left camera, and a right camera. The upper camera is used to shoot the top of the grain bowl, and the left camera and the right camera are used to shoot the right and left sides of the inner wall of the grain bowl on the left and right sides of the grain outlet respectively. The pet feeding analysis module is used to perform feeding analysis on each feeding amount using a feeding analysis method based on the image captured by the recognition camera, and obtain the pet habit parameters corresponding to each feeding amount based on the analysis results; analyze the pet habit parameters, and obtain the pet feeding timing corresponding to each feeding amount based on the analysis results; The feeding recognition module is used to record the feeding amount set by the user as the user feeding amount when the pet machine is working, obtain the pet habit parameters and pet feeding timing corresponding to the user feeding amount, and control the pet machine to feed and refill based on the real-time image taken by the recognition camera.

Images