Feeding image visual identification method and system of intelligent pet machine

Abstract

The invention discloses a feeding image visual identification method and system of an intelligent pet machine, and relates to the technical field of feeding identification, and the method comprises the steps: obtaining a plurality of feeding amounts, and installing an identification camera in the pet machine; obtaining a full grain identification area and a residual grain identification area by using a feeding analysis method; when the pet machine works, the user feeding amount is obtained, and the pet machine is controlled to feed and supplement feed based on the real-time image; according to the feeding identification method of the intelligent pet machine, the problems that in an existing feeding identification method of the intelligent pet machine, when a feeding pet has feeding preference for food in a food bowl, for example, only food in a middle area is fed or only food on one side is fed, excessive feeding or small feeding can be caused only through timing feeding or radar induction feeding and other modes, and the feeding efficiency is poor are solved. Therefore, the feeding demand of the pet is influenced.

Description

Technical Field

[0001] The present invention relates to the technical field of feeding recognition, and specifically to a feeding image visual recognition method and system for an intelligent pet machine. Background Art

[0002] An intelligent pet machine refers to a pet care and management tool that combines artificial intelligence technology and intelligent device functions; they help pet owners better care for and manage their pets in various ways, enhancing the interaction and emotional connection between the pet and the owner; the feeding methods of intelligent pet machines usually include timed feeding, gravity feeding, metered feeding, radar induction feeding, etc.

[0003] Existing methods for feeding recognition in intelligent pet machines usually improve feeding on the basis of existing feeding methods. For example, on the basis of existing timed feeding or gravity feeding, by adding a discharge port in the intelligent pet machine and controlling each discharge port to achieve the problem of meeting different feeding requirements, or by sensing whether the pet is approaching for feeding. This feeding method is relatively basic and can only feed according to existing settings. When the feeding pet has a feeding preference for the food in the food bowl, such as only eating the food in the middle area or only eating the food on one side, only feeding by timed feeding or radar induction feeding will cause problems of overfeeding or underfeeding, thus affecting the pet's feeding needs. For example, in the patent application with the publication number CN117409447A, a pet recognition feeding system and method are disclosed. This solution uses image recognition to identify the pet and quantitatively feeds according to a preset feeding instruction. Other improvements in feeding recognition for intelligent pet machines are usually improvements after analyzing the pet's food intake. They still cannot solve the problem that when the feeding pet has a feeding preference for the food in the food bowl, such as only eating the food in the middle area or only eating the food on one side, only feeding by timed feeding or radar induction feeding will cause overfeeding or underfeeding, thus affecting the pet's feeding needs. In view of this, it is necessary to improve the existing feeding recognition method for pet intelligent machines. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems in the existing technology to some extent. By proposing a feeding image visual recognition method and system for an intelligent pet machine, it is used to solve the problem that in the existing feeding recognition method for pet intelligent machines, when the feeding pet has a feeding preference for the food in the food bowl, such as only eating the food in the middle area or only eating the food on one side, only feeding by timed feeding or radar induction feeding will cause overfeeding or underfeeding, thus affecting the pet's feeding needs.

[0005] To achieve the above object, in a first aspect, the present application provides a feeding image vision recognition method for an intelligent pet machine, including the following steps:

[0006] Based on the feeding settings of the pet machine, obtain multiple feeding amounts of the pet machine, and install an identification camera in the pet machine; the identification camera includes a front camera, a left camera, and a right camera. Among them, the front upper camera is used to take pictures directly above the food bowl, and the left camera and the right camera are respectively used to take pictures of the right side and the left side of the inner wall of the food bowl on the left and right sides of the food outlet.

[0007] Based on the images taken by the identification camera, use the feeding analysis method to analyze each feeding amount, and obtain the full food recognition area and the residual food recognition area corresponding to each feeding amount based on the analysis results.

[0008] When the pet machine is working, record the feeding amount set by the user as the user feeding amount, obtain the full food recognition area and the residual food recognition area corresponding to the user feeding amount, and control the pet machine to feed and refill based on the real-time images taken by the identification camera.

[0009] Further, obtaining multiple feeding amounts of the pet machine based on the feeding settings of the pet machine and installing an identification camera in the pet machine includes:

[0010] Respectively record the weights corresponding to the multiple feeding amounts in the feeding settings of the pet machine as pet feeding amount CT1 to pet feeding amount CT n , where n is the number of feeding amounts in the feeding settings of the pet machine; obtain the three-dimensional size data of the pet machine.

[0011] Install a camera directly above the food bowl outside the pet machine, denoted as the front upper camera, and adjust the orientation of the front upper camera to the center of the inner wall of the food bowl.

[0012] Further, installing an identification camera in the pet machine further includes:

[0013] Establish a spatial coordinate system with the unit of each axis being cm, denoted as the model building coordinate system. Based on the three-dimensional size data of the pet machine, build a corresponding model of the pet machine in the model building coordinate system, denoted as the pet machine model; denote the food outlet and the food bowl in the pet machine model as the virtual food outlet and the virtual food bowl respectively.

[0014] Based on the left view and the right view of the pet machine model, obtain the positions on the left surface and the right surface of the pet machine model that are closest to the virtual food outlet and satisfy diagonal shooting, and denote them as the left installation point and the right installation point respectively.

[0015] Further, the diagonal shooting is as follows: for the left surface of the pet machine, when the line connecting any point to the rightmost point on the inner wall of the virtual food bowl does not coincide with the pet machine model for any length, this point is recorded as a point satisfying diagonal shooting; for the right surface of the pet machine, when the line connecting any point to the leftmost point on the inner wall of the virtual food bowl does not coincide with the pet machine model for any length, this point is recorded as a point satisfying diagonal shooting.

[0016] Install cameras in the pet machine based on the positions of the left installation point and the right installation point, and denote them as the left camera and the right camera respectively; adjust the shooting center of the left camera to the rightmost side of the inner wall of the food bowl, and adjust the shooting center of the right camera to the leftmost side of the inner wall of the food bowl.

[0017] Further, the feeding analysis method includes empty quantity analysis and feeding quantity analysis. The empty quantity analysis includes:

[0018] Empty the food bowl of the pet machine, and use the top camera, the left camera, and the right camera to take pictures respectively, and denote the obtained images as the top panoramic image, the left panoramic image, and the right panoramic image; denote the centers of the top panoramic image, the left panoramic image, and the right panoramic image as the panoramic centers corresponding to the top panoramic image, the left panoramic image, and the right panoramic image respectively.

[0019] Based on image recognition, remove the images other than the inner wall of the food bowl in the top panoramic image, the left panoramic image, and the right panoramic image, and denote the obtained images as the top food image, the left food image, and the right food image respectively. Denote the regions of the top food image, the left food image, and the right food image in the top panoramic image, the left panoramic image, and the right panoramic image as the top recognition region, the left recognition region, and the right recognition region respectively.

[0020] Further, the empty quantity analysis method also includes:

[0021] For any one of the top food image, the left food image, and the right food image: obtain the distances between all points in the image and the camera based on image ranging, establish a spatial coordinate system with the unit of the coordinate axis being cm, and denote it as the ranging analysis coordinate system; denote the distance corresponding to the panoramic center obtained based on image ranging as L.

[0022] Denote the coordinate origin of the ranging analysis coordinate system and the point (L, 0, 0) as the point corresponding to the camera and the point corresponding to the panoramic center of the image respectively; based on the positional relationship between the panoramic center and the points other than the panoramic center in the image and the positional relationship between the camera and the points other than the panoramic center in the image, obtain the points corresponding to all points in the image within the ranging analysis coordinate system, and denote the surface formed by all points as the image surface; connect the points on the edge of the image surface to the coordinate origin in sequence, and denote the obtained closed region as the feeding relationship region.

[0023] Obtain the feeding relationship areas corresponding to the top grain image, left grain image, and right grain image, and record them as the empty grain relationship areas corresponding to the top grain image, left grain image, and right grain image.

[0024] Furthermore, the feeding analysis method includes:

[0025] For any pet feeding amount CT, put the grain with a weight equal to the pet feeding amount CT into the food bowl, take a picture using the recognition camera, and obtain the feeding relationship areas corresponding to the top grain image, left grain image, and right grain image at this time based on the empty amount analysis method, top recognition area, left recognition area, and right recognition area, and record them as the full grain relationship areas corresponding to the top grain image, left grain image, and right grain image.

[0026] Furthermore, the feeding analysis method also includes:

[0027] Let the pet eat the grain in the food bowl, take a picture using the recognition camera after one feeding, and obtain the feeding relationship areas corresponding to the top grain image, left grain image, and right grain image at this time based on the empty amount analysis method, top recognition area, left recognition area, and right recognition area, and record them as the remaining grain relationship areas corresponding to the top grain image, left grain image, and right grain image. Among them, when obtaining the remaining grain relationship areas, the pet can eat from the food bowl with a grain content equal to the pet feeding amount CT multiple times, and obtain the remaining grain relationship areas after one feeding, and use the average value of the feeding relationship areas obtained multiple times as the remaining grain relationship areas corresponding to the top grain image, left grain image, and right grain image;

[0028] For any one of the top grain image, left grain image, and right grain image, record the non - overlapping area between the full grain relationship area corresponding to the image and the empty grain relationship area as the full grain recognition area; record the non - overlapping area between the remaining grain relationship area corresponding to the image and the empty grain relationship area as the remaining grain recognition area;

[0029] Obtain the full grain recognition areas and remaining grain recognition areas corresponding to all pet feeding amounts CT.

[0030] Furthermore, controlling the pet machine to feed and replenish based on the real - time image captured by the recognition camera includes:

[0031] When the pet machine is working, after releasing the amount of grain put by the user at the discharge port, based on image recognition, after the pet has one feeding, obtain the feeding relationship areas corresponding to the top grain image, left grain image, and right grain image, and record them as the feeding relationship areas corresponding to the top grain image, left grain image, and right grain image;

[0032] For any one of the front grain feeding image, left grain feeding image, and right grain feeding image, the non-overlapping area between the feeding relationship area corresponding to the image and the empty grain relationship area is denoted as the feeding recognition area; when the volume of the feeding recognition area is less than or equal to the full grain recognition area and greater than or equal to the remaining grain recognition area, the image is denoted as a supplementary grain image; when the feeding recognition area is less than the remaining grain recognition area, the image is denoted as a grain feeding image;

[0033] When the number of supplementary grain images among the front grain feeding image, left grain feeding image, and right grain feeding image is greater than or equal to 2, grain is fed into the food bowl until the grain in the food bowl is the amount of grain set by the user; when the number of grain feeding images among the front grain feeding image, left grain feeding image, and right grain feeding image is greater than or equal to 2, the amount of grain set by the user is fed into the food bowl; in other cases, no grain is fed.

[0034] In a second aspect, the present application also provides a visual recognition system for the feeding image of an intelligent pet machine, including a feeding camera installation module, a pet feeding analysis module, and a recognition feeding module;

[0035] The feeding camera installation module is used to obtain multiple feeding amounts of the pet machine based on the feeding settings of the pet machine and install a recognition camera in the pet machine; the recognition camera includes a front camera, a left camera, and a right camera. Among them, the front upper camera is used to take pictures directly above the food bowl, and the left camera and the right camera are used to take pictures of the right side and the left side of the inner wall of the food bowl on the left and right sides of the grain outlet respectively;

[0036] The pet feeding analysis module is used to perform feeding analysis on each feeding amount using the feeding analysis method based on the images taken by the recognition camera, and obtain the pet habit parameters corresponding to each feeding amount based on the analysis results; analyze the pet habit parameters, and obtain the pet feeding timing corresponding to each feeding amount based on the analysis results;

[0037] The recognition feeding module is used to, when the pet machine is working, record the feeding amount set by the user as the user-set amount, obtain the pet habit parameters and the pet feeding timing corresponding to the user-set amount, and control the pet machine to feed and supplement grain based on the real-time images taken by the recognition camera.

[0038] Advantages of the present invention: First, based on the feeding settings of the pet machine, multiple feeding amounts of the pet machine are obtained, and an identification camera is installed in the pet machine; then, based on the images captured by the identification camera, feeding analysis is performed on each feeding amount using the feeding analysis method, and based on the analysis results, the full-grain identification area and the remaining-grain identification area corresponding to each feeding amount are obtained. The advantage of this is that by installing the identification camera and obtaining the full-grain identification area and the remaining-grain identification area through the identification camera, it is possible to obtain the pet's feeding preferences under different feeding amounts and the image information in the food bowl after the pet has eaten under different feeding amounts, which helps to judge the pet's feeding situation when feeding the pet subsequently, so as to feed and replenish the food bowl in a timely manner to meet the pet's feeding needs;

[0039] In addition, when the pet machine is working, the feeding amount set by the user is recorded as the user feeding amount, the full-grain identification area and the remaining-grain identification area corresponding to the user feeding amount are obtained, and based on the real-time images captured by the identification camera, the pet machine is controlled to feed and replenish the food. The advantage of this is that by feeding and replenishing the food based on the full-grain identification area and the remaining-grain identification area, it is possible to replenish the food bowl in a timely manner when the food in the food bowl is less or not conducive to the pet's feeding on the premise of determining the pet's feeding preferences, so as to avoid the problems of overfeeding or underfeeding caused by conventional feeding methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic block diagram of the system of the present invention;

[0041] Figure 2 is a flowchart of the steps of the method of the present invention;

[0042] Figure 3 is a schematic diagram of the positional relationship between the virtual food outlet and the virtual food bowl of the present invention;

[0043] Figure 4 is a schematic diagram of the top-view food image of the present invention;

[0044] Figure 5 is a schematic diagram of the structure of the electronic device of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0046] Example 1, please refer to Figure 1As shown, the present application provides a feeding image vision recognition system for an intelligent pet machine, including a feeding camera installation module, a pet feeding analysis module, and a recognition feeding module;

[0047] The feeding camera installation module is used to obtain multiple feeding amounts of the pet machine based on the feeding settings of the pet machine, and install a recognition camera in the pet machine; the recognition camera includes a front camera, a left camera, and a right camera. Among them, the front upper camera is used to take pictures directly above the food bowl, and the left camera and the right camera are used to take pictures of the right side and the left side of the inner wall of the food bowl on the left side and the right side of the food outlet respectively;

[0048] The feeding camera installation module includes a camera installation unit, and the camera installation unit is configured with a camera installation strategy, and the camera installation strategy includes:

[0049] Respectively record the weights corresponding to the multiple feeding amounts in the feeding settings of the pet machine as pet feeding amount CT1 to pet feeding amount CT n , where n is the number of feeding amounts in the feeding settings of the pet machine; obtain the three-dimensional dimension data of the pet machine;

[0050] In the specific implementation process, the pet feeding amount CT can be obtained according to the settings of the pet machine or the settings of the user. For example, in a data processing, the pet feeding amounts CT set by the user are 50g, 100g, and 200g respectively, then the value of n can be 3, and the pet feeding amounts CT1 to CT3 are 50g, 100g, and 200g in sequence;

[0051] Install a camera directly above the food bowl outside the pet machine, denoted as the front upper camera, and adjust the orientation of the front upper camera to the center of the inner wall of the food bowl;

[0052] Establish a space coordinate system with the unit of the coordinate axis being cm, denoted as the model building coordinate system, and build a model corresponding to the pet machine in the model building coordinate system based on the three-dimensional dimension data of the pet machine, and denote it as the pet machine model; denote the food outlet and the food bowl in the pet machine model as the virtual food outlet and the virtual food bowl respectively;

[0053] Based on the left view and the right view of the pet machine model, obtain the positions on the left surface and the right surface of the pet machine model that are closest to the virtual food outlet and satisfy diagonal shooting, and denote them as the left installation point and the right installation point respectively;

[0054] In the specific implementation process, for example, in a data processing, the positional relationship between the virtual food outlet and the virtual food bowl obtained is as Figure 3As shown in the figure, where CLK is the virtual grain outlet and LW is the virtual grain bowl. Through analysis, it can be obtained that the positions of the centers of the circles of ○ from left to right are the left mounting point, the mounting point of the directly above camera, and the right mounting point. By obtaining the left mounting point and the right mounting point that satisfy diagonal shooting, it can be ensured that in subsequent shooting and image analysis, the obtained images can analyze all areas in the grain bowl, thereby making the analysis of the pet's eating habits more accurate and effective, and further improving the accuracy of feeding and replenishing the grain bowl;

[0055] Diagonal shooting means that for the left surface of the pet machine, when the connection line between any point and the rightmost point on the inner wall of the virtual grain bowl does not coincide with the pet machine model for any length, this point is recorded as a point that satisfies diagonal shooting; for the right surface of the pet machine, when the connection line between any point and the leftmost point on the inner wall of the virtual grain bowl does not coincide with the pet machine model for any length, this point is recorded as a point that satisfies diagonal shooting;

[0056] Install cameras in the pet machine based on the positions of the left mounting point and the right mounting point, and record them as the left camera and the right camera respectively; adjust the shooting center of the left camera to the rightmost side of the inner wall of the grain bowl, and adjust the shooting center of the right camera to the leftmost side of the inner wall of the grain bowl.

[0057] The pet feeding analysis module is used to perform feeding analysis on each feeding amount using the feeding analysis method based on the images captured by the recognition cameras, and obtain the pet habit parameters corresponding to each feeding amount based on the analysis results; analyze the pet habit parameters, and obtain the pet feeding timing corresponding to each feeding amount based on the analysis results;

[0058] The feeding analysis method includes the empty amount analysis method and the feeding amount analysis method. The empty amount analysis method includes: emptying the grain bowl of the pet machine, and using the directly above camera, the left camera, and the right camera to take pictures respectively, and record the obtained images as the directly above panoramic image, the left panoramic image, and the right panoramic image; record the centers of the directly above panoramic image, the left panoramic image, and the right panoramic image as the panoramic centers corresponding to the directly above panoramic image, the left panoramic image, and the right panoramic image;

[0059] In the specific implementation process, for example, in a data processing, the obtained directly above panoramic image is as Figure 4 shown, where TX1 is the directly above panoramic image, QZ is the center point of the directly above panoramic image, and TX2 is the directly above grain image corresponding to the directly above panoramic image after image recognition processing; by obtaining the directly above grain image, the left grain image, and the right grain image, it is possible to perform image analysis on the grain in the grain bowl from three directions, so as to obtain the characteristics in the empty bowl state in the empty amount analysis method and the characteristics of the full grain and the remaining grain in the bowl after being eaten by the pet when obtaining different pet feeding amounts CT in the feeding amount analysis method, which helps to control the actual grain feeding of the pet machine in subsequent analysis;

[0060] Based on image recognition, images other than the inner wall of the food bowl are removed from the top panoramic image, left panoramic image, and right panoramic image. The images obtained after removal are respectively denoted as the top food image, left food image, and right food image. The regions of the top food image, left food image, and right food image in the top panoramic image, left panoramic image, and right panoramic image are respectively denoted as the top recognition region, left recognition region, and right recognition region;

[0061] For any one of the top food image, left food image, and right food image: Based on image ranging, the distances between all points in the image and the camera are obtained. A spatial coordinate system with the unit of the coordinate axis being cm is established and denoted as the ranging analysis coordinate system; The distance corresponding to the panoramic center obtained based on image ranging is denoted as L;

[0062] The coordinate origin of the ranging analysis coordinate system and the point (L, 0, 0) are respectively denoted as the point corresponding to the camera and the point corresponding to the panoramic center of the image; Based on the positional relationship between the panoramic center and the points other than the panoramic center in the image and the positional relationship between the camera and the points other than the panoramic center in the image, the points corresponding to all points in the image are obtained within the ranging analysis coordinate system, and the surface formed by all points is denoted as the image surface; The points on the edge of the image surface are sequentially connected to the coordinate origin, and the obtained closed region is denoted as the feeding relationship region;

[0063] In the specific implementation process, by obtaining the image surface, the positional relationship between the orientation of the camera and the food in the food bowl can be obtained, and by obtaining the feeding relationship region, the spatial characteristics formed by the orientation of the camera and the food in the food bowl can be obtained, which helps to analyze the feeding situation of the pet in the food bowl based on the feeding relationship region during subsequent analysis;

[0064] Obtain the feeding relationship regions corresponding to the top food image, left food image, and right food image, and denote them as the empty food relationship regions corresponding to the top food image, left food image, and right food image.

[0065] The feeding analysis method includes: For any pet feeding amount CT, put food with a weight of the pet feeding amount CT into the food bowl, use the recognition camera to take pictures, and based on the empty amount analysis method, top recognition region, left recognition region, and right recognition region, obtain the feeding relationship regions corresponding to the top food image, left food image, and right food image at this time, and denote them as the full food relationship regions corresponding to the top food image, left food image, and right food image;

[0066] The pet eats the food in the food bowl, and after one feeding, an identification camera is used to take pictures. Based on the empty quantity analysis method, the directly above recognition area, the left recognition area, and the right recognition area, the corresponding feeding relationship areas of the directly above food image, the left food image, and the right food image are obtained, and are recorded as the remaining food relationship areas corresponding to the directly above food image, the left food image, and the right food image. Among them, when obtaining the remaining food relationship area, the pet can eat from the food bowl with a food content of the pet feeding amount CT multiple times, and after one feeding, the remaining food relationship area is obtained, and the average value of the feeding relationship areas obtained multiple times is recorded as the remaining food relationship area corresponding to the directly above food image, the left food image, and the right food image;

[0067] For any one of the directly above food image, the left food image, and the right food image, the area that does not overlap between the full food relationship area and the empty food relationship area corresponding to the image is recorded as the full food recognition area; the area that does not overlap between the remaining food relationship area corresponding to the image and the empty food relationship area is recorded as the remaining food recognition area;

[0068] Obtain the full food recognition area and the remaining food recognition area corresponding to all pet feeding amounts CT.

[0069] The identification feeding module is used to, when the pet machine is working, record the feeding amount set by the user as the user feeding amount, obtain the pet habit parameters and the pet feeding timing corresponding to the user feeding amount, and control the pet machine to feed and replenish based on the real-time image taken by the identification camera; the identification feeding module includes a feeding identification unit, and the feeding identification unit is configured with a feeding identification strategy, and the feeding identification strategy includes:

[0070] When the pet machine is working, after discharging the food of the user feeding amount at the discharge port, based on image recognition, after the pet has one feeding, obtain the feeding relationship areas corresponding to the directly above food image, the left food image, and the right food image, and record them as the feeding relationship areas corresponding to the directly above food image, the left food image, and the right food image;

[0071] For any one of the directly above food image, the left food image, and the right food image, the area that does not overlap between the feeding relationship area corresponding to the image and the empty food relationship area is recorded as the feeding recognition area; when the volume of the feeding recognition area is less than or equal to the full food recognition area and greater than or equal to the remaining food recognition area, the image is recorded as a replenishment food image; when the feeding recognition area is less than the remaining food recognition area, the image is recorded as a feeding image;

[0072] In the specific implementation process, when the volume of the feeding recognition area is less than or equal to the full grain recognition area and greater than or equal to the residual grain recognition area, it indicates that the pet has not completed a full feeding at this time. The grain in the food bowl is not full but there is a relatively large amount. Therefore, the grain in the food bowl can be replenished; when the feeding recognition area is less than the residual grain recognition area, it indicates that the pet has completed a full feeding at this time. The grain in the food bowl is not full and there is a small amount. A feeding should be carried out in the food bowl to meet the pet's feeding needs;

[0073] When the number of replenishing images in the front upward grain image, the left grain image, and the right grain image is greater than or equal to 2, feed the food bowl until the grain in the food bowl is the amount of grain set by the user; when the number of feeding images in the front upward grain image, the left grain image, and the right grain image is greater than or equal to 2, feed the amount of grain set by the user into the food bowl; in other cases, no feeding is carried out;

[0074] In the specific implementation process, by feeding and replenishing materials based on the full grain recognition area and the residual grain recognition area, it is possible to replenish the food bowl in a timely manner when the grain in the food bowl is less or not conducive to the pet's feeding on the premise of determining the pet's feeding preference, so as to avoid the problems of overfeeding or underfeeding caused by conventional feeding methods.

[0075] Example 2, please refer to Figure 2 As shown, the present application also provides a visual recognition method for the feeding image of an intelligent pet machine, including the following steps:

[0076] Step S1, obtain multiple feeding amounts of the pet machine based on the feeding setting of the pet machine, and install an identification camera in the pet machine; the identification camera includes a front camera, a left camera, and a right camera. Among them, the front upward camera is used to take pictures directly above the food bowl, and the left camera and the right camera are used to take pictures of the right side and the left side of the inner wall of the food bowl on the left side and the right side of the food outlet respectively;

[0077] Step S1 includes: Step S101, record the weights corresponding to the multiple feeding amounts in the feeding setting of the pet machine as pet feeding amount CT1 to pet feeding amount CT n , where n is the number of feeding amounts set in the feeding setting of the pet machine; obtain the three-dimensional size data of the pet machine;

[0078] Step S102, install a camera directly above the food bowl outside the pet machine, denoted as the front upward camera, and adjust the orientation of the front upward camera to the center of the inner wall of the food bowl;

[0079] Step S103, establish a spatial coordinate system with the unit of the coordinate axis being cm, denoted as the model building coordinate system, and build a model corresponding to the pet machine in the model building coordinate system based on the three-dimensional size data of the pet machine, denoted as the pet machine model; denote the food outlet and the food bowl in the pet machine model as the virtual food outlet and the virtual food bowl respectively;

[0080] Step S104, based on the left view and the right view of the pet machine model, respectively obtain the positions on the left surface and the right surface of the pet machine model that are closest to the virtual food outlet and meet the requirements of diagonal shooting, and record them as the left installation point and the right installation point respectively;

[0081] Step S105, diagonal shooting is as follows: Step S1051, for the left side surface of the pet machine, when the line connecting any point and the rightmost point on the inner wall of the virtual food bowl does not overlap with the pet machine model at any length, the point is recorded as a point that satisfies diagonal shooting; for the right side surface of the pet machine, when the line connecting any point and the leftmost point on the inner wall of the virtual food bowl does not overlap with the pet machine model at any length, the point is recorded as a point that satisfies diagonal shooting;

[0082] Step S1052, install cameras in the pet machine based on the positions of the left installation point and the right installation point, and record them as the left camera and the right camera respectively; adjust the shooting center of the left camera to the rightmost side of the inner wall of the grain bowl, and adjust the shooting center of the right camera to the leftmost side of the inner wall of the grain bowl.

[0083] Step S2, performing feeding analysis on each feeding amount using a feeding analysis method based on the image captured by the recognition camera, and obtaining a full grain recognition area and a residual grain recognition area corresponding to each feeding amount based on the analysis result;

[0084] The feeding analysis method includes an empty quantity analysis method and a feeding analysis method. The empty quantity analysis method includes: step S2011, emptying the food bowl of the pet machine, and using the upper camera, the left camera and the right camera to take pictures respectively, and recording the obtained images as the upper panoramic image, the left panoramic image and the right panoramic image; recording the centers of the upper panoramic image, the left panoramic image and the right panoramic image as the panoramic centers corresponding to the upper panoramic image, the left panoramic image and the right panoramic image;

[0085] Step S2012, based on image recognition, the images other than the inner wall of the grain bowl are eliminated from the upper panoramic image, the left panoramic image and the right panoramic image, and the images obtained after the elimination are recorded as the upper grain image, the left grain image and the right grain image, respectively, and the areas of the upper grain image, the left grain image and the right grain image in the upper panoramic image, the left panoramic image and the right panoramic image are recorded as the upper recognition area, the left recognition area and the right recognition area, respectively;

[0086] Step S2013, for any one of the top grain image, the left grain image and the right grain image: based on the image ranging, the distance between all points in the image and the camera is obtained, a spatial coordinate system with coordinate axes in units of cm is established, and recorded as the ranging analysis coordinate system; the distance corresponding to the panoramic center obtained based on the image ranging is recorded as L;

[0087] In step S2014, the coordinate origin of the ranging analysis coordinate system and the point (L, 0, 0) are respectively denoted as the point corresponding to the camera and the point corresponding to the panoramic center of the image; based on the positional relationship between the panoramic center and the points other than the panoramic center in the image and the positional relationship between the camera and the points other than the panoramic center in the image, obtain the points corresponding to all points in the image within the ranging analysis coordinate system, and denote the surface formed by all the points as the image surface; connect the points on the edge of the image surface to the coordinate origin in sequence, and denote the obtained closed area as the feeding relationship area;

[0088] In step S2015, obtain the feeding relationship areas corresponding to the top - feeding image, the left - feeding image, and the right - feeding image, and denote them as the empty - grain relationship areas corresponding to the top - feeding image, the left - feeding image, and the right - feeding image.

[0089] The feeding analysis method includes: step S2021, for any pet feeding amount CT, put food with a weight of the pet feeding amount CT into the food bowl, use the recognition camera to take a picture, and based on the empty - amount analysis method, the top - recognition area, the left - recognition area, and the right - recognition area, obtain the feeding relationship areas corresponding to the top - feeding image, the left - feeding image, and the right - feeding image at this time, and denote them as the full - grain relationship areas corresponding to the top - feeding image, the left - feeding image, and the right - feeding image;

[0090] In step S2022, let the pet eat the food in the food bowl, and after one meal, use the recognition camera to take a picture, and based on the empty - amount analysis method, the top - recognition area, the left - recognition area, and the right - recognition area, obtain the feeding relationship areas corresponding to the top - feeding image, the left - feeding image, and the right - feeding image at this time, and denote them as the remaining - grain relationship areas corresponding to the top - feeding image, the left - feeding image, and the right - feeding image. Among them, when obtaining the remaining - grain relationship area, the pet can eat from the food bowl with a food content of the pet feeding amount CT multiple times, and obtain the remaining - grain relationship area after one meal, and use the average value of the feeding relationship areas obtained multiple times as the remaining - grain relationship area corresponding to the top - feeding image, the left - feeding image, and the right - feeding image;

[0091] In step S2023, for any one of the top - feeding image, the left - feeding image, and the right - feeding image, denote the non - overlapping area between the full - grain relationship area corresponding to the image and the empty - grain relationship area as the full - grain recognition area; denote the non - overlapping area between the remaining - grain relationship area corresponding to the image and the empty - grain relationship area as the remaining - grain recognition area;

[0092] In step S2024, obtain the full - grain recognition areas and the remaining - grain recognition areas corresponding to all pet feeding amounts CT.

[0093] Step S3: When the pet machine is working, record the feeding amount set by the user as the user feeding amount, obtain the full grain recognition area and the residual grain recognition area corresponding to the user feeding amount, and control the pet machine to feed and replenish based on the real-time image captured by the recognition camera.

[0094] Step S3 includes: Step S301: When the pet machine is working, after discharging the user feeding amount of grain at the discharge port, based on image recognition, after the pet has eaten once, obtain the feeding relationship areas corresponding to the front-up grain image, the left grain image, and the right grain image, and record them as the eating relationship areas corresponding to the front-up grain image, the left grain image, and the right grain image.

[0095] Step S302: For any one of the front-up grain image, the left grain image, and the right grain image, record the area that does not overlap between the eating relationship area corresponding to the image and the empty grain relationship area as the eating recognition area; when the volume of the eating recognition area is less than or equal to the full grain recognition area and greater than or equal to the residual grain recognition area, record the image as a replenishment grain image; when the eating recognition area is less than the residual grain recognition area, record the image as a feeding grain image.

[0096] Step S303: When the number of replenishment grain images among the front-up grain image, the left grain image, and the right grain image is greater than or equal to 2, put grain into the food bowl until the grain in the food bowl is the user feeding amount of grain; when the number of feeding grain images among the front-up grain image, the left grain image, and the right grain image is greater than or equal to 2, put the user feeding amount of grain into the food bowl; in other cases, no grain is put.

[0097] Example 3, please refer to Figure 5 as shown Figure 5 illustrates a schematic structural diagram of an electronic device. The electronic device may include: a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus. The memory stores computer-readable instructions, and the processor can call the instructions in the memory. When the computer-readable instructions are executed by the processor, the steps in a feeding image visual recognition method of an intelligent pet machine are run to achieve the following functions: First, obtain multiple feeding amounts of the pet machine based on the feeding setting of the pet machine, and install a recognition camera in the pet machine; then use the feeding analysis method to analyze each feeding amount based on the image captured by the recognition camera, and obtain the full grain recognition area and the residual grain recognition area corresponding to each feeding amount based on the analysis results; finally, when the pet machine is working, record the feeding amount set by the user as the user feeding amount, obtain the full grain recognition area and the residual grain recognition area corresponding to the user feeding amount, and control the pet machine to feed and replenish based on the real-time image captured by the recognition camera.

[0098] In addition, when the logical instructions in the above-mentioned memory are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0099] Embodiment 4, this application also provides a computer-readable storage medium. This application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, it runs the steps in the above-mentioned method for visual recognition of feeding images of an intelligent pet machine to achieve the following functions: First, obtain multiple feeding amounts of the pet machine based on the feeding settings of the pet machine, and install an identification camera in the pet machine; then use the feeding analysis method to analyze each feeding amount based on the images captured by the identification camera, and obtain the full-grain recognition area and the residual-grain recognition area corresponding to each feeding amount based on the analysis results; finally, when the pet machine is working, record the feeding amount set by the user as the user input amount, obtain the full-grain recognition area and the residual-grain recognition area corresponding to the user input amount, and control the pet machine to perform feeding and replenishment based on the real-time images captured by the identification camera.

[0100] Through the description of the above embodiments, the embodiments of the present invention can be provided as a method, a system, or a computer program product. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disks, optical discs, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments.

[0101] In the embodiments provided in the present application, it should be understood that the disclosed system or method can be implemented in other ways. The above-described embodiments are merely illustrative. For example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of systems, modules, and units can be in electrical, mechanical, or other forms.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for visual recognition of feeding images of an intelligent pet machine, characterized in that, It includes the following steps: Based on the feeding setting of the pet machine, obtain multiple feeding amounts of the pet machine, and install an identification camera in the pet machine; the identification camera includes a front camera, a left camera, and a right camera. Among them, the front upper camera is used to take pictures directly above the food bowl, and the left camera and the right camera are used to take pictures of the right side and the left side of the inner wall of the food bowl on the left side and the right side of the food outlet respectively; Based on the images taken by the identification camera, use the feeding analysis method to analyze the feeding of each feeding amount, and obtain the full food recognition area and the residual food recognition area corresponding to each feeding amount based on the analysis results; When the pet machine is working, record the feeding amount set by the user as the user feeding amount, obtain the full food recognition area and the residual food recognition area corresponding to the user feeding amount, and control the pet machine to feed and replenish food based on the real-time images taken by the identification camera.

2. The feeding image vision recognition method of an intelligent pet machine according to claim 1, characterized in that Based on the feeding setting of the pet machine, obtaining multiple feeding amounts of the pet machine and installing an identification camera in the pet machine includes: The weights corresponding to multiple feeding amounts in the feeding setting of the pet machine are respectively recorded as pet feeding amount CT1 to pet feeding amount CT n , where n is the number of feeding amounts in the feeding setting of the pet machine; obtain the three-dimensional dimension data of the pet machine; Install a camera directly above the food bowl outside the pet machine, denoted as the front upper camera, and adjust the orientation of the front upper camera to the center of the inner wall of the food bowl.

3. The feeding image visual recognition method of an intelligent pet machine according to claim 2, characterized in that, Installing an identification camera in the pet machine further includes: Establish a spatial coordinate system with the unit of the coordinate axis being cm, denoted as the model building coordinate system. Based on the three-dimensional dimension data of the pet machine, build a corresponding model of the pet machine in the model building coordinate system, denoted as the pet machine model; denote the food outlet and the food bowl in the pet machine model as the virtual food outlet and the virtual food bowl respectively; Based on the left view and the right view of the pet machine model, obtain the positions on the left surface and the right surface of the pet machine model that are closest to the virtual food outlet and satisfy diagonal shooting, and denote them as the left installation point and the right installation point respectively.

4. The feeding image visual recognition method of an intelligent pet machine according to claim 3, characterized in that, Diagonal shooting means: for the left surface of the pet machine, when the connection line between any point and the rightmost point on the inner wall of the virtual food bowl does not coincide with the pet machine model for any length, this point is denoted as the point that satisfies diagonal shooting; for the right surface of the pet machine, when the connection line between any point and the leftmost point on the inner wall of the virtual food bowl does not coincide with the pet machine model for any length, this point is denoted as the point that satisfies diagonal shooting; Install cameras in the pet machine based on the positions of the left installation point and the right installation point, and denote them as the left camera and the right camera respectively; adjust the shooting center of the left camera to the rightmost side of the inner wall of the food bowl, and adjust the shooting center of the right camera to the leftmost side of the inner wall of the food bowl.

5. The feeding image visual recognition method of an intelligent pet machine according to claim 4, characterized in that, The feeding analysis method includes the empty amount analysis method and the feeding analysis method. The empty amount analysis method includes: Empty the food bowl of the pet machine, and use the front camera, the left camera, and the right camera to take pictures respectively, and denote the obtained images as the front panoramic image, the left panoramic image, and the right panoramic image; denote the centers of the front panoramic image, the left panoramic image, and the right panoramic image as the panoramic centers corresponding to the front panoramic image, the left panoramic image, and the right panoramic image; Based on image recognition, the images other than the inner wall of the food bowl in the top panoramic image, left panoramic image, and right panoramic image are removed. The images obtained after removal are respectively denoted as the top food image, left food image, and right food image. The regions of the top food image, left food image, and right food image in the top panoramic image, left panoramic image, and right panoramic image are respectively denoted as the top recognition region, left recognition region, and right recognition region.

6. The feeding image vision recognition method of an intelligent pet machine according to claim 5, characterized in that The empty quantity analysis method further includes: For any one of the top food image, left food image, and right food image: Based on image ranging, the distances between all points in the image and the camera are obtained, and a spatial coordinate system with the unit of the coordinate axis being cm is established and denoted as the ranging analysis coordinate system; The distance corresponding to the panoramic center obtained based on image ranging is denoted as L; The coordinate origin of the ranging analysis coordinate system and the point (L, 0, 0) are respectively denoted as the point corresponding to the camera and the point corresponding to the panoramic center of the image; Based on the positional relationship between the panoramic center and the points other than the panoramic center in the image and the positional relationship between the camera and the points other than the panoramic center in the image, all points corresponding to the points in the image are obtained in the ranging analysis coordinate system, and the surface formed by all points is denoted as the image surface; The points on the edge of the image surface are sequentially connected to the coordinate origin, and the obtained closed region is denoted as the feeding relationship region; The feeding relationship regions corresponding to the top food image, left food image, and right food image are obtained and denoted as the empty food relationship regions corresponding to the top food image, left food image, and right food image.

7. A feeding image visual recognition method for an intelligent pet machine according to claim 6, characterized in that, The feeding analysis method includes: For any pet feeding quantity CT, food with a weight of the pet feeding quantity CT is put into the food bowl, and an identification camera is used for shooting. Based on the empty quantity analysis method, the top recognition region, left recognition region, and right recognition region, the feeding relationship regions corresponding to the top food image, left food image, and right food image at this time are obtained and denoted as the full food relationship regions corresponding to the top food image, left food image, and right food image.

8. A feeding image vision recognition method for an intelligent pet machine according to claim 7, characterized in that, The feeding analysis method further includes: The pet eats the food in the food bowl, and after one feeding, an identification camera is used for shooting. Based on the empty quantity analysis method, the top recognition region, left recognition region, and right recognition region, the feeding relationship regions corresponding to the top food image, left food image, and right food image at this time are obtained and denoted as the remaining food relationship regions corresponding to the top food image, left food image, and right food image. Among them, when obtaining the remaining food relationship region, the pet can eat from the food bowl with a food content of the pet feeding quantity CT multiple times, and the remaining food relationship region is obtained after one feeding. The average value of the feeding relationship regions obtained multiple times is denoted as the remaining food relationship region corresponding to the top food image, left food image, and right food image; For any one of the top food image, left food image, and right food image, the non - overlapping region between the full food relationship region and the empty food relationship region corresponding to the image is denoted as the full food recognition region; The non - overlapping region between the remaining food relationship region and the empty food relationship region corresponding to the image is denoted as the remaining food recognition region; The full food recognition regions and remaining food recognition regions corresponding to all pet feeding quantities CT are obtained.

9. The feeding image visual recognition method of an intelligent pet machine according to claim 8, characterized in that Controlling a pet machine to feed and refill based on the real-time image captured by an identification camera includes: When the pet machine is working, after discharging the amount of food set by the user at the discharge port, based on image recognition, after the pet has eaten once, obtain the feeding relationship areas corresponding to the directly above food image, left food image, and right food image, and record them as the feeding relationship areas corresponding to the directly above food image, left food image, and right food image; For any one of the directly above food image, left food image, and right food image, record the area that does not overlap between the feeding relationship area corresponding to the image and the empty food relationship area as the feeding recognition area; when the volume of the feeding recognition area is less than or equal to the full food recognition area and greater than or equal to the remaining food recognition area, record the image as a refilling image; when the feeding recognition area is less than the remaining food recognition area, record the image as a feeding image; When the number of refilling images among the directly above food image, left food image, and right food image is greater than or equal to 2, feed the food into the food bowl until the food in the food bowl is the amount of food set by the user; when the number of feeding images among the directly above food image, left food image, and right food image is greater than or equal to 2, feed the amount of food set by the user into the food bowl; in other cases, do not feed the food.

10. A feeding image visual recognition system for an intelligent pet machine, which is used to implement the feeding image visual recognition method for an intelligent pet machine according to any one of claims 1-9, and is characterized in that, It includes a feeding camera installation module, a pet feeding analysis module, and an identification feeding module; The feeding camera installation module is used to obtain multiple feeding amounts of the pet machine based on the feeding settings of the pet machine and install an identification camera in the pet machine; The identification camera includes a front camera, a left camera, and a right camera. Among them, the directly above front camera is used to take pictures directly above the food bowl, and the left camera and the right camera are used to take pictures of the right side and the left side of the inner wall of the food bowl on the left and right sides of the food outlet respectively; The pet feeding analysis module is used to perform feeding analysis on each feeding amount using the feeding analysis method based on the images captured by the identification camera, and obtain the pet habit parameters corresponding to each feeding amount based on the analysis results; analyze the pet habit parameters, and obtain the pet feeding timing corresponding to each feeding amount based on the analysis results; The identification feeding module is used to, when the pet machine is working, record the feeding amount set by the user as the user-set feeding amount, obtain the pet habit parameters and pet feeding timing corresponding to the user-set feeding amount, and control the pet machine to feed and refill based on the real-time image captured by the identification camera.

Images