Remaining food detection method, device and equipment for pet dinner plate and medium

By semantic segmentation and target weight analysis of the pictures to be detected by pet meal plates, the problems of low detection accuracy of pet meal plates and easy equipment damage in the existing technology are solved, and higher detection accuracy and less food are achieved waste.

CN119068193APending Publication Date: 2024-12-03XIAOPEI NETWORK TECH (SHANGHAI) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411121232.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing pet meal tray detection methods are greatly affected by environmental factors and have low accuracy. The weight detection device is susceptible to pet collision damage, which affects the detection accuracy.

Method used

The pre-trained detection model is used to semantically segment the pictures to be detected in pet meal plates, and the amount of residual grain is determined by segmenting the area and positional relationship of the sub-region, and the target weight is used to improve the detection accuracy.

Benefits of technology

It improves the accuracy of the detection of surplus food for pet meal plates, reduces the waste of pet food, and avoids the impact of environmental factors and equipment damage on the detection results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068193A_ABST
    Figure CN119068193A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a remaining food detection method and device for a pet dinner plate, equipment and a medium, and the method comprises the steps: inputting a to-be-detected picture of the pet dinner plate into a pre-trained detection model, carrying out the semantic segmentation of the to-be-detected picture through the detection model, and outputting a remaining food region and a pet dinner plate region in the to-be-detected picture; segmenting the pet dinner plate area according to a preset segmentation condition, determining each segmented sub-area of the pet dinner plate area, and determining the area weight of each segmented sub-area according to the area of each segmented sub-area; determining the area of the remaining grain sub-region in each segmented sub-region according to the position relationship between the remaining grain region and each segmented sub-region; and for any communication region formed by the remaining food sub-regions, determining the remaining food amount in the communication region according to the area of the remaining food sub-region in the communication region and the target weight, and determining the remaining food amount in the pet dinner plate according to the remaining food amount in each communication region. According to the technical scheme, the residual grain detection accuracy can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of deep learning, and in particular to a method, device, equipment and medium for detecting remaining food in a pet plate. Background Art

[0002] As the number of people raising pets increases, smart pet feeding equipment can automatically monitor and replenish pet food, and is therefore becoming more and more popular among pet owners compared to traditional manual pet feeding equipment.

[0003] The current method of detecting remaining food mainly relies on two methods. First, it relies on infrared scanning to scan the remaining food on the plate. However, the detection results of this method are greatly affected by environmental factors, such as ambient light, ambient brightness, etc. Therefore, when the environmental conditions are poor, the detection accuracy is low. Second, it relies on a weight detection device to detect the remaining food. However, the weight sensor device is installed inside the plate. Due to the collision of the pet with the plate, it is easy to cause the weighing sensor device to be damaged, affecting the detection accuracy. Summary of the invention

[0004] The embodiment of the present invention discloses a method, device, equipment and medium for detecting remaining food in a pet meal tray. Through the technical solution of the embodiment of the present invention, the accuracy of detecting remaining food in a pet meal tray can be improved, and the waste of pet food can be reduced.

[0005] In a first aspect, an embodiment of the present invention discloses a method for detecting remaining food in a pet meal tray, comprising:

[0006] Inputting a to-be-detected picture of a pet meal plate into a pre-trained detection model, performing semantic segmentation on the to-be-detected picture through the detection model, and outputting a remaining food area and a pet meal plate area in the to-be-detected picture, wherein the pet meal plate area includes the remaining food area;

[0007] Segmenting the pet plate area according to a preset segmentation condition, determining each segmented sub-area of ​​the pet plate area, and determining the area weight of each segmented sub-area according to the area of ​​each segmented sub-area;

[0008] Determining the area of ​​the surplus grain sub-region within each divided sub-region according to the positional relationship between the surplus grain region and each divided sub-region;

[0009] For any connected area formed by the remaining food sub-areas, the remaining food amount in the connected area is determined according to the area of ​​the remaining food sub-areas in the connected area and the target weight, and the remaining food amount in the pet plate is determined according to the remaining food amount in each connected area, wherein the target weight includes the area weight of the segmented sub-area to which the remaining food sub-area belongs.

[0010] In a second aspect, an embodiment of the present invention discloses a remaining food detection device for a pet meal tray, comprising:

[0011] a segmentation module, for inputting a to-be-detected picture of a pet meal plate into a pre-trained detection model, performing semantic segmentation on the to-be-detected picture through the detection model, and outputting a remaining food area and a pet meal plate area in the to-be-detected picture, wherein the pet meal plate area includes the remaining food area;

[0012] A sub-region determination module, configured to segment the pet plate region according to a preset segmentation condition, determine each segmented sub-region of the pet plate region, and determine a region weight of each segmented sub-region according to an area of ​​each segmented sub-region;

[0013] An area determination module, used for determining the area of ​​the surplus grain sub-region in each divided sub-region according to the positional relationship between the surplus grain region and each divided sub-region;

[0014] The remaining food determination module is used to determine the remaining food amount in any connected area formed by the remaining food sub-areas according to the area of ​​the remaining food sub-areas in the connected area and the target weight, and determine the remaining food amount in the pet plate according to the remaining food amount in each connected area, wherein the target weight includes the area weight of the segmented sub-area to which the remaining food sub-area belongs.

[0015] The third invention, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0016] at least one processor; and,

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the remaining food detection method for a pet meal plate according to any one of the embodiments of the present invention.

[0019] A fourth invention is provided in an embodiment of the present invention. A computer-readable storage medium is provided. The computer-readable storage medium stores computer instructions. The computer instructions are used to enable a processor to implement the remaining food detection method for a pet meal plate as described in any one of the embodiments of the present invention when executed.

[0020] The embodiment of the present invention provides a method, device, equipment and medium for detecting remaining food in a pet plate, the method comprising: inputting a to-be-detected picture of a pet plate into a pre-trained detection model, performing semantic segmentation on the to-be-detected picture through the detection model, and outputting a remaining food area and a pet plate area in the to-be-detected picture, wherein the pet plate area includes the remaining food area; segmenting the pet plate area according to a preset segmentation condition, determining each segmented sub-area of ​​the pet plate area, and determining an area weight of each segmented sub-area according to an area of ​​each segmented sub-area; determining an area of ​​a remaining food sub-area in each segmented sub-area according to a positional relationship between the remaining food area and each segmented sub-area; for any connected area formed by the remaining food sub-area, determining the remaining food amount in the connected area according to the area of ​​the remaining food sub-area in the connected area and a target weight, and determining the remaining food amount in the pet plate according to the remaining food amount in each connected area, wherein the target weight includes the area weight of the segmented sub-area to which the remaining food sub-area belongs. Specifically, by determining the area of ​​the remaining food sub-region in the connected region and the region weight of the segmented sub-region to which the remaining food sub-region belongs, the remaining food amount in the remaining food region in the pet plate can be accurately determined. The method of the embodiment of the present invention can improve the accuracy of pet remaining food detection and reduce pet food waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flow chart of a method for detecting remaining food in a pet meal tray provided by an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of the structure of a detection model provided by an embodiment of the present invention;

[0024] Figure 3 A flowchart of a method for detecting remaining food in a pet meal tray provided in Embodiment 2 of the present invention;

[0025] Figure 4 A schematic diagram of segmenting sub-regions provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of the structure of a remaining food detection device for a pet meal tray provided in a third embodiment of the present invention;

[0027] Figure 6 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] It should be noted that the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0031] Embodiment 1

[0032] Figure 1 A flowchart of a method for detecting remaining food in a pet meal plate provided in an embodiment of the present invention is applicable to the case of detecting remaining food in a pet meal plate. The method can be executed by a device for detecting remaining food in a pet meal plate, which can be composed of software and / or hardware and is set in various computers or servers.

[0033] like Figure 1 As shown, including:

[0034] Step 110: input the to-be-detected picture of the pet plate into a pre-trained detection model, perform semantic segmentation on the to-be-detected picture through the detection model, and output the remaining food area and the pet plate area in the to-be-detected picture, wherein the pet plate area includes the remaining food area.

[0035] The detection model may be a neural network model for performing semantic segmentation on the image of the pet plate to be detected, wherein the semantic segmentation is used to assign each pixel in the image of the pet plate to be detected to a predefined category label, such as pet plate, leftover food, or background. Furthermore, the leftover food area is located in the pet plate area.

[0036] For example, Figure 2 A structural diagram of a detection model provided for an embodiment of the present invention, specifically, the detection model includes a feature map generation module 210, an attention fusion module 220 and a decoder 230. Among them, the feature map generation module 210 is designed to be lightweight on the basis of a conventional network model, including reducing the number of channels of the depth-separable convolution and replacing the ordinary convolution layer with a hole convolution. When the input image data size is the same, this method can reduce the computational complexity of the model and improve its operating efficiency. After the detection image is input into the feature map generation module 210, feature maps of different sizes are extracted and input into the attention fusion module 220, thereby completing feature screening and fusion at different semantic levels; finally, multi-channel feature fusion and segmentation map generation are performed through the decoder 230, and the segmentation map is upsampled to obtain the final semantic segmentation result.

[0037] Furthermore, the training process of the detection model includes:

[0038] Different categories of segmentation and annotation are performed on the targets (plates, pet food, etc.) that need to be semantically segmented in the pictures of pet plates under different environmental conditions. The annotation method can be manual annotation or automatic annotation by algorithm. Specifically, the annotation result is single-channel image data, and the data size of the annotation result is consistent with the original image data size. The annotated data under the pixel coordinates contained in the target area in the original image is assigned to the label value, such as the annotated data values ​​in the pet food pixel coordinate area are all 1, the annotated data values ​​in the plate pixel coordinate area are all 2, and the annotated data values ​​in the remaining pixel coordinate areas are all 0. Furthermore, the detection model to be trained is trained using the annotated pictures. When the training is completed, the detection model described in the embodiment of the present invention is obtained.

[0039] Specifically, according to the remaining food area and the pet dish area in the output image to be detected, the remaining food amount in the pet dish area can be determined and detected by the method of the embodiment of the present invention.

[0040] Optionally, step 110 includes:

[0041] Performing semantic segmentation on the image to be detected by using the detection model to determine a segmentation mask for each pixel in the image to be detected;

[0042] According to the mask value of the segmentation mask of each pixel, the remaining food area and the pet plate area in the image to be detected are determined.

[0043] The segmentation mask is the category code corresponding to the pixel, and the category of the image content corresponding to the pixel can be determined according to the segmentation mask. For example, it can be set that the value of the labeled data in the pet food pixel coordinate area is 1, the value of the labeled data in the pet plate pixel coordinate area is 2, and the value of the labeled data in the background area is 0.

[0044] Step 120 : Segment the pet plate area according to a preset segmentation condition, determine each segmented sub-area of ​​the pet plate area, and determine the area weight of each segmented sub-area according to the area of ​​each segmented sub-area.

[0045] Specifically, the preset segmentation condition is used to segment the plate area, specifically, the pet plate area is segmented into different segmented sub-areas, and the regional weights of different sub-areas can be determined according to their regional areas. The regional weight of the segmented sub-area can be the area ratio of the segmented sub-area to the overall area of ​​the pet plate. It can be understood that the larger the area, the more residual food there may be. Therefore, the regional weight of the segmented sub-area can be determined by the area, and then the residual food amount in different segmented sub-areas can be determined by the regional weight, and then the residual food amount in the pet plate can be accurately determined.

[0046] Optionally, determining the area weight of each segmented sub-region according to the area of ​​each segmented sub-region includes:

[0047] Determining the area ratio of the segmented sub-regions according to the area of ​​the segmented sub-regions and the total area of ​​the pet plate;

[0048] The area weight of the segmented sub-region is determined according to the area ratio of the segmented sub-region.

[0049] Step 130: Determine the area of ​​the surplus grain sub-region within each divided sub-region according to the positional relationship between the surplus grain region and each divided sub-region.

[0050] Specifically, since the remaining food is located in the pet dish area, the remaining food area and the pet dish area have an intersecting portion, that is, the remaining food area and the plurality of segmented sub-areas may have overlapping areas, and the positional relationship is the overlapping relationship between the remaining food area and the segmented sub-areas. Further, the remaining food sub-area is the overlapping area between the remaining food area and the segmented sub-areas.

[0051] Specifically, by determining the area of ​​the remaining food sub-region, the remaining food amount in the segmented sub-regions with different area weights can be accurately determined, and then the remaining food amount in the pet dish can be determined by the remaining food amount in each remaining food sub-region. In this way, the accuracy of remaining food detection can be improved.

[0052] Step 140: for any connected region formed by the remaining food sub-regions, determine the remaining food amount in the connected region according to the area of ​​the remaining food sub-region in the connected region and the target weight, and determine the remaining food amount in the pet plate according to the remaining food amount in each connected region, wherein the target weight includes the area weight of the segmented sub-region to which the remaining food sub-region belongs.

[0053] Among them, the connected area is used to characterize the area where different piles of leftover food are located in the pet plate. The leftover food piles may include large aggregated piles of leftover food and discrete piles of leftover food represented by individual leftover food particles scattered in the pet plate. The connected area is composed of leftover food sub-areas. Since the connected area may span multiple segmented sub-areas, different segmented sub-areas have different regional weights. Therefore, for any connected area, it is necessary to determine the area and target weight of the leftover food sub-area in the connected area respectively, and then determine the amount of leftover food in the connected area. Among them, the target weight includes the regional weight of the segmented sub-area to which the leftover food sub-area belongs, and can also include a preset ratio parameter for scattered leftover food, wherein the preset ratio parameter is used to characterize the correspondence between the discrete leftover food area and the actual amount of scattered leftover food. If the leftover food area is S and the preset ratio parameter is A, the actual amount of scattered leftover food is M=S*A.

[0054] Furthermore, the remaining food amounts in each connected area may be summed to determine the remaining food amount in the pet meal tray.

[0055] The embodiment of the present invention provides a method for detecting residual food in a pet plate. The method comprises: inputting a to-be-detected picture of a pet plate into a pre-trained detection model, performing semantic segmentation on the to-be-detected picture through the detection model, and outputting a residual food area and a pet plate area in the to-be-detected picture, wherein the pet plate area includes the residual food area; segmenting the pet plate area according to a preset segmentation condition, determining each segmented sub-area of ​​the pet plate area, and determining the area weight of each segmented sub-area according to the area of ​​each segmented sub-area; determining the area of ​​the residual food sub-area in each segmented sub-area according to the positional relationship between the residual food area and each segmented sub-area; for any connected area formed by the residual food sub-area, determining the residual food amount in the connected area according to the area of ​​the residual food sub-area in the connected area and the target weight, and determining the residual food amount in the pet plate according to the residual food amount in each connected area, wherein the target weight includes the area weight of the segmented sub-area to which the residual food sub-area belongs. Specifically, by determining the area of ​​the residual food sub-area in the connected area and the area weight of the segmented sub-area to which the residual food sub-area belongs, the residual food amount in the residual food area in the pet plate can be accurately determined. The method of the embodiment of the present invention can improve the accuracy of pet food surplus detection and reduce pet food waste. Furthermore, by performing semantic segmentation on the image to be detected through a preset detection model and determining the segmentation mask of each pixel in the image to be detected, the position of each image content in the image to be detected can be accurately determined, thereby improving the accuracy of surplus food detection.

[0056] Embodiment 2

[0057] Figure 3 This is a flow chart of a method for detecting remaining food in a pet dish provided in the second embodiment of the present invention. This embodiment is based on the above embodiments and further defines a method for determining each segmented sub-region and a method for determining the remaining food amount in a connected region. Figure 3 As shown, including:

[0058] Step 310: input the to-be-detected picture of the pet plate into a pre-trained detection model, perform semantic segmentation on the to-be-detected picture through the detection model, and output the remaining food area and the pet plate area in the to-be-detected picture, wherein the pet plate area includes the remaining food area.

[0059] Step 320: Determine the area center of the pet plate area, and generate at least one corresponding circular area according to the area center and at least one preset radius.

[0060] Step 330: determine the circular rings generated by the intersection of each adjacent circular area and the circular area corresponding to the minimum radius as each segmented sub-area.

[0061] The area center is the center of the pet plate area. Furthermore, at least one corresponding circular area can be generated according to the area center and at least one preset radius.

[0062] For example, Figure 4 A schematic diagram of segmented sub-regions provided in an embodiment of the present invention, wherein region 1 is a circular region corresponding to a minimum radius, and regions 2-n are circular rings generated by the intersection of adjacent circular regions.

[0063] Step 340: Determine the region weight of each segmented sub-region according to the area of ​​each segmented sub-region.

[0064] Specifically, the area weight of each segmented sub-region may be determined according to the area ratio of each segmented sub-region in the pet plate.

[0065] Step 350: Determine the intersection of the surplus grain area and each segmented sub-region according to the positional relationship between the surplus grain area and each segmented sub-region, and determine the area represented by the intersection as the surplus grain sub-region of the segmented sub-region.

[0066] Specifically, since the surplus food area may span across multiple segmented sub-areas, the surplus food area will have an intersection area with the segmented sub-areas, and the intersection area is the surplus food sub-area of ​​the segmented sub-areas. Furthermore, since there may be multiple surplus food areas on the plate, one segmented sub-area may include multiple surplus food sub-areas.

[0067] Step 360: Determine the area of ​​the remaining grain sub-region according to the number of pixels in the remaining grain sub-region and the area of ​​each pixel.

[0068] Specifically, the image to be detected has a preset resolution and image area. Therefore, the number of pixels of the image to be detected can be determined by the resolution of the image to be detected, and then the area of ​​each pixel can be determined according to the image area and the number of pixels of the image to be detected, and then the area of ​​the surplus sub-region can be determined according to the number of pixels in the surplus sub-region and the area of ​​each pixel.

[0069] Step 370: If the area of ​​the connected region is smaller than a preset area threshold, the connected region is determined as a scattered surplus grain region, and the amount of surplus grain in the scattered surplus grain region is determined according to a preset ratio parameter and the scattered surplus grain region.

[0070] Step 380: If the area of ​​the connected area is greater than a preset area threshold, the connected area is determined as a surplus grain accumulation area, and the surplus grain amount of the surplus grain accumulation area is determined based on the areas of each surplus grain sub-area constituting the surplus grain accumulation area and the area weights of the divided sub-areas to which each surplus grain sub-area constituting the surplus grain accumulation area belongs.

[0071] Among them, since the remaining food in the pet meal plate may include a large area of ​​aggregated remaining food piles, it may also include scattered remaining food representing individual particles. The preset area threshold is used to determine whether the remaining food pile area represented by the connected area is a scattered remaining food area or a concentrated remaining food area. Specifically, if the area of ​​the connected area is less than the preset area threshold, the connected area is determined as a scattered remaining food area; if the area of ​​the connected area is greater than the preset area threshold, the connected area is determined as a concentrated remaining food area.

[0072] Specifically, since there is no overlap, coverage, etc. between individual surplus grain particles, the surplus grain amount in the surplus grain scattered area can be determined by the preset ratio parameter and the surplus grain scattered area, wherein the preset ratio parameter is used to characterize the correspondence between the discrete surplus grain area and the actual surplus grain amount. If the surplus grain area is S and the preset ratio parameter is A, the surplus grain amount in the surplus grain scattered area is M=S*A.

[0073] Specifically, if the connected area is a surplus grain gathering area, the surplus grain amount of the surplus grain gathering area needs to be determined according to the area weights of the segmented sub-areas to which each surplus grain sub-area of ​​the surplus grain gathering area belongs. Optionally, the surplus grain amount of the surplus grain sub-area is determined according to the area of ​​the surplus grain sub-area and the area weights of the segmented sub-areas to which the surplus grain sub-area belongs; and the surplus grain amount of the surplus grain gathering area is determined according to the surplus grain amount of each surplus grain sub-area.

[0074] Specifically, each surplus grain sub-region of the connected region belongs to a different segmented sub-region, and different segmented sub-regions correspond to different regional weights. Therefore, different surplus grain sub-regions also correspond to different regional weights. Furthermore, the connected region is composed of multiple surplus grain sub-regions. Therefore, the surplus grain amount of the surplus grain aggregation area can be determined by determining the surplus grain amount of each surplus grain sub-region.

[0075] For example, if the region weight of the segmented sub-region to which the surplus grain sub-region belongs is R i , the area of ​​the surplus grain region is S i , then the amount of surplus grain in the surplus grain concentration area is Here, n represents the number of surplus grain sub-regions.

[0076] Step 390: Determine the remaining food amount in the pet dish according to the remaining food amount in each connected area.

[0077] Specifically, the sum of the remaining food amounts in each remaining food gathering area and the remaining food amounts in each remaining food discrete area may be determined as the remaining food amount in the pet meal plate.

[0078] The embodiment of the present invention provides a method for detecting remaining food in a pet meal plate. The method can determine whether a connected area is a remaining food gathering area or a remaining food discrete area by the area of ​​any connected area formed by the remaining food sub-areas, and then accurately determine the remaining food amount in the pet meal plate by the remaining food amount in the remaining food gathering area and the remaining food discrete area. The method of the embodiment of the present invention can improve the accuracy of remaining food detection in the pet meal plate and save food. The remaining food amount in the remaining food gathering area is determined according to the area weight of the segmented sub-area to which each remaining food sub-area of ​​the remaining food gathering area belongs and the area area of ​​each remaining food sub-area.

[0079] Embodiment 3

[0080] Figure 5 This is a schematic diagram of the structure of a device for detecting residual food in a pet meal tray provided in the third embodiment of the present invention. Figure 5 As shown, the device comprises:

[0081] The segmentation module 510 is used to input the to-be-detected picture of the pet plate into the pre-trained detection model, perform semantic segmentation on the to-be-detected picture through the detection model, and output the remaining food area and the pet plate area in the to-be-detected picture, wherein the pet plate area includes the remaining food area.

[0082] The sub-region determination module 520 is used to segment the pet plate region according to a preset segmentation condition, determine each segmented sub-region of the pet plate region, and determine the region weight of each segmented sub-region according to the area of ​​each segmented sub-region.

[0083] The area determination module 530 is used to determine the area of ​​the surplus grain sub-region within each segmented sub-region according to the positional relationship between the surplus grain region and each segmented sub-region.

[0084] The remaining food determination module 540 is used to determine the remaining food amount in any connected area formed by the remaining food sub-areas according to the area of ​​the remaining food sub-areas in the connected area and the target weight, and determine the remaining food amount in the pet plate according to the remaining food amount in each connected area, wherein the target weight includes the area weight of the segmented sub-area to which the remaining food sub-area belongs.

[0085] The embodiment of the present invention provides a device for detecting a remaining food in a pet dish, the device comprising: inputting a to-be-detected picture of a pet dish into a pre-trained detection model, performing semantic segmentation on the to-be-detected picture through the detection model, and outputting a remaining food area and a pet dish area in the to-be-detected picture, wherein the pet dish area includes the remaining food area; segmenting the pet dish area according to a preset segmentation condition, determining each segmented sub-area of ​​the pet dish area, and determining a region weight of each segmented sub-area according to the area of ​​each segmented sub-area; determining the area of ​​a remaining food sub-area in each segmented sub-area according to a positional relationship between the remaining food area and each segmented sub-area; for any connected area formed by the remaining food sub-area, determining the remaining food amount in the connected area according to the area of ​​the remaining food sub-area in the connected area and a target weight, and determining the remaining food amount in the pet dish according to the remaining food amount in each connected area, wherein the target weight includes the region weight of the segmented sub-area to which the remaining food sub-area belongs. Specifically, by segmenting the pet meal plate area according to the preset segmentation conditions, the segmented sub-areas of the pet meal plate area are determined, and then according to the area weights of the remaining food area and the different segmented sub-areas, the area of ​​the remaining food sub-areas in each segmented sub-area can be accurately determined, and then according to the area of ​​the remaining food sub-areas in the connected area and the target weight, the remaining food amount of the pet meal plate can be determined. Through the device of the embodiment of the present invention, the accuracy of pet remaining food detection can be improved and the waste of pet food can be reduced.

[0086] Optionally, the segmentation module 510 includes:

[0087] A segmentation mask determination unit, used to perform semantic segmentation on the image to be detected by using the detection model, and determine a segmentation mask for each pixel in the image to be detected;

[0088] The area determination module is used to determine the remaining food area and the pet plate area in the image to be detected according to the mask value of the segmentation mask of each pixel.

[0089] Optionally, the sub-region determination module 520 includes a segmentation unit and a weight determination unit.

[0090] The segmentation unit is used to: determine the area center of the pet plate area, generate at least one corresponding circular area according to the area center and at least one preset radius; and determine the circular rings generated by the intersection of each adjacent circular area and the circular area corresponding to the minimum radius as each segmented sub-area.

[0091] The weight determination unit is used to determine the area ratio of the segmented sub-region according to the area of ​​the segmented sub-region and the total area of ​​the pet plate; and determine the area weight of the segmented sub-region according to the area ratio of the segmented sub-region.

[0092] Optionally, the area determination module 530 includes:

[0093] a surplus grain sub-region determining unit, configured to determine an intersection of the surplus grain region and each segmented sub-region according to a positional relationship between the surplus grain region and each segmented sub-region, and determine an area represented by the intersection as a surplus grain sub-region of the segmented sub-region;

[0094] The area unit is used to determine the area of ​​the remaining grain sub-region according to the number of pixels in the remaining grain sub-region and the area of ​​each pixel.

[0095] Optionally, the surplus food determination module 540 includes:

[0096] A scattered surplus grain area unit, for determining the connected area as a scattered surplus grain area if the area of ​​the connected area is less than a preset area threshold, and determining the amount of surplus grain in the scattered surplus grain area according to a preset ratio parameter and the scattered surplus grain area;

[0097] The surplus grain accumulation area unit is used to determine the connected area as a surplus grain accumulation area if the area of ​​the connected area is greater than a preset area threshold, and determine the surplus grain amount in the surplus grain accumulation area according to the areas of each surplus grain sub-area constituting the surplus grain accumulation area and the area weights of the divided sub-areas to which each surplus grain sub-area constituting the surplus grain accumulation area belongs.

[0098] The scattered surplus grain area unit is specifically used to determine the surplus grain amount of the surplus grain sub-area according to the area of ​​the surplus grain sub-area and the area weight of the divided sub-area to which the surplus grain sub-area belongs;

[0099] The amount of surplus grain in the surplus grain gathering area is determined according to the amount of surplus grain in each surplus grain sub-area.

[0100] The remaining food detection device for a pet meal tray provided in the embodiment of the present invention can execute the remaining food detection method for a pet meal tray provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0101] Embodiment 4

[0102] Figure 6A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0103] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0104] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0105] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for detecting remaining food in a pet dish.

[0106] In some embodiments, the remaining food detection method for a pet dish may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the remaining food detection method for a pet dish described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the remaining food detection method for a pet dish in any other appropriate manner (e.g., by means of firmware).

[0107] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0109] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0110] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0111] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0112] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0113] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0114] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting remaining food in a pet meal tray, characterized in that: include: Inputting a to-be-detected picture of a pet meal plate into a pre-trained detection model, performing semantic segmentation on the to-be-detected picture through the detection model, and outputting a remaining food area and a pet meal plate area in the to-be-detected picture, wherein the pet meal plate area includes the remaining food area; Segmenting the pet plate area according to a preset segmentation condition, determining each segmented sub-area of ​​the pet plate area, and determining the area weight of each segmented sub-area according to the area of ​​each segmented sub-area; Determining the area of ​​the surplus grain sub-region within each divided sub-region according to the positional relationship between the surplus grain region and each divided sub-region; For any connected area formed by the remaining food sub-areas, the remaining food amount in the connected area is determined according to the area of ​​the remaining food sub-areas in the connected area and the target weight, and the remaining food amount in the pet plate is determined according to the remaining food amount in each connected area, wherein the target weight includes the area weight of the segmented sub-area to which the remaining food sub-area belongs.

2. The method according to claim 1, characterized in that The semantic segmentation of the image to be detected by the detection model and outputting the remaining food area and the pet plate area in the image to be detected includes: Performing semantic segmentation on the image to be detected by using the detection model to determine a segmentation mask for each pixel in the image to be detected; According to the mask value of the segmentation mask of each pixel, the remaining food area and the pet plate area in the image to be detected are determined.

3. The method according to claim 1, characterized in that The step of segmenting the pet plate area according to a preset segmentation condition to determine each segmented sub-area of ​​the pet plate area includes: Determine the area center of the pet plate area, and generate at least one corresponding circular area according to the area center and at least one preset radius; The circular rings generated by the intersection of each adjacent circular area and the circular area corresponding to the minimum radius are determined as each segmentation sub-area.

4. The method according to claim 1, characterized in that: The determining the area weight of each segmented sub-region according to the area of ​​each segmented sub-region comprises: Determining the area ratio of the segmented sub-regions according to the area of ​​the segmented sub-regions and the total area of ​​the pet plate; The area weight of the segmented sub-region is determined according to the area ratio of the segmented sub-region.

5. The method according to claim 1, characterized in that The determining the area of ​​the surplus grain sub-region within each divided sub-region according to the positional relationship between the surplus grain region and each divided sub-region comprises: According to the positional relationship between the surplus grain area and each segmented sub-area, determining the intersection of the surplus grain area and each segmented sub-area, and determining the area represented by the intersection as the surplus grain sub-area of ​​the segmented sub-area; The area of ​​the remaining grain sub-region is determined according to the number of pixels in the remaining grain sub-region and the area of ​​each pixel.

6. The method according to claim 1, characterized in that The types of the connected areas include: scattered surplus grain areas or concentrated surplus grain areas. For any connected area formed by the surplus grain sub-areas, the amount of surplus grain in the connected area is determined according to the area and target weight of the surplus grain sub-areas in the connected area, including: If the area of ​​the connected region is smaller than a preset area threshold, the connected region is determined as a scattered surplus grain region, and the amount of surplus grain in the scattered surplus grain region is determined according to a preset ratio parameter and the scattered surplus grain region; If the area of ​​the connected area is greater than the preset area threshold, the connected area is determined as a surplus grain accumulation area, and the surplus grain amount in the surplus grain accumulation area is determined based on the areas of each surplus grain sub-area constituting the surplus grain accumulation area and the area weights of the divided sub-areas to which each surplus grain sub-area constituting the surplus grain accumulation area belongs.

7. The method according to claim 6, characterized in that The method of determining the amount of surplus grain in the surplus grain gathering area according to the area of ​​each surplus grain sub-area constituting the surplus grain gathering area and the area weight of the divided sub-area to which each surplus grain sub-area constituting the surplus grain gathering area belongs comprises: Determining the amount of surplus grain in the surplus grain sub-region according to the area of ​​the surplus grain sub-region and the regional weight of the segmented sub-region to which the surplus grain sub-region belongs; The amount of surplus grain in the surplus grain gathering area is determined according to the amount of surplus grain in each surplus grain sub-area.

8. A device for detecting residual food in a pet food tray, characterized in that: include: a segmentation module, for inputting a to-be-detected picture of a pet meal plate into a pre-trained detection model, performing semantic segmentation on the to-be-detected picture through the detection model, and outputting a remaining food area and a pet meal plate area in the to-be-detected picture, wherein the pet meal plate area includes the remaining food area; A sub-region determination module, configured to segment the pet plate region according to a preset segmentation condition, determine each segmented sub-region of the pet plate region, and determine a region weight of each segmented sub-region according to an area of ​​each segmented sub-region; An area determination module, used for determining the area of ​​the surplus grain sub-region in each divided sub-region according to the positional relationship between the surplus grain region and each divided sub-region; The remaining food determination module is used to determine the remaining food amount in any connected area formed by the remaining food sub-areas according to the area of ​​the remaining food sub-areas in the connected area and the target weight, and determine the remaining food amount in the pet plate according to the remaining food amount in each connected area, wherein the target weight includes the area weight of the segmented sub-area to which the remaining food sub-area belongs.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the remaining food detection method for a pet meal plate according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the remaining food detection method for a pet meal plate according to any one of claims 1 to 7 when executed.