A hybrid target image recognition method and system based on cloud platform

Abstract

The present invention discloses a hybrid target image recognition method and system based on a cloud platform, which belongs to the field of image recognition technology. The key points of its technical solution include: identifying the attribute information of each food monomer based on the original image of the multi-food monomer combination collected by the terminal device; determining the food monomer reference corresponding to each food monomer based on the attribute information of each food monomer; determining the content of each nutrient component in the multi-food monomer combination based on each food monomer and the corresponding each food monomer reference. The present invention can accurately obtain the content of each nutrient component in the multi-food monomer combination when the attribute information of each food monomer is unknown, so that the user can formulate a diet plan based on the content of each nutrient component.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and more specifically to a hybrid target image recognition method and system based on a cloud platform. Background Art

[0002] For patients with certain diseases, strict diet control is required. However, there are many types of food available today, and it is often difficult to accurately judge the nutritional content of unknown foods and develop a diet plan.

[0003] Chinese patent application publication number CN109856345A provides a fruit quality identification method and system. The invention establishes a fruit quality identification model based on historical fruit parameters and historical fruit quality, and determines the fruit quality of the tested fruit based on the fruit quality identification model, thereby realizing automatic identification of fruit quality. However, the invention can only identify the size and maturity of the fruit and cannot obtain the nutritional components of the fruit. Therefore, there are deficiencies in the existing technology. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the purpose of the present invention is to provide a hybrid target image recognition method and system based on a cloud platform, which can obtain the attribute information of each food monomer and the food monomer reference corresponding to each food monomer through image recognition technology when the attribute information of each food monomer is unknown, and calculate the content of each nutrient component in a multi-food monomer combination based on the volume difference between each food monomer and the corresponding food monomer reference and the proportion of each nutrient component of the food corresponding to the food monomer reference, so that users can formulate a diet plan based on the content of each nutrient component.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The present invention provides a hybrid target image recognition method based on a cloud platform. The hybrid target image recognition method is executed by a cloud platform server. The hybrid target image recognition method is applied to the recognition of a multi-food monomer combination, wherein the multi-food monomer combination includes a plurality of food monomers. The hybrid target image recognition method includes:

[0007] Identifying attribute information of each food unit according to the original image of the combination of multiple food units captured by the terminal device;

[0008] Determining a food monomer reference object corresponding to each food monomer according to the attribute information of each food monomer;

[0009] The content of each nutrient component in the multi-food monomer combination is determined based on each food monomer and the corresponding food monomer reference.

[0010] As a further improvement of the present invention, if the food monomer reference object is a three-dimensional physical model of a standard food monomer, determining the content of each nutrient component in the multi-food monomer combination based on each food monomer and the corresponding food monomer reference object includes:

[0011] Acquire a first mixed image, where the first mixed image is an image including the combination of multiple food units and the three-dimensional entity model;

[0012] The content of each nutrient component in the multi-food monomer combination is obtained according to the first mixed image.

[0013] As a further improvement of the present invention, if the food monomer reference object is a standard food monomer image, determining the content of each nutrient component in the multi-food monomer combination based on each food monomer and the corresponding food monomer reference object includes:

[0014] splicing the original image and the standard food monomer image to obtain a second mixed image;

[0015] The content of each nutrient component in the multi-food monomer combination is obtained according to the second mixed image.

[0016] As a further improvement of the present invention, obtaining the content of each nutrient component in the multi-food monomer combination according to the first mixed image or the second mixed image includes:

[0017] Obtaining a volume difference between each of the food monomers and its corresponding food monomer reference object according to the first mixed image or the second mixed image;

[0018] The content of each nutrient component in the multi-food monomer combination is obtained based on the proportion of each nutrient component in the food corresponding to the food monomer reference and the volume difference.

[0019] As a further improvement of the present invention, identifying the attribute information of each food unit based on the original image of the combination of multiple food units captured by the terminal device includes:

[0020] Obtaining contour features, color features, and texture features of each food unit according to the original image;

[0021] Attribute information of each food unit is obtained according to the outline feature, the color feature, the texture feature and a preset database.

[0022] As a further improvement of the present invention, obtaining the contour feature of each food unit according to the original image includes:

[0023] Obtain the contour of each food monomer based on the original image;

[0024] Based on the contour points on the contour of each food monomer, obtain multiple contour segments corresponding to the contour of each food monomer;

[0025] Calculate the feature description matrix of each contour segment to obtain the contour features of each food monomer.

[0026] As a further improvement of the present invention, the obtaining the contour of each food monomer based on the original image includes:

[0027] Based on the gradient magnitude of each pixel point in the original image, select multiple contour points in the original image;

[0028] Based on the distances between the multiple contour points, divide the contour points into multiple groups, each group containing γ contour points, where γ is a positive integer greater than 2;

[0029] Connect the contour points within each group in sequence to obtain multiple edge segments;

[0030] Based on the turning direction of each edge segment, select some edge segments from the multiple edge segments;

[0031] Connect the selected edge segments to obtain the contour of each food monomer.

[0032] As a further improvement of the present invention, the obtaining multiple contour segments corresponding to the contour of each food monomer based on the contour points on the contour of each food monomer includes:

[0033] For the contour of each food monomer, perform iterative operations respectively. The iterative operations include calculating the contribution degree of each contour point among all current contour points respectively to obtain the first contour point, and replacing the first line segment with the second line segment until all current line segments meet the preset termination condition; the first contour point is the contour point with the smallest contribution degree among all current contour points, the first line segment is the line segment with the first contour point as an end point, the second line segment is the line segment with the second contour point as an end point, and each of the two second contour points is one of the two adjacent contour points of the first contour point;

[0034] Based on the finally output all current line segments, obtain multiple contour segments corresponding to the contour of each food monomer; where the preset termination condition is D < d, where d is a preset value, A0 is the average value of the turning angles of all current line segments, A is the average value of the turning angles of all initial line segments, and D represents the difference degree between A0 and A.

[0035] As a further improvement of the present invention, the step of calculating a feature description matrix for each contour segment includes:

[0036] Selecting a plurality of sampling points in each of the contour segments, and determining a reference axis corresponding to each of the sampling points;

[0037] Obtaining a height feature matrix of each contour segment according to each sampling point and the reference axis;

[0038] A feature description matrix for each contour segment is obtained according to the height feature matrix and discrete Fourier transform.

[0039] The present invention provides a hybrid target image recognition system based on a cloud platform, the system comprising a cloud platform server and a terminal device, the cloud platform server comprising:

[0040] An acquisition module is used to collect original images of a combination of multiple food monomers through a terminal device;

[0041] an identification module, configured to identify attribute information of each food monomer and determine a food monomer reference object corresponding to each food monomer based on the attribute information of each food monomer;

[0042] The calculation module is used to determine the content of each nutrient component in the multi-food monomer combination based on each food monomer and the corresponding food monomer reference.

[0043] Based on the contour features, color features, texture features and a preset database of each food monomer, the present invention can accurately identify the attribute information of each food monomer when there is occlusion between food monomers. At the same time, the corresponding food monomer reference object is determined according to the attribute information of each food monomer, and further according to the volume difference between each food monomer and the corresponding food monomer reference object and the proportion of each nutrient component of the food corresponding to the food monomer reference object, the content of each nutrient component in the multi-food monomer combination is calculated, so that the user can formulate a diet plan according to the content of each nutrient component. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flow chart of the method steps of the present invention;

[0045] Figure 2 is a schematic diagram of an ellipsoid;

[0046] Figure 3 A schematic diagram of a scene containing multiple food monomers;

[0047] Figure 4 is a schematic diagram of the contour points;

[0048] Figure 5is a schematic diagram of edge segments;

[0049] Figure 6 It is a schematic diagram of the outline of a food monomer;

[0050] Figure 7 It is a structural diagram of line segments and corners;

[0051] Figure 8 Schematic diagram of the outline after multiple cycles. DETAILED DESCRIPTION

[0052] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations of the technical solution of the present invention.

[0053] The term "and / or" in the following text simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " generally indicates an "or" relationship between the related objects.

[0054] like Figure 1 As shown, the embodiment of the present application provides a hybrid target image recognition method based on a cloud platform. The hybrid target image recognition method is executed by a cloud platform server. The hybrid target image recognition method is applied to the recognition of a combination of multiple food monomers. The multiple food monomer combination includes multiple food monomers. The hybrid target image recognition method includes:

[0055] According to the original image of the combination of multiple food units collected by the terminal device, the attribute information of each food unit is identified;

[0056] Determining a food monomer reference object corresponding to each food monomer based on the attribute information of each food monomer;

[0057] The content of each nutrient in the multi-food monomer combination is determined based on each food monomer and the corresponding food monomer reference.

[0058] The food monomer reference object may be a standard food monomer image or a three-dimensional entity model of a standard food monomer.

[0059] Exemplarily, when the mixed target is multiple fruits and vegetables, the original image of the mixed target is an image containing multiple fruits and vegetables, each food monomer is each fruit and vegetable contained in the mixed target, the attribute information of each food monomer includes the type and specific food name of each food monomer, and the food monomer reference is a standard image or standard three-dimensional model of each fruit and vegetable.

[0060] The method provided in this embodiment can obtain the attribute information of each food monomer through image recognition technology when the attribute information of each food monomer is unknown, and determine the corresponding food monomer reference object. According to the volume difference between each food monomer and the corresponding food monomer reference object and the proportion of each nutrient component of the food corresponding to the food monomer reference object, the content of each nutrient component in the multi-food monomer combination is calculated, so that the user can formulate a diet plan based on the content of each nutrient component.

[0061] Furthermore, if the food unit reference object is a three-dimensional physical model of a standard food unit, this embodiment provides a step of determining the content of each nutrient component in a combination of multiple food units based on each food unit and the corresponding food unit reference object, including:

[0062] Acquire a first mixed image, where the first mixed image is an image including a combination of multiple food entities and a three-dimensional entity model;

[0063] According to the first mixed image, the content of each nutrient component in the multi-food monomer combination is obtained.

[0064] There may be multiple first mixed images, and each first mixed image corresponds to a shooting angle.

[0065] Furthermore, if the food monomer reference object is a standard food monomer image, this embodiment provides a method for determining the content of each nutrient component in a combination of multiple food monomers based on each food monomer and the corresponding food monomer reference object, including:

[0066] splicing the original image and the standard food monomer image to obtain a second mixed image;

[0067] According to the second mixed image, the content of each nutrient component in the multi-food monomer combination is obtained.

[0068] Among them, the shooting conditions of the standard food monomer image and the original image must be consistent, that is, the scaling ratio of each food monomer must be consistent, and there can be multiple standard food monomer images, original images and second mixed images. Each standard food monomer image and original image corresponds to a shooting angle. The standard food monomer images and original images with the same shooting angle are spliced ​​together, so that each second mixed image obtained by splicing corresponds to a shooting angle.

[0069] Furthermore, this embodiment provides a method for obtaining the content of each nutrient component in a combination of multiple food monomers based on the first mixed image or the second mixed image, including:

[0070] Obtaining a volume difference between each food monomer and its corresponding food monomer reference object according to the first mixed image or the second mixed image;

[0071] The content of each nutrient in the multi-food monomer combination is obtained based on the proportion and volume difference of each nutrient in the food corresponding to the food monomer reference.

[0072] For example, assuming that a food unit is an orange, and its corresponding food unit reference object is a standard-sized orange model, and the size data of the orange model is known, the first mixed image is an image that includes both the orange and the orange model. In this case, the volume difference between the orange and the orange model is calculated as follows:

[0073] The orange and the orange model are approximated as spheres. Based on image recognition technology, the projected areas s1 and s2 of the orange and the orange model on the plane can be obtained. According to the circle area formula, the projected radii r1 and r2 of the orange and the orange model can be calculated. Since the radius of a sphere is directly related to its volume, the volume of the orange can be calculated based on the proportional relationship between the volume and radius of the sphere. Then we get the volume difference ΔV=|V1-V0|, where V0 is the volume of the orange model, which is known data.

[0074] For example, assuming that a food unit is a winter melon, and its corresponding food unit reference is an image of a standard-sized winter melon, and the size data of the standard-sized winter melon is known, the second mixed image is a front view and a top view stitched together by the original image of the winter melon and the image of the standard-sized winter melon. In this case, the volume difference is calculated as follows:

[0075] like Figure 2 As shown, the winter melon is approximated as an ellipsoid. The three semi-axis lengths a1, b1, and c1 of the winter melon can be obtained based on the second mixed image. The three semi-axis lengths a2, b2, and c2 of the standard-sized winter melon can be obtained based on the size data of the standard-sized winter melon. Since the ratio of the volumes of similar ellipsoids is equal to the cube of the ratio of the corresponding semi-axis lengths, the volume of the winter melon can be obtained according to this proportional relationship: V2 = V3(k1k2k3) 3 ,in They represent the proportional relationship of the three semi-axes respectively, and then the volume difference ΔV1=|V2-V3| is obtained, where V3 is the volume of a standard-sized winter melon.

[0076] Specifically, the nutritional components of the food corresponding to the food monomer reference include sugar content, vitamin content, protein content, etc. The proportion of each nutritional component is the mass of each nutritional component in every 100g of food.

[0077] Furthermore, based on the volume difference and the density of the food, the mass difference between the food unit and the standard-sized food can be calculated. Based on the mass difference and the proportion of each nutrient, the content of each nutrient in each food unit can be calculated, and then the content of each nutrient in the combination of multiple food units can be obtained.

[0078] Exemplarily, following the above example of an orange, assuming the density of the orange is ρ, the mass difference m = ρΔV can be calculated according to the relationship between volume, density, and mass. Assuming the sugar content of the orange is β g / 100g, then compare the sizes of V1 and V0. If V1 > V0, the sugar content of the orange is If V1 < V0, the sugar content of the orange is If V1 = V0, the sugar content of the orange is where m1 = ρV0, representing the mass of an orange with a volume of V0.

[0079] The method provided in this embodiment approximates food as common geometric solids, and combines image recognition technology and known size data to calculate the volume and mass of each food monomer more accurately. Then, by combining the proportion of each nutrient component in the food of standard size, the content of each nutrient component in each food monomer can be calculated more accurately.

[0080] However, the example in this embodiment only introduces the calculation method for a single food monomer. Therefore, there is no phenomenon that the image recognition is inaccurate due to occlusion between food monomers. For example Figure 3 As shown, in actual recognition, there are multiple food monomers, and occlusion is likely to occur between multiple food monomers. Therefore, it is necessary to separate the occluded area to obtain the complete part and the occluded part. The complete part can still be calculated by the above method. For the occluded part, according to its contour features, color features, and texture features, it needs to be replaced with a complete food pattern in the preset database, and then calculated by the above method.

[0081] Furthermore, this embodiment provides a method for identifying the attribute information of each food monomer based on the original image of a multi-food monomer combination collected by a terminal device, including:

[0082] Obtain the contour features, color features, and texture features of each food monomer from the original image;

[0083] Obtain the attribute information of each food monomer according to the contour features, color features, texture features, and preset database.

[0084] Specifically, this embodiment uses the HSV color space to extract the color features of each food monomer. Since the original image belongs to the RGB color space, it is necessary to convert it from the RGB color space to the HSV color space. The specific method is:

[0085] R' = R / 255

[0086] G' = G / 255

[0087] B' = B / 255

[0088] U MAX =MAX(R',G',B')

[0089] U MIN =MIN(R',G',B')

[0090]

[0091] V=U MAX

[0092] Among them, H is the hue value, ranging from 0° to 360°, S is the saturation value, ranging from 0 to 100%, the larger the saturation value, the more saturated the color, V is the brightness value, ranging from 0 (black) to 100% (white), T, G, and B represent red, green, and blue in the RGB color space respectively, R', G', and B' are the results of normalization of R, G, and B respectively, and U MAX Indicates the maximum value among R, G, and B. MAX(·) indicates the maximum value function. U MIN Indicates the minimum value among R, G, and B, and MIN(·) represents the minimum value function.

[0093] This embodiment uses the HSV color space to extract the color features of each food unit. Compared with the RGB color space, the HSV color space is more suitable for the color expression of human vision and is easier to adjust.

[0094] Specifically, the texture features of any pixel in each food monomer in the original image It can be expressed as:

[0095]

[0096] Among them, g c Represents the gray value of any pixel in each food monomer, g t Represents the grayscale value of any pixel in the preset neighborhood of the pixel. The preset neighborhood is a circle. δ(g t -g c ) is a sign function, when g t -g c ≥0, δ(g t -g c )=1, when g t -g c <0, δ(g t -g c )=0, T represents the number of pixels contained in the preset neighborhood, i represents the radius of the preset neighborhood, and the texture features of each pixel in each food monomer are combined to obtain the texture features of each food monomer.

[0097] The texture features provided in this embodiment are rotationally invariant and grayscale invariant, that is, the texture features are not affected by image rotation or illumination changes, and have a certain degree of robustness in processing illumination effects and noise. At the same time, the calculation steps of this method are relatively simple, and texture features can be effectively captured with a small amount of calculation. Moreover, this method can capture texture features of different scales by adjusting the size of the preset neighborhood, which can meet multi-scale needs.

[0098] Furthermore, this embodiment provides a step of obtaining the contour features of each food unit based on the original image, including:

[0099] Obtain the outline of each food monomer based on the original image;

[0100] According to the contour points on the contour of each food monomer, a plurality of contour segments corresponding to the contour of each food monomer are obtained;

[0101] Calculate the feature description matrix of each contour segment to obtain the contour features of each food monomer.

[0102] Furthermore, this embodiment provides a method for obtaining the outline of each food unit according to the original image, including:

[0103] According to the gradient amplitude of each pixel in the original image, multiple contour points are selected in the original image;

[0104] According to the distances between multiple contour points, the contour points are divided into multiple groups, each group contains γ contour points, and γ is a positive integer greater than 2;

[0105] Connect the contour points in each group in sequence to obtain multiple edge segments;

[0106] Selecting some edge segments from the plurality of edge segments according to the corner direction of each edge segment;

[0107] Connect some edge segments to obtain the outline of each food unit.

[0108] Furthermore, this embodiment provides a method for obtaining multiple contour segments corresponding to the contour of each food unit based on the contour points on the contour of each food unit, including:

[0109] For the contour of each food monomer, an iterative operation is performed separately. The iterative operation includes calculating the contribution degree of each contour point among all current contour points respectively to obtain the first contour point, replacing the first line segment with the second line segment until all current line segments meet the preset termination condition; the first contour point is the contour point with the smallest contribution degree among all current contour points, the first line segment is the line segment with the first contour point as an end point, the second line segment is the line segment with the second contour point as an end point, and each of the two second contour points is one of the two adjacent contour points of the first contour point;

[0110] According to all the current line segments finally output, multiple contour segments corresponding to the contour of each food monomer are obtained; where the preset termination condition is D < d, where d is a preset value, A0 is the average value of the turning angles of all current line segments, A is the average value of the turning angles of all initial line segments, and D represents the difference degree between A0 and A.

[0111] Specifically, in an image, the boundary position between a food monomer and the background or between food monomers usually shows a sudden change in gray value. This sudden change in gray value can be measured using gradients. Therefore, in this embodiment, multiple contour points are selected in the original image through the gradient magnitude of each pixel point. However, due to problems such as background interference, among the multiple selected contour points, there may be interference points that do not belong to the contour of the food monomer, such as Figure 4 shown. In order to accurately obtain the contour points of the contour of each food monomer from the multiple selected contour points, the contour points can be divided into multiple groups according to the distance between the contour points, such as Figure 5 shown. Each group contains 3 contour points. The contour points within each group are connected in sequence to obtain multiple edge line segments. Then, according to the turning direction of each edge line segment, some edge line segments that can form the contour of the food monomer are selected from the multiple edge line segments, and the selected edge line segments are connected to obtain the contour of the food monomer, such as Figure 6 shown.

[0112] Specifically, the method of selecting some edge line segments that can form the contour of the food monomer from the multiple edge line segments according to the turning direction of each edge line segment includes: first, judging whether they can match each other according to the turning directions of two adjacent edge line segments to achieve a smooth transition of the contour. Generally, the turning directions of two adjacent edge line segments of a closed contour should be the same. Then, calculate the turning angle at the connection of the two edge line segments and judge whether it conforms to the angle change law that the object contour should have at this position. At the same time, in order to ensure the accuracy of the obtained contour of the food monomer, the obtained contour of the food monomer can be further inspected, that is, judging from a geometric perspective whether the obtained contour has a strange shape and does not conform to the geometric contour of common objects.

[0113] Next, for each food monomer, the outline of the food monomer is recorded as outline C = {S σ}, (σ=1,2,…,M), where S σ is the σth line segment in the contour C, M is the total number of line segments contained in the contour C, P n (n=1, 2, ..., N) represents the nth contour point of the contour C, and N is the total number of contour points.

[0114] Afterwards, in order to reduce the number of contour points and improve recognition efficiency, it is necessary to calculate the contour point p with the smallest contribution in each cycle. i (i=1,2,…,n), and delete the contour point. i Replace the line segments with endpoints with p i-1 and p i+1 is a line segment with endpoints, where the contribution K(S i ,S i+1 ) is calculated as follows:

[0115]

[0116] like Figure 7 As shown, where w(S i ,S i+1 ) is the line segment S i and S i+1 The turning angle, S i Indicates that p i and p i-1 is the line segment with endpoints, S i+1 Indicates that p i and p i+1 is the line segment with endpoints, l(S i ) and l(S i+1 ) is line segment S i and S i+1 Compared to the normalized length of the contour perimeter, for example, assuming the contour perimeter is C1, the line segment S i The length of is C2,

[0117] This embodiment uses line segment length and rotation angle to measure the contribution of contour points. Line segment length reflects the extent of the line segment in a local area and plays a key role in describing the boundary characteristics of an object. For example, when describing the contour of an object, the areas containing long line segments often represent the boundary regions where the object's shape is more prominent and stable, contributing significantly to defining the object's overall form. Rotation angles, on the other hand, describe the degree of directional change between adjacent line segments and can effectively capture key details of contour shape changes.

[0118] Then calculate the average angle A0 of all corners in the updated multiple line segments after deleting the contour point, and calculate the difference D between A0 and the average angle A of all corners in the initial multiple line segments:

[0119]

[0120] If the value of the difference D is less than the preset value d, the loop is terminated to obtain multiple contour segments of the food monomer. The preset value d can be set according to the number of contour points. Repeat the above steps for the contour of each food monomer to obtain multiple contour segments corresponding to each food monomer.

[0121] like Figure 8 As shown in the figure, when the number of cycles is too many, the originally smooth contour will become sharp as the contour points are deleted. At this time, the turning angles of adjacent line segments will become smaller. Therefore, this embodiment avoids the occurrence of the above situation through the preset termination condition. While deleting the contour points that do not contribute much to the contour features of the food monomer, the shape of the original contour can be retained as much as possible to avoid the phenomenon of excessive evolution.

[0122] Furthermore, the embodiment of the present application provides a step of calculating a feature description matrix for each contour segment, including:

[0123] Select multiple sampling points in each contour segment and determine the reference axis corresponding to each sampling point;

[0124] According to each sampling point and the reference axis, the height feature matrix of each contour segment is obtained;

[0125] According to the height feature matrix and discrete Fourier transform, the feature description matrix of each contour segment is obtained.

[0126] Specifically, for each contour segment P in the contour of each food monomer, select J sampling points in P, and for each sampling point P in P j , j=1,2,…,J, passing through point P j Make a horizontal line parallel to the x-axis as the reference axis, and then calculate the j The distance between all sampling points other than the reference axis is obtained by sampling point P j The height eigenvector H j ,Repeat the above steps for each sampling point to obtain the height feature vector corresponding to each sampling point. Then, combine the height feature vectors corresponding to each sampling point to obtain the height feature matrix of the contour segment P.

[0127] Use f v Represents the vth row of the height feature matrix, for f v Performing discrete Fourier transform yields:

[0128]

[0129] Where q is the imaginary unit, F v (ω) represents the discrete Fourier transform coefficient, ω=1,2,…,Y, Y is the Fourier order, f v (t) is the element in the vth row and tth column of the height feature matrix. According to the discrete Fourier transform coefficient, the feature description matrix F(P) of the contour segment P is obtained as follows:

[0130]

[0131] Where abs(·) represents the absolute value, and F1(1) represents the discrete Fourier transform coefficient obtained by performing discrete Fourier transform on the first row of the height feature matrix when ω=1.

[0132] Then compare the similarity between the contour segment P and the contour segment Q in the preset database:

[0133]

[0134] Where S(P,Q) represents the similarity function of the contour segments P and Q, e is used to describe the alignment relationship of the sampling points between the contour segments P and Q, and R P,Q F is a sequence that records the alignment relationship between the sampling points of the contour segment P and the contour segment Q. h (ω) is f h The discrete Fourier coefficients obtained after discrete Fourier transform, f h is the hth row of the height feature matrix of contour segment Q. The dimensions of the height feature matrices of contour segments P and Q are the same.

[0135] By repeating the above steps for each contour segment in the contour of each food monomer, multiple line segments with high similarity to the contour segments in the contour of each food monomer can be obtained in the preset database. According to the type of these contour segments and combined with the color characteristics and texture characteristics of each food monomer, the attribute information of each food monomer can be obtained.

[0136] For example, apples and oranges are similar in size and can both be approximated as spheres. If matching is performed based solely on contour features, large errors are likely to occur. Therefore, it is necessary to combine color features and texture features to more accurately determine the attribute information of the food monomer.

[0137] The pre-set database contains food patterns of various types and sizes, captured from various angles, along with the contour segments, color features, texture features, and a feature description matrix for each contour segment. When multiple food units are occluded, the specific pattern of the occluded portion is determined based on the patterns corresponding to the contour segments with high similarity and the color and texture features of each food unit. This specific pattern is then used to replace the occluded portion, facilitating subsequent processing.

[0138] The embodiment of the present application provides a cloud platform-based hybrid target image recognition method, which is applied to the cloud platform. When occlusion occurs between food monomers, the attribute information of the food monomer can be determined based on the contour features, color features, texture features and a preset database of each food monomer, and the occluded part can be updated. Thereafter, the corresponding food monomer reference object can be determined based on the attribute information of each food monomer, and the content of each nutrient component in the multi-food monomer combination can be calculated based on the volume difference between each food monomer and the corresponding food monomer reference object and the proportion of each nutrient component of the food corresponding to the food monomer reference object, so that the user can formulate a diet plan based on the content of each nutrient component.

[0139] Furthermore, an embodiment of the present application provides a hybrid target image recognition system based on a cloud platform, the system including a cloud platform server and a terminal device, wherein the cloud platform server includes:

[0140] An acquisition module is used to collect original images of a combination of multiple food monomers through a terminal device;

[0141] an identification module, configured to identify the attribute information of each food monomer and determine the food monomer reference object corresponding to each food monomer based on the attribute information of each food monomer;

[0142] The calculation module is used to determine the content of each nutrient component in a combination of multiple food monomers based on each food monomer and the corresponding food monomer reference.

[0143] The cloud platform-based hybrid target image recognition method and system provided in the embodiments of the present application can accurately identify the attribute information of each food monomer when there is occlusion between food monomers, and at the same time determine the corresponding food monomer reference object based on the attribute information of each food monomer, and further calculate the content of each nutrient component in the multi-food monomer combination based on the volume difference between each food monomer and the corresponding food monomer reference object and the proportion of each nutrient component of the food corresponding to the food monomer reference object, so that users can formulate a diet plan based on the content of each nutrient component.

[0144] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A hybrid target image recognition method based on a cloud platform, characterized in that: The mixed target image recognition method is executed by a cloud platform server and is applied to the recognition of a combination of multiple food monomers, wherein the combination of multiple food monomers includes multiple food monomers. The mixed target image recognition method includes: Identifying attribute information of each food unit according to the original image of the combination of multiple food units captured by the terminal device; Determining a food monomer reference object corresponding to each food monomer according to the attribute information of each food monomer; Determining the content of each nutrient component in the multi-food monomer combination according to each food monomer and the corresponding food monomer reference; The identifying of the attribute information of each food unit based on the original image of the combination of multiple food units captured by the terminal device includes: Obtaining contour features, color features, and texture features of each food unit according to the original image; Obtaining attribute information of each food unit according to the outline feature, the color feature, the texture feature and a preset database; The step of obtaining the contour features of each food unit according to the original image includes: For each of the food monomer contours, an iterative operation is performed, the iterative operation comprising: calculating the contribution of each contour point among all current contour points to obtain a first contour point, and replacing the first line segment with the second line segment until all current line segments meet a preset termination condition; the first contour point is the contour point with the smallest contribution among all current contour points, the first line segment is the line segment with the first contour point as an endpoint, the second line segment is the line segment with the second contour point as an endpoint, and each of the two second contour points is one of two contour points adjacent to the first contour point; According to all the current line segments finally outputted, multiple contour segments corresponding to the contour of each food monomer are obtained; wherein, the preset termination condition is ,in is the default value, , is the average value of the rotation angles of all current line segments, is the average value of the initial rotation angles of all line segments, express and degree of difference.

2. The hybrid target image recognition method based on a cloud platform according to claim 1, characterized in that: If the food monomer reference object is a three-dimensional physical model of a standard food monomer, determining the content of each nutrient component in the multi-food monomer combination based on each food monomer and the corresponding food monomer reference object includes: Acquire a first mixed image, where the first mixed image is an image including the combination of multiple food units and the three-dimensional entity model; The content of each nutrient component in the multi-food monomer combination is obtained according to the first mixed image.

3. The hybrid target image recognition method based on a cloud platform according to claim 2, characterized in that: If the food monomer reference object is a standard food monomer image, determining the content of each nutrient component in the multi-food monomer combination according to each food monomer and the corresponding food monomer reference object includes: splicing the original image and the standard food monomer image to obtain a second mixed image; The content of each nutrient component in the multi-food monomer combination is obtained according to the second mixed image.

4. The hybrid target image recognition method based on a cloud platform according to claim 3, characterized in that: Obtaining the content of each nutrient component in the multi-food monomer combination according to the first mixed image or the second mixed image includes: Obtaining a volume difference between each of the food monomers and its corresponding food monomer reference object according to the first mixed image or the second mixed image; The content of each nutrient component in the multi-food monomer combination is obtained based on the proportion of each nutrient component in the food corresponding to the food monomer reference and the volume difference.

5. The hybrid target image recognition method based on a cloud platform according to claim 1, characterized in that: Obtaining the contour feature of each food unit according to the original image includes: Obtaining the outline of each food monomer according to the original image; Obtaining a plurality of contour segments corresponding to the contour of each food unit according to the contour points on the contour of each food unit; The feature description matrix of each contour segment is calculated to obtain the contour feature of each food monomer.

6. The hybrid target image recognition method based on a cloud platform according to claim 5, characterized in that: Obtaining the outline of each food unit according to the original image includes: Selecting a plurality of contour points in the original image according to the gradient magnitude of each pixel in the original image; According to the distances between the multiple contour points, the contour points are divided into multiple groups, each group including contour points, is a positive integer greater than 2; Connect the contour points in each group in sequence to obtain multiple edge segments; selecting a portion of edge segments from the plurality of edge segments according to a corner direction of each edge segment; The partial edge segments are connected to obtain the outline of each food unit.

7. The hybrid target image recognition method based on a cloud platform according to claim 5, characterized in that: The calculating of the feature description matrix of each contour segment comprises: Selecting a plurality of sampling points in each of the contour segments, and determining a reference axis corresponding to each of the sampling points; Obtaining a height feature matrix of each contour segment according to each sampling point and the reference axis; A feature description matrix for each contour segment is obtained according to the height feature matrix and discrete Fourier transform.

8. A cloud-based hybrid target image recognition system, used to implement the cloud-based hybrid target image recognition method according to any one of claims 1 to 7, characterized in that: The system includes a cloud platform server and a terminal device, wherein the cloud platform server includes: An acquisition module is used to collect original images of a combination of multiple food monomers through a terminal device; an identification module, configured to identify attribute information of each food monomer and determine a food monomer reference object corresponding to each food monomer based on the attribute information of each food monomer; a calculation module, configured to determine the content of each nutrient component in the multi-food monomer combination based on each food monomer and the corresponding food monomer reference; The identifying of the attribute information of each food unit based on the original image of the combination of multiple food units captured by the terminal device includes: Obtaining contour features, color features, and texture features of each food unit according to the original image; Obtaining attribute information of each food unit according to the outline feature, the color feature, the texture feature and a preset database; The step of obtaining the contour features of each food unit according to the original image includes: For each of the food monomer contours, an iterative operation is performed, the iterative operation comprising: calculating the contribution of each contour point among all current contour points to obtain a first contour point, and replacing the first line segment with the second line segment until all current line segments meet a preset termination condition; the first contour point is the contour point with the smallest contribution among all current contour points, the first line segment is the line segment with the first contour point as an endpoint, the second line segment is the line segment with the second contour point as an endpoint, and each of the two second contour points is one of two contour points adjacent to the first contour point; According to all the current line segments finally outputted, multiple contour segments corresponding to the contour of each food monomer are obtained; wherein, the preset termination condition is ,in is the default value, , is the average value of the rotation angles of all current line segments, is the average value of the initial rotation angles of all line segments, express and degree of difference.

Images