Hybrid target image recognition method and system based on cloud platform

Abstract

The invention discloses a mixed target image recognition method and system based on a cloud platform, and belongs to the technical field of image recognition, and the key points of the technical scheme are that the method comprises the steps: recognizing the attribute information of each food monomer according to an original image of a multi-food monomer combination collected by terminal equipment; determining a food monomer reference object corresponding to each food monomer according to the attribute information of each food monomer; according to each food monomer and each corresponding food monomer reference substance, the content of each nutritional ingredient in the multi-food monomer combination is determined, and the content of each nutritional ingredient in the multi-food monomer combination can be accurately obtained under the condition that the attribute information of each food monomer is unknown; therefore, the user can make a diet plan according to the content of each nutritional ingredient.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and more particularly to a method and system for hybrid target image recognition based on a cloud platform. Background Art

[0002] For patients with certain diseases, it is necessary to strictly control their diet. However, there are currently a wide variety of foods, and it is usually difficult to accurately determine the nutritional components of unknown foods and formulate a diet plan.

[0003] The Chinese patent application with the publication number CN109856345A provides a method and system for fruit quality recognition. The invention establishes a fruit quality recognition model based on historical fruit parameters and historical fruit quality, and determines the fruit quality of the measured fruit according to the fruit quality recognition model, thereby realizing automatic recognition of fruit quality. However, this invention can only recognize the size and maturity of fruits and cannot obtain the nutritional components of fruits. Therefore, there are deficiencies in the prior art. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the purpose of the present invention is to provide a method and system for hybrid target image recognition based on a cloud platform, which can obtain the attribute information of each food monomer and the food monomer reference object corresponding to each food monomer through image recognition technology when the attribute information of each food monomer is unknown, and calculate the content of each nutritional component in the multi-food monomer combination according to the volume difference between each food monomer and the corresponding food monomer reference object and the proportion of each nutritional component of the food corresponding to the food monomer reference object, so that users can formulate a diet plan according to the content of each nutritional component.

[0005] To achieve the above purpose, the present invention provides the following technical solutions:

[0006] The present invention provides a method for hybrid target image recognition based on a cloud platform. The hybrid target image recognition method is executed by a cloud platform server. The hybrid target image recognition method is applied to the recognition of a multi-food monomer combination, and the multi-food monomer combination includes a plurality of food monomers. The hybrid target image recognition method includes:

[0007] Identifying the attribute information of each food monomer according to the original image of the multi-food monomer combination collected by the terminal device;

[0008] Determining the food monomer reference object corresponding to each food monomer according to the attribute information of each food monomer;

[0009] Determining the content of each nutritional component in the multi-food monomer combination according to each food monomer and the corresponding food monomer reference object.

[0010] As a further improvement of the present invention, if the food monomer reference object is a three-dimensional solid model of a standard food monomer, determining the content of each nutrient component in the multi-food monomer combination according to each of the food monomers and the corresponding food monomer reference object includes:

[0011] Obtaining a first mixed image, where the first mixed image is an image that simultaneously includes the multi-food monomer combination and the three-dimensional solid model;

[0012] According to the first mixed image, obtaining the content of each nutrient component in the multi-food monomer combination.

[0013] As a further improvement of the present invention, if the food monomer reference object is a standard food monomer image, determining the content of each nutrient component in the multi-food monomer combination according to each of the food monomers and the corresponding food monomer reference object includes:

[0014] Stitching the original image and the standard food monomer image to obtain a second mixed image;

[0015] According to the second mixed image, obtaining the content of each nutrient component in the multi-food monomer combination.

[0016] As a further improvement of the present invention, obtaining the content of each nutrient component in the multi-food monomer combination according to the first mixed image or the second mixed image includes:

[0017] According to the first mixed image or the second mixed image, obtaining the volume difference between each food monomer and its corresponding food monomer reference object;

[0018] According to the proportion of each nutrient component in the food corresponding to the food monomer reference object and the volume difference, obtaining the content of each nutrient component in the multi-food monomer combination.

[0019] As a further improvement of the present invention, identifying the attribute information of each food monomer according to the original image of the multi-food monomer combination collected by the terminal device includes:

[0020] Obtaining the contour feature, color feature, and texture feature of each food monomer according to the original image;

[0021] According to the contour feature, the color feature, the texture feature, and a preset database, obtaining the attribute information of each food monomer.

[0022] As a further improvement of the present invention, obtaining the contour feature of each food monomer according to the original image includes:

[0023] Obtain the contour of each food monomer according to the original image;

[0024] Obtain a plurality of contour segments corresponding to the contour of each food monomer according to the contour points on the contour of each food monomer;

[0025] Calculate the feature description matrix of each contour segment to obtain the contour features of each food monomer.

[0026] As a further improvement of the present invention, the obtaining the contour of each food monomer according to the original image includes:

[0027] Select a plurality of contour points in the original image according to the gradient magnitude of each pixel point in the original image;

[0028] Divide the contour points into multiple groups according to the distance between the plurality of contour points, each group containing γ contour points, where γ is a positive integer greater than 2;

[0029] Connect the contour points within each group in sequence to obtain a plurality of edge segments;

[0030] Select some edge segments from the plurality of edge segments according to the turning direction of each edge segment;

[0031] Connect the partial edge segments to obtain the contour of each food monomer.

[0032] As a further improvement of the present invention, the obtaining a plurality of contour segments corresponding to the contour of each food monomer according to the contour points on the contour of each food monomer includes:

[0033] For the contour of each food monomer, perform an iterative operation respectively. The iterative operation includes calculating the contribution degree of each contour point among all current contour points respectively to obtain a first contour point, and replacing the first line segment with a second line segment until all current line segments meet a preset termination condition; the first contour point is the contour point with the smallest contribution degree among all current contour points, the first line segment is the line segment with the first contour point as an end point, the second line segment is the line segment with a second contour point as an end point, and each of the two second contour points is one of the two adjacent contour points of the first contour point;

[0034] Obtain a plurality of contour segments corresponding to the contour of each food monomer according to the finally output all current line segments; wherein, the preset termination condition is D < d, where d is a preset value, A 0 is the average value of the turning angles of all current line segments, A is the average value of the turning angles of all initial line segments, and D represents the 0 difference degree between A and A.

[0035] As a further improvement of the present invention, calculating the feature description matrix of each of the contour segments includes:

[0036] Selecting a plurality of sampling points in each of the contour segments and determining a reference axis corresponding to each of the sampling points;

[0037] Obtaining a height feature matrix of each of the contour segments according to each of the sampling points and the reference axis;

[0038] Obtaining the feature description matrix of each of the contour segments according to the height feature matrix and the discrete Fourier transform.

[0039] The present invention provides a hybrid target image recognition system based on a cloud platform. The system includes a cloud platform server and a terminal device. The cloud platform server includes:

[0040] An acquisition module, configured to collect an original image of a multi-food monomer combination through a terminal device;

[0041] An identification module, configured to identify the attribute information of each food monomer, and determine a food monomer reference object corresponding to each food monomer according to the attribute information of each food monomer;

[0042] A calculation module, configured to determine the content of each nutrient component in the multi-food monomer combination according to each food monomer and the corresponding food monomer reference object.

[0043] Based on the contour features, color features, texture features of each food monomer and a preset database, the present invention can accurately identify the attribute information of each food monomer in the case of occlusion between food monomers. At the same time, according to the attribute information of each food monomer, a food monomer reference object corresponding thereto is determined, and further, according to the volume difference between each food monomer and the corresponding food monomer reference object and the proportion of each nutrient component of the food corresponding to the food monomer reference object, the content of each nutrient component in the multi-food monomer combination is calculated, so that a user can formulate a diet plan according to the content of each nutrient component. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Is a flowchart of the method steps in the present invention;

[0045] Figure 2 Is a schematic diagram of an ellipsoid;

[0046] Figure 3 Is a schematic diagram of a scene including multiple food monomers;

[0047] Figure 4 Is a schematic diagram of contour points;

[0048] Figure 5 Schematic diagram of edge segments;

[0049] Figure 6 Schematic diagram of the contour of a single food item;

[0050] Figure 7 Schematic diagram of the structure of line segments and corners;

[0051] Figure 8 Schematic diagram of the contour after multiple cycles. Detailed implementation manners

[0052] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention.

[0053] The term "and / or" in the following text is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " generally represents an "or" relationship between the associated objects before and after.

[0054] As Figure 1 shown, an embodiment of the present application provides a hybrid target image recognition method based on a cloud platform. The hybrid target image recognition method is executed by a cloud platform server. The hybrid target image recognition method is applied to the recognition of a multi-food monomer combination. The multi-food monomer combination includes multiple food monomers. The hybrid target image recognition method includes:

[0055] Identifying the attribute information of each food monomer according to the original image of the multi-food monomer combination collected by the terminal device;

[0056] Determining a food monomer reference object corresponding to each food monomer according to the attribute information of each food monomer;

[0057] Determining the content of each nutrient component in the multi-food monomer combination according to each food monomer and the corresponding food monomer reference object.

[0058] Among them, the food monomer reference object can be a standard food monomer image or a three-dimensional solid model of a standard food monomer.

[0059] Exemplarily, when the hybrid target is multiple fruits and vegetables, the original image of the hybrid target is an image containing multiple fruits and vegetables. Each food monomer is each fruit and vegetable included in the hybrid target. The attribute information of each food monomer includes the type and specific food name to which each food monomer belongs. The food monomer reference object is a standard image or a standard three-dimensional model of each fruit and vegetable.

[0060] The method provided in this embodiment can obtain the attribute information of each food monomer through image recognition technology when the attribute information of each food monomer is unknown, determine the corresponding food monomer reference object, and calculate the content of each nutrient component in the multi-food monomer combination according to the volume difference between each food monomer and the corresponding food monomer reference object and the proportion of each nutrient component of the food corresponding to the food monomer reference object, so that users can formulate a diet plan based on the content of each nutrient component.

[0061] Further, if the food monomer reference object is a three-dimensional solid model of a standard food monomer, this embodiment provides a step of determining the content of each nutrient component in the multi-food monomer combination according to each food monomer and the corresponding food monomer reference object, including:

[0062] Obtain a first mixed image, where the first mixed image is an image that simultaneously includes the multi-food monomer combination and the three-dimensional solid model;

[0063] According to the first mixed image, obtain the content of each nutrient component in the multi-food monomer combination.

[0064] There can be multiple first mixed images, and each first mixed image corresponds to a shooting angle.

[0065] Further, if the food monomer reference object is a standard food monomer image, this embodiment provides a method for determining the content of each nutrient component in the multi-food monomer combination according to each food monomer and the corresponding food monomer reference object, including:

[0066] Stitch the original image and the standard food monomer image to obtain a second mixed image;

[0067] According to the second mixed image, obtain the content of each nutrient component in the multi-food monomer combination.

[0068] Among them, the shooting conditions of the standard food monomer image and the original image need to be the same, that is, it is necessary to ensure that the scaling ratio of each food monomer is the same, and the standard food monomer image, the original image, and the second mixed image can all be multiple. Each standard food monomer image and the original image correspond to a shooting angle. Stitch the standard food monomer image and the original image with the same shooting angle, so that each stitched second mixed image corresponds to a shooting angle.

[0069] Further, this embodiment provides a method for obtaining the content of each nutrient component in the multi-food monomer combination according to the first mixed image or the second mixed image, including:

[0070] According to the first mixed image or the second mixed image, obtain the volume difference between each food monomer and its corresponding food monomer reference object;

[0071] The content of each nutrient in the multi-food monomer combination is obtained according to the proportion and volume difference of each nutrient in the food corresponding to the food monomer reference object.

[0072] Exemplarily, assume that a food monomer is an orange, and its corresponding food monomer reference object is an orange model of standard size, and the size data of the orange model is known. The first mixed image is an image that includes both the orange and the orange model. At this time, the calculation method of the volume difference between the orange and the orange model is as follows:

[0073] Approximate the orange and the orange model as spheres. According to image recognition technology, the projected areas s 1 and s 2 of the orange and the orange model on the plane can be obtained. According to the circle area formula, the projected radii r 1 and r 2 of the orange and the orange model can be calculated. Since the radius of a sphere has a direct relationship with its volume, the volume of the orange can be calculated according to the proportional relationship between the volume and the radius of the sphere. Furthermore, the volume difference ΔV = |V 1 -V 0 | is obtained, where V 0 is the volume of the orange model, which is known data.

[0074] Exemplarily, assume that a food monomer is a wax gourd, and its corresponding food monomer reference object is an image of a wax gourd of standard size, and the size data of the wax gourd of standard size is known. The second mixed image is the front view and top view formed by splicing the original image of the wax gourd and the image of the wax gourd of standard size. At this time, the calculation method of the volume difference is as follows:

[0075] As Figure 2 shown, approximate the wax gourd as an ellipsoid. According to the second mixed image, the three semi-axis lengths a 1 , b 1 , c 1 of the wax gourd can be obtained. According to the size data of the wax gourd of standard size, the three semi-axis lengths a 2 , b 2 , c 2 of the wax gourd of standard size can be obtained. Since the volume ratio of similar ellipsoids is equal to the cube of the ratio of the corresponding semi-axis lengths, the volume V 2 =V 3 (k 1 k 2 k 3 ) 3 is obtained, where respectively represent the proportional relationships of the three semi-axes. Furthermore, the volume difference ΔV 1 =|V 2 -V3 |, where V 3 is the volume of a wax gourd of standard size.

[0076] Specifically, the nutrients in the food corresponding to the food monomer reference object include sugar content, vitamin content, protein content, etc. The proportion of each nutrient is the mass of each nutrient in every 100g of food.

[0077] Furthermore, based on the volume difference and the density of the food, the mass difference between the food monomer and the food of standard size can be calculated. Based on the mass difference and the proportion of each nutrient, the content of each nutrient in each food monomer can be calculated, and then the content of each nutrient in the multi-food monomer combination can be obtained.

[0078] Exemplarily, continuing with the above example of an orange, assuming the density of the orange is ρ, the mass difference m = ρυ can be calculated according to the relationship between volume, density, and mass. Assuming the sugar content of the orange is βg / 100g, then compare V 1 with V 0 If V 1 > V 0 , then the sugar content of the orange is If V 1 < V 0 , then the sugar content of the orange is If V 1 = V 0 , then the sugar content of the orange is where m 1 = ρV 0 represents the mass of an orange with a volume of V 0 .

[0079] The method provided in this embodiment approximates the food as a common geometric solid figure, and combines image recognition technology and known size data to calculate the volume and mass of each food monomer more accurately. Then, combined with the proportion of each nutrient in the food of standard size, the content of each nutrient in each food monomer can be calculated more accurately.

[0080] However, the examples in this embodiment only introduce the calculation method for a single food monomer. Therefore, there is no phenomenon of inaccurate image recognition caused by occlusion between food monomers. As Figure 3 shown, in actual recognition, there are multiple food monomers, and occlusion is likely to occur between multiple food monomers. Therefore, the occluded area needs to be separated to obtain the complete part and the occluded part. The complete part can still be calculated by the above method. For the occluded part, according to its contour features, color features, and texture features, it needs to be replaced with a complete food pattern in the preset database, and then calculated by the above method.

[0081] Further, this embodiment provides a method for identifying the attribute information of each food monomer according to the original image of a multi-food monomer combination collected by a terminal device, including:

[0082] Obtaining the contour feature, color feature, and texture feature of each food monomer from the original image;

[0083] Obtaining the attribute information of each food monomer according to the contour feature, color feature, texture feature, and a preset database.

[0084] Specifically, this embodiment uses the HSV color space to extract the color feature of each food monomer. Since the original image belongs to the RGB color space, it is necessary to convert it from the RGB color space to the HSV color space. The specific method is as follows:

[0085] R' = R / 255

[0086] G' = G / 255

[0087] B' = B / 255

[0088] U MAX = MAX(R', G', B')

[0089] U MIN = MIN(R', G', B')

[0090]

[0091] V = U MAX

[0092] Among them, H is the hue value, and its value range is 0° to 360°. S is the saturation value, and its value range is 0 to 100%. The larger the saturation value, the more saturated the color. V is the brightness value, and its value is 0 (black) or 100% (white). R, G, and B respectively represent red, green, and blue in the RGB color space. R', G', and B' are the results of normalizing R, G, and B. U MAX represents the maximum value among R, G, and B. MAX(·) represents the maximum value function. U MIN represents the minimum value among R, G, and B. MIN(·) represents the minimum value function.

[0093] This embodiment uses the HSV color space to extract the color feature of each food monomer. Compared with the RGB color space, the HSV color space is more suitable for the color expression of the human eye vision and is easier to adjust.

[0094] Specifically, the texture feature of any pixel point in each food monomer in the original image can be expressed as:

[0095]

[0096] Among them, g c represents the gray value of any pixel point in each food monomer, and g t represents the gray value of any pixel point within the preset neighborhood of this pixel point. The preset neighborhood is circular. δ(g t - g c ) is the sign function. When g t - g c ≥ 0, δ(g t - g c ) = 1. When g t - g c < 0, δ(g t - g c ) = 0. T represents the number of pixel points included in the preset neighborhood, and i represents the radius of the preset neighborhood. By combining the texture features of each pixel point in each food monomer respectively, the texture feature of each food monomer can be obtained.

[0097] The texture feature provided in this embodiment has rotational invariance and gray invariance, that is, this texture feature is not affected by image rotation or illumination change, has a certain robustness in dealing with illumination influence and noise. At the same time, the calculation steps of this method are relatively simple, can effectively capture texture features with a relatively small amount of calculation. And this method can capture texture features of different scales by adjusting the size of the preset neighborhood, and can meet the needs of multiple scales.

[0098] Furthermore, this embodiment provides a step of obtaining the contour feature of each food monomer from the original image, including:

[0099] Obtaining the contour of each food monomer from the original image;

[0100] Obtaining multiple contour segments corresponding to the contour of each food monomer according to the contour points on the contour of each food monomer;

[0101] Calculating the feature description matrix of each contour segment to obtain the contour feature of each food monomer.

[0102] Furthermore, this embodiment provides a method for obtaining the contour of each food monomer from the original image, including:

[0103] Selecting multiple contour points in the original image according to the gradient amplitude of each pixel point in the original image;

[0104] Dividing the contour points into multiple groups according to the distance between the multiple contour points, and each group contains γ contour points, where γ is a positive integer greater than 2;

[0105] Connect the contour points within each group in sequence to obtain multiple edge segments;

[0106] Select some edge segments from the multiple edge segments according to the turning direction of each edge segment;

[0107] Connect the selected edge segments to obtain the contour of each food monomer.

[0108] Furthermore, this embodiment provides a method for obtaining multiple contour segments corresponding to the contour of each food monomer according to the contour points on the contour of each food monomer, including:

[0109] For the contour of each food monomer, perform iterative operations respectively. The iterative operations include calculating the contribution degree of each contour point among all current contour points respectively to obtain the first contour point, replacing the first line segment with the second line segment until all current line segments meet the preset termination condition; the first contour point is the contour point with the smallest contribution degree among all current contour points, the first line segment is the line segment with the first contour point as an endpoint, the second line segment is the line segment with the second contour point as an endpoint, and each of the two second contour points is one of the two adjacent contour points of the first contour point;

[0110] Obtain multiple contour segments corresponding to the contour of each food monomer according to all the current line segments finally output; where the preset termination condition is D < d, where d is a preset value, A 0 is the average value of the turning angles of all current line segments, A is the average value of the turning angles of all initial line segments, and D represents the difference degree between A 0 and A.

[0111] Specifically, in an image, the boundary position between a food monomer and the background or between food monomers usually shows a sudden change in gray value. This sudden change in gray value can be measured by gradient. Therefore, in this embodiment, multiple contour points are selected in the original image through the gradient amplitude of each pixel point. However, due to problems such as background interference, some of the selected multiple contour points may include interference points that do not belong to the contour of the food monomer, as Figure 4 shown. In order to accurately obtain the contour points of the contour of each food monomer from the selected multiple contour points, the contour points can be divided into multiple groups according to the distance between the contour points, as Figure 5 shown. Each group contains 3 contour points. Connect the contour points within each group in sequence to obtain multiple edge segments. Then, according to the turning direction of each edge segment, select some edge segments that can form the contour of the food monomer from the multiple edge segments, and connect the selected edge segments to obtain the contour of the food monomer, as Figure 6 shown.

[0112] Specifically, the method of selecting partial edge segments that can form the contour of a food monomer from multiple edge segments according to the turning direction of each edge segment includes: First, according to the turning directions of two adjacent edge segments, determine whether they can match each other to achieve a smooth transition of the contour. Generally, the turning directions of two adjacent edge segments of a closed contour should be the same. Then, calculate the turning angle between the two edge segments at the connection point and determine whether it conforms to the angle change law that the object contour should have at this position. At the same time, to ensure the accuracy of the contour of the obtained food monomer, the obtained contour of the food monomer can be further inspected, that is, from a geometric perspective, determine whether the obtained contour is strange in shape and does not conform to the geometric contour of common objects.

[0113] Next, for one of each food monomer, record the contour of this food monomer as contour C = {S σ}, (σ = 1, 2, …, M), where S σ is the σ-th line segment in contour C, M is the total number of line segments included in contour C, and P n (n = 1, 2, …, N) represents the n-th contour point of contour C, and N is the total number of contour points.

[0114] After that, in order to reduce the number of contour points and improve the recognition efficiency, it is necessary to calculate the contour point p i (i = 1, 2, …, n) with the smallest contribution in each loop, delete this contour point, and at the same time replace the line segment with p i as the endpoint with the line segment with p i-1 and p i+1 as the endpoints. The specific calculation formula for the contribution degree K(S i , S i+1 ) is as follows:

[0115]

[0116] As Figure 7 shown, where w(S i , S i+1 ) is the turning angle between line segments S i and S i+1 , S i represents the line segment with p i and p i-1 as the endpoints, S i+1 represents the line segment with p i and p i+1 as the endpoints, l(S i ) and l(S i+1 ) are the lengths of line segments S i and S i+1 normalized with respect to the contour perimeter. Exemplarily, assume the contour perimeter is C 1, line segment S i has a length of C 2 ,

[0117] In this embodiment, the contribution degree of contour points is measured by the length and turning angle of line segments. The length of the line segment reflects the stretching degree of the line segment in the local area and plays a key role in describing the boundary characteristics of the object. For example, when describing the contour of an object, the parts where the long line segments are located on the contour are often the prominent and stable boundary regions of the object shape and contribute greatly to defining the overall shape of the object. The turning angle describes the degree of direction change between adjacent line segments and can well capture the key details of the contour shape change.

[0118] Next, calculate the average angle A of all turning angles in the updated multiple line segments after deleting this contour point 0 , and calculate A 0 The difference degree D from the average angle A of all turning angles in the initial multiple line segments:

[0119]

[0120] If the value of the difference degree D is less than the preset value d, terminate the loop to obtain multiple contour segments of this food monomer. The preset value d can be set according to the number of contour points. Repeat the above steps for the contour of each food monomer to obtain multiple contour segments corresponding to each food monomer.

[0121] As Figure 8 shown, when the number of loop times is too large, the originally relatively smooth contour will become sharp with the deletion of contour points. At this time, the turning angles of adjacent line segments will become smaller. Therefore, this embodiment avoids the above situation through the preset termination condition, can delete the contour points that contribute little to the contour characteristics of the food monomer while trying to retain the shape of the original contour and avoid the phenomenon of excessive evolution.

[0122] Furthermore, the embodiment of the present application provides a step of calculating the feature description matrix of each contour segment, including:

[0123] Select multiple sampling points in each contour segment respectively and determine the reference axis corresponding to each sampling point;

[0124] According to each sampling point and the reference axis, obtain the height feature matrix of each contour segment;

[0125] According to the height feature matrix and the discrete Fourier transform, obtain the feature description matrix of each contour segment.

[0126] Specifically, for each contour segment P in the contour of each food monomer, select J sampling points in P. For each sampling point P in P j, for \(j = 1, 2, \ldots, J\), passing through point \(P\) j Make a horizontal line parallel to the \(x\)-axis as the reference axis, and then calculate the distances between all sampling points except \(P\) j and the reference axis, obtaining the height feature vector \(H\) of sampling point \(P\) j . Repeat the above steps for each sampling point, and the height feature vectors corresponding to each sampling point can be obtained. After that, combine the height feature vectors corresponding to each sampling point to obtain the height feature matrix of contour segment \(P\). j . Repeat the above steps for each sampling point, and the height feature vectors corresponding to each sampling point can be obtained. After that, combine the height feature vectors corresponding to each sampling point to obtain the height feature matrix of contour segment \(P\).

[0127] Let \(f\) v represent the \(v\)-th row of the height feature matrix. Perform the discrete Fourier transform on \(f\) v to obtain:

[0128]

[0129] where \(q\) is the imaginary unit, \(F\) v (\(\omega\)) represents the discrete Fourier transform coefficient, \(\omega = 1, 2, \ldots, Y\), \(Y\) is the Fourier order, \(f\) v (\(t\)) is the element in the \(v\)-th row and \(t\)-th column of the height feature matrix. According to the discrete Fourier transform coefficient, the feature description matrix \(F(P)\) of contour segment \(P\) is:

[0130]

[0131] where \(abs(\cdot)\) represents taking the absolute value, \(F\) 1 (1) represents the discrete Fourier transform coefficient obtained by performing the discrete Fourier transform on the first row of the height feature matrix when \(\omega = 1\).

[0132] Then, compare the similarity between contour segment \(P\) and contour segment \(Q\) in the preset database:

[0133]

[0134] where \(S(P, Q)\) represents the similarity function between contour segment \(P\) and contour segment \(Q\), \(e\) is used to describe the alignment relationship of sampling points between contour segment \(P\) and contour segment \(Q\), \(R\) P,Q is the sequence recording the alignment relationship of sampling points between contour segment \(P\) and contour segment \(Q\), \(F\) h (\(\omega\)) is the discrete Fourier coefficient obtained after performing the discrete Fourier transform on \(f\) h , \(f\) h is the \(h\)-th row of the height feature matrix of contour segment \(Q\). The dimensions of the height feature matrices of contour segment \(P\) and contour segment \(Q\) are the same.

[0135] Repeat the above steps for each contour segment in the contour of each food monomer, and multiple line segments with high similarity to the contour segments in the contour of each food monomer can be obtained in the preset database. According to the types to which these contour segments belong, combined with the color features and texture features of each food monomer, the attribute information of each food monomer can be obtained.

[0136] Exemplarily, apples and oranges are similar in size and can both be approximated as spheres. If only matching based on contour features, large errors are likely to occur. Therefore, it is necessary to combine color features and texture features to more accurately determine the attribute information of food monomers.

[0137] The preset database stores food patterns of different types and specifications taken at multiple angles, the contour segments of each pattern, color features, texture features, and the feature description matrix of each contour segment. When there are multiple food monomers and there is an occlusion situation, it is necessary to determine the specific pattern of the occluded part according to the patterns corresponding to these contour segments with high similarity and the color features and texture features of each food monomer, and replace the occluded part with the specific pattern to facilitate the subsequent work.

[0138] The hybrid target image recognition method based on a cloud platform provided by the embodiments of the present application is applied to the cloud platform. It can, in the case of occlusion between food monomers, determine the attribute information of food monomers according to the contour features, color features, texture features of each food monomer and the preset database, and update the occluded part. Then, according to the attribute information of each food monomer, determine the food monomer reference object corresponding to it, and calculate the content of each nutrient component in the multi-food monomer combination according to the volume difference between each food monomer and the corresponding food monomer reference object and the proportion of each nutrient component of the food corresponding to the food monomer reference object, so that users can formulate a diet plan according to the content of each nutrient component.

[0139] Further, the embodiments of the present application provide a hybrid target image recognition system based on a cloud platform. The system includes a cloud platform server and a terminal device. The cloud platform server includes:

[0140] An acquisition module, configured to acquire the original image of the multi-food monomer combination through the terminal device;

[0141] An identification module, configured to identify the attribute information of each food monomer, and determine the food monomer reference object corresponding to each food monomer according to the attribute information of each food monomer;

[0142] A calculation module, configured to determine the content of each nutrient component in the multi-food monomer combination according to each food monomer and the corresponding food monomer reference object.

[0143] The hybrid target image recognition method and system based on a cloud platform provided by the embodiments of the present application can accurately identify the attribute information of each food monomer when there is occlusion between food monomers, and at the same time determine the corresponding food monomer reference object according to the attribute information of each food monomer, and further calculate the content of each nutrient component in the multi-food monomer combination according to the volume difference between each food monomer and the corresponding food monomer reference object and the proportion of each nutrient component of the food corresponding to the food monomer reference object, so that users can formulate a diet plan according to the content of each nutrient component.

[0144] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 a device for realizing the functions specified in one block or multiple blocks.

[0146] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device realizes the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 a device for realizing the functions specified in one block or multiple blocks.

[0147] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, several improvements and retouches made without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A hybrid target image recognition method based on a cloud platform, characterized in that: The mixed target image recognition method is executed by a cloud platform server, and the mixed target image recognition method is applied to the recognition of a combination of multiple food monomers, wherein the combination of multiple food monomers includes multiple food monomers, and the mixed target image recognition method includes: According to the original image of the combination of multiple food monomers collected by the terminal device, identifying the attribute information of each of the food monomers; Determining a food monomer reference object corresponding to each food monomer according to the attribute information of each food monomer; The content of each nutrient component in the multi-food monomer combination is determined based on each of the food monomers and the corresponding food monomer reference.

2. The hybrid target image recognition method based on a cloud platform according to claim 1, characterized in that: If the food monomer reference object is a three-dimensional physical model of a standard food monomer, determining the content of each nutrient component in the multi-food monomer combination according to each food monomer and the corresponding food monomer reference object includes: Acquire a first mixed image, wherein the first mixed image is an image including the combination of multiple food monomers and the three-dimensional entity model; The content of each nutrient component in the multi-food monomer combination is obtained according to the first mixed image.

3. The hybrid target image recognition method based on a cloud platform according to claim 2, characterized in that: If the food monomer reference is a standard food monomer image, determining the content of each nutrient component in the multi-food monomer combination according to each food monomer and the corresponding food monomer reference includes: splicing the original image and the standard food monomer image to obtain a second mixed image; According to the second mixed image, the content of each nutrient component in the multi-food monomer combination is obtained.

4. The hybrid target image recognition method based on a cloud platform according to claim 3, characterized in that: Obtaining the content of each nutrient component in the multi-food monomer combination according to the first mixed image or the second mixed image includes: Obtaining a volume difference between each of the food monomers and its corresponding food monomer reference object according to the first mixed image or the second mixed image; The content of each nutrient component in the multi-food monomer combination is obtained according to the proportion of each nutrient component in the food corresponding to the food monomer reference and the volume difference.

5. The hybrid target image recognition method based on a cloud platform according to claim 1, characterized in that: The identifying of the attribute information of each food monomer according to the original image of the combination of multiple food monomers collected by the terminal device includes: Obtaining contour features, color features and texture features of each food monomer according to the original image; According to the outline feature, the color feature, the texture feature and a preset database, the attribute information of each food monomer is obtained.

6. The hybrid target image recognition method based on a cloud platform according to claim 5, characterized in that: The step of obtaining the contour feature of each food monomer according to the original image includes: Obtaining the outline of each food monomer according to the original image; According to the contour points on the contour of each food monomer, a plurality of contour segments corresponding to the contour of each food monomer are obtained; The feature description matrix of each contour segment is calculated to obtain the contour feature of each food monomer.

7. The hybrid target image recognition method based on a cloud platform according to claim 6, characterized in that: The step of obtaining the outline of each food monomer according to the original image comprises: Selecting a plurality of contour points in the original image according to the gradient amplitude of each pixel in the original image; According to the distances between the plurality of contour points, the contour points are divided into a plurality of groups, each group comprising γ contour points, where γ is a positive integer greater than 2; Connect the contour points in each group in sequence to obtain multiple edge segments; Selecting some edge segments from the plurality of edge segments according to the corner direction of each edge segment; The partial edge segments are connected to obtain the outline of each food monomer.

8. The hybrid target image recognition method based on a cloud platform according to claim 6, characterized in that: The step of obtaining a plurality of contour segments corresponding to the contour of each food monomer according to the contour points on the contour of each food monomer comprises: For each of the contours of the food monomer, an iterative operation is performed respectively, the iterative operation includes respectively calculating the contribution of each contour point among all the current contour points to obtain a first contour point, replacing the first line segment with the second line segment until all the current line segments meet a preset termination condition; the first contour point is the contour point with the smallest contribution among all the current contour points, the first line segment is the line segment with the first contour point as an endpoint, the second line segment is the line segment with the second contour point as an endpoint, and the two second contour points are each one of two adjacent contour points of the first contour point; Based on all the current line segments in the final output, multiple contour segments corresponding to the contour of each food monomer are obtained; wherein, the preset termination condition is D < d, where d is a preset value. A0 is the average value of the turning angles of all the current line segments, A is the average value of the turning angles of all the initial line segments, and D represents the degree of difference between A0 and A.

9. The hybrid target image recognition method based on a cloud platform according to claim 6, characterized in that: The calculating of the feature description matrix of each contour segment comprises: Selecting a plurality of sampling points in each of the contour segments, and determining a reference axis corresponding to each of the sampling points; Obtaining a height feature matrix of each contour segment according to each sampling point and the reference axis; A feature description matrix for each contour segment is obtained according to the height feature matrix and discrete Fourier transform.

10. A hybrid target image recognition system based on a cloud platform, used to implement a hybrid target image recognition method based on a cloud platform as claimed in any one of claims 1 to 9, characterized in that: The system includes a cloud platform server and a terminal device, wherein the cloud platform server includes: An acquisition module, used for acquiring original images of a combination of multiple food monomers through a terminal device; An identification module, used to identify the attribute information of each food monomer, and determine the food monomer reference corresponding to each food monomer according to the attribute information of each food monomer; The calculation module is used to determine the content of each nutrient component in the multi-food monomer combination according to each of the food monomers and the corresponding food monomer reference.

Images