Artificial Intelligence-Based Monitoring Method, System and Cloud Platform for Pre-Made Food Production

Abstract

The artificial intelligence-based pre-prepared food production monitoring method, system and cloud platform of the present invention start from the idea of ​​image processing, first obtain the production monitoring image of the target pre-prepared food, and then use the debugged image processing network to perform quality analysis of the production line link on the production monitoring image to obtain the quality supervision analysis result. In this way, the production quality under multiple production line links can be considered based on the image level, so as to reflect whether the production of the target pre-prepared food is compliant or meets the quality requirements from the overall / full assembly line level. After obtaining the quality supervision analysis results, the production line of the automated kitchen can be improved and processed in combination with the quality supervision analysis results, so as to adaptively adjust the overall production line of the automated kitchen, thereby achieving quality control of the target pre-prepared food. The whole process is efficient and highly operational, reducing the monitoring error caused by the subjective experience of human monitoring, and avoiding production quality fluctuations of large quantities of target pre-prepared dishes as much as possible.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly relates to a method, system and cloud platform for monitoring the production of prefabricated dishes based on artificial intelligence. Background Art

[0002] Prepared dishes, also known as pre-prepared food products, are made from one or more agricultural products as the main raw materials, using standardized assembly line operations, and are pre-processed (such as cutting, mixing, marinating, tumbling, shaping, seasoning, etc.) and / or pre-cooked (such as stir-frying, frying, roasting, boiling, steaming, etc.), and are pre-packaged finished or semi-finished dishes. With the accelerating pace of life, prepared dishes are favored by more and more office workers. To ensure food safety and market competitiveness, it is particularly important to control the production quality of prepared dishes. However, in the face of large-scale production, the traditional monitoring methods for prepared dish production are inefficient and lack operability, and there will also be monitoring errors due to human subjective experience. Summary of the Invention

[0003] To improve the technical problems existing in the related art, the present invention provides a method, system and cloud platform for monitoring the production of prefabricated dishes based on artificial intelligence.

[0004] In a first aspect, an embodiment of the present invention provides a method for monitoring the production of prefabricated dishes based on artificial intelligence, which is applied to a cloud platform for monitoring the production of prefabricated dishes. The method includes: obtaining a production monitoring image of a target prefabricated dish collected and sent by a camera arranged in an automated kitchen;

[0005] Using an image processing network that has been debugged, performing quality analysis on the production monitoring image for production line links to obtain a quality supervision analysis result;

[0006] According to the quality supervision analysis result, performing improvement processing on the production line of the automated kitchen.

[0007] In some possible examples, the performing improvement processing on the production line of the automated kitchen according to the quality supervision analysis result includes:

[0008] If the quality supervision analysis result indicates that the quality label of the target prefabricated dish is an abnormal label, obtaining the current cooking parameters of at least one cooking device in the automated kitchen; wherein the current cooking parameters include food ingredient quantitative parameters, food ingredient cutting parameters, mixing and stirring parameters, and pickling temperature and humidity parameters;

[0009] Performing improvement processing on the current cooking parameters.

[0010] In some possible examples, the debugging method of the image processing network includes:

[0011] Call the image processing network to perform image description mining on the production monitoring image example and the noise monitoring image example of the production monitoring image example, make a dish quality supervision decision through the mined production monitoring image descriptors, generate the dish quality supervision estimation results of the production monitoring image example and the noise monitoring image example, and the production monitoring image example and the corresponding noise monitoring image example both contain the same target dish quality supervision decision result;

[0012] Obtain the first dish quality supervision decision offset and the second dish quality supervision decision offset. The first dish quality supervision decision offset is the offset between the dish quality supervision estimation result of the production monitoring image example and the target dish quality supervision decision result, and the second dish quality supervision decision offset is the offset between the dish quality supervision estimation result of the noise monitoring image example and the target dish quality supervision decision result;

[0013] Call the image processing network to parse the production monitoring image descriptors of the noise monitoring image example, and generate the production monitoring image parsing results corresponding to the production monitoring image descriptors;

[0014] Combine the production monitoring image parsing results with the production monitoring image example to obtain the parsing offset;

[0015] Combine the first dish quality supervision decision offset, the second dish quality supervision decision offset and the parsing offset to optimize the network variables of the image processing network.

[0016] In some possible examples, the calling the image processing network to parse the production monitoring image descriptors of the noise monitoring image example and generate the production monitoring image parsing results corresponding to the production monitoring image descriptors includes:

[0017] Call the image processing network to project the production monitoring image descriptors of the noise monitoring image example into a preset feature space to obtain the linear feature array corresponding to the production monitoring image descriptors;

[0018] Pair the linear feature array with the dish quality feature set to generate at least one paired dish quality feature, and use the at least one dish quality feature as the production monitoring image parsing results corresponding to the production monitoring image descriptors.

[0019] In some possible examples, the image processing network includes two-level AI sub-models. The first-level AI sub-model is used to project the production monitoring image descriptors of the noise monitoring image example into a preset feature space, and the second-level AI sub-model is used to pair the linear feature array with the dish quality feature set.

[0020] In some possible examples, invoking the image processing network to parse the production monitoring image descriptor of the noise monitoring image example, and generating a production monitoring image parsing result corresponding to the production monitoring image descriptor, includes:

[0021] Invoking the image processing network to perform normalization processing on the production monitoring image descriptor of the noise monitoring image example, and based on the production monitoring image descriptor after the normalization processing, implementing the steps of projecting to a preset feature space and pairing with the dish quality feature set.

[0022] In some possible examples, combining the first dish quality supervision decision offset, the second dish quality supervision decision offset, and the parsing offset to optimize the network variables of the image processing network includes one of the following:

[0023] Obtaining the set operation result of the parsing offset and the confidence coefficient of the parsing offset, obtaining the sum of the set operation result and the sum of the first dish quality supervision decision offset and the second dish quality supervision decision offset as the global offset, and combining the global offset to optimize the network variables of the image processing network;

[0024] Combining the confidence coefficients of the first dish quality supervision decision offset, the second dish quality supervision decision offset, and the parsing offset respectively, summing up the first dish quality supervision decision offset, the second dish quality supervision decision offset, and the parsing offset to obtain the global offset, and combining the global offset to optimize the network variables of the image processing network.

[0025] In some possible examples, invoking the image processing network to perform image description mining on the production monitoring image example and the noise monitoring image example of the production monitoring image example, and making a dish quality supervision decision through the mined production monitoring image descriptor, and generating a dish quality supervision estimation result of the production monitoring image example and the noise monitoring image example, includes:

[0026] Loading the production monitoring image example into the image processing network, having the image processing network perform image description mining on the production monitoring image example, and making a dish quality supervision decision through the mined production monitoring image descriptor, and generating a dish quality supervision estimation result of the production monitoring image example;

[0027] Combining the production monitoring image example, the dish quality supervision estimation result of the production monitoring image example, and the target dish quality supervision decision result to generate a corresponding noise monitoring image example;

[0028] Perform image description mining on the noise monitoring image example, and generate the dish quality supervision estimation result of the noise monitoring image example through the mined production monitoring image descriptors for dish quality supervision decision-making.

[0029] In some possible examples, the image description mining of the production monitoring image example by the image processing network includes:

[0030] The image processing network projects the image blocks covered in the production monitoring image content of the production monitoring image example into a preset feature space to obtain the linear feature array of the production monitoring image example;

[0031] Perform image description mining on the linear feature array of the production monitoring image example to obtain the production monitoring image descriptor of the production monitoring image example.

[0032] In some possible examples, the generation of the corresponding noise monitoring image example by combining the production monitoring image example, the dish quality supervision estimation result of the production monitoring image example, and the target dish quality supervision decision result includes:

[0033] Combine the dish quality supervision estimation result of the production monitoring image example and the target dish quality supervision decision result to determine the decision noise factor of the production monitoring image example;

[0034] Add the decision noise factor to the production monitoring image example to obtain the noise monitoring image example corresponding to the production monitoring image example.

[0035] In some possible examples, the combination of the dish quality supervision estimation result of the production monitoring image example and the target dish quality supervision decision result to determine the decision noise factor of the production monitoring image example includes:

[0036] Utilize the dish quality supervision estimation result of the production monitoring image example and the target dish quality supervision decision result to obtain the first dish quality supervision decision offset of the production monitoring image example;

[0037] Combine the change vector of the first dish quality supervision decision offset to obtain the alternative decision noise factor of the production monitoring image example;

[0038] Add the alternative decision noise factor to the production monitoring image example to obtain the alternative noise monitoring image example corresponding to the production monitoring image example;

[0039] Obtain the deviation of the dish quality supervision decision for the alternative noise monitoring image example by combining the obtained dish quality supervision estimation result and the target dish quality supervision decision result for the dish quality supervision decision made again with the alternative noise monitoring image example;

[0040] Optimize the alternative decision noise factors of the production monitoring image example by combining the change vector of the deviation of the dish quality supervision decision for the alternative noise monitoring image example until the preset requirements are met and then terminate to obtain the decision noise factors of the production monitoring image example.

[0041] In some possible examples, adding the decision noise factors to the production monitoring image example to obtain the corresponding noise monitoring image example of the production monitoring image example includes: adding the decision noise factors to the production monitoring image content of the production monitoring image example to obtain the production monitoring image content of the corresponding noise monitoring image example of the production monitoring image example;

[0042] The image description mining process of the noise monitoring image example includes: projecting the image blocks covered in the production monitoring image content of the noise monitoring image example onto a preset feature space to obtain the linear feature array of the noise monitoring image example; performing image description mining on the linear feature array of the noise monitoring image example to obtain the production monitoring image descriptor of the noise monitoring image example.

[0043] In some possible examples, adding the decision noise factors to the production monitoring image example to obtain the corresponding noise monitoring image example of the production monitoring image example includes: adding the decision noise factors to the linear feature array of the production monitoring image example to obtain the linear feature array of the corresponding noise monitoring image example of the production monitoring image example;

[0044] The image description mining process of the noise monitoring image example includes: performing image description mining on the linear feature array of the noise monitoring image example to obtain the production monitoring image descriptor of the noise monitoring image example.

[0045] In some possible examples, the image processing network includes a description feature mining unit, a quality supervision decision unit, and a monitoring image analysis unit;

[0046] Among them, the description feature mining unit is used for image description mining; the quality supervision decision unit is used to implement dish quality supervision decision-making through the mined production monitoring image descriptors; the monitoring image analysis unit is used to implement the analysis of the production monitoring image descriptors of the noise monitoring image example to generate the production monitoring image analysis result corresponding to the production monitoring image descriptor.

[0047] In a second aspect, an embodiment of the present invention provides an artificial intelligence-based pre-prepared food production monitoring system, comprising a pre-prepared food production monitoring cloud platform and a camera arranged in an automated kitchen that communicate with each other; the camera is used to: collect production monitoring images of target pre-prepared dishes and send the production monitoring images to the pre-prepared food production monitoring cloud platform; the pre-prepared food production monitoring cloud platform is used to: use a debugged image processing network to perform quality analysis of the production line links on the production monitoring images to obtain quality supervision analysis results; and based on the quality supervision analysis results, improve the production line of the automated kitchen.

[0048] In a third aspect, the present invention further provides a pre-prepared food production monitoring cloud platform, comprising a processor and a memory; the processor and the memory are communicatively connected, and the processor is used to read a computer program from the memory and execute it to implement the above method.

[0049] In a fourth aspect, the present invention further provides a computer-readable storage medium having a program stored thereon, which implements the above method when executed by a processor.

[0050] The technical solution provided by the embodiment of the present invention starts from the idea of ​​image processing, first obtains the production monitoring image of the target pre-prepared dish, and then uses the debugged image processing network to perform quality analysis of the production line link on the production monitoring image to obtain the quality supervision analysis result. In this way, the production quality under multiple production line links can be considered based on the image level, so as to reflect whether the production of the target pre-prepared dish is compliant or meets the quality requirements from the overall / full assembly line level. Furthermore, after obtaining the quality supervision analysis results, the production line of the automated kitchen can be improved and processed in combination with the quality supervision analysis results, so as to adaptively adjust the overall production line of the automated kitchen, thereby achieving quality control of the target pre-prepared dish. The whole process is efficient and highly operational, reducing the monitoring error caused by the subjective experience of human monitoring, and avoiding production quality fluctuations of large quantities of target pre-prepared dishes as much as possible.

[0051] Further, during the debugging process of the image processing network, a noise monitoring image example was added. The production monitoring image example and the noise monitoring image example were used to debug the image processing network, enabling the image processing network to absorb the dish quality supervision decision-making method for production monitoring images with added noise interference, improving the stability of the image processing network, and enhancing the accuracy of dish quality supervision decision-making analysis based on production monitoring images. In addition, the image processing network was able to adjust the production monitoring image descriptors of the noise monitoring image examples mined during dish quality supervision decision-making and restore them to the content of the production monitoring images, reducing the understanding difficulty of noise-non-noise linkage debugging. By combining the offset between the adjusted production monitoring image content and the production monitoring image content of the production monitoring image example to debug network variables, the image processing network was able to mine more accurate production monitoring image descriptors, ensuring the accuracy and credibility of the detail output of the production monitoring image content and improving the stability and accuracy of the image description mining of the image processing network. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are incorporated herein and form a part of this specification, showing embodiments consistent with the present invention and, together with the specification, are used to explain the principles of the present invention.

[0053] Figure 1 It is a schematic flowchart of a prefabricated dish production monitoring method based on artificial intelligence provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0055] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above accompanying drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence.

[0056] The method embodiments provided by the embodiments of the present invention can be executed in a prefabricated food production monitoring cloud platform, a computer device, or a similar computing device. Taking the operation on the prefabricated food production monitoring cloud platform as an example, the prefabricated food production monitoring cloud platform may include one or more processors (the processors may include, but are not limited to, processing devices such as a microprocessor MCU or a programmable logic device FPGA), and a memory for storing data. Optionally, the above-mentioned prefabricated food production monitoring cloud platform may further include a transmission device for communication functions. Those of ordinary skill in the art can understand that the above structure is only illustrative and does not limit the structure of the above-mentioned prefabricated food production monitoring cloud platform. For example, the prefabricated food production monitoring cloud platform may further include more or fewer components than those shown above, or have a different configuration from those shown above.

[0057] The memory can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to a prefabricated food production monitoring method based on artificial intelligence in the embodiments of the present invention. The processor executes various functional applications and data processing by running the computer program stored in the memory, that is, the above-mentioned method is implemented. The memory may include a high-speed random access memory, and may further include a non-volatile memory, such as one or more magnetic storage devices, a flash memory, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely set relative to the processor, and these remote memories can be connected to the prefabricated food production monitoring cloud platform through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and their combinations.

[0058] The transmission device is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the prefabricated food production monitoring cloud platform. In one instance, the transmission device includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one instance, the transmission device may be a radio frequency (abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0059] Based on this, please refer to Figure 1 , Figure 1 is a schematic flowchart of a prefabricated food production monitoring method based on artificial intelligence provided by the embodiments of the present invention. This method is applied to the prefabricated food production monitoring cloud platform and may further include Step 1 - Step 3.

[0060] Step 1: The prefabricated food production monitoring cloud platform acquires the production monitoring image of the target prefabricated food collected and sent by the camera set in the automated kitchen.

[0061] In an embodiment of the present invention, multiple cooking devices are installed in an automated kitchen, and these cooking devices are uniformly controlled by a prefabricated dish production monitoring cloud platform. The production monitoring image of the target prefabricated dish may include multiple image frames. After the camera sends the production monitoring image to the prefabricated dish production monitoring cloud platform, the prefabricated dish production monitoring cloud platform can first perform adaptive preprocessing on the production monitoring image to ensure image quality. For example, it can perform defogging processing, distortion correction processing, jitter elimination processing, etc., and then use Step 2 to perform quality analysis on the production line link.

[0062] Step 2: The prefabricated dish production monitoring cloud platform uses the image processing network that has been debugged to perform quality analysis on the production monitoring image for the production line link, and obtains a quality supervision analysis result.

[0063] It can be understood that when the prefabricated dish production monitoring cloud platform uses the image processing network that has been debugged, the input production monitoring image is after preprocessing. The quality analysis of the production line link can consider the production quality under multiple production line links. For example, it can judge the standardization of different cooking stages based on image-level recognition and analysis. Based on this, the quality supervision analysis result can reflect whether the production of the target prefabricated dish is compliant or meets the quality requirements from the overall / full production line level.

[0064] Step 3: The prefabricated dish production monitoring cloud platform improves the production line of the automated kitchen according to the quality supervision analysis result.

[0065] In an embodiment of the present invention, the quality supervision analysis result is characterized by different quality labels. For example, the quality labels may include a good label, a general label, and an abnormal label. Based on this, the improvement process of the production line of the automated kitchen according to the quality supervision analysis result in Step 3 may include Step 31 and Step 32.

[0066] Step 31: If the quality label of the target prefabricated dish characterized by the quality supervision analysis result is an abnormal label, obtain the current cooking parameters of at least one cooking device in the automated kitchen.

[0067] Wherein, the current cooking parameters include food ingredient quantification parameters, food ingredient cutting parameters, mixing and stirring parameters, and pickling temperature and humidity parameters.

[0068] Step 32: Improve the current cooking parameters.

[0069] In an embodiment of the present invention, ingredients from different sources may differ in properties and quality. If the cooking method of ingredients from a certain source is blindly followed, ingredients from other sources may not match the current cooking parameters, making it difficult to meet the quality requirements for dish production. To this end, when the quality label of a target pre-prepared dish is an abnormal label, the current cooking parameters of at least one cooking device in the automated kitchen can be obtained and improved. In this way, the overall production line of the automated kitchen can be adaptively adjusted to achieve quality control of the target pre-prepared dish.

[0070] It can be seen that by applying the above steps 1 to 3, starting from the idea of ​​image processing, we first obtain the production monitoring image of the target pre-prepared dish, and then use the debugged image processing network to perform quality analysis of the production line link on the production monitoring image to obtain the quality supervision analysis result. In this way, the production quality of multiple production line links can be considered based on the image level, so as to reflect whether the production of the target pre-prepared dish is compliant or meets the quality requirements from the overall / full assembly line level. Furthermore, after obtaining the quality supervision analysis results, the production line of the automated kitchen can be improved and processed in combination with the quality supervision analysis results, so as to adaptively adjust the overall production line of the automated kitchen, thereby achieving quality control of the target pre-prepared dish. The whole process is efficient and highly operational, reducing the monitoring error caused by the subjective experience of human monitoring, and avoiding production quality fluctuations of large quantities of target pre-prepared dishes as much as possible.

[0071] In the actual implementation process, the pre-prepared food production monitoring cloud platform described in step 2 uses the debugged image processing network to perform quality analysis on the production line links of the production monitoring image to obtain quality supervision analysis results, which is the core step of the embodiment of the present invention. That is, by using artificial intelligence technology to assist quality supervision and analysis, the interference of human subjective experience can be minimized, and the accuracy and credibility of food quality supervision and analysis can be improved. The operating performance of the image processing network determines the quality of the quality supervision and analysis results. On this basis, an independently implementable design scheme of an embodiment of the present invention shows a debugging and training method for the image processing network, including steps 41-45.

[0072] Step 41, the pre-prepared food production monitoring cloud platform calls the image processing network to perform image description mining on the production monitoring image example and the noise monitoring image example of the production monitoring image example, and makes dish quality supervision decisions through the mined production monitoring image descriptors to generate dish quality supervision estimation results for the production monitoring image example and the noise monitoring image example. The production monitoring image example and the corresponding noise monitoring image example both contain the same target dish quality supervision decision results.

[0073] In the embodiments of the present invention, a production monitoring image example refers to a training sample of an image format / type. In the embodiments of the present invention, this production monitoring image example is used to debug and train an image processing network. The corresponding noise monitoring image example of the production monitoring image example refers to the noise monitoring image example obtained based on this production monitoring image example. For example, by adding a decision noise factor to the production monitoring image example, the corresponding noise monitoring image example can be generated. The debugging method of actively introducing training noise is to enable the image processing network to accurately and reliably implement the decision-making analysis of dish quality supervision for the noise monitoring image example with the added decision noise factor. Therefore, the target dish quality supervision decision result carried by this noise monitoring image example is consistent with the target dish quality supervision decision result carried by the corresponding production monitoring image example.

[0074] Furthermore, the dish quality supervision estimation result refers to the dish quality supervision decision result generated through the dish quality supervision decision-making of the image processing network. This dish quality supervision decision-making process is the process of estimating the dish quality supervision category corresponding to the production monitoring image. Then, the dish quality supervision decision result generated by this image processing network is the estimation result. The target dish quality supervision decision result refers to the certified and correct dish quality supervision decision result, which can also be understood as the "true value". The purpose of debugging this image processing network is to make the dish quality supervision estimation result obtained by this image processing network approach the target dish quality supervision decision result by optimizing the network variables of this image processing network, so that this image processing network can accurately perform the dish quality supervision decision-making process on the production monitoring image.

[0075] In this step 41, the network variables of the image processing network are default values, and better network variables still need to be obtained through network debugging. During network debugging, the prefabricated dish production monitoring cloud platform can obtain the production monitoring image example, input the production monitoring image example into this image processing network, and the image processing network makes a dish quality supervision decision on the production monitoring image example to obtain the dish quality supervision estimation result.

[0076] This dish quality supervision decision-making process includes two stages: mining the production monitoring image description from the image description and making a dish quality supervision decision based on the production monitoring image description. Among them, the image description refers to a series of image features such as the color, visual texture, volume, and dish arrangement of the dishes in a production monitoring image. The production monitoring image description is the difference between this production monitoring image and other production monitoring images. The production monitoring image description expresses the image features of the production monitoring image through feature vectors.

[0077] Step 42: The prefabricated food production monitoring cloud platform obtains the first food quality supervision decision deviation and the second food quality supervision decision deviation. The first food quality supervision decision deviation is the deviation between the food quality supervision estimation result of the production monitoring image example and the target food quality supervision decision result. The second food quality supervision decision deviation is the deviation between the food quality supervision estimation result of the noise monitoring image example and the target food quality supervision decision result.

[0078] Among them, after the prefabricated food production monitoring cloud platform makes food quality supervision decisions on each example (including the production monitoring image example and the noise monitoring image example) to obtain the estimated production monitoring image food quality supervision decision result, it can obtain the food quality supervision decision deviation according to the food quality supervision estimation result and the target food quality supervision decision result, and use this food quality supervision decision deviation to judge the current food quality supervision decision quality of the image processing network. It can be understood that the food quality supervision decision deviation corresponding to the production monitoring image example is the first food quality supervision decision deviation, and the food quality supervision decision deviation corresponding to the noise monitoring image example is the second food quality supervision decision deviation. The deviation in the embodiments of the present invention can be understood as the error of the quality decision or the deviation of the quality discrimination.

[0079] Further, if the food quality supervision decision deviation is large, the food quality supervision decision quality of the image processing network is poor; if the food quality supervision decision deviation is small, the food quality supervision decision quality of the image processing network is good.

[0080] Step 43: The prefabricated food production monitoring cloud platform parses the production monitoring image descriptor of the noise monitoring image example based on the image processing network, and generates a production monitoring image parsing result corresponding to the production monitoring image descriptor.

[0081] Step 44: The prefabricated food production monitoring cloud platform obtains a parsing deviation based on the production monitoring image parsing result and the production monitoring image example.

[0082] In steps 43 and 44, for the production monitoring image descriptor of the noise monitoring image example, the prefabricated food production monitoring cloud platform can also perform adjustment processing on it to restore its production monitoring image content. By comparing the production monitoring image example with the production monitoring image parsing result obtained after adding decision noise factors to the production monitoring image example and then adjusting, the parsing deviation is obtained. The parsing deviation is used to indicate the difference between the production monitoring image parsing result and the production monitoring image example, and can judge the stability and accuracy of the image processing network in image description mining. Exemplarily, the process of parsing the production monitoring image descriptor into the corresponding production monitoring image content is the adjustment process. Correspondingly, the parsing deviation can be understood as the adjustment deviation.

[0083] Furthermore, if the parsing offset is small, that is, the difference between the adjusted result and the production monitoring image example corresponding to the noise monitoring image example is small, which means that even if the decision noise factor is added, the original production monitoring image content can still be accurately restored, indicating that the production monitoring image descriptor obtained by image description mining for this production monitoring image example can accurately represent this production monitoring image example, and the detail output performance of the current descriptor is better. If the parsing offset is large, that is, the difference between the adjusted result and the production monitoring image example corresponding to the noise monitoring image example is large, which means that it is difficult to restore the original production monitoring image content even if the decision noise factor is added, and the original production monitoring image content has been changed, indicating that the accuracy of the production monitoring image descriptor obtained by image description mining for this production monitoring image example is poor.

[0084] Step 45: The prefabricated food production monitoring cloud platform optimizes the network variables of the image processing network based on the first food quality supervision decision offset, the second food quality supervision decision offset, and the parsing offset.

[0085] It can be understood that optimizing the network variables of the image processing network by combining the two offsets of the food quality supervision decision offset and the parsing offset can not only improve the stability and accuracy of the food quality supervision decision of the image processing network, but also improve the stability and accuracy of the image description mining of the image processing network. On the one hand, during the debugging process of the image processing network, noise monitoring image examples are added. The debugging samples include production monitoring image examples and noise monitoring image examples. The image processing network can make accurate food quality supervision decisions for both the original production monitoring image examples and the production monitoring image examples with the added decision noise factor, thus improving the stability and accuracy of the food quality supervision decision of the image processing network. In addition, the parsing offset is added in this debugging process. Through offset parsing, the image processing network can still accurately restore the original production monitoring image content after adding the decision noise factor to the input production monitoring image content. It can be seen that the image processing network has absorbed an accurate and reliable feature detail mining mode, and the image description mining step also has strong stability.

[0086] The technical solution provided by the embodiment of the present invention adds noise monitoring image examples during the debugging process of the image processing network, and uses the production monitoring image examples and the noise monitoring image examples to debug the image processing network, so that the image processing network absorbs the food quality supervision decision method for the production monitoring image with added noise interference, improves the stability of the image processing network, and improves the accuracy of the food quality supervision decision analysis based on the production monitoring image.

[0087] In addition, the image processing network can adjust the production monitoring image descriptors of the noise monitoring image examples mined during the decision-making of dish quality supervision, restore them to the content of the production monitoring images, and reduce the difficulty of understanding the noise-non-noise linkage debugging. By combining the offset between the adjusted production monitoring image content and the production monitoring image content of the production monitoring image examples to debug the network variables, the image processing network can mine more accurate production monitoring image descriptors, ensure the accuracy and reliability of the detail output of the production monitoring image content, and improve the stability and accuracy of the image descriptor mining of the image processing network.

[0088] Another implementation manner of the prefabricated dish production monitoring method based on artificial intelligence provided by the embodiments of the present invention includes the following steps.

[0089] Step 51, the prefabricated dish production monitoring cloud platform obtains a production monitoring image example.

[0090] In the embodiments of the present invention, the prefabricated dish production monitoring cloud platform can obtain a production monitoring image example and debug the image processing network based on the production monitoring image example. The production monitoring image example carries the target dish quality supervision decision result. By comparing the dish quality supervision estimation result of the production monitoring image example processed by the image processing network with the target dish quality supervision decision result, the dish quality supervision decision performance of the image processing network can be determined, so as to optimize the network variables in multiple rounds of loops to improve the dish quality supervision decision-making ability of the image processing network.

[0091] Exemplarily, according to the different cache spaces of the production monitoring image example, the prefabricated dish production monitoring cloud platform can obtain the production monitoring image example through various ideas. For example, the production monitoring image example can be recorded in the cloud storage space (image database). When the prefabricated dish production monitoring cloud platform needs to debug the image processing network, it can extract the production monitoring image example from the cloud storage space.

[0092] In another example, the production monitoring image example can be recorded in the prefabricated dish production monitoring cloud platform. For example, it is the historical production monitoring image sent by a third-party server to the prefabricated dish production monitoring cloud platform, or the production monitoring image generated by the prefabricated dish production monitoring cloud platform. The prefabricated dish production monitoring cloud platform can extract the production monitoring image example from the local database.

[0093] It can be understood that the prefabricated dish production monitoring cloud platform can also obtain the production monitoring image example through other ideas, and the embodiments of the present invention are not limited to the exemplary acquisition ideas of the production monitoring image example.

[0094] Step 52: The prefabricated food production monitoring cloud platform loads the production monitoring image example into the image processing network, and the image processing network projects the image patches covered in the production monitoring image content of the production monitoring image example into a preset feature space to obtain the linear feature array of the production monitoring image example.

[0095] After the prefabricated food production monitoring cloud platform obtains the production monitoring image example, it can use the production monitoring image example to debug the image processing network. Among them, the form of the production monitoring image example is the image format, that is, the production monitoring image example exists in the form of production monitoring image content. The image processing network can perform feature downsampling on the production monitoring image content and record it with a feature vector (linear feature array). The idea of using the linear feature array to represent the production monitoring image can improve the subsequent analysis effect of the production monitoring image.

[0096] For example, the production monitoring image example includes at least one dish quality feature, and the image processing network can project each of the at least one dish quality feature in the production monitoring image content of the production monitoring image example into a preset feature space to obtain the corresponding linear feature array, and the linear feature array includes the linear feature array corresponding to each image patch.

[0097] Exemplarily, the linear feature array can be a hash code or a one-hot code, or other representation methods can also be used.

[0098] Step 53: The prefabricated food production monitoring cloud platform performs image description mining on the linear feature array of the production monitoring image example to obtain the production monitoring image descriptor of the production monitoring image example.

[0099] After the prefabricated food production monitoring cloud platform determines the linear feature array of the production monitoring image example, the linear feature array contains the representation of the image patches in the production monitoring image content, but the connection between each image patch cannot be accurately determined through the linear feature array. The prefabricated food production monitoring cloud platform can perform image description mining on the linear feature array to obtain a production monitoring image descriptor that can more accurately represent the production monitoring image example.

[0100] For example, the linear feature array includes the linear feature arrays of one or more image patches. For the linear feature array of each image patch, the prefabricated food production monitoring cloud platform can determine the production monitoring image descriptor corresponding to the image patch based on the image area of the image patch.

[0101] Exemplarily, the prefabricated food production monitoring cloud platform can determine the production monitoring image descriptor corresponding to the image block based on the linear feature array of the image block, the linear feature array of the first image block, and the linear feature array of the second image block. Among them, the first image block refers to the image block ranked before the image block in the content of the production monitoring image. The second image block refers to the image block ranked after the image block in the content of the production monitoring image. Of course, for the first image block and the last image block in the content of the production monitoring image, they include the second image block and the first image block respectively.

[0102] It can be understood that the prefabricated food production monitoring cloud platform can also adopt another idea for image description mining. For example, in addition to based on the image area, the prefabricated food production monitoring cloud platform can also determine the production monitoring image descriptor corresponding to the image block according to the distribution label of the image block in the content of the production monitoring image, or can determine the corresponding production monitoring image descriptor according to the image area of the image block and its distribution label in the content of the production monitoring image, etc.

[0103] Steps 52 and 53 are the process of image description mining for the production monitoring image example by the image processing network. In this image description mining process, the features are first downsampled to obtain a linear feature array, and then the production monitoring image descriptor is extracted from the linear feature array, which can more accurately analyze the production monitoring image content of the production monitoring image example to obtain an accurate production monitoring image descriptor. For example, steps 52 and 53 can be implemented through two levels of AI sub-model layers, which are respectively called the linear feature extraction layer and the image description mining layer. The prefabricated food production monitoring cloud platform can process the production monitoring image example through the linear feature extraction layer to generate a linear feature array, and input the linear feature array into the image description mining layer, and the image description mining layer conducts image description mining on the linear feature array to generate the production monitoring image descriptor. The processing performed by each AI sub-model layer can be moving average processing (convolution operation).

[0104] Among them, this image description mining process can also be realized through another idea. For example, directly conduct image description mining on the production monitoring image content of the production monitoring image example to obtain the production monitoring image descriptor.

[0105] Step 54, the prefabricated food production monitoring cloud platform makes a decision on the quality supervision of the dishes through the mined production monitoring image descriptor, and generates the quality supervision estimation result of the dishes for this production monitoring image example.

[0106] After the prefabricated food production monitoring cloud platform extracts the production monitoring image descriptors, it can make decisions on dish quality supervision based on the production monitoring image descriptors. This dish quality supervision decision-making process is used to pair the production monitoring image descriptors with multiple alternative decision types of the production monitoring image example, determine the matching degree of the production monitoring image descriptors with each alternative decision type, and generate a dish quality supervision estimation result.

[0107] Furthermore, the dish quality supervision estimation result can include multiple modes. In one possible mode, the dish quality supervision estimation result is a feature vector mode. The feature vector includes multiple feature members (feature elements), and each feature member corresponds to an alternative decision type. The feature member is used to represent the matching degree of the production monitoring image descriptor with the alternative decision type.

[0108] In another possible mode, the dish quality supervision estimation result is the alternative decision type with the highest matching degree with the production monitoring image descriptor, or the dish quality supervision estimation result is the alternative decision type with the highest matching degree with the production monitoring image descriptor and the matching degree. Among them, the matching degree can be recorded using a quantitative possibility or a matching level. The embodiments of the present invention do not limit the recording method of the dish quality supervision estimation result and the matching degree.

[0109] For example, this dish quality supervision decision-making process can be implemented through a dish quality supervision decision algorithm. The dish quality supervision decision algorithm can be any dish quality supervision decision algorithm, such as a regression analysis model, a multi-layer perceptron model, etc.

[0110] Exemplarily, this dish quality supervision decision-making process can be implemented through a logistic regression idea. The prefabricated food production monitoring cloud platform uses the logistic regression idea to process the production monitoring image descriptor and obtains the dish quality supervision estimation result of the production monitoring image descriptor.

[0111] For example, this dish quality supervision decision-making step can be implemented through a quality supervision decision unit. The quality supervision decision unit (which can be understood as a classifier) can include multiple types, such as a binary classification quality supervision decision unit, a multi-classification quality supervision decision unit, etc. The quality supervision decision unit can implement any dish quality supervision decision algorithm, and those skilled in the art can flexibly select the type of the quality supervision decision unit and the dish quality supervision decision algorithm used.

[0112] Step 55: The prefabricated food production monitoring cloud platform determines the decision noise factor of the production monitoring image example based on the dish quality supervision estimation result of the production monitoring image example and the target dish quality supervision decision result.

[0113] In the embodiment of the present invention, after the prefabricated food production monitoring cloud platform conducts a decision-making estimation on the production monitoring image example for the quality supervision of the dishes and obtains the estimation result of the quality supervision of the dishes, it can generate a corresponding noise monitoring image example for the production monitoring image example based on the estimation result of the quality supervision of the dishes in the production monitoring image example and the target decision result of the quality supervision of the dishes. In this way, the noise monitoring image example can be added to the production monitoring image example to jointly debug the image processing network. When generating the noise monitoring image example, the prefabricated food production monitoring cloud platform can first determine the decision noise factor of the production monitoring image example, and then add the decision noise factor to the production monitoring image example to obtain the corresponding noise monitoring image example. Further, the decision noise factor can be understood as an adversarial factor or an adversarial sample.

[0114] For example, the determination process of the decision noise factor can be implemented through subsequent S1 to S5.

[0115] S1. The prefabricated food production monitoring cloud platform obtains the first deviation of the quality supervision decision of the dishes in the production monitoring image example according to the estimation result of the quality supervision of the dishes in the production monitoring image example and the target decision result of the quality supervision of the dishes.

[0116] After the prefabricated food production monitoring cloud platform conducts a decision-making estimation on the production monitoring image example for the quality supervision of the dishes and obtains the estimation result of the quality supervision of the dishes, it can compare with the target decision result of the quality supervision of the dishes to determine the quality supervision decision-making ability of the image processing network, and this quality supervision decision-making ability can be represented by the first deviation of the quality supervision decision of the dishes.

[0117] For example, the first deviation of the quality supervision decision of the dishes can be obtained through the training cost (loss function), and the training cost can be any kind of training cost, such as the cross-entropy loss function, the exponential training cost, etc. The exemplary acquisition idea of the first deviation of the quality supervision decision of the dishes in the embodiment of the present invention is not limited.

[0118] S2. The prefabricated food production monitoring cloud platform obtains the alternative decision noise factor of the production monitoring image example based on the change vector of the first deviation of the quality supervision decision of the dishes.

[0119] In the embodiment of the present invention, after the prefabricated food production monitoring cloud platform determines the first deviation of the quality supervision decision of the dishes, it can determine the decision noise factor of the production monitoring image example based on the first deviation of the quality supervision decision of the dishes. In the process of obtaining the decision noise factor, the prefabricated food production monitoring cloud platform can first determine an alternative decision noise factor based on the first deviation of the quality supervision decision of the dishes, add it to the production monitoring image example, and then determine a new alternative decision noise factor again based on the deviation of the quality supervision decision of the dishes in the alternative noise monitoring image example with the alternative decision noise factor added subsequently.

[0120] S3. The prefabricated food production monitoring cloud platform adds the alternative decision noise factor to the production monitoring image example to obtain an alternative noise monitoring image example corresponding to the production monitoring image example.

[0121] Furthermore, the process of adding the alternative decision noise factor can be realized through various ideas. The embodiment of the present invention can be realized by adopting one of the following two ideas.

[0122] Idea 1: Add the alternative decision noise factor to the linear feature array of the production monitoring image example to obtain a linear feature array of an alternative noise monitoring image example corresponding to the production monitoring image example.

[0123] Exemplarily, when the alternative noise monitoring image example is generated in the linear feature array mode, in the subsequent S4, the prefabricated food production monitoring cloud platform can directly perform image description mining on the linear feature array of the alternative noise monitoring image example to obtain a production monitoring image descriptor of the alternative noise monitoring image example, and then make a dish quality supervision decision based on the production monitoring image descriptor to obtain a dish quality supervision estimation result.

[0124] Idea 2: Add the alternative decision noise factor to the production monitoring image content of the production monitoring image example to obtain the production monitoring image content of an alternative noise monitoring image example corresponding to the production monitoring image example.

[0125] Exemplarily, when the alternative noise monitoring image example is generated in the image format, in the subsequent S4, the prefabricated food production monitoring cloud platform can first project the images covered in the production monitoring image content of the alternative noise monitoring image example onto a preset feature space to obtain a linear feature array of the alternative noise monitoring image example, and then perform image description mining on the linear feature array of the alternative noise monitoring image example to obtain a production monitoring image descriptor of the alternative noise monitoring image example.

[0126] S4. The prefabricated food production monitoring cloud platform obtains the dish quality supervision decision deviation of the alternative noise monitoring image example based on the dish quality supervision estimation result and the target dish quality supervision decision result obtained by making a dish quality supervision decision on the alternative noise monitoring image example again.

[0127] Among them, the process of making a dish quality supervision decision on the alternative noise monitoring image example and the process of obtaining the dish quality supervision decision deviation in S4 are similar to the above steps 52 to 54 and the implementation manner of the above S1.

[0128] S5. The prefabricated food production monitoring cloud platform optimizes the alternative decision noise factors of the production monitoring image example based on the change vector of the decision deviation of the dish quality supervision for the alternative noise monitoring image example, and terminates until the preset requirements are met, obtaining the decision noise factors of the production monitoring image example.

[0129] Among them, the preset requirement is that the number of loop rounds meets the set rounds, or the preset requirement is that the decision deviation of the dish quality supervision converges. The set rounds can be set by those skilled in the art according to needs and can be hyperparameters of the image processing network. For example, the set rounds can be 3 or 5, etc.

[0130] The above relevant content can be understood as multiple rounds of loops. During each round of loop, it is expected that the alternative decision noise factors can be amplified so that the decision deviation of the dish quality supervision becomes larger and larger. In this way, the interference caused by the obtained noise monitoring image example to the image processing network may be greater, and the stability of the image processing network debugged through such noise monitoring image examples is better.

[0131] For example, the prefabricated food production monitoring cloud platform can default to set an alternative decision noise factor, optimize the alternative decision noise factor during each subsequent round of loop, find the alternative decision noise factor for which the change vector (gradient) of the decision deviation of the dish quality supervision rises, and make the decision deviation of the dish quality supervision the largest through multiple rounds of loops. In this way, the interference of the obtained noise monitoring image example to the image processing network is greater.

[0132] During each round of loop, the prefabricated food production monitoring cloud platform determines the optimized value (adjustment value) of the alternative decision noise factor based on the change vector of the decision deviation of the dish quality supervision, adds the optimized value to the alternative decision noise factor in the previous round of loop, and converts the adjusted alternative decision noise factor into the numerical range of the decision noise factor to obtain the alternative decision noise factor to be added in the next round of loop.

[0133] Step 56. The prefabricated food production monitoring cloud platform adds the decision noise factor to the production monitoring image example to obtain the corresponding noise monitoring image example of the production monitoring image example.

[0134] Among them, step 56 is the process of generating a noise monitoring image example based on the decision noise factor. According to different patterns of the decision noise factor, the prefabricated food production monitoring cloud platform can add it to data of different patterns of the production monitoring image example to generate noise monitoring image examples of different patterns. The prefabricated food production monitoring cloud platform can implement the generation process of the noise monitoring image example based on any of the following schemes.

[0135] Solution 1: Add the decision noise factor to the linear feature array of the production monitoring image example to obtain the linear feature array of the noise monitoring image example corresponding to the production monitoring image example.

[0136] Among them, when the noise monitoring image example is generated, it is in the linear feature array mode. Correspondingly, in subsequent step 57, the prefabricated dish production monitoring cloud platform can perform image description mining on the linear feature array of the noise monitoring image example to obtain the production monitoring image descriptor of the noise monitoring image example.

[0137] Solution 2: Add the decision noise factor to the production monitoring image content of the production monitoring image example to obtain the production monitoring image content of the noise monitoring image example corresponding to the production monitoring image example.

[0138] Among them, when the noise monitoring image example is generated, it is in the linear feature array mode. Correspondingly, in subsequent step 57, the prefabricated dish production monitoring cloud platform can first project the images covered in the production monitoring image content of the noise monitoring image example into a preset feature space to obtain the linear feature array of the noise monitoring image example, and then perform image description mining on the linear feature array of the noise monitoring image example to obtain the production monitoring image descriptor of the noise monitoring image example.

[0139] In the embodiment of the present invention, steps 55 to 56 are the processes of generating corresponding noise monitoring image examples based on the production monitoring image example, the dish quality supervision estimation result of the production monitoring image example, and the target dish quality supervision decision result. Both the production monitoring image example and the corresponding noise monitoring image example contain the same target dish quality supervision decision result.

[0140] Step 57: The prefabricated dish production monitoring cloud platform performs image description mining on the noise monitoring image example, makes a dish quality supervision decision through the mined production monitoring image descriptor, and generates the dish quality supervision estimation result of the noise monitoring image example.

[0141] The design idea of this step 57 is similar to that of the above steps 52 to 54.

[0142] Steps 52 to 57 are the processes of calling an image processing network to perform image description mining on the production monitoring image example and the noise monitoring image example of the production monitoring image example, making a dish quality supervision decision through the mined production monitoring image descriptor, and generating the dish quality supervision estimation results of the production monitoring image example and the noise monitoring image example. For example, in the above step 55, the prefabricated dish production monitoring cloud platform can obtain the first dish quality supervision decision offset, so that in subsequent step 508, the prefabricated dish production monitoring cloud platform can no longer obtain the first dish quality supervision decision offset.

[0143] In addition, the prefabricated food production monitoring cloud platform can also repeatedly implement the step of obtaining the first food quality supervision decision deviation. Optionally, in step 55, the decision noise factors can also be obtained through another idea, and the prefabricated food production monitoring cloud platform obtains the first food quality supervision decision deviation and the second food quality supervision decision deviation in step 58.

[0144] Step 58: The prefabricated food production monitoring cloud platform obtains the first food quality supervision decision deviation and the second food quality supervision decision deviation. The first food quality supervision decision deviation is the deviation between the food quality supervision estimation result of the production monitoring image example and the target food quality supervision decision result, and the second food quality supervision decision deviation is the deviation between the food quality supervision estimation result of the noise monitoring image example and the target food quality supervision decision result.

[0145] Among them, step 508 is similar to S1 in the above step 55.

[0146] Step 59: The prefabricated food production monitoring cloud platform analyzes the production monitoring image descriptor of the noise monitoring image example based on the image processing network to generate a production monitoring image analysis result corresponding to the production monitoring image descriptor.

[0147] Among them, the analysis process is used to analyze the production monitoring image descriptor into the corresponding production monitoring image content, and the analyzed production monitoring image content is called the production monitoring image analysis result. The production monitoring image analysis result can include multiple modes. For example, it can be the production monitoring image and the quantization possibility corresponding to the production monitoring image descriptor, or it can be the production monitoring image corresponding to the production monitoring image descriptor.

[0148] Furthermore, step 59 can also be understood as feature translation, which is used to restore the image descriptor obtained by feature encoding (i.e., the image description mining process) back to the image format.

[0149] Exemplarily, step 59 can be implemented through subsequent steps 591 and 592.

[0150] Step 591: Based on the image processing network, project the production monitoring image descriptor of the noise monitoring image example into a preset feature space to obtain a linear feature array corresponding to the production monitoring image descriptor.

[0151] Step 592: Pair the linear feature array with the food quality feature set to generate at least one paired food quality feature, and use the at least one food quality feature as the production monitoring image analysis result corresponding to the production monitoring image descriptor.

[0152] For example, the image processing network includes two levels of AI sub-models. The first-level AI sub-model is used to project the production monitoring image descriptor of the noise monitoring image example into a preset feature space, and the second-level AI sub-model is used to pair the linear feature array with the dish quality feature set.

[0153] Exemplarily, in step 59, the prefabricated dish production monitoring cloud platform can call the first-level AI sub-model of the image processing network to project the production monitoring image descriptor of the noise monitoring image example into a preset feature space, obtaining a linear feature array corresponding to the production monitoring image descriptor. The model variables of the first-level AI sub-model can be synchronized with the model variables of the AI sub-model layer that executes step 52. The prefabricated dish production monitoring cloud platform calls the first-level AI sub-model of the image processing network to pair the linear feature array with the dish quality feature set, generating at least one paired dish quality feature, and using the at least one dish quality feature as the production monitoring image parsing result corresponding to the production monitoring image descriptor.

[0154] For example, the prefabricated dish production monitoring cloud platform can also first perform a normalization process on the production monitoring image descriptor of the noise monitoring image example, and then execute the subsequent projection and image description mining steps. Exemplarily, the prefabricated dish production monitoring cloud platform performs a normalization process on the production monitoring image descriptor of the noise monitoring image example based on the image processing network, and based on the normalized production monitoring image descriptor, executes the steps of projecting into a preset feature space and pairing with the dish quality feature set. Normalizing the production monitoring image descriptor (normalization operation) can transform the production monitoring image descriptor into the variable range that the subsequent AI sub-model can handle, improving the adjustment accuracy and also reducing the subsequent resource overhead.

[0155] Exemplarily, the image processing network includes a normalization layer and the above two levels of AI sub-models. The normalization layer is used to implement the above normalization process steps, and the two levels of AI sub-models are respectively used to implement the steps of projecting into a preset feature space and pairing with the dish quality feature set.

[0156] Step 510: The prefabricated dish production monitoring cloud platform obtains an analysis offset based on the production monitoring image parsing result and the production monitoring image example.

[0157] Among them, the analysis offset can be understood as an adjustment offset. The production monitoring image parsing result is an estimated value, and the production monitoring image example is a true value. The analysis offset is used to distinguish the difference between the estimated value and the true value, similar to S1 in step 55 above, and can be implemented using the training cost.

[0158] Step 511, the prefabricated food production monitoring cloud platform optimizes the network variables of the image processing network based on the first food quality supervision decision deviation, the second food quality supervision decision deviation, and the parsing deviation.

[0159] For the production monitoring image example, the prefabricated food production monitoring cloud platform obtains three kinds of deviations. One is the first food quality supervision decision deviation obtained from the production monitoring image example itself, another is the second food quality supervision decision deviation obtained from the corresponding noise monitoring image example of the production monitoring image example, and there is also a parsing deviation obtained by adjusting the noise monitoring image example.

[0160] Next, optimizing the network variables in combination with the three kinds of deviations can take into account both the stability and accuracy of the food quality supervision decision of the image processing network, and also take into account the stability and accuracy of the image description mining of the image processing network. In this way, the food quality supervision decision example of the production monitoring image debugged can be improved in the functions of both dimensions.

[0161] The optimization process combining the above three kinds of deviation losses can include two options, and the embodiments of the present invention can implement the optimization update process based on any one of the options.

[0162] Option 1: The prefabricated food production monitoring cloud platform obtains the setting operation result of the parsing deviation and the confidence coefficient of the parsing deviation, obtains the sum of the setting operation result, the first food quality supervision decision deviation, and the second food quality supervision decision deviation as the global deviation, and optimizes the network variables of the image processing network based on the global deviation.

[0163] In Option 1, a confidence coefficient can be set for the parsing deviation. The confidence coefficient of the parsing deviation can be set by those skilled in the art according to needs. The confidence coefficient of the parsing deviation can be a hyperparameter of the image processing network, and the confidence coefficient (weight value) can be 0.1. In other embodiments, the confidence coefficient can also be optimized together with the network variables in this round of network debugging.

[0164] Option 2: The prefabricated food production monitoring cloud platform sums up the first food quality supervision decision deviation, the second food quality supervision decision deviation, and the parsing deviation based on their respective confidence coefficients, obtains the global deviation, and optimizes the network variables of the image processing network based on the global deviation.

[0165] In Option 2, a confidence coefficient is set for each kind of deviation, and the setting idea of the confidence coefficient is similar to that of Option 1.

[0166] For example, the image processing network includes a description feature mining unit, a quality supervision decision-making unit, and a monitoring image parsing unit. Among them, the description feature mining unit is used for image description mining; the quality supervision decision-making unit is used to implement dish quality supervision decision-making through the mined production monitoring image descriptors; the monitoring image parsing unit is used to implement parsing of the production monitoring image descriptors of the noise monitoring image example to generate a production monitoring image parsing result corresponding to the production monitoring image descriptor.

[0167] It can be understood that the structure of this image processing network is different from that of a conventional image processing network. Compared with a conventional image processing network that includes two parts, an image description mining layer and a quality supervision decision-making unit, this image processing network performs image description mining through a description feature mining unit and adds a monitoring image parsing unit to perform image description translation / image parsing on the production monitoring image descriptors of the noise monitoring image example to obtain the corresponding production monitoring image parsing result. This image description translation / image parsing process can adjust the content of the production monitoring image, so as to be able to compare with the original production monitoring image example to analyze whether the image descriptors obtained by image description mining are accurate and reliable.

[0168] It can be seen that by adding this monitoring image parsing unit, the stability and accuracy of the image description mining of the image processing network can be determined, and the stability and accuracy of the image description mining of the image processing network can be strengthened in the above loop process. In this way, the stability and accuracy of the dish quality supervision decision-making and image description mining of the obtained image processing network can be improved.

[0169] Exemplarily, the description feature mining unit is used to perform image description mining on a linear feature array, and the image processing network further includes a linear feature extraction layer. The linear feature extraction layer is used to project the content of the production monitoring image into a preset feature space to obtain a linear feature array.

[0170] It can be seen that after the image processing network is debugged by the above method, the image processing network can provide a dish quality supervision decision-making function for the production monitoring image. For example, in response to a dish quality supervision decision-making command for a production monitoring image, the prefabricated dish production monitoring cloud platform can call this image processing network, input the production monitoring image to be subject to dish quality supervision decision-making into this image processing network, and this image processing network performs image description mining on the production monitoring image and performs dish quality supervision decision-making through the mined production monitoring image descriptors to generate a quality label corresponding to this production monitoring image.

[0171] In summary, during the debugging process of the image processing network, a noise monitoring image example is added. The production monitoring image example and the noise monitoring image example are used to debug the image processing network, enabling the image processing network to absorb the dish quality supervision decision-making method for production monitoring images with added noise interference, improving the stability of the image processing network, and enhancing the accuracy of dish quality supervision decision-making analysis based on production monitoring images. In addition, the image processing network can adjust the production monitoring image descriptors of the noise monitoring image examples mined during dish quality supervision decision-making and restore them to the production monitoring image content, reducing the understanding difficulty of noise-non-noise linkage debugging. By combining the offset between the adjusted production monitoring image content and the production monitoring image content of the production monitoring image example to debug the network variables, the image processing network can mine more accurate production monitoring image descriptors, ensuring the accuracy and credibility of the detail output of the production monitoring image content and improving the stability and accuracy of the image description mining of the image processing network.

[0172] Based on the same or similar inventive concept as described above, a prefabricated dish production monitoring system based on artificial intelligence is further provided, including a prefabricated dish production monitoring cloud platform and a camera disposed in an automated kitchen that communicate with each other; the camera is configured to: collect production monitoring images of target prefabricated dishes and send the production monitoring images to the prefabricated dish production monitoring cloud platform; the prefabricated dish production monitoring cloud platform is configured to: use the image processing network that has been debugged to perform quality analysis on the production monitoring images for production line links to obtain a quality supervision analysis result; and based on the quality supervision analysis result, perform improvement processing on the production line of the automated kitchen.

[0173] Furthermore, a computer-readable storage medium is provided, on which a program is stored, and when the program is executed by a processor, the above method is implemented.

[0174] In several embodiments provided by the embodiments of the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the part of the module, program segment, or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0175] In addition, each functional module in various embodiments of the present invention may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

[0176] If the described functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs. It should be noted that in this document, the term "including", "comprising", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a process, method, article, or device. Without further limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device including the said element.

[0177] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for monitoring the production of prefabricated dishes based on artificial intelligence, characterized in that, Applied to the prefabricated food production monitoring cloud platform, the method includes: Obtaining a production monitoring image of a target prefabricated food collected and sent by a camera installed in an automated kitchen; Using the image processing network that has been debugged to perform quality analysis on the production monitoring image for the production line link, and obtaining a quality supervision analysis result; According to the quality supervision analysis result, performing improvement processing on the production line of the automated kitchen; The debugging method of the image processing network includes: calling the image processing network to perform image description mining on a production monitoring image example and a noise monitoring image example of the production monitoring image example, making a dish quality supervision decision through the mined production monitoring image descriptors, generating a dish quality supervision estimation result of the production monitoring image example and the noise monitoring image example, and the production monitoring image example and the corresponding noise monitoring image example both contain the same target dish quality supervision decision result; obtaining a first dish quality supervision decision offset and a second dish quality supervision decision offset, where the first dish quality supervision decision offset is the offset between the dish quality supervision estimation result of the production monitoring image example and the target dish quality supervision decision result, and the second dish quality supervision decision offset is the offset between the dish quality supervision estimation result of the noise monitoring image example and the target dish quality supervision decision result; calling the image processing network to parse the production monitoring image descriptors of the noise monitoring image example, and generating a production monitoring image parsing result corresponding to the production monitoring image descriptors; combining the production monitoring image parsing result with the production monitoring image example to obtain a parsing offset; combining the first dish quality supervision decision offset, the second dish quality supervision decision offset and the parsing offset to optimize the network variables of the image processing network; Invoking the image processing network to parse the production monitoring image descriptor of the noise monitoring image example, and generating a production monitoring image parsing result corresponding to the production monitoring image descriptor, including: invoking the image processing network to project the production monitoring image descriptor of the noise monitoring image example into a preset feature space to obtain a linear feature array corresponding to the production monitoring image descriptor; pairing the linear feature array with a set of dish quality features to generate at least one paired dish quality feature, and using the at least one dish quality feature as the production monitoring image parsing result corresponding to the production monitoring image descriptor; wherein, the image processing network includes two levels of AI sub-models, the first-level AI sub-model is used to project the production monitoring image descriptor of the noise monitoring image example into a preset feature space, and the second-level AI sub-model is used to pair the linear feature array with the set of dish quality features; wherein, invoking the image processing network to parse the production monitoring image descriptor of the noise monitoring image example and generating a production monitoring image parsing result corresponding to the production monitoring image descriptor includes: invoking the image processing network to perform normalization processing on the production monitoring image descriptor of the noise monitoring image example, and based on the normalized production monitoring image descriptor, performing the steps of projecting into a preset feature space and pairing with the set of dish quality features.

2. The method according to claim 1, wherein Improving the production line of the automated kitchen according to the quality supervision analysis result, including: If the quality supervision analysis result indicates that the quality label of the target prefabricated dish is an abnormal label, obtaining the current cooking parameters of at least one cooking device in the automated kitchen; wherein, the current cooking parameters include food ingredient quantification parameters, food ingredient cutting parameters, mixing and stirring parameters, and pickling temperature and humidity parameters; Performing improvement processing on the current cooking parameters.

3. The method according to claim 1, wherein Combining the first dish quality supervision decision deviation, the second dish quality supervision decision deviation, and the parsing deviation to optimize the network variables of the image processing network, including one of the following: Obtaining the set operation result of the parsing deviation and the confidence coefficient of the parsing deviation, obtaining the sum of the set operation result, the first dish quality supervision decision deviation, and the second dish quality supervision decision deviation as the global deviation, and combining the global deviation to optimize the network variables of the image processing network; Combining the confidence coefficients of the first dish quality supervision decision deviation, the second dish quality supervision decision deviation, and the parsing deviation respectively, summing the first dish quality supervision decision deviation, the second dish quality supervision decision deviation, and the parsing deviation to obtain the global deviation, and combining the global deviation to optimize the network variables of the image processing network.

4. The method according to claim 1, wherein The image processing network is called to perform image description mining on the production monitoring image example and the noise monitoring image example of the production monitoring image example, and the dish quality supervision decision is made through the mined production monitoring image descriptors, generating the dish quality supervision estimation results of the production monitoring image example and the noise monitoring image example, including: Loading the production monitoring image example into the image processing network, the image processing network performs image description mining on the production monitoring image example, and the dish quality supervision decision is made through the mined production monitoring image descriptors, generating the dish quality supervision estimation result of the production monitoring image example; Combining the production monitoring image example, the dish quality supervision estimation result of the production monitoring image example, and the target dish quality supervision decision result to generate the corresponding noise monitoring image example; Performing image description mining on the noise monitoring image example, and the dish quality supervision decision is made through the mined production monitoring image descriptors, generating the dish quality supervision estimation result of the noise monitoring image example; Among them, the image processing network performing image description mining on the production monitoring image example includes: the image processing network projects the image blocks covered in the production monitoring image content of the production monitoring image example into a preset feature space to obtain the linear feature array of the production monitoring image example; performing image description mining on the linear feature array of the production monitoring image example to obtain the production monitoring image descriptors of the production monitoring image example; Among them, combining the production monitoring image example, the dish quality supervision estimation result of the production monitoring image example, and the target dish quality supervision decision result to generate the corresponding noise monitoring image example includes: combining the dish quality supervision estimation result and the target dish quality supervision decision result of the production monitoring image example to determine the decision noise factor of the production monitoring image example; adding the decision noise factor to the production monitoring image example to obtain the noise monitoring image example corresponding to the production monitoring image example; Among them, combining the dish quality supervision estimation result and the target dish quality supervision decision result of the production monitoring image example to determine the decision noise factor of the production monitoring image example includes: Using the dish quality supervision estimation result and the target dish quality supervision decision result of the production monitoring image example to obtain the first dish quality supervision decision offset of the production monitoring image example; Combining the change vector of the first dish quality supervision decision offset to obtain the alternative decision noise factor of the production monitoring image example; Adding the alternative decision noise factor to the production monitoring image example to obtain the alternative noise monitoring image example corresponding to the production monitoring image example; Again combining the dish quality supervision estimation result obtained by the dish quality supervision decision on the alternative noise monitoring image example and the target dish quality supervision decision result to obtain the dish quality supervision decision offset of the alternative noise monitoring image example; Optimize the alternative decision noise factors of the production monitoring image example in combination with the change vector of the decision deviation of the dish quality supervision for the alternative noise monitoring image example, and terminate until the preset requirements are met to obtain the decision noise factors of the production monitoring image example.

5. The method according to claim 4, wherein Adding the decision noise factor to the production monitoring image example to obtain the corresponding noise monitoring image example of the production monitoring image example includes: adding the decision noise factor to the production monitoring image content of the production monitoring image example to obtain the production monitoring image content of the corresponding noise monitoring image example of the production monitoring image example; the image description mining process of the noise monitoring image example includes: projecting the image blocks covered in the production monitoring image content of the noise monitoring image example into a preset feature space to obtain the linear feature array of the noise monitoring image example; performing image description mining on the linear feature array of the noise monitoring image example to obtain the production monitoring image descriptor of the noise monitoring image example. Alternatively, adding the decision noise factor to the production monitoring image example to obtain the corresponding noise monitoring image example of the production monitoring image example includes: adding the decision noise factor to the linear feature array of the production monitoring image example to obtain the linear feature array of the corresponding noise monitoring image example of the production monitoring image example; the image description mining process of the noise monitoring image example includes: performing image description mining on the linear feature array of the noise monitoring image example to obtain the production monitoring image descriptor of the noise monitoring image example.

6. The method according to claim 1, wherein The image processing network includes a description feature mining unit, a quality supervision decision unit, and a monitoring image analysis unit; among them, the description feature mining unit is used for image description mining; the quality supervision decision unit is used to implement dish quality supervision decision-making through the mined production monitoring image descriptor; the monitoring image analysis unit is used to implement the analysis of the production monitoring image descriptor of the noise monitoring image example to generate the production monitoring image analysis result corresponding to the production monitoring image descriptor.

7. An artificial intelligence-based prefabricated food production monitoring system, characterized in that, It includes a prefabricated dish production monitoring cloud platform and a camera set in an automated kitchen that communicate with each other; The camera is used for: Collecting the production monitoring image of the target prefabricated dish and sending the production monitoring image to the prefabricated dish production monitoring cloud platform; The prefabricated dish production monitoring cloud platform is used for: Using the completed and debugged image processing network to perform quality analysis on the production monitoring image for the production line link to obtain a quality supervision analysis result; Improving the production line of the automated kitchen according to the quality supervision analysis result; The debugging method of the image processing network includes: calling the image processing network, performing image description mining on the production monitoring image example and the noise monitoring image example of the production monitoring image example, making a dish quality supervision decision through the mined production monitoring image descriptors, generating the dish quality supervision estimation results of the production monitoring image example and the noise monitoring image example, where the production monitoring image example and the corresponding noise monitoring image example both contain the same target dish quality supervision decision result; obtaining a first dish quality supervision decision offset and a second dish quality supervision decision offset, where the first dish quality supervision decision offset is the offset between the dish quality supervision estimation result of the production monitoring image example and the target dish quality supervision decision result, and the second dish quality supervision decision offset is the offset between the dish quality supervision estimation result of the noise monitoring image example and the target dish quality supervision decision result; calling the image processing network to parse the production monitoring image descriptors of the noise monitoring image example, generating a production monitoring image parsing result corresponding to the production monitoring image descriptors; combining the production monitoring image parsing result with the production monitoring image example to obtain a parsing offset; combining the first dish quality supervision decision offset, the second dish quality supervision decision offset and the parsing offset to optimize the network variables of the image processing network; The step of calling the image processing network to parse the production monitoring image descriptors of the noise monitoring image example and generating a production monitoring image parsing result corresponding to the production monitoring image descriptors includes: calling the image processing network to project the production monitoring image descriptors of the noise monitoring image example into a preset feature space to obtain a linear feature array corresponding to the production monitoring image descriptors; pairing the linear feature array with a dish quality feature set to generate at least one paired dish quality feature, and using the at least one dish quality feature as the production monitoring image parsing result corresponding to the production monitoring image descriptors; where the image processing network includes two levels of AI sub-models, the first-level AI sub-model is used to project the production monitoring image descriptors of the noise monitoring image example into a preset feature space, and the second-level AI sub-model is used to pair the linear feature array with the dish quality feature set; where the step of calling the image processing network to parse the production monitoring image descriptors of the noise monitoring image example and generating a production monitoring image parsing result corresponding to the production monitoring image descriptors includes: calling the image processing network to perform normalization processing on the production monitoring image descriptors of the noise monitoring image example, and based on the normalized production monitoring image descriptors, implementing the steps of projecting into a preset feature space and pairing with the dish quality feature set.

8. A prefabricated food production monitoring cloud platform, characterized in that, It includes a processor and a memory; the processor and the memory are communicatively connected, and the processor is configured to read and execute a computer program from the memory to implement the method according to any one of claims 1-6.

Images