Methods, apparatus, baking equipment, and storage media for determining the baking stage

Abstract

This application proposes a method, apparatus, baking equipment, and storage medium for determining the baking stage. It can extract baking state features of the baking target from an image of the baking target, and then determine the relationship between the baking stage of the baking target and each set baking stage based on these features. The relationship between the baking stage of the baking target and each set baking stage includes the baking stage of the baking target being before any set baking stage, or the baking stage of the baking target being after any set baking stage. Finally, based on the relationship between the baking stage of the baking target and each set baking stage, the baking stage of the baking target is determined. This achieves the purpose of automatically determining the baking stage of the baking target, avoiding the influence of the baking personnel's experience and skill level on the baking quality, and ensuring the quality of the baked target.

Description

Technical Field

[0001] This application relates to the field of automatic control technology, and in particular to a method, apparatus, baking equipment, and storage medium for determining the baking stage. Background Technology

[0002] For some products, roasting and dehydrating the tobacco leaves used in their production is a necessary step in the manufacturing process. For example, tobacco leaf roasting is an essential step in tobacco processing; by roasting the tobacco leaves, the moisture content is reduced to meet the requirements of subsequent production steps such as tobacco packaging.

[0003] Currently, in the process of baking and dehydrating the target, in order to ensure the baking quality, the baking personnel need to check the target regularly and judge the baking stage based on experience, and then adjust the drying temperature, humidity and other parameters according to the baking stage of the target.

[0004] However, this method of determining the baking stage makes the baking quality of the target baking target greatly affected by the experience and skill of the baking personnel, resulting in inconsistent baking quality and affecting product quality. Summary of the Invention

[0005] Based on the above needs, this application proposes a method, apparatus, baking equipment, and storage medium for determining the baking stage, in order to solve the problem that the method of determining the baking stage in the prior art affects product quality.

[0006] The technical solution proposed in this application is as follows:

[0007] On the one hand, this application provides a method for determining the baking stage, including:

[0008] Extract the baking state features of the baking target from the baking target image;

[0009] Based on the baking state characteristics of the baking target, the relationship between the baking stage in which the baking target is located and each set baking stage is determined; the relationship between the baking stage in which the baking target is located and any baking stage includes whether the baking stage in which the baking target is located is before or after the baking stage.

[0010] The baking stage of the target is determined based on its relationship with each set baking stage.

[0011] Furthermore, in the method described above, the baking target image includes images of multiple baking areas of the baking target;

[0012] Extracting baking state features of the baking target from the baking target image, including:

[0013] The baking status features of each baking area are extracted from the image of each baking area.

[0014] The baking state characteristics of all baking areas are fused to obtain the baking state characteristics of the baking target.

[0015] Furthermore, in the method described above, the baking state features of each baking area are extracted from the image of each baking area, including:

[0016] The image of each baking area is segmented to obtain at least two regions of interest images corresponding to each baking area;

[0017] Extract the region-of-interest (ROI) image features from each ROI image;

[0018] The features of the region of interest (ROI) images in each baking region are fused to obtain the baking state features of each baking region.

[0019] Furthermore, in the method described above, the region of interest image features in the region of interest image of each baking area are fused to obtain the baking state features of each baking area, including:

[0020] The region of interest (ROI) image features in the ROI image of each baking region are weighted and fused to obtain the baking state features of each baking region; wherein, the weight of the ROI image features changes as the baking time of the target increases.

[0021] Furthermore, in the method described above, feature fusion is performed on the baking state features of all baking areas to obtain the baking state features of the baking target, including:

[0022] Connect the baking state features of all baking areas to obtain the connection features;

[0023] The baking state features of the baking target are extracted from the connection features.

[0024] Furthermore, in the method described above, extracting the baking state features of the baking target from the connection features includes:

[0025] S1. Obtain the first feature corresponding to the Nth baking region; where N is a positive integer;

[0026] S2. Based on the first feature corresponding to the Nth baking region, extract the first feature corresponding to the (N+1)th baking region from the connection features; wherein, the (N+1)th baking region is the baking region adjacent to the Nth baking region;

[0027] S3. Let N = N + 1, and repeat steps S1 and S2 until N equals the total number of baking areas. Then, determine the baking state characteristics of the baking target through the first characteristics corresponding to each baking area.

[0028] Furthermore, in the method described above, determining the baking state characteristics of the baking target through the first feature corresponding to each baking area includes:

[0029] The first feature corresponding to each baking area is subjected to time-series-based feature extraction processing to obtain the second feature corresponding to each baking area;

[0030] The second features corresponding to all baking areas are superimposed and fused to obtain the baking state features of the baking target.

[0031] Furthermore, in the method described above, determining the relationship between the baking stage of the baking target and each set baking stage based on the baking state characteristics of the baking target includes:

[0032] The baking state characteristics of the baking target are compared with the baking state characteristics of each set baking stage to determine the relationship between the baking stage of the baking target and each set baking stage.

[0033] Furthermore, in the method described above, determining the baking stage of the target baking objective based on its relationship with each set baking stage includes:

[0034] Based on the relationship between the baking stage where the baking target is located and each set baking stage, the relationship parameter corresponding to each baking stage is determined; wherein, if the baking stage where the baking target is located is before any set baking stage, the relationship parameter corresponding to that baking stage is a first set value, and if the baking stage where the baking target is located is after any set baking stage, the relationship parameter corresponding to that baking stage is a second set value.

[0035] The baking stage of the target is determined by the range of the sum of the relational parameters corresponding to all baking stages.

[0036] Furthermore, in the method described above, determining the baking stage of the target baking object by using the interval containing the sum of the relational parameters corresponding to all baking stages includes:

[0037] Calculate the sum of the relational parameters corresponding to all baking stages;

[0038] Determine the interval of the sum of the relational parameters; where different intervals of relational parameters correspond to different baking stages;

[0039] The baking stage corresponding to the interval of relational parameters in which the sum of the relational parameters lies is determined as the baking stage of the baking target.

[0040] Furthermore, in the method described above, feature fusion is performed on the baking state features of all baking areas to obtain the baking state features of the baking target. Based on the baking state features of the baking target, the relationship between the baking stage of the baking target and each set baking stage is determined. The baking stage of the baking target is determined according to the relationship between the baking stage of the baking target and each set baking stage, including:

[0041] The baking state features of the target area in each image acquisition region are input into a pre-trained baking stage detection model, so that the first network in the baking stage detection model performs feature fusion on the baking state features of all baking regions to obtain the baking state features of the target. The second network in the baking stage detection model determines the relationship between the baking stage of the target and each set baking stage based on the baking state features of the target. Based on the relationship between the baking stage of the target and each set baking stage, the baking stage of the target is determined.

[0042] On the other hand, this application provides an apparatus for determining the baking stage, comprising:

[0043] An extraction module is used to extract the baking state features of the baking target from the baking target image;

[0044] The first determining module is used to determine the relationship between the baking stage of the baking target and each set baking stage based on the baking state characteristics of the baking target; the relationship between the baking stage of the baking target and each set baking stage includes the baking stage of the baking target being located before any set baking stage, or the baking stage of the baking target being located after any set baking stage.

[0045] The second determining module is used to determine the baking stage of the baking target based on the relationship between the baking stage of the baking target and each set baking stage.

[0046] On the other hand, this application provides an electronic device, including:

[0047] Memory and processor;

[0048] The memory is used to store programs;

[0049] The processor is configured to implement the method for determining the baking stage as described above by running a program in the memory.

[0050] On the other hand, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the method for determining the baking stage as described in any of the above claims.

[0051] The method for determining the baking stage proposed in this application can extract the baking state features of the baking target from the baking target image, and then determine the relationship between the baking stage of the baking target and each set baking stage based on the baking state features. The relationship between the baking stage of the baking target and each set baking stage includes the baking stage of the baking target being located before any set baking stage, or the baking stage of the baking target being located after any set baking stage. Finally, the baking stage of the baking target is determined based on the relationship between the baking stage of the baking target and each set baking stage, thereby achieving the purpose of automatically determining the baking stage of the baking target, avoiding the influence of the baking personnel's experience and skill level on the baking quality, and ensuring the quality of the baking target.

[0052] Furthermore, by comprehensively considering the baking stage of the target and the relationship between each baking stage, the current baking stage of the target can be determined more accurately, effectively improving the baking quality of the target. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating a method for determining a baking stage provided in an embodiment of this application;

[0055] Figure 2 This is a schematic flowchart illustrating the process of extracting the baking state features of a baking target according to an embodiment of this application;

[0056] Figure 3 This is a schematic diagram of the process for extracting the baking state features of the baking area according to an embodiment of this application;

[0057] Figure 4 This is a flowchart illustrating the baking state characteristics of the fusion baking region provided in an embodiment of this application;

[0058] Figure 5 This is a flowchart illustrating the relationship between determining the baking target and each set baking stage, provided in an embodiment of this application.

[0059] Figure 6 This is a schematic diagram of the structure of the first network in the baking stage detection model provided in the embodiments of this application;

[0060] Figure 7 This is a schematic diagram of the structure of the second network in the baking stage detection model provided in the embodiments of this application;

[0061] Figure 8 This is a schematic diagram of a baking stage determination device provided in an embodiment of this application;

[0062] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0063] Application Overview

[0064] The technical solution of this application embodiment is applicable to the application scenario of determining the baking stage of the baking target. By adopting the technical solution of this application embodiment, the baking stage of the baking target can be automatically determined, thus ensuring the quality of the baked product.

[0065] For some products, roasting and dehydration of the tobacco leaves used in their production is a necessary step in the manufacturing process. For example, tobacco curing is an important step in tobacco production, aimed at promoting the yellowing and drying of the leaves, reducing the moisture content, and meeting the requirements of subsequent production steps such as tobacco packaging.

[0066] During the curing process, curing personnel need to periodically check the status of the tobacco leaves to determine their curing stage and adjust the temperature and humidity levels in the curing chamber accordingly. For example, in traditional tobacco curing processes, there are generally three stages: yellowing, color setting, and drying. Each stage is further subdivided into several smaller stages. The temperature and humidity settings during tobacco curing correspond to these smaller stages. During curing, personnel use observation windows to periodically check the condition of the tobacco leaves and adjust the temperature and humidity levels in the curing chamber accordingly.

[0067] However, the current baking method not only places an excessive workload on baking personnel, but also relies too heavily on their work experience. If baking personnel misjudge the baking stage of the target based on their work experience, resulting in premature or delayed temperature and humidity index adjustments, it will cause significant economic losses and waste of resources.

[0068] Based on this, this application proposes a method, apparatus, baking equipment, and storage medium for determining the baking stage. This technical solution can automatically determine the baking stage of the baking target by comprehensively considering the relationship between the baking stage of the baking target and each baking stage, thereby avoiding the influence of the baking personnel's experience and skill level on the baking quality and ensuring the quality of the baking target.

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] Exemplary methods

[0071] This application proposes a method for determining the baking stage. This method can be executed by an electronic device, which can be any device with data and instruction processing capabilities, such as a computer, smart terminal, or server. See also Figure 1 As shown, the method includes:

[0072] S101. Extract the baking state features of the baking target from the baking target image.

[0073] The aforementioned roasting target refers to the object being roasted. The roasting target can be placed inside a roasting chamber, and roasting can be achieved by controlling parameters such as temperature, humidity, airflow, and heating rate within the chamber. For example, in the tobacco production process, tobacco leaves can be used as the roasting target.

[0074] The image of the baking target refers to an image of the target being baked, which can be captured by an image acquisition device, such as a camera. For example, to ensure the clarity of the image, the image acquisition device can be installed on the inner wall of the baking chamber to capture the image from inside the chamber.

[0075] When acquiring images of the target to be baked, one image acquisition device can be set up to capture the entire image of the target's image acquisition surface as the target image. It should be noted that the aforementioned image acquisition surface is generally the baking surface of the target near the observation window of the baking chamber, and can be set by the staff according to the actual situation; this embodiment does not impose any limitations.

[0076] After obtaining the image of the target to be baked, the baking state features of the target to be baked are extracted from the image.

[0077] For example, if an image acquisition device is set up to capture the entire image of the target surface being baked as the target image, when extracting the baking state features of the target from the target image, the target image can be input into a pre-trained feature extraction network model. This feature extraction network model can be trained using CNN, VGG, Resnt, or other similar networks as its base model. The training samples are the target images, and the training labels are the baking state features of the target corresponding to each training sample. During training, the training samples are input into the feature extraction network model to obtain its output. Based on the output and the training labels, the loss value of the feature extraction network model is determined. The parameters of the feature extraction network model are adjusted to reduce the loss value. This training process is repeated until the loss value of the feature extraction network model is less than a set value, at which point training is complete.

[0078] During the roasting process, different parts of the target material change differently, and the changes in each part are crucial for determining the roasting stage. If feature extraction is performed on the entire image of the target material as described in the example above, it will be impossible to effectively locate the feature changes in smaller areas, leading to inaccurate feature extraction results. For instance, if the target material is tobacco leaf, the leaf mesophyll and veins are both important indicators of the roasting stage. Since the veins occupy a relatively small area of ​​the entire tobacco leaf, directly extracting features from the entire image may not effectively locate the feature changes in the veins.

[0079] Based on this, another example is to use an image acquisition device to capture the entire image of the target being baked as the target image. When extracting the baking state features from the target image, the target image can be segmented first to obtain images of different parts of the target. Then, a trained feature extraction network model is used to extract features from the images of different parts of the target, obtaining the baking state features of different parts of the target. Finally, the baking state features of different parts of the target are fused together to obtain the baking state features of the target. This setup ensures that feature changes in any part of the target are not ignored, improving the accuracy of feature extraction. For example, if the target being baked is tobacco leaf, the image of the tobacco leaf can be segmented first to obtain images of the leaf mesophyll region and the leaf vein region. Then, a trained feature extraction network model is used to extract features from the images of the leaf mesophyll region and the leaf vein region, obtaining the baking state features of the leaf mesophyll region and the leaf vein region, respectively. Finally, the baking state features of the leaf mesophyll region and the leaf vein region are fused together to obtain the baking state features of the tobacco.

[0080] In this embodiment, a pre-trained semantic segmentation network model can be used to segment the baking target image. This semantic segmentation network model can be trained using a segformer network as the base network model; this embodiment is not limited to this. The training samples are the baking target images, and the training labels are the semantic segmentation results corresponding to each training sample. The specific training process of the semantic segmentation network model is the same as the training process of the feature extraction network model in the above embodiments. Those skilled in the art can refer to the descriptions in the above embodiments; the training process of the semantic segmentation network model will not be described in detail here.

[0081] When fusing the baking state features of different parts of a baking target, a pre-trained feature fusion network model can be used to splice and fuse or superimpose the baking state features of different parts of the baking target. This embodiment does not impose any limitations. When training the feature fusion network model, the training samples are the baking state features of different parts of the baking target, and the training labels are the fused features corresponding to each training sample. The specific training process of the feature fusion network model is the same as the training process of the feature extraction network model in the above embodiments. Those skilled in the art can refer to the description in the above embodiments, and the training process of the feature fusion network model will not be described in detail here.

[0082] It should be noted that different parts of the target material exhibit different characteristic changes at different stages of the baking process, and their contributions to determining the baking stage also vary. For example, during the baking of tobacco leaves, the leaf mesophyll undergoes greater changes in the early stages, thus the baking state characteristics of the leaf mesophyll region contribute more significantly to determining the baking stage of the tobacco leaf. Conversely, the veins undergo greater changes in the later stages, thus the baking state characteristics of the veins region contribute more significantly to determining the baking stage of the tobacco leaf. Based on this, when training the aforementioned feature fusion network model, different weights can be assigned to the baking state characteristics of different parts according to the baking stage of each training sample. That is, if a certain part in the baking stage of a training sample contributes significantly to determining the baking stage of the target material, then the baking state characteristics of that part can be assigned a larger weight; conversely, if a certain part in the baking stage of a training sample contributes less significantly, then the baking state characteristics of that part can be assigned a smaller weight. The feature fusion network model trained in this way can amplify the features that contribute significantly to the determination of the baking stage of the target during feature fusion, thereby improving the accuracy of the determination of the baking stage of the target.

[0083] For example, in the early stage of curing, the curing status features of the leaf mesophyll region contribute significantly to determining the curing stage of the tobacco leaf. Therefore, when training the feature fusion network model, a larger weight can be assigned to the curing status features of the leaf mesophyll region in the early stage of curing. In the later stage of curing, the curing status features of the leaf vein region contribute significantly to determining the curing stage of the tobacco leaf. Therefore, when training the feature fusion network model, a larger weight can be assigned to the curing status features of the leaf vein region in the later stage of curing.

[0084] S102. Based on the baking state characteristics of the baking target, determine the relationship between the baking stage of the baking target and each set baking stage.

[0085] In this embodiment, the baking target is divided into different baking stages. The division of the baking stages can refer to existing technology, and this embodiment does not limit it. For example, if the baking target is tobacco leaves, the baking stages can be divided into three stages: yellowing stage, color fixing stage, and dry rib stage. Each stage can be further subdivided into several smaller stages. The baking temperature and humidity settings during tobacco leaf baking correspond to the smaller stage in which the tobacco leaves are in.

[0086] The relationship between the current baking stage of the target tobacco leaf and each set baking stage can be determined. This relationship includes whether the target tobacco leaf is in a baking stage before or after that stage. For example, if the target tobacco leaf is tobacco leaves, and the baking stages are divided into three phases: yellowing, color fixing, and drying, then this embodiment needs to determine, based on the characteristics of the tobacco leaf's baking state, whether the current baking state is before or after the yellowing stage, before or after the color fixing stage, and before or after the drying stage.

[0087] Specifically, the baking state features of the baking target extracted in the above embodiments can be compared with the standard baking state features corresponding to each set baking stage to determine the relationship between the baking stage of the baking target and each set baking stage. For example, a feature comparison network model can be trained to compare the baking state features of the baking target extracted in the above embodiments with the standard baking state features corresponding to each set baking stage; this embodiment does not impose any limitations on this.

[0088] S103. Determine the baking stage of the target based on the relationship between the baking stage of the target and each set baking stage.

[0089] Based on a comprehensive judgment of the relationship between the baking target's current baking stage and each set baking stage as determined in the above steps, the baking stage of the baking target can be further determined.

[0090] Specifically, if there are T baking stages, where T is a positive integer, then the baking stages are the first baking stage, the second baking stage, ..., the Tth baking stage. If, based on the above steps, it is determined that the target is located after the first baking stage to the Tth baking stage, then the baking stage of the target is determined to be the Tth baking stage; if, based on the above steps, it is determined that the target is located after the first baking stage to the (T-1)th baking stage and before the Tth baking stage, then the baking stage of the target is determined to be the (T-1)th baking stage; if, based on the above steps, it is determined that the target is located after the first baking stage to the (T-2)th baking stage and before the Tth baking stage, then the baking stage of the target is determined to be the (T-2)th baking stage; if, based on the above steps, it is determined that the target is located after the first baking stage to the Nth baking stage and before the (N+1)th baking stage and the Tth baking stage, then the baking stage of the target is determined to be the Nth baking stage, where N is a positive integer and N is less than T.

[0091] For example, if the target of the baking is tobacco leaves, the baking stage is divided into three stages: yellowing stage, color fixing stage, and drying stage. Based on the relationship between the baking stage of the target determined by the above steps and each set baking stage, if the baking stage of the target is after the yellowing stage, after the color fixing stage, and after the drying stage, then the baking stage of the target can be determined as the drying stage. Based on the relationship between the baking stage of the target determined by the above steps and each set baking stage, if the baking stage of the target is after the yellowing stage, after the color fixing stage, and before the drying stage, then the baking stage of the target can be determined as the color fixing stage. Based on the relationship between the baking stage of the target determined by the above steps and each set baking stage, if the baking stage of the target is after the yellowing stage, before the color fixing stage, and before the drying stage, then the baking stage of the target can be determined as the yellowing stage.

[0092] In the above embodiments, baking state features of the target object can be extracted from the image of the target object. Then, based on these baking state features, the relationship between the current baking stage of the target object and each set baking stage is determined. This relationship includes the target object being in a baking stage before or after any set baking stage. Finally, based on the relationship between the target object and each set baking stage, the current baking stage of the target object is determined. This achieves the goal of automatically determining the baking stage of the target object, avoiding the influence of the baking personnel's experience and skill level on the baking quality, and ensuring the quality of the baked object. Furthermore, by comprehensively considering the relationship between the current baking stage of the target object and each baking stage, the current baking stage of the target object can be determined more accurately, effectively improving the baking quality of the target object.

[0093] As an optional implementation method, such as Figure 2 As shown in another embodiment of this application, the baking target image includes images of multiple baking areas of the baking target. The steps in the above embodiments to extract the baking state features of the baking target from the baking target image may specifically include the following steps:

[0094] S201. Extract the baking state features of each baking area from the image of each baking area.

[0095] Because the temperature and humidity may vary at different locations within the baking chamber, the state changes of the baking target located at different locations within the baking chamber may also differ during the baking process. Therefore, in this embodiment, the image acquisition surface of the baking target is divided into multiple baking areas according to the differences in temperature and humidity. Multiple image acquisition devices are set up to simultaneously acquire images of each baking area as images of the baking target, so as to determine the baking stage of the baking target by combining the differences in the state changes of the baking target in each baking area.

[0096] For example, if the target for curing is tobacco leaves, traditional intensive curing barns are currently mostly used. These barns are characterized by forced ventilation and hot air circulation, with heat and airflow controlled by staff, improving the stability of the curing process. However, traditional intensive curing barns typically have air inlets at the top and outlets at the bottom, with air circulating downwards. This results in higher temperatures at the top and lower temperatures at the bottom. Furthermore, the tobacco leaves are usually stacked in three layers (top, middle, and bottom). Therefore, due to differences in temperature and humidity between these layers, subtle variations in the tobacco leaves' condition during curing can occur. Thus, the image acquisition area can be divided into three curing zones (top, middle, and bottom), and images of the tobacco leaves in all three layers can be simultaneously captured using a three-channel image acquisition device as the target image for curing.

[0097] If multiple image acquisition devices are set up to simultaneously acquire images of each baking area as baking target images, when extracting the baking state features of the baking target from the baking target images, the baking state features of each baking area can be extracted from the images of each baking area as described in the above example. For example, a feature extraction network model can be used to extract the baking state features of each baking area from the images of each baking area. Those skilled in the art can refer to the description in the above embodiments, and it will not be elaborated here.

[0098] S202. Perform feature fusion on the baking state characteristics of all baking areas to obtain the baking state characteristics of the baking target.

[0099] After obtaining the baking state characteristics of the baking areas, the baking state characteristics of the baking areas are fused. For example, when fusing the baking state characteristics of all baking areas, splicing fusion or superposition fusion can be performed, and this embodiment is not limited to this.

[0100] In the above embodiments, the image acquisition surface of the baking target is divided into multiple baking regions. The baking state features of each baking region are extracted and then fused to obtain the baking state features of the baking target. This setup, combined with the differences in the state changes of the baking target in each baking region, can determine the baking stage of the baking target, thereby improving the accuracy of determining the baking stage of the baking target.

[0101] As an optional implementation, another embodiment of this application discloses that the steps of the above embodiments, which extract the baking state features of each baking area from the image of each baking area, may specifically include the following steps:

[0102] The image of each baking area is segmented to obtain at least two regions of interest (ROI) images corresponding to each baking area; ROI image features are extracted from each ROI image; and the ROI image features of each baking area are fused to obtain the baking state features of each baking area.

[0103] As described in the above embodiments, different parts of the baking target change differently during the baking process, and the changes in each part of the baking target are important bases for determining the baking stage of the baking target. If the baking state features of each baking area are directly extracted from the image of each baking area, it may not be possible to effectively locate the feature changes of the small-area parts in the image of the baking area, resulting in inaccurate feature extraction results.

[0104] Based on this, in this embodiment, when extracting the baking state features of the baking target from the images of the baking areas, the images of each baking area can be segmented first to obtain regions of interest (ROI) images of different parts of the baking target. Then, a trained feature extraction network model is used to extract features for each ROI image, obtaining ROI image features in each baking area's ROI image. Finally, the ROI image features in each baking area's ROI image are fused together to obtain the baking state features of each baking area. This setup ensures that feature changes in any part of the baking target are not ignored, improving the accuracy of feature extraction.

[0105] For example, if the target for roasting is tobacco leaves, the tobacco leaf image for each roasting area can be segmented first to obtain the leaf mesophyll image and leaf vein image for each roasting area. Then, a trained feature extraction network model is used to extract features from the leaf mesophyll image and leaf vein image respectively to obtain the leaf mesophyll roasting state features and leaf vein roasting state features for each roasting area. Finally, the leaf mesophyll roasting state features and leaf vein roasting state features of each roasting area are fused together to obtain the roasting state features of each roasting area.

[0106] In this embodiment, a pre-trained semantic segmentation network model can be used to segment the image of the baking region. This semantic segmentation network model can be trained using a segformer network as the base network model; this embodiment is not limited to this. The training samples are images of the baking region, and the training labels are the semantic segmentation results corresponding to each training sample. The specific training process of the semantic segmentation network model is the same as the training process of the feature extraction network model in the above embodiments. Those skilled in the art can refer to the descriptions in the above embodiments; the training process of the semantic segmentation network model will not be described in detail here.

[0107] When fusing the region of interest (ROI) image features in the ROI images of each baking region, a pre-trained feature fusion network model can be used to stitch and fuse or overlay the ROI image features in the ROI images of each baking region. This embodiment does not impose any limitations. When training the feature fusion network model, the training samples are the ROI image features in the ROI images of each baking region, and the training labels are the fused features corresponding to each training sample. The specific training process of the feature fusion network model is the same as the training process of the feature extraction network model in the above embodiments. Those skilled in the art can refer to the description in the above embodiments, and the training process of the feature fusion network model will not be described in detail here.

[0108] In the above embodiments, feature changes in areas with a small area ratio in the baking target can be effectively located, improving the accuracy of feature extraction results.

[0109] As an optional implementation, another embodiment of this application discloses that the steps of the above embodiments fuse the region of interest image features in the region of interest image of each baking area to obtain the baking state features of each baking area, which may specifically include the following steps:

[0110] The region of interest (ROI) image features in each baking region are weighted and fused to obtain the baking state features of each baking region; the weight of the ROI image features changes as the baking time of the target increases.

[0111] As described in the above embodiments, different parts of the baking target exhibit different feature changes at different baking stages, and their contributions to determining the baking stage of the target also vary. Therefore, in this embodiment, when training the feature fusion network model used to stitch and fuse or overlay the region of interest (ROI) image features in the ROI images of each baking region, different weights can be assigned to the ROI image features in different ROI images according to the baking stage of each training sample. That is, if the ROI image feature in a certain ROI image of the baking stage of the training sample contributes significantly to determining the baking stage of the target, then a larger weight can be assigned to that ROI image feature; conversely, if the ROI image feature in a certain ROI image of the baking stage of the training sample contributes less significantly, then a smaller weight can be assigned to that ROI image feature. The feature fusion network model trained in this way can amplify features that contribute significantly to determining the baking stage of the target during feature fusion, thereby improving the accuracy of determining the baking stage of the target.

[0112] For example, such as Figure 3 As shown, if the target of baking is tobacco leaves, semantic segmentation network models can be used to perform semantic segmentation on the tobacco leaf images of each baking area, obtaining images of interest for the leaf mesophyll region and the leaf vein region of interest for each baking area. Then, feature extraction network models are used to extract baking state features from the images of interest, obtaining the extraction result Q. i Q i This includes leaf baking state features Q1 extracted from images of interest in the leaf mesophyll region, and leaf vein baking state features Q2 extracted from images of interest in the leaf vein region. Then, the leaf mesophyll baking state features and leaf vein baking state features of each baking region are weighted and fused, where the weight score Y... i It can be represented as:

[0113] Y i =SoftMax(Q i )

[0114] After determining the weighted scores of the leaf mesophyll baking state characteristics and leaf vein baking state characteristics, a set of weighted scores Y can be obtained. Normalizing the set Y yields...

[0115] Finally, the weighted features of the two regions are fused to obtain the fused feature R. * :

[0116]

[0117] In the above embodiments, features that contribute significantly to determining the baking stage of the target can be amplified during feature fusion, thereby improving the accuracy of determining the baking stage of the target.

[0118] As an optional implementation, another embodiment of this application discloses that the steps of the above embodiments perform feature fusion on the baking state features of all baking areas to obtain the baking state features of the baking target, which may specifically include the following steps:

[0119] Connect the baking state features of all baking areas to obtain connection features; extract the baking state features of the baking target from the connection features.

[0120] When performing feature fusion on the baking state features of all baking areas, in addition to directly splicing and fusion or superimposing and fusion as described in the above embodiments, in this embodiment, the baking state features of all baking areas can be connected first to obtain connection features, and then feature extraction can be performed again from the connection features to obtain the baking state features of the baking target.

[0121] Specifically, such as Figure 4 As shown, extracting the baking state features of the baking target from the connectivity features includes the following steps:

[0122] S401. Obtain the first feature corresponding to the Nth baking area.

[0123] In this embodiment, the baking areas are sorted according to their positions during the actual baking process. For example, an area of ​​any size can be set as the baking area, and then the baking areas are sorted from top to bottom and from right to left according to their positions during the actual baking process.

[0124] For example, if the target to be baked is tobacco leaves, the image acquisition surface of the tobacco leaves can be divided into three baking areas: upper, middle and lower. The three baking areas can also be sorted, with the upper baking area defined as the first baking area, the middle baking area defined as the second baking area, and the lower baking area defined as the third baking area.

[0125] In the embodiments of this application, the first feature corresponding to the Nth baking region is first extracted. This can be achieved by extracting features from the connectivity features to obtain the first feature corresponding to the Nth baking region. If the first feature corresponding to the Nth baking region has already been obtained through other steps, then the first feature corresponding to the Nth baking region obtained through those other steps can be directly acquired.

[0126] It should be noted that if feature extraction is performed on the connection features to obtain the first feature corresponding to the Nth baking region, then a feature extraction model can be used for feature extraction, such as training a Long Short-Term Memory (LSTM) network to obtain the feature extraction model. This embodiment does not limit this.

[0127] The value of N mentioned above is a positive integer. In this embodiment, the initial value of N is 1, that is, in this embodiment, the first feature corresponding to the first baking region is first obtained. Furthermore, in this embodiment, feature extraction is performed on the connection features to obtain the first feature corresponding to the first baking region.

[0128] S402. Based on the first feature corresponding to the Nth baking region, extract the first feature corresponding to the (N+1)th baking region from the connection features.

[0129] In this embodiment, after obtaining the first feature corresponding to the Nth baking region, the first feature and connectivity features corresponding to the Nth baking region are used as input to extract the first feature corresponding to the (N+1)th baking region. The (N+1)th baking region is the baking region adjacent to the Nth baking region. For example, the first feature and connectivity features corresponding to the Nth baking region can be input into a pre-trained feature extraction model, so that the feature extraction model extracts the first feature corresponding to the (N+1)th baking region from the connectivity features based on the first feature corresponding to the Nth baking region.

[0130] For example, if N is 1, after extracting the first feature corresponding to the first baking region, the first feature corresponding to the second baking region is extracted based on the first feature and the connectivity feature. If N is 3, after extracting the first feature corresponding to the third baking region, the first feature corresponding to the fourth baking region is extracted based on the first feature and the connectivity feature.

[0131] S403. Check if N+1 is equal to the total number of baking areas; if N+1 is equal to the total number of baking areas, proceed to step S404; if N+1 is less than the total number of baking areas, set N = N+1 and repeat step S401.

[0132] In this step, it is checked whether the value of N+1 is equal to the total number of baking areas. If the value of N+1 is equal to the total number of baking areas, then step S404 is executed; if the value of N+1 is less than the total number of baking areas, then N = N+1, and steps S401 and S402 are repeated.

[0133] For example, if the total number of baking areas is 10 and N is 1, in this embodiment, after extracting the first feature corresponding to the first baking area, the first feature corresponding to the second baking area is extracted based on the first feature corresponding to the first baking area and the connection feature.

[0134] At this time, N is 1 and N+1 is 2. Since the value of N+1 is less than the total number of baking areas, let N = 2 and repeat steps S401 and S402 to obtain the first feature corresponding to the second baking area extracted in the above steps. Based on the connection feature and the first feature corresponding to the second baking area, the first feature corresponding to the third baking area is extracted.

[0135] At this time, N is 2 and N+1 is 3. Since the value of N+1 is less than the total number of baking areas, let N = 3 and repeat steps S401 and S402 to obtain the first feature corresponding to the third baking area obtained in the above steps. Based on the connection feature and the first feature corresponding to the third baking area, the first feature corresponding to the fourth baking area is extracted.

[0136] At this point, N is 3 and N+1 is 4. Since the value of N+1 is less than the total number of baking areas, N is set to 4, and steps S401 and S402 are repeated. This process is repeated continuously.

[0137] The above steps are repeated until N is 8. When N+1 is 9, and N+1 is less than the total number of baking areas, N is set to 9. Steps S401 and S402 are repeated. The first feature corresponding to the ninth baking area obtained in the above steps is extracted. Based on the connection feature and the first feature corresponding to the ninth baking area, the first feature corresponding to the tenth baking area is extracted.

[0138] At this point, N is 9 and N+1 is 10. The value of N+1 is equal to the total number of baking areas, which is 10. The first feature extraction corresponding to the baking area is completed, and step S404 is executed.

[0139] S404. Determine the baking state characteristics of the baking target through the first characteristics corresponding to each baking area.

[0140] The first features corresponding to each baking area can be superimposed and fused or spliced ​​together to obtain the baking state features of the baking target.

[0141] In the above embodiments, when extracting the baking state features of the current baking area, the baking state features of adjacent baking areas can be combined, thereby preserving the correlation features of each baking area and improving the accuracy of feature extraction.

[0142] As an optional implementation, another embodiment of this application discloses that the steps of the above embodiments for determining the baking state characteristics of the baking target through the first features corresponding to each baking area may specifically include the following steps:

[0143] The first feature corresponding to each baking area is subjected to time-series-based feature extraction processing to obtain the second feature corresponding to each baking area; the second features corresponding to all baking areas are superimposed and fused to obtain the baking state feature of the baking target.

[0144] Specifically, after determining the first feature corresponding to each baking area through the above steps, time-based feature extraction processing can be performed on the first feature corresponding to each baking area to obtain the second feature corresponding to each baking area. Then, the second features corresponding to all baking areas are superimposed and fused, and the superimposed and fused features are used as the baking state features of the baking target. Specifically, when performing time-based feature extraction processing on the first feature corresponding to each baking area, a time recurrent neural network can be used for feature extraction to capture fine-grained features of time-related changes in tobacco leaves.

[0145] In the above embodiments, the correlation features of each baking area are preserved while capturing the fine-grained features of time-related tobacco leaf changes, thereby improving the accuracy of feature extraction.

[0146] As an optional implementation, another embodiment of this application discloses that the steps of the above embodiments determine the relationship between the baking stage of the baking target and each set baking stage based on the baking state characteristics of the baking target, which may specifically include the following steps:

[0147] The baking state characteristics of the target baking state are compared with the baking state characteristics of each set baking stage to determine the relationship between the baking stage of the target baking state and each set baking stage.

[0148] Specifically, in this embodiment, the baking state characteristics of the baking target are compared with the baking state characteristics of each set baking stage, thereby determining the relationship between the baking stage of the baking target and each set baking stage.

[0149] For example, a state feature comparison model can be pre-trained. The training samples are the baking state features of baking targets at any two baking stages, and the training labels are the sequential relationship of the baking stages of the baking targets in each training sample. The specific training process of the state feature comparison model is the same as the training process of the feature extraction network model in the above embodiments. Those skilled in the art can refer to the description in the above embodiments, and the training process of the state feature comparison model will not be described in detail here.

[0150] The baking state features of the target tobacco leaf and the standard baking state features corresponding to each set baking stage can be input into a trained state feature comparison model. This allows the model to determine the relationship between the baking stage of the target tobacco leaf and each set baking stage. For example, if the target tobacco leaf is a tobacco leaf, and the baking stages are divided into three stages: yellowing stage, color fixing stage, and brisket drying stage, the baking state features extracted from the tobacco leaf image and the baking state features of the yellowing stage can be input into the trained state feature comparison model to determine whether the tobacco leaf is in a baking stage before or after the yellowing stage. Similarly, the baking state features extracted from the tobacco leaf image and the baking state features of the color fixing stage can be input into the trained state feature comparison model to determine whether the tobacco leaf is in a baking stage before or after the color fixing stage. Finally, the baking state features extracted from the tobacco leaf image and the baking state features of the brisket drying stage can be input into the trained state feature comparison model to determine whether the tobacco leaf is in a baking stage before or after the brisket drying stage.

[0151] In the above embodiments, by comparing the baking state characteristics of the baking target with the baking state characteristics of each set baking stage, the relationship between the baking stage of the baking target and each set baking stage can be effectively and quickly determined.

[0152] As an optional implementation method, such as Figure 5 As shown in another embodiment of this application, the steps of the above embodiments determine the baking stage of the target based on the relationship between the baking stage of the target and each set baking stage, and may specifically include the following steps:

[0153] S501. Based on the relationship between the baking stage of the baking target and each set baking stage, determine the relationship parameters corresponding to each baking stage.

[0154] Specifically, if the baking stage of the target object is located before any of the set baking stages, the relationship parameter corresponding to that baking stage is a first set value; if the baking stage of the target object is located after any of the set baking stages, the relationship parameter corresponding to that baking stage is a second set value. The specific values ​​of the first and second set values ​​can be set by those skilled in the art according to actual needs, and are not limited here.

[0155] For example, if the target of the baking is tobacco leaves, the baking stage is divided into three stages: yellowing stage, color fixing stage, and dry bar stage. If the baking stage of the target is before the yellowing stage, color fixing stage, or dry bar stage, the relational parameter corresponding to the baking stage can be 1. If the baking stage of the target is after the yellowing stage, color fixing stage, or dry bar stage, the relational parameter corresponding to the baking stage can be 0.

[0156] S502. Determine the baking stage of the target by the interval containing the sum of the relational parameters corresponding to all baking stages.

[0157] By calculating the sum of the relational parameters corresponding to all baking stages, the interval in which the sum of the relational parameters corresponding to each baking stage falls can be determined.

[0158] Specifically, this may include the following steps:

[0159] Calculate the sum of the relational parameters corresponding to all baking stages;

[0160] Determine the interval of relation parameters containing the sum of relation parameters;

[0161] The baking stage corresponding to the interval of relational parameters where the sum of relational parameters is located is determined as the baking stage of the baking target.

[0162] In this embodiment, different relational parameter intervals correspond to different baking stages. Therefore, after calculating the sum of relational parameters corresponding to all baking stages, the baking stage corresponding to the relational parameter interval where the sum of relational parameters lies can be determined as the baking stage of the target being baked.

[0163] For example, if the baking stage of the target is before the yellowing stage, color setting stage, or drying stage, the relational parameter corresponding to this baking stage can be 1. If the baking stage of the target is after the yellowing stage, color setting stage, or drying stage, the relational parameter corresponding to this baking stage can be 0. If, after detection, it is determined that the baking stage of the target is after the yellowing stage, before the color setting stage, and before the drying stage, the relational parameter corresponding to the yellowing stage is 0, the relational parameter corresponding to the color setting stage is 1, the relational parameter corresponding to the drying stage is 1, and the sum of the relational parameters corresponding to all baking stages is 2.

[0164] In the embodiments of this application, different intervals can be set to correspond to different baking stages. Therefore, the baking stage corresponding to the interval of relational parameters where the sum of relational parameters is located can be determined as the baking stage of the baking target.

[0165] For example, if the sum of relational parameters is set to 0 for the drying period, 1 for the color fixing period, and 2 for the yellowing period, then in the above embodiment, the sum of relational parameters corresponding to all baking stages is determined to be 2, and the baking stage of the tobacco leaves can be further determined to be the yellowing period.

[0166] In the above embodiments, by comprehensively considering the baking stage of the baking target and the relationship between each baking stage, the current baking stage of the baking target can be determined more accurately, thereby effectively improving the baking quality of the baking target.

[0167] As an optional implementation, another embodiment of this application discloses that the steps of the above embodiments perform feature fusion on the baking state features of all baking areas to obtain the baking state features of the baking target. Based on the baking state features of the baking target, the relationship between the baking stage of the baking target and each set baking stage is determined. According to the relationship between the baking stage of the baking target and each set baking stage, the baking stage of the baking target is determined. Specifically, this may include the following steps:

[0168] The baking state features of the target area in each image acquisition region are input into the pre-trained baking stage detection model. The first network in the baking stage detection model performs feature fusion on the baking state features of all baking regions to obtain the baking state features of the target. The second network in the baking stage detection model determines the relationship between the baking stage of the target and each set baking stage based on the baking state features of the target. Based on the relationship between the baking stage of the target and each set baking stage, the baking stage of the target is determined.

[0169] Specifically, in this embodiment, a baking stage detection model can be pre-trained to detect the baking stage of the target being baked. This trained baking stage detection model may include a first network and a second network.

[0170] like Figure 6 As shown, the first network includes a first fusion network structure, a second fusion network structure, and a third fusion network structure.

[0171] The first fusion network structure is used to integrate and connect the baking state features of each baking zone to obtain connection features. For example, Figure 6 In the embodiment shown, three baking regions are set up. The first fusion network structure is used to integrate and connect the baking state features of the first baking region, the baking state features of the second baking region, and the baking state features of the third baking region to obtain connection features.

[0172] The second fusion network structure is a two-layer fusion network built based on a recurrent temporal neural network. The recurrent temporal neural network can be an LSTM network, such as... Figure 6As shown, this embodiment is not limited. The number of first-layer fusion networks in the second fusion network structure is the same as the total number of baking regions. Each first-layer fusion network is used to extract the first feature of the corresponding baking region. As described in the above embodiment, the first-layer fusion network corresponding to the first baking region is used to extract the first feature corresponding to the first baking region from the connection features; the first-layer fusion network corresponding to the second baking region is used to extract the first feature corresponding to the first baking region from the connection features based on the first feature corresponding to the first baking region; the first-layer fusion network corresponding to the third baking region is used to extract the first feature corresponding to the third baking region from the connection features based on the first feature corresponding to the second baking region, and so on. For example, Figure 6 In the embodiment shown, the number of first-layer fusion networks is three, namely LSTM1, LSTM2 and LSTM3. LSTM1 is used to extract features from the connection features to obtain the first feature corresponding to the first baking region. LSTM2 is used to extract features from the connection features based on the first feature corresponding to the first baking region to obtain the first feature corresponding to the second baking region. LSTM3 is used to extract features from the connection features based on the first feature corresponding to the second baking region to obtain the first feature corresponding to the third baking region.

[0173] The number of second-layer fusion networks in the second fusion network structure is the same as the total number of baking regions. These second-layer fusion networks are used to perform time-series-based feature extraction based on the first feature, obtaining the second feature corresponding to each baking region. For example, Figure 6 In the illustrated embodiment, the second-layer fusion network consists of three networks: LSTM4, LSTM5, and LSTM6. LSTM4 is used to perform time-based feature extraction on the first feature corresponding to the first baking region output by LSTM1 to obtain the second feature corresponding to the first baking region. LSTM5 is used to perform time-based feature extraction on the first feature corresponding to the second baking region output by LSTM2 to obtain the second feature corresponding to the second baking region. LSTM6 is used to perform time-based feature extraction on the first feature corresponding to the third baking region output by LSTM3 to obtain the second feature corresponding to the third baking region.

[0174] The third fusion network structure is a fully connected layer, which is used to superimpose the second features corresponding to each baking area to obtain the baking state features of the baking target. Figure 6 In the embodiment shown, the third fusion network structure fuses the second feature corresponding to the first baking region, the second feature corresponding to the second baking region, and the second feature corresponding to the third baking region to obtain the baking state feature of the baking target.

[0175] The baking state characteristics of the baking target output by the first network are the input of the second network.

[0176] like Figure 7 As shown, the second network includes an aggregation network structure and n state feature comparison network structures, where n equals the set number of baking stages. Each state feature comparison network structure corresponds to a set baking stage, and each state feature comparison network structure is trained independently to ensure that the learned features have more effective representation capabilities. Based on each state feature comparison network structure, the relationship between the baking stage of the target and the baking stage corresponding to that state feature comparison network structure is compared, thereby determining the relationship between the baking stage of the target and each set baking stage. Based on the relationship between the baking stage of the target and each set baking stage, the relationship parameters corresponding to each baking stage are determined.

[0177] For example, if the state feature comparison network structure determines, through comparison, that the baking stage of the target is located before the baking stage corresponding to the state feature comparison network structure, the relationship parameter of the baking stage corresponding to the state feature comparison network structure can be determined as a first set value. If the state feature comparison network structure determines, through comparison, that the baking stage of the target is located after the baking stage corresponding to the state feature comparison network structure, the relationship parameter of the baking stage corresponding to the state feature comparison network structure can be determined as a second set value. For example, the first set value can be set to 1, and the second set value can be set to 2; this embodiment does not impose any limitations.

[0178] The aggregation network structure is used to integrate the output of the state feature comparison network to determine the baking stage of the baking target.

[0179] For example, the aggregated network structure can be used to calculate the sum of relational parameters corresponding to all baking stages, determine the relational parameter interval in which the sum of relational parameters lies, and determine the baking stage in which the baking target is located as the baking stage in the relational parameter interval in which the sum of relational parameters lies.

[0180] Another example is that the aggregation network structure can be used to calculate the predicted stage value according to the following formula, and then determine the baking stage of the target baking stage based on the baking stage corresponding to the predicted stage interval in which the predicted stage value is located:

[0181]

[0182] D(x i ) represents the predicted value, x i H represents the baking state characteristics of the baking target. n (x i ) represents the relational parameter, n represents the number of the set baking stages, and [] represents the truth judgment.n (x i When ) > 0 is true, the output value of [] is 1, otherwise it is 0.

[0183] For example, if the target to be roasted is tobacco, the set roasting stages include the yellowing stage, the color-fixing stage, or the drying stage. If the roasting stage of the target to be roasted is before the yellowing stage, the color-fixing stage, or the drying stage, the corresponding relational parameter for that roasting stage can be 1; if the roasting stage of the target to be roasted is after the yellowing stage, the color-fixing stage, or the drying stage, the corresponding relational parameter for that roasting stage can be 0.

[0184] If, after testing, it is determined that the baking target is in the baking stage after the yellowing stage, before the color-fixing stage, and before the drying stage, the relational parameter corresponding to the yellowing stage is 0, the relational parameter corresponding to the color-fixing stage is 1, and the relational parameter corresponding to the drying stage is 1, then according to the above formula, D(x i The value of ) is 3.

[0185] If D(x) is set i A value of 1 corresponds to the dry bond stage, D(x) i A value of 2 corresponds to the color fixation period, D(x) i The value of ) is 3 for the strain yellow period, then in the above embodiments, D(x) is determined to be 3. i If the value of ) is 3, then the curing stage of the tobacco leaves can be further determined to be the yellowing stage.

[0186] Furthermore, it should be noted that when training the baking stage detection model, the first and second networks can be trained jointly. The training samples are the baking state features of each baking region, and the relational parameters of each training sample for each set baking stage. Specifically, the training samples are input into the first network, and the output of the comparison network structure for each state feature in the second network is obtained. The output of each state feature comparison network structure in the second network is compared with the training labels to determine the loss value of the baking stage detection model. The parameters of the baking stage detection model are then adjusted in the direction of reducing the loss value. This training process is repeated until the loss value of the baking stage detection model is less than the set value.

[0187] Furthermore, the processes of semantic segmentation of the image of each baking region, extraction of region of interest image features from the segmented region of interest images, and weighted fusion of the region of interest image features in the region of interest images of each baking region can all be performed using models. The above models can be trained individually or in combination, and this embodiment does not impose any limitations.

[0188] In the above example, the pre-trained baking stage detection model can quickly and efficiently determine the baking stage of the target, so that the baking personnel can adjust the temperature, humidity and other parameters in the baking room according to the baking stage of the target, effectively improving the baking quality of the target.

[0189] Exemplary devices, electronic devices, computer program products, and storage media

[0190] Corresponding to the method for determining the baking stage described above, this application also discloses a device for determining the baking stage, see [link to relevant documentation]. Figure 8 As shown, the device includes:

[0191] Extraction module 100 is used to extract the baking state features of the baking target from the baking target image of the baking target;

[0192] The first determining module 110 is used to determine the relationship between the baking stage of the baking target and each set baking stage based on the baking state characteristics of the baking target; the relationship between the baking stage of the baking target and each set baking stage includes the baking stage of the baking target being located before any set baking stage, or the baking stage of the baking target being located after any set baking stage.

[0193] The second determining module 120 is used to determine the baking stage of the baking target based on the relationship between the baking stage of the baking target and each set baking stage.

[0194] As an optional implementation, another embodiment of this application discloses that the baking target image of the above embodiments includes images of multiple baking areas of the baking target;

[0195] Extraction module 100 includes:

[0196] The extraction unit is used to extract the baking state features of each baking area from the image of each baking area;

[0197] The fusion unit is used to fuse the baking state features of all baking areas to obtain the baking state features of the baking target.

[0198] As an optional implementation, another embodiment of this application discloses an extraction unit, comprising:

[0199] The segmentation subunit is used to segment the image of each baking area to obtain at least two regions of interest images corresponding to each baking area;

[0200] The first extraction subunit is used to extract the region-of-interest (ROI) image features from each ROI image;

[0201] The fusion subunit is used to fuse the region of interest image features in the region of interest image of each baking area to obtain the baking state features of each baking area.

[0202] As an optional implementation, another embodiment of this application discloses that when the fusion subunit fuses the region-of-interest image features in the region-of-interest image of each baking region to obtain the baking state features of each baking region, it is specifically used for:

[0203] The region of interest (ROI) image features in each baking region are weighted and fused to obtain the baking state features of each baking region; the weight of the ROI image features changes as the baking time of the target increases.

[0204] As an optional implementation, another embodiment of this application discloses a fusion unit, comprising:

[0205] The connection subunit is used to connect the baking state features of all baking areas to obtain the connection features;

[0206] The second extraction subunit is used to extract the baking state features of the baking target from the connection features.

[0207] As an optional implementation, another embodiment of this application discloses that when the second extraction subunit extracts the baking state features of the baking target from the connection features, it is specifically used for:

[0208] S1. Obtain the first feature corresponding to the Nth baking region; where N is a positive integer;

[0209] S2. Based on the first feature corresponding to the Nth baking region, extract the first feature corresponding to the (N+1)th baking region from the connection features; wherein, the (N+1)th baking region is the baking region adjacent to the Nth baking region;

[0210] S3. Let N = N + 1, and repeat steps S1 and S2 until N equals the total number of baking areas. Then, determine the baking state characteristics of the baking target through the first characteristics corresponding to each baking area.

[0211] As an optional implementation, another embodiment of this application discloses that when the second extraction subunit determines the baking state features of the baking target through the first features corresponding to each baking area, it is specifically used for:

[0212] The first feature corresponding to each baking area is subjected to time-series-based feature extraction processing to obtain the second feature corresponding to each baking area; the second features corresponding to all baking areas are superimposed and fused to obtain the baking state feature of the baking target.

[0213] As an optional implementation, another embodiment of this application discloses that when the first determining module 110 determines the relationship between the baking stage of the baking target and each set baking stage based on the baking state characteristics of the baking target, it is specifically used for:

[0214] The baking state characteristics of the target baking state are compared with the baking state characteristics of each set baking stage to determine the relationship between the baking stage of the target baking state and each set baking stage.

[0215] As an optional implementation, another embodiment of this application discloses a second determining module 120, comprising:

[0216] The first determining unit is used to determine the relationship parameter corresponding to each baking stage based on the relationship between the baking stage where the baking target is located and each set baking stage; wherein, if the baking stage where the baking target is located is before any set baking stage, the relationship parameter corresponding to that baking stage is a first set value, and if the baking stage where the baking target is located is after any set baking stage, the relationship parameter corresponding to that baking stage is a second set value.

[0217] The second determining unit is used to determine the baking stage of the baking target by the interval containing the sum of the relational parameters corresponding to all baking stages.

[0218] As an optional implementation, another embodiment of this application discloses that when the second determining unit determines the baking stage of the baking target by the interval containing the sum of the relational parameters corresponding to all baking stages, it is specifically used for:

[0219] Calculate the sum of relational parameters corresponding to all baking stages; determine the relational parameter interval in which the sum of relational parameters lies; different relational parameter intervals correspond to different baking stages; determine the baking stage corresponding to the relational parameter interval in which the sum of relational parameters lies as the baking stage of the baking target.

[0220] As an optional implementation, another embodiment of this application discloses that the fusion unit performs feature fusion on the baking state features of all baking areas to obtain the baking state features of the baking target. The first determining module 110 determines the relationship between the baking stage of the baking target and each set baking stage based on the baking state features of the baking target. The second determining module 120, when determining the baking stage of the baking target based on the relationship between the baking stage of the baking target and each set baking stage, is specifically used for:

[0221] The baking state features of the target area in each image acquisition region are input into the pre-trained baking stage detection model. The first network in the baking stage detection model performs feature fusion on the baking state features of all baking regions to obtain the baking state features of the target. The second network in the baking stage detection model determines the relationship between the baking stage of the target and each set baking stage based on the baking state features of the target. Based on the relationship between the baking stage of the target and each set baking stage, the baking stage of the target is determined.

[0222] For details on the specific working content of each unit of the baking stage determination device, please refer to the above method embodiment; it will not be repeated here.

[0223] Another embodiment of this application also provides an electronic device, see [link to relevant documentation] Figure 9 As shown, the electronic device includes:

[0224] Memory 200 and processor 210;

[0225] The memory 200 is connected to the processor 210 and is used to store programs;

[0226] The processor 210 is configured to implement the method for determining the baking stage disclosed in any of the above embodiments by running a program stored in the memory 200.

[0227] Specifically, the aforementioned electronic device may also include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0228] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them:

[0229] A bus can include a pathway for transmitting information between various components of a computer system.

[0230] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0231] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0232] The memory 200 stores a program for executing the technical solution of this application, and may also store an operating system and other critical business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0233] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0234] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0235] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0236] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the various steps of the baking stage determination method provided in the above embodiments of this application.

[0237] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by processor 210, cause processor 210 to perform the various steps of the method for determining the baking stage provided in the above embodiments.

[0238] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0239] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions, which, when executed by a processor, cause the processor 210 to perform the various steps of the baking stage determination method provided in the above embodiments.

[0240] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0241] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0242] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0243] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0244] The modules and sub-modules in the various embodiments of the present application's devices and terminals can be merged, divided, and deleted according to actual needs.

[0245] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0246] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0247] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0248] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0249] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0250] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0251] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining a baking stage, characterized in that, include: The baking status features of each baking area are extracted from the image of each baking area. The baking area is obtained by dividing the image acquisition surface of the baking target according to the difference in temperature and humidity. The baking state characteristics of all baking areas are fused to obtain the baking state characteristics of the baking target. Based on the baking state characteristics of the baking target, the relationship between the baking stage in which the baking target is located and each set baking stage is determined; the relationship between the baking stage in which the baking target is located and any baking stage includes whether the baking stage in which the baking target is located is before or after the baking stage. The baking stage of the target is determined based on its relationship with each set baking stage.

2. The method according to claim 1, characterized in that, The baking status features of each baking area were extracted from the image of each baking area, including: The image of each baking area is segmented to obtain at least two regions of interest images corresponding to each baking area; Extract the region-of-interest (ROI) image features from each ROI image; The features of the region of interest (ROI) images in each baking region are fused to obtain the baking state features of each baking region.

3. The method according to claim 2, characterized in that, The region-of-interest (ROI) image features of each baking region are fused to obtain the baking state features of each baking region, including: The region of interest (ROI) image features in the ROI image of each baking region are weighted and fused to obtain the baking state features of each baking region; wherein, the weight of the ROI image features changes as the baking time of the target increases.

4. The method according to claim 1, characterized in that, The baking state features of all baking areas are fused to obtain the baking state features of the target baking area, including: Connect the baking state features of all baking areas to obtain the connection features; The baking state features of the baking target are extracted from the connection features.

5. The method according to claim 4, characterized in that, The baking state features of the baking target are extracted from the connection features, including: S1. Obtain the first feature corresponding to the Nth baking region; where N is a positive integer; S2. Based on the first feature corresponding to the Nth baking region, extract the first feature corresponding to the (N+1)th baking region from the connection features; wherein, the (N+1)th baking region is the baking region adjacent to the Nth baking region; S3. Let N = N + 1, and repeat steps S1 and S2 until N equals the total number of baking areas. Then, determine the baking state characteristics of the baking target through the first characteristics corresponding to each baking area.

6. The method according to claim 5, characterized in that, The baking state characteristics of the baking target are determined by using the first feature corresponding to each baking area, including: The first feature corresponding to each baking area is subjected to time-series-based feature extraction processing to obtain the second feature corresponding to each baking area; The second features corresponding to all baking areas are superimposed and fused to obtain the baking state features of the baking target.

7. The method according to claim 1, characterized in that, Based on the baking state characteristics of the baking target, the relationship between the baking stage of the baking target and each set baking stage is determined, including: The baking state characteristics of the baking target are compared with the baking state characteristics of each set baking stage to determine the relationship between the baking stage of the baking target and each set baking stage.

8. The method according to claim 1, characterized in that, The baking stage of the target baking object is determined based on its relationship with each set baking stage, including: Based on the relationship between the baking stage where the baking target is located and each set baking stage, the relationship parameter corresponding to each baking stage is determined; wherein, if the baking stage where the baking target is located is before any set baking stage, the relationship parameter corresponding to that baking stage is a first set value, and if the baking stage where the baking target is located is after any set baking stage, the relationship parameter corresponding to that baking stage is a second set value. The baking stage of the target is determined by the interval containing the sum of the relational parameters corresponding to all baking stages.

9. The method according to claim 8, characterized in that, Determining the baking stage of the target baking target by identifying the interval containing the sum of the relational parameters corresponding to all baking stages includes: Calculate the sum of the relational parameters corresponding to all baking stages; Determine the interval of the sum of the relational parameters; where different intervals of relational parameters correspond to different baking stages; The baking stage corresponding to the interval of relational parameters in which the sum of the relational parameters lies is determined as the baking stage of the baking target.

10. The method according to claim 1, characterized in that, Feature fusion is performed on the baking state characteristics of all baking areas to obtain the baking state characteristics of the baking target. Based on the baking state characteristics of the baking target, the relationship between the baking stage of the baking target and each set baking stage is determined. The baking stage of the baking target is determined according to the relationship between the baking stage of the baking target and each set baking stage, including: The baking state features of the target area in each image acquisition region are input into a pre-trained baking stage detection model, so that the first network in the baking stage detection model performs feature fusion on the baking state features of all baking regions to obtain the baking state features of the target. The second network in the baking stage detection model determines the relationship between the baking stage of the target and each set baking stage based on the baking state features of the target. Based on the relationship between the baking stage of the target and each set baking stage, the baking stage of the target is determined.

11. A device for determining a baking stage, characterized in that, include: The extraction module is used to extract the baking state features of each baking area from the image of each baking area; The baking area is obtained by dividing the image acquisition surface of the baking target according to the difference in temperature and humidity; the baking state features of all baking areas are fused to obtain the baking state features of the baking target; The first determining module is used to determine the relationship between the baking stage of the baking target and each set baking stage based on the baking state characteristics of the baking target; the relationship between the baking stage of the baking target and each set baking stage includes the baking stage of the baking target being located before any set baking stage, or the baking stage of the baking target being located after any set baking stage. The second determining module is used to determine the baking stage of the baking target based on the relationship between the baking stage of the baking target and each set baking stage.

12. An electronic device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is configured to implement the method for determining the baking stage as described in any one of claims 1 to 10 by running a program in the memory.

13. A storage medium, characterized in that, include: The storage medium stores a computer program, which, when executed by a processor, implements the method for determining the baking stage as described in any one of claims 1 to 10.

Images