Shielding reminding method and device for camera and electronic equipment

Abstract

The invention provides a shielding reminding method and device for a camera and electronic equipment. The shielding reminding method comprises the following steps: acquiring image data and environment state data; performing feature extraction processing on the image data to obtain image feature data; performing identification processing based on the environment state data and the image feature data to obtain image quality information and occlusion category information; and under the condition that the image quality information indicates that the image quality of the image data is in a preset quality range, generating occlusion reminding information according to the occlusion category information. Therefore, in camera shielding detection, the accuracy and robustness of camera shielding detection are improved by fusing the image data and the environment state data; in combination with image quality evaluation, the method is used for assisting camera shielding detection and determining the camera shielding reminding opportunity, it is ensured that the camera can continuously provide high-quality images, and too frequent shielding reminding is avoided.

Description

Technical Field

[0001] The present disclosure relates to the field of image acquisition, and in particular, to an occlusion reminder method, device, and electronic device for a camera. Background Art

[0002] In the field of image acquisition, image acquisition is performed through a camera. If the camera is occluded, the camera cannot capture the desired image, and camera occlusion reminder becomes an urgent problem to be solved. The application fields of image acquisition are extensive and the application environments are diverse. Currently, in the cooking field, image acquisition is used to improve the intelligence of cooking devices. However, the recognition of the occlusion state of the camera in the cooking field often has the defect of low accuracy, which is not conducive to reminding users of camera occlusion. Summary of the Invention

[0003] To solve the above technical problems, the present disclosure provides an occlusion reminder method, device, and electronic device for a camera.

[0004] According to a first aspect of the present disclosure, there is provided an occlusion reminder method for a camera, including:

[0005] Obtaining image data and environmental state data, where the image data is collected by the camera, and the environmental state data represents the environmental state corresponding to the image data;

[0006] Performing feature extraction processing on the image data to obtain image feature data;

[0007] Performing recognition processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information;

[0008] When the image quality information indicates that the image quality of the image data is within a preset quality range, generating occlusion reminder information according to the occlusion category information.

[0009] Optionally, the environmental state data includes temperature data and humidity data. The performing recognition processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information includes:

[0010] Performing extraction processing on the temperature data and the humidity data respectively to obtain environmental feature data, where the environmental feature data includes temperature feature data and humidity feature data;

[0011] Performing recognition processing according to the environmental feature data and the image feature data to obtain image quality information and occlusion category information.

[0012] Optionally, the performing recognition processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information includes:

[0013] Perform a conversion process based on the environmental state data and the image feature data to obtain feature conversion data;

[0014] Based on the feature conversion data, obtain the image quality information and the occlusion category information, where the occlusion category information includes at least one of wiping state information, cover state information, attachment state information, and lid-lifting state information.

[0015] Optionally, the obtaining the image quality information and the occlusion category information based on the feature conversion data includes:

[0016] Obtain target parameter information corresponding to a target category, where the target category is one of image quality, wiping state, cover state, attachment state, and lid-lifting state, and the target parameter information includes a target weight parameter and a target bias parameter;

[0017] Perform an identification conversion based on the target parameter information and the feature conversion data to obtain target category information, where the target category information is one of image quality information, wiping state information, cover state information, attachment state information, and lid-lifting state information.

[0018] Optionally, the generating the occlusion reminder information based on the occlusion category information includes:

[0019] Generate an occlusion reminder information when the occlusion category information indicates that the camera is in a target occlusion state, where the target occlusion state is at least one of a real-time wiping state, a cover covering state, an attachment attachment state, and a lid-lifting state.

[0020] Optionally, the reminder method further includes:

[0021] When the image quality information indicates that the image quality of the image data is within a preset quality range, if the occlusion category information meets a preset occlusion condition, no occlusion reminder information is generated.

[0022] Optionally, the performing an identification process based on the environmental state data and the image feature data to obtain image quality information and occlusion category information includes:

[0023] Use a state detection model to perform an identification process based on the environmental state data and the image feature data to obtain image quality information and occlusion category information;

[0024] The method further includes:

[0025] Based on the sample data, the initial detection model is trained to obtain the state detection model, and the loss parameter of the training is obtained from the cross-entropy loss obtained from the mean square error losses corresponding to the occlusion category information and the image quality information respectively.

[0026] Optionally, the reminder method further includes:

[0027] Generating an occlusion risk reminder message when the temperature data and the humidity data meet the preset environmental state conditions.

[0028] According to a second aspect of the present disclosure, there is provided an occlusion reminder device for a camera, including:

[0029] A data acquisition module for acquiring image data and environmental state data, where the image data is acquired by the camera, and the environmental state data characterizes the environmental state corresponding to the image data;

[0030] An extraction processing module for performing feature extraction processing on the image data to obtain image feature data;

[0031] An identification processing module for performing identification processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information;

[0032] An occlusion reminder module for generating an occlusion reminder message according to the occlusion category information when the image quality information indicates that the image quality of the image data is within a preset quality range.

[0033] According to a third aspect of the present disclosure, there is provided an electronic device, the electronic device including:

[0034] A processor;

[0035] A memory for storing instructions executable by the processor;

[0036] Wherein, the processor is configured to execute the instructions to implement the occlusion reminder method for a camera as described in the above technical solution.

[0037] According to a fourth aspect of the present disclosure, there is provided an extractor hood, including an extractor hood component, an image acquisition component, a temperature and humidity sensing component, and at least one processor, and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and at least one of the processors is used for communicatively connecting to the image acquisition component and the temperature and humidity sensing component, and at least one of the processors realizes the occlusion reminder method as described in the above technical solution by executing the instructions stored in the memory.

[0038] According to a fifth aspect of the present disclosure, there is provided a computer-readable storage medium, which, when the instructions stored therein are executed by a processor of an electronic device, enables the electronic device to execute the occlusion reminder method as described in the above technical solution.

[0039] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure.

[0040] Implementing the present disclosure has the following beneficial effects:

[0041] The present disclosure provides an occlusion reminder method, apparatus, and electronic device for a camera, including: obtaining image data and environmental state data; performing feature extraction processing on the image data to obtain image feature data; performing recognition processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information; and generating an occlusion reminder information according to the occlusion category information when the image quality information indicates that the image quality of the image data is within a preset quality range. Thus, in the camera occlusion detection, by fusing the image data and the environmental state data, the accuracy and robustness of the camera occlusion detection are improved, and the false alarm rate and the missed alarm rate are reduced; the image quality evaluation is combined to assist the camera occlusion detection and determine the camera occlusion reminder timing, ensuring that the camera can continuously provide high-quality images and avoiding overly frequent occlusion reminders.

[0042] According to the following detailed description of exemplary embodiments with reference to the accompanying drawings, other features and aspects of the present disclosure will become clear. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present specification or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present specification, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0044] Figure 1 A flowchart showing an occlusion reminder method for a camera according to an embodiment of the present disclosure;

[0045] Figure 2 A flowchart showing the process of determining image quality information and occlusion category information according to an embodiment of the present disclosure;

[0046] Figure 3 A flowchart showing the process of recognition conversion processing for determining image quality information and occlusion category information according to an embodiment of the present disclosure;

[0047] Figure 4A flowchart showing the process of determining target category information according to an embodiment of the present disclosure;

[0048] Figure 5 A structural diagram showing an occlusion reminder device for a camera according to an embodiment of the present disclosure. Detailed implementation manners

[0049] The technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present specification. Obviously, the described embodiments are only a part of the embodiments of the present specification, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present specification without creative efforts shall fall within the scope of protection of the present invention.

[0050] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0051] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the drawings. The same reference numerals in the drawings denote elements having the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings do not have to be drawn to scale unless otherwise specified.

[0052] The special term "exemplary" herein means "serving as an example, embodiment, or illustration". Any embodiment described as "exemplary" herein does not have to be construed as superior to or better than other embodiments.

[0053] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" in this article means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set composed of A, B, and C.

[0054] In addition, to better illustrate the present disclosure, numerous specific details are given in the following detailed implementation manners. Those skilled in the art should understand that the present disclosure can be implemented without some specific details. In some instances, methods, means, elements, and circuits well-known to those skilled in the art are not described in detail to highlight the gist of the present disclosure.

[0055] Figure 1 FIG. shows a schematic flowchart of a method for reminding of camera occlusion according to an embodiment of the present disclosure. The method for reminding of camera occlusion in the embodiments of the present invention can be applied to an image acquisition device, a cooking device, a smart terminal, or a background server, such as a control unit of an image acquisition device, including a control unit of a cooking device of an image acquisition device, or a smart terminal or a background server communicatively connected to the image acquisition device; the cooking device is a device for cooking food, for example, the cooking device includes a range hood, an oven, or a steam oven; the smart terminal can be a PC, a mobile phone, a tablet computer, or other smart electronic devices, and the background server is, for example, a cloud server. This specification provides method operation steps such as in the embodiments or flowcharts, but based on routine or non-creative labor, it can include more or fewer operation steps. The order of steps listed in the embodiments is only one way among the execution orders of numerous steps and does not represent the only execution order. When the actual system or server product executes, it can execute in the order of the embodiments or as shown in the drawings or execute in parallel (for example, in an environment of parallel processors or multi-threaded processing). As Figure 1 shown, the above-mentioned method for reminding of camera occlusion includes:

[0056] Step S101, obtaining image data and environmental state data, where the image data is collected by the camera, and the environmental state data characterizes the environmental state corresponding to the image data.

[0057] Optionally, the image data is collected by the camera. Preferably, the image data is collected by the camera in real time. Alternatively, the image data is collected by the camera at a target moment. Exemplarily, the camera is installed on a range hood, an oven, or a steam oven for image acquisition of a stove top, an oven cavity, or a steam oven.

[0058] Optionally, the environmental state data characterizes the environmental state corresponding to the image data, that is, the environmental state where the camera is located or corresponding to when generating the image data. Optionally, the environmental state data is generated by an environmental state sensor arranged adjacent to the camera, and the generation time of the environmental state data corresponds to the generation time of the image data.

[0059] Step S102, performing feature extraction processing on the image data to obtain image feature data.

[0060] Specifically, perform feature extraction processing on the image data, for example, perform convolutional processing on the image data to extract image features or visual features, and obtain image feature data.

[0061] Optionally, before performing the feature extraction processing, preprocess the image data, such as performing normalization, denoising, etc.

[0062] Step S103, perform recognition processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information.

[0063] Specifically, use a state detection model to perform the recognition processing, and use the environmental state data and the image feature data as the inputs of the state detection model for recognition processing to obtain corresponding image quality information and occlusion category information. Thus, in the recognition processing, using both the environmental state data and the image feature data as input variables can identify the image quality and occlusion category information of image data in different environmental states, avoid interference caused by different environmental conditions to the detection of image data, and improve the accuracy and robustness of camera occlusion detection.

[0064] Optionally, the image quality information represents the image quality evaluation result of the image data. Optionally, the image quality information is image quality category information or image quality quantization information. Exemplarily, the image quality category information includes excellent, good, qualified, and unqualified, or the first level, the second level... the nth level, where n is a positive integer, and optionally, 4 ≤ n ≤ 20. For example, the value of n is 5, 6, 8, 10, 12, or 15. The higher the level, the higher the image quality, or the higher the level, the lower the image quality; the image quality quantization information includes a quantization value for indicating the image quality. For example, the image quality quantization information is a value between 0 - 1, between 1 - 10, or between 1 - 100. The larger the value, the higher the image quality.

[0065] Optionally, the occlusion category information is used to indicate the occlusion category that occludes the camera, such as attachment occlusion or covering occlusion. The attachment is an object attached to the camera lens, such as particles, fibers, water droplets, water stains, oil droplets, or oil stains, etc. The attachment makes the camera dirty and affects the light transmission of the camera. The covering is an object located on the camera lens or adjacent to the camera lens. The covering blocks light from entering the camera. Exemplarily, the covering is a vegetable leaf, a block, a cooking container adjacent to the camera lens, or other coverings located on the camera optical path that block light from entering the camera. The cooking container is, for example, a bread mold.

[0066] Step S104, when the image quality information indicates that the image quality of the image data is within a preset quality range, generating occlusion reminder information according to the occlusion category information.

[0067] Specifically, if the image quality information indicates that the image quality of the image data is within a preset quality range, the image quality of the image data is in a low image quality state. In other words, the image data is in an abnormal quality state, which means there is a possibility of camera abnormality. Generate corresponding occlusion reminder information according to the camera occlusion state indicated by the occlusion category information to remind the user of camera occlusion.

[0068] Thus, the present disclosure performs image recognition detection based on image data and environmental state data to obtain image quality information and occlusion category information, and when the image quality information indicates that the image quality of the image data is within a preset quality range, generating occlusion reminder information according to the occlusion category information. Thus, in camera occlusion detection, by fusing image data and environmental state data, the accuracy and robustness of camera occlusion detection are improved, and the false alarm rate and missed alarm rate are reduced; combining image quality evaluation is used to assist camera occlusion detection and determine the timing of camera occlusion reminder, ensuring that the camera can continuously provide high-quality images and avoiding overly frequent occlusion reminders.

[0069] In an alternative embodiment, as Figure 2 shown, the environmental state data includes temperature data and humidity data. The performing recognition processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information includes:

[0070] Step S11, respectively performing extraction processing on the temperature data and the humidity data to obtain environmental feature data, where the environmental feature data includes temperature feature data and humidity feature data.

[0071] Specifically, the temperature data is generated by a temperature sensor, and the humidity data is generated by a humidity sensor. The temperature sensor and the humidity sensor are separately arranged or integrally combined. Optionally, temperature feature data is extracted from the temperature data generated by the temperature sensor. For example, statistical processing or temperature data point extraction is performed on the temperature data to obtain temperature feature data corresponding to the generation time of the image data. The temperature feature data characterizes the temperature corresponding to the generation time of the image data. Exemplarily, the temperature feature data can be generated based on the temperature data generated within a time range preset at a time interval from the generation time. For example, the average temperature or weighted temperature value corresponding to the temperature data generated within the time range is used as the temperature feature data. The weighted temperature value is obtained based on the temperature data and the corresponding temperature weight. The closer the temperature generation time of the temperature data is to the generation time of the image data, the greater the temperature weight corresponding to the temperature data. The preset time interval Δt is 0.01 - 1 s, such as 0.05 s, 0.1 s, 0.2 s, 0.4 s, or 0.5 s. Then, based on the temperature data generated between the first moment t - Δt, which is the preset time interval before the generation time, and the second moment t + Δt, which is the preset time interval after the generation time, the temperature feature data is obtained. In an alternative embodiment, the temperature data point value closest to the generation time of the image data is used as the temperature feature data, or the average value of the two temperature data closest to the generation time of the image data is used as the temperature feature data.

[0072] Optionally, humidity feature data is extracted from the humidity data generated by the humidity sensor. For example, statistical processing or humidity data point extraction is performed on the humidity data to obtain humidity feature data corresponding to the generation time of the image data. The humidity feature data characterizes the humidity corresponding to the generation time of the image data. Exemplarily, the humidity feature data can be generated based on the humidity data generated within a time range preset at a time interval from the generation time. For example, the average humidity or weighted humidity value corresponding to the humidity data generated within the time range is used as the humidity feature data. The weighted humidity value is obtained based on the humidity data and the corresponding humidity weight. The closer the humidity generation time of the humidity data is to the generation time of the image data, the greater the humidity weight corresponding to the humidity data. The preset time interval Δt is 0.01 - 1 s, such as 0.05 s, 0.1 s, 0.2 s, 0.4 s, or 0.5 s. Then, based on the humidity data generated between the first moment t - Δt, which is the preset time interval before the generation time, and the second moment t + Δt, which is the preset time interval after the generation time, the humidity feature data is obtained. In an alternative embodiment, the humidity data point value closest to the generation time of the image data is used as the humidity feature data, or the average value of the two humidity data closest to the generation time of the image data is used as the humidity feature data.

[0073] Step S12: Perform recognition and conversion processing based on the environmental feature data and the image feature data to obtain image quality information and occlusion category information.

[0074] Specifically, perform recognition processing based on the environmental feature data including temperature feature data and humidity feature data and the image feature data to obtain image quality information and occlusion category information. Thus, in this embodiment, the image quality information and the occlusion category information are obtained through recognition processing based on the temperature data, the humidity data, and the image data, which is particularly beneficial for accurately identifying the camera occlusion state in a cooking scenario with changing temperature and humidity. For example, it can improve the accuracy of the camera occlusion state in a humidity-saturated environment, and avoid misidentifying the camera occlusion state caused by poor image quality collected due to the specific temperature and humidity environment itself. For example, the obstruction of light propagation in the camera's field of view by water mist or oil fume results in poor image quality and leads to misjudgment that the camera is occluded.

[0075] In an alternative embodiment, as Figure 3 shown, the recognition processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information includes:

[0076] Step S21: Perform conversion processing based on the environmental state data and the image feature data to obtain feature conversion data.

[0077] Specifically, use the environmental state data and the image feature data as inputs for conversion processing to obtain feature conversion data.

[0078] Exemplarily, perform image feature extraction on the image data I to obtain image feature data F img , where F img = FE(I), and FE represents image feature extraction processing. For example, FE is a convolutional neural network; perform conversion processing based on the input value F s composed of the environmental state data F img and F f . Optionally, F f = [F s , F img . Exemplarily, the environmental state data F s includes temperature data T and humidity data H. Optionally, F s = [T, H].

[0079] Based on the input value F f , obtain feature conversion data X, where X = QT(F f ), and QT represents conversion processing.

[0080] Step S22, obtaining the image quality information and the occlusion category information according to the feature conversion data, where the occlusion category information includes at least one of wiping state information, cover state information, attachment state information, and pot lid lifting state information.

[0081] Specifically, the image quality information and the occlusion category information are obtained according to the feature conversion data X. Among them, the occlusion category information includes at least one of wiping state information, cover state information, attachment state information, and pot lid lifting state information. The wiping state information is used to indicate that the camera is in a wiping state or a non-wiping state; the cover state information is used to indicate that the camera is in a state of being covered by a cover or a non-covered state; the attachment state information is used to indicate that the camera is in a state of being attached by an attachment or a non-attached state. It should be noted that the camera being in a non-attached state means that the attachments on the camera have not reached the degree that can be recognized (for example, the volume and / or quantity of the attached objects are large), rather than there being no attachments on the camera at all. For example, a very small amount of dust on the camera will not cause the camera to be recognized as being in an attached state by an attachment, and the recognition result is that the camera is in a non-attached state. At this time, the camera has no wiping and cleaning requirements; the pot lid lifting state information is used to indicate that the pot lid in the camera's field of view is in a lifted state or a non-lifted state. Since the pot lid has a large volume, a lifted pot lid is likely to cause a large proportion of the camera's field of view to be blocked by the pot lid, affecting the normal image acquisition of the camera.

[0082] Optionally, the occlusion category information is represented numerically. Exemplarily, the output value corresponding to the wiping state information is 1 (the camera is in a wiping state) or 0 (the camera is in a non-wiping state), the output value corresponding to the cover state information is 1 (the camera is in a state of being covered by a cover) or 0 (the camera is in a non-covered state), the output value corresponding to the attachment state information is 1 (the camera is in a state of being attached by an attachment) or 0 (the camera is in a non-attached state), and the output value corresponding to the pot lid lifting state information is 1 (the pot lid in the camera's field of view is in a lifted state) or 0 (the pot lid in the camera's field of view is in a non-lifted state).

[0083] In an alternative embodiment, as Figure 4 shown, the obtaining of the image quality information and the occlusion category information according to the feature conversion data includes:

[0084] Step S31, obtaining target parameter information corresponding to a target category, where the target category is one of image quality, wiping state, cover state, attachment state, and pot lid lifting state, and the target parameter information includes a target weight parameter and a target bias parameter.

[0085] Step S32: Perform identification conversion based on the target parameter information and the feature conversion data to obtain target category information, where the target category information is one of image quality information, wiping state information, cover state information, attachment state information, and lid-lifting state information.

[0086] Specifically, each different target category corresponds to a set of target parameter information. Different target categories are identified and converted through each set of target parameter information, thereby obtaining the target category information.

[0087] Exemplarily, the target parameter information includes a target weight parameter W m and a target bias parameter b m , and the target category information is M, M = ST(X, W m , b m ), where ST represents identification conversion.

[0088] In a specific example, perform eigenvalue conversion processing on the feature conversion data X to obtain eigenvalue conversion data P(X). The eigenvalue conversion processing can be pooling processing or feature dimensionality reduction processing. The pooling processing can be global average pooling processing or max pooling processing. Perform conversion mapping based on the target parameter information and the eigenvalue conversion data P(X) to obtain the target category information. For example, the image quality information Q, the wiping state information P w , the cover state information P o , the attachment state information P d , and the lid-lifting state information P l are calculated respectively according to the following formulas:

[0089] Q = TR(W q ·P(X) + b q )

[0090] P w = TR(W w ·P(X) + b w )

[0091] P o = TR(W o ·P(X) + b o )

[0092] P d = TR(W d ·P(X) + b d )

[0093] P l = TR(W l ·P(X) + b l )

[0094] where, Wq , b q are respectively the target weight parameter and the target bias parameter corresponding to the image quality, W w , b w are respectively the target weight parameter and the target bias parameter corresponding to the wiping state, W o , b o are respectively the target weight parameter and the target bias parameter corresponding to the cover state, W d , b d are respectively the target weight parameter and the target bias parameter corresponding to the attachment state, W l , b l are respectively the target weight parameter and the target bias parameter corresponding to the state of lifting the pot lid, and TR is the conversion mapping relationship, which can be a Sigmoid function or a Tanh function.

[0095] In an optional implementation manner, the generating of the occlusion reminder information according to the occlusion category information includes:

[0096] When the occlusion category information indicates that the camera is in a target occlusion state, generating occlusion reminder information, where the target occlusion state is at least one of a real-time wiping state, a cover covering state, an attachment attachment state, and a pot lid lifting state.

[0097] Specifically, when the occlusion category information indicates that the camera is in at least one of a real-time wiping state, a cover covering state, an attachment attachment state, and a pot lid lifting state, that is, when the output value corresponding to at least one of the wiping state information, the cover state information, the attachment state information, and the pot lid lifting state information is 1, occlusion reminder information is generated. Optionally, the occlusion reminder information includes occlusion type information, and the occlusion type information represents the occlusion state of the camera. For example, the camera is in at least one of a state of being wiped, being covered by a cover, being attached by an attachment, and a state where the pot lid in the field of view is lifted. Thus, while reminding the user of the camera occlusion, the occlusion state of the camera can be provided to the user.

[0098] In an optional implementation manner, the reminder method further includes:

[0099] When the image quality information indicates that the image quality of the image data is within a preset quality range, if the occlusion category information meets a preset occlusion condition, no occlusion reminder information is generated.

[0100] Specifically, if the occlusion category information meets the preset occlusion condition, the camera is not in the target occlusion state, there is no need for an occlusion reminder, and no occlusion reminder information is generated. For example, when the image quality information indicates that the image quality of the image data is within the preset quality range, the image data is in an abnormal quality state, and the occlusion category information meets the preset occlusion condition, and the camera is not in the target occlusion state, then the abnormality of the image data is not caused by the target occlusion state, and no occlusion reminder information is generated.

[0101] In an alternative embodiment, the performing identification processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information includes:

[0102] Using a state detection model to perform identification processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information.

[0103] Specifically, the state detection model can be a neural network model or a machine learning model.

[0104] The method further includes:

[0105] Training an initial detection model according to sample data to obtain the state detection model, where the loss parameter of the training is obtained from the cross-entropy loss obtained from the mean squared error losses corresponding to the occlusion category information and the image quality information respectively.

[0106] Specifically, the sample data includes image sample data and annotation data corresponding to the image sample data. The image sample data includes a plurality of sample images, which are collected and generated in a target scene. The target scene preferably includes a plurality of scenes, such as scenes corresponding to various lighting conditions, various ingredient situations, and various environmental states. The annotation data includes environmental state annotation data, image quality annotation information, and occlusion category annotation information. The environmental state annotation data is used to indicate the environmental state corresponding to the sample image, such as the temperature and humidity when the sample image is collected. The image quality annotation information is used to indicate the image quality corresponding to the sample image. The occlusion category annotation information is used to indicate the occlusion state corresponding to the sample image.

[0107] In model training, the loss parameter used is obtained from the cross-entropy loss obtained from the mean squared error losses corresponding to the occlusion category information and the image quality information respectively. The loss parameter Loss is obtained according to the following formula:

[0108] Loss = λ q ·MSELoss(Q, Q t ) + λ w ·BCELoss(P w, P wt ) + λ o ·BCELoss(P o , P ot )

[0109] + λ d ·BCELoss(P d , P dt ) + λ l ·BCELoss(P l , P lt )

[0110] Among them, λ q , λ w , λ o , λ d , λ l are the weight coefficients of the losses corresponding to Q, P w , P o , P d and P l respectively, and Q t , P wt , P ot , P dt , P lt are the target values corresponding to Q, P w , P o , P d and P l respectively, that is, the image quality annotation information and occlusion category annotation information corresponding to the sample image.

[0111] In an alternative embodiment, the reminder method further includes:

[0112] Generating an occlusion risk reminder message when the temperature data and the humidity data meet the preset environmental state conditions.

[0113] Specifically, if the temperature data and the humidity data meet the preset environmental state conditions, there is a risk of occluding the camera in the temperature and humidity environment. For example, in a high-humidity environment, there are water droplets on the camera, or in a frosting environment, there is frost on the camera. Exemplarily, obtaining the humidity threshold corresponding to the temperature data, generating an occlusion risk reminder message when the humidity corresponding to the humidity data is greater than the humidity threshold, or generating an occlusion risk reminder message when the temperature corresponding to the temperature data is within the target temperature range and the humidity corresponding to the humidity data is within the target humidity range.

[0114] In an alternative embodiment, an occlusion risk reminder message is generated when the image quality information indicates that the image quality of the image data is within the preset quality range, the temperature corresponding to the temperature data is within the target temperature range, and the humidity corresponding to the humidity data is within the target humidity range.

[0115] Figure 5 A block diagram showing an occlusion reminder device for a camera according to an embodiment of the present disclosure; as Figure 5 shown, the above device includes:

[0116] A data acquisition module, configured to acquire image data and environmental status data, where the image data is acquired by the camera, and the environmental status data characterizes the environmental status corresponding to the image data;

[0117] An extraction processing module, configured to perform feature extraction processing on the image data to obtain image feature data;

[0118] An identification processing module, configured to perform identification processing based on the environmental status data and the image feature data to obtain image quality information and occlusion category information;

[0119] An occlusion reminder module, configured to generate an occlusion reminder information according to the occlusion category information in the case that the image quality information indicates that the image quality of the image data is within a preset quality range.

[0120] In some embodiments, the functions or modules / units included in the cooking control device provided by the embodiments of the present disclosure can be used to execute the cooking control method described in the above embodiments. Its specific implementation can refer to the description of the above embodiments. For the sake of brevity, it will not be repeated here.

[0121] The embodiments of the present disclosure also propose an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein, the above processor realizes the above occlusion reminder method for a camera by executing the instructions stored in the memory.

[0122] The electronic device can be provided as a terminal, a server or other forms of devices. The server can be a cloud server, and the cloud server is communicatively connected to an image acquisition device and an environmental status sensor, or communicatively connected to a control unit of a cooking device to receive image data and environmental status data through the control unit. The image acquisition device and the environmental status sensor, or the control unit of the cooking device send the image data and environmental status data acquired in real time to the cloud server. The cloud server performs data processing and identification processing to obtain image quality information, occlusion category information and other analysis results (such as occlusion reminder information or occlusion risk reminder information), and sends the analysis results to the local control unit, so that the local control unit performs information display and reminder according to the analysis results.

[0123] An embodiment of the present disclosure also provides a range hood, which includes an oil extraction component, an image acquisition component, a temperature and humidity sensing component, and at least one processor, as well as a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the at least one processor is used to communicatively connect with the image acquisition component and the temperature and humidity sensing component, and the at least one processor realizes the occlusion reminder method as described in the above technical solution by executing the instructions stored in the memory.

[0124] An embodiment of the present disclosure also provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the occlusion reminder method as described in the above technical solution.

[0125] The embodiments of the present disclosure have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to technologies in the market, or to enable other ordinary skill in the art in the technical field to understand the embodiments disclosed herein.

Claims

1. A camera occlusion reminder method, characterized in that: include: Acquire image data and environmental status data, wherein the image data is acquired by the camera, and the environmental status data represents the environmental status corresponding to the image data; Performing feature extraction processing on the image data to obtain image feature data; Performing recognition processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information; When the image quality information indicates that the image quality of the image data is within a preset quality range, occlusion reminder information is generated according to the occlusion category information.

2. The reminder method according to claim 1, characterized in that: The environmental state data includes temperature data and humidity data, and the identification processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information includes: Extracting and processing the temperature data and the humidity data respectively to obtain environmental characteristic data, wherein the environmental characteristic data includes temperature characteristic data and humidity characteristic data; Identification and conversion processing is performed according to the environmental feature data and the image feature data to obtain image quality information and occlusion category information.

3. The reminder method according to claim 1 or 2, characterized in that: The performing identification processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information includes: Performing conversion processing based on the environmental state data and the image feature data to obtain feature conversion data; The image quality information and the occlusion category information are obtained according to the feature conversion data, and the occlusion category information includes at least one of wiping state information, covering state information, attachment state information and pot lid lifting state information.

4. The reminder method according to claim 3, characterized in that: The obtaining the image quality information and the occlusion category information according to the feature conversion data includes: Obtaining target parameter information corresponding to a target category, wherein the target category is one of image quality, wiping state, covering state, attachment state, and pot lid lifting state, and the target parameter information includes a target weight parameter and a target bias parameter; According to the target parameter information and the feature conversion data, identification conversion is performed to obtain target category information, where the target category information is one of image quality information, wiping status information, covering status information, attachment status information and pot lid lifting status information.

5. The reminder method according to claim 1, characterized in that: The generating of the occlusion reminder information according to the occlusion category information includes: When the occlusion category information indicates that the camera is in a target occlusion state, occlusion reminder information is generated, and the target occlusion state is at least one of a real-time wiping state, a covering state, an attachment state, and a pot lid lifted state.

6. The reminder method according to claim 1, characterized in that: Also includes: In a case where the image quality information indicates that the image quality of the image data is within a preset quality range, if the occlusion category information satisfies a preset occlusion condition, no occlusion reminder information is generated.

7. The reminder method according to claim 1, characterized in that: The performing identification processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information includes: Using the state detection model to perform recognition processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information; The method further comprises: The initial detection model is trained according to the sample data to obtain the state detection model, and the loss parameter of the training is obtained by the cross entropy loss obtained according to the mean square error loss corresponding to the occlusion category information and the image quality information respectively.

8. The reminder method according to claim 2, characterized in that: Also includes: When the temperature data and the humidity data meet the preset environmental state conditions, occlusion risk reminder information is generated.

9. A camera obstruction reminder device, characterized in that: include: A data acquisition module, used to acquire image data and environmental status data, wherein the image data is acquired by the camera, and the environmental status data represents the environmental status corresponding to the image data; An extraction processing module, used for performing feature extraction processing on the image data to obtain image feature data; An identification processing module, used for performing identification processing based on the environmental state data and the image feature data to obtain image quality information and occlusion category information; The occlusion reminder module is used to generate occlusion reminder information according to the occlusion category information when the image quality information indicates that the image quality of the image data is within a preset quality range.

10. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instruction to implement the camera occlusion reminder method as described in any one of claims 1 to 8.

Images