Kitchen environment monitoring method and apparatus, storage medium and electronic device

Abstract

Disclosed in the present application are a kitchen environment monitoring method and apparatus, a storage medium and an electronic device. The method comprises: on the basis of a first set of kitchen environment images collected by a collection device provided in a kitchen, determining a static reference base image corresponding to the collection device; on the basis of the static reference base image, extracting an image of a target area from an acquired second set of kitchen environment images; performing type prediction on the image of the target area used as a candidate recognition image, so as to determine a predicted type of an object in the target area; and inputting the image of the target area as an image under test into a type detection model corresponding to the predicted type for detection, so as to determine the safety state of the object in a kitchen environment. Thus, the present application avoids computing resource consumption and can provide recognition accuracy.

Description

Kitchen environment detection method and device, storage medium and electronic equipment Technical Field

[0001] The present application relates to the field of machine vision, and more particularly to a method and device for detecting a kitchen environment. The present application also relates to a computer storage medium and an electronic device. Background Art

[0002] A bright kitchen and stove is a commonly used kitchen safety and hygiene symbol. Originally, it meant keeping the kitchen light box illuminated during cooking, ensuring the chef's work area is clearly visible. This symbol emphasizes the importance of food safety and hygiene, allowing customers to see the entire cooking process, increasing transparency and reliability. With a bright kitchen and stove, customers can confidently observe the chef's actions and ensure that food quality and hygiene standards are met.

[0003] An open kitchen is more than just a symbol; it's a concept that encompasses the entire restaurant industry and food safety. Its implementation not only enhances public awareness and trust in food safety and hygiene, but also promotes management and enhances service quality for restaurant operators. In today's restaurant market, an increasing number of restaurants and à la carte services are adopting open kitchens to enhance customer dining experience and satisfaction.

[0004] With the continuous development of internet technology, open kitchens have expanded from allowing customers to observe the internal kitchen environment and operating processes for safety supervision to identifying safety hazards in real time through the recognition of image information collected by equipment installed in the kitchen (i.e., computer vision). This helps businesses improve safety management and hygiene standards, reduce food safety risks, and ensure food quality and safety. In short, open kitchens have played a significant role in raising food safety awareness and service quality.

[0005] Summary of the Invention

[0006] The present application provides a kitchen environment detection method to solve the problem of excessive consumption of computing resources during the detection process in the prior art.

[0007] This application provides a kitchen environment detection method, comprising:

[0008] Determining a static reference base image corresponding to a first kitchen environment image set acquired by an acquisition device deployed in the kitchen;

[0009] Extracting an image of a target area from the acquired second kitchen environment image set based on the static reference base image; performing type prediction on the image of the target area as a candidate recognition image to determine a predicted type of an object in the target area;

[0010] The image of the target area is input as an image to be detected into a type detection model corresponding to the predicted type for detection, so as to determine the safety status of the object in the back kitchen environment.

[0011] In some embodiments, determining a static reference base image corresponding to a first set of back kitchen environment images captured by a capture device arranged in the back kitchen environment includes:

[0012] Performing object recognition on the images in the first kitchen environment image set to obtain a recognition image;

[0013] determining whether there is a difference image in the recognition images according to the comparison between the recognition images;

[0014] If not, an image randomly selected from the first kitchen environment image set is determined as the static reference base image.

[0015] In some embodiments, performing object recognition on images in the first kitchen environment image set to obtain a recognized image includes:

[0016] Performing object edge detection on the images in the first back kitchen environment image set to determine object segmentation areas in the images;

[0017] The object recognition is performed on the segmented area to obtain a recognition image corresponding to the segmented area.

[0018] In some embodiments, further comprising:

[0019] When the determination of whether there is a difference image in the identified image is yes, determining whether the difference is within a difference threshold range determined according to the acquisition device specification and / or acquisition angle;

[0020] If so, the randomly selected image in the first kitchen environment image is determined as the static reference base image.

[0021] In some embodiments, further comprising:

[0022] When the determination of whether the difference is within the difference threshold range determined according to the acquisition device specification and / or acquisition angle is negative, the image with the least number of objects in the recognition image is determined as the static reference base image corresponding to the acquisition device.

[0023] In some embodiments, determining a static reference base image corresponding to a first kitchen environment image set acquired by an acquisition device deployed in the kitchen includes:

[0024] Randomly select an image from the first kitchen environment image set as a candidate static reference base image;

[0025] performing object recognition on the images in the first kitchen environment image set;

[0026] updating the candidate static reference base image according to an image in which the number of objects recognized in the first kitchen environment image set is less than the number of objects in the candidate static reference base image;

[0027] Determine whether the candidate static reference base image has the least number of objects in the first kitchen environment image set; if so, determine the updated candidate static reference base image as the static reference base image.

[0028] In some embodiments, the step of using the image of the target area as a candidate recognition image, performing type prediction on an object corresponding to the target area, and determining a predicted type of the object in the target area includes:

[0029] determining whether an area ratio between the target region and the static reference base image is greater than or equal to a preset threshold range;

[0030] If yes, the image of the target area is used as a candidate recognition image and input into the type prediction model for type prediction;

[0031] The object prediction type of the target area is determined according to the prediction result.

[0032] In some embodiments, performing type prediction on the image of the target area as a candidate recognition image and determining the predicted type of the object corresponding to the target area includes:

[0033] Extracting feature data of an object in the candidate recognition image according to a feature extraction method corresponding to the acquisition time of the candidate recognition image;

[0034] A type prediction is performed based on the feature data to obtain a predicted type of the object corresponding to the target area.

[0035] In some embodiments, extracting feature data of an object in the candidate recognition image according to a feature extraction method corresponding to the acquisition time of the candidate recognition image includes:

[0036] When the acquisition time is during the daytime, extracting local feature data of the object in the candidate recognition image according to a local feature extraction method;

[0037] When the acquisition time is during the night time period, the reflective feature data of the object in the candidate recognition image is extracted in accordance with the light reflection feature extraction method.

[0038] In some embodiments, determining a static reference base image corresponding to a first kitchen environment image set acquired by an acquisition device deployed in the kitchen includes:

[0039] Acquire a first back kitchen environment image set from the acquisition device according to the configured acquisition dimension of the back kitchen environment image set; wherein the acquisition dimension includes at least one of a time dimension, a food type dimension, and a back kitchen location dimension;

[0040] A static reference base image corresponding to the acquisition device is determined according to the first kitchen environment image set.

[0041] The present application also provides a kitchen environment detection device, comprising:

[0042] A first determining unit is configured to determine a static reference base image corresponding to a first kitchen environment image set acquired by an acquisition device deployed in the kitchen;

[0043] an extraction unit, configured to extract an image of a target area from the acquired second back kitchen environment image set according to the static reference base image;

[0044] a second determining unit, configured to perform type prediction on the image of the target area as a candidate recognition image, and determine a predicted type of an object corresponding to the target area;

[0045] The third determining unit is configured to input the image of the target area as an image to be detected into a type detection model corresponding to the predicted type of the object for detection, so as to determine the safety status of the object in the back kitchen environment.

[0046] The present application also provides a computer storage medium for storing a computer program;

[0047] The program executes the above-mentioned kitchen environment detection method.

[0048] The present application also provides an electronic device, comprising:

[0049] processor;

[0050] The memory is used to store a computer program, wherein the program executes the above-mentioned back kitchen environment detection method.

[0051] Compared with the prior art, this application has the following advantages.

[0052] The present application provides a back kitchen environment detection method that first screens images from a first back kitchen environment image set to determine a static reference base image. The static reference base image is then compared with images from a second back kitchen environment image set to extract images containing a target area, thereby reducing the consumption of computing resources for subsequent prediction of the type of objects in the target area. This is because, on the one hand, there is no need to perform global recognition on the second back kitchen environment image; it is only necessary to compare it with the static reference base image to determine whether the target area exists. On the other hand, type prediction is only for the extracted target area portion rather than the entire image, thus eliminating the need to consume a large amount of computing resources. Based on the type prediction results for the object corresponding to the target area, type detection is performed again, thereby ensuring not only the accuracy of the target area object type detection but also the accuracy of the safety status identification in the back kitchen environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] FIG1 is a flow chart of an embodiment of a kitchen environment detection method provided by the present application;

[0054] FIG2 is a schematic diagram of object edge segmentation in an embodiment of a kitchen environment detection method provided by the present application;

[0055] FIG3 is a schematic structural diagram of an embodiment of a kitchen environment detection device provided by the present application;

[0056] FIG4 is a schematic structural diagram of an electronic device embodiment provided by the present application. DETAILED DESCRIPTION

[0057] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0058] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. Descriptive terms such as "a," "a," "a first," and "a second," etc., used in this application and the appended claims, are not intended to limit quantity or sequence, but are used to distinguish information of the same type from one another.

[0059] According to the above background technology, it can be known that the invention concept of this application originates from the application scenario of open kitchen and bright stove. Specifically, in the application scenario of open kitchen and bright stove, the existing technology is to deploy cameras in the back kitchen of catering businesses to detect the operating specifications of the back kitchen staff, the back kitchen environment, etc. However, in the existing technology, on the one hand, because the cameras are provided by different manufacturers, abnormal situations are identified locally by the camera, and this method is limited by the performance and hardware conditions of the camera itself, and different camera specifications have different image ratios. In the case of a large amount of collected data, it will not be possible to achieve more refined recognition due to performance reasons; on the other hand, because the hardware and manufacturers of the cameras involved are different, the local recognition capabilities are also different, resulting in inconsistent recognition standards, and then there is a problem of poor recognition consistency. Therefore, this application provides a back kitchen environment detection method to avoid the problems of low detection accuracy and large amount of calculation caused by problems such as camera performance and hardware. A back kitchen environment detection method provided by this application is described below.

[0060] As shown in FIG1 , FIG1 is a flowchart of an embodiment of a back kitchen environment detection method provided by the present application, and the method includes steps S101 to S104 .

[0061] Step S101: determining a static reference base image corresponding to a first kitchen environment image set acquired by an acquisition device deployed in the kitchen.

[0062] Step S102: extracting an image of a target area from the acquired second kitchen environment image set according to the static reference base image.

[0063] Step S103: performing type prediction on the image of the target area as a candidate recognition image to determine the predicted type of the object corresponding to the target area.

[0064] Step S104: inputting the image of the target area as the image to be detected into a type detection model corresponding to the predicted type of the object for detection, and determining the safety status of the object in the back kitchen environment.

[0065] The above steps S101 to S104 are described in detail below with reference to specific examples.

[0066] Regarding the step S101: according to a first kitchen environment image set acquired by an acquisition device arranged in the kitchen, a static reference base image corresponding to the acquisition device is determined.

[0067] The acquisition device may be a camera or other device for capturing image or video information. The first kitchen environment image set may include multiple continuous images, such as sequential frame images in video data, or multiple non-continuous images, such as images sent separately according to a set acquisition time. The multiple continuous and non-continuous images can also be understood as having different time intervals. For example, continuous images may have a short time interval between images, such as one frame per second, while non-continuous images may have a longer time interval between images, such as one image every five seconds. The images in the first kitchen environment image set may be video stream data captured by a camera, and the video stream data may include frame images at different time intervals. Of course, it can also be images sent by the acquisition device according to a set transmission time. In this embodiment, the static reference base image can be understood as a reference base image determined in the first kitchen environment image set.

[0068] The purpose of step S101 is to determine a static reference base image from the first kitchen environment image set. Because different acquisition devices have different signal specifications and deployment angles, when multiple acquisition devices are deployed in the kitchen environment, corresponding static reference base images need to be established for each acquisition device.

[0069] In this embodiment, when the collection device collects the kitchen environment information, it can collect according to different collection dimensions, which can specifically include steps S101-1 to S101-2.

[0070] Step S101-1: acquiring a first set of back kitchen environment images from the acquisition device according to the configured acquisition dimensions of the back kitchen environment image set; wherein the acquisition dimensions include at least one of a time dimension, a food type dimension, and a back kitchen location dimension.

[0071] Step S101 - 2 : Determine a static reference base image corresponding to the acquisition device based on the first kitchen environment image set.

[0072] The specific implementation of step S101-1 may include setting the acquisition frequency based on periods of high food safety risk. For example, in summer, which is a period of high food safety risk, the acquisition frequency may be increased to increase the frequency of back-kitchen environment testing during this period. The acquisition frequency may also be set based on industries with high food safety risk, i.e., types of food safety risk. For example, the acquisition frequency may be increased for snack foods such as spicy hot pot and barbecue to increase the frequency of back-kitchen environment testing for these types of catering businesses. The acquisition frequency may also be set based on the location of the business's back kitchen. For example, the acquisition frequency may be increased if the business's back kitchen is located on the street, or if the business's back kitchen is located on the second floor or below in a shopping mall. Of course, the acquisition frequency may also be set based on the number of back-kitchen workers. The specific acquisition dimensions are not limited to those shown in the above examples and can be set in conjunction with specific food safety scenarios.

[0073] In this embodiment, it should be noted that the acquisition devices deployed within the kitchen environment can include devices of various specifications, and can come from different providers or the same provider. Regardless of whether the providers are different or the same, the image sets collected from the kitchen environment may be in various formats, necessitating adaptation processing to convert continuous or discontinuous images obtained from the acquisition devices into images that can be recognized by a large CV model or a multimodal large model. A large CV model refers to a large deep learning model used for computer vision tasks, typically implemented using deep learning algorithms such as convolutional neural networks (CNNs). In recent years, with the development of deep learning technology and the improvement of computing power, large CV models have achieved many important results in the field of computer vision, such as image classification, object detection, and image segmentation. The basic idea of ​​a large CV model is to convert an input image into an output result by learning a mapping from the input image to the output result, for example, identifying information such as the category or location of an object in an image.

[0074] It is known from existing technologies that the acquisition equipment is limited by its own performance, resulting in certain limitations in image processing, which in turn makes the recognition results inaccurate. In this embodiment, in order to reduce the consumption of computing resources during anomaly detection, the processing efficiency is improved from the service side (or processing side) of image detection. Specifically, a matching static reference base map image is established for each acquisition equipment deployed in the back kitchen environment, thereby reducing the amount of calculation during the subsequent back kitchen safety risk detection process. In other words, the early preparation is taken as the focus to provide a reliable foundation for subsequent detection, while improving the detection accuracy and reducing the consumption of computing resources. The specific implementation process can include multiple implementation methods, which are described in turn below.

[0075] Method 1 includes steps S101 - 11 to S101 - 13 .

[0076] Step S101 - 11 : performing object recognition on the images in the first back kitchen environment image set to obtain a recognition image.

[0077] Step S101 - 12 : Determine whether there is a difference image in the recognition images based on the comparison between the recognition images.

[0078] Step S101 - 13 : If not, an image randomly selected from the first kitchen environment image set is determined as the static reference base image.

[0079] There are at least two situations for step S101-11 in this embodiment.

[0080] The first case is that when the first kitchen environment image set is a continuous sequence frame image in a video stream obtained by an acquisition device, object recognition can be performed based on the context between the images to obtain a recognition image.

[0081] The second case is when the first back kitchen environment image set is a non-continuous image acquired by an acquisition device, the edges of objects in the image can be determined by edge detection (as shown in FIG. 2 ), which can specifically include steps S101 - 111 to S101 - 112 .

[0082] Step S101 - 111 : performing object edge detection on the images in the first back kitchen environment image set to determine object segmentation areas in the images.

[0083] Steps S101-112: performing object recognition on the segmented area to obtain a recognition image corresponding to the segmented area. The object recognition may be performed to determine the outline of the object in the segmented area, and the outline and the object within the outline are used as the recognition image.

[0084] It should be noted that object recognition can be achieved through steps S101-111 and S101-112 for both continuous and non-continuous images. In order to improve detection efficiency, continuous images can be recognized in a context-based manner.

[0085] The purpose of step S101 - 12 is to determine whether there are differences or changes between the recognized images, which can be done by comparing contours and / or pixel values, without limitation.

[0086] Method 1 performs object recognition on images in the first back-kitchen environment image set, then compares the recognized images to determine whether there are any differences between the images. If there are no differences, it indicates that no objects have changed or no living creatures have appeared in the first back-kitchen environment image set. Therefore, a randomly selected image from the first back-kitchen environment image set can be used as a static reference base image. Conversely, if there are differences, it indicates that objects have changed or living creatures have appeared in the images. In this case, further analysis is required, as differences do not necessarily indicate the presence of living creatures. In back-kitchen environments, object changes occur more frequently during food processing. For example, if the images are of the operating area, there is a high probability of image differences. Therefore, the likelihood of changes is related to the specifications and / or installation angle of the acquisition equipment. For example, wide-angle lenses and fisheye lenses select different proportions of the image range for the same area.

[0087] In order to improve the accuracy of the judgment, it may also include: when the determination of whether there is a difference image with a difference in the identified image is yes, determining whether the difference is within a difference threshold range determined according to the acquisition device specifications and / or acquisition angle; wherein the difference threshold range can be set according to the acquisition device specifications and / or acquisition angle, and of course can also be determined based on historical experience values, for example: it can be between 5% and 30%.

[0088] If so, the randomly selected image in the first kitchen environment image is determined as the static reference base image.

[0089] If not, the image with the least number of objects in the recognition image is determined as the static reference base image corresponding to the acquisition device.

[0090] Method 2 includes steps S101 - 21 to S101 - 24 .

[0091] Step S101 - 21 : Randomly select an image from the first kitchen environment image set as a candidate static reference base image.

[0092] Step S101 - 22 : performing object recognition on the images in the first kitchen environment image set.

[0093] Step S101 - 23 : updating the candidate static reference base image according to the image in which the number of objects recognized in the first kitchen environment image set is less than the number of objects in the candidate static reference base image.

[0094] Step S101-24: Determine whether the candidate static reference base image has the least number of objects in the first kitchen environment image set. If so, determine the updated candidate static reference base image as the static reference base image.

[0095] In the second method, the object identification can also be performed in the same manner as in the first method.

[0096] In both Method 1 and Method 2, the static reference base image can be the image with the fewest objects. When comparing recognition images, iterative updates can be performed based on the number of objects in the image. After comparing all images in the first set of kitchen environment images, a static reference base image is determined. Method 1 aims to determine whether there are differences between the recognition images by comparing them. If there are no differences, the static reference base image is determined. Method 2 primarily compares randomly selected candidate static reference base images, determines the image with the fewest objects, and then updates the candidate static reference base image.

[0097] The above describes the determination of a static reference base image. It is understood that the above determination process can be applied to different sets of first kitchen environment images, for example, where the images in the first kitchen environment image set are continuous or discontinuous. The following method can also be used to determine a static reference base image within a continuous image:

[0098] Extracting the first frame image from the sequence of frame images as a reference image;

[0099] According to the order of the sequence frame images, the second frame image is compared with the first frame image to extract the difference portion;

[0100] Object recognition is performed on the difference. If the target object is present, the portion outside the target object region is used as a static reference base image. This process is repeated until all consecutive images have been recognized. The region outside the region where the target object was identified can be saved as a static reference base image. It is understood that the static reference base image for consecutive images can be determined by stitching together portions of images selected at different times, which is faster than identifying contours and regions for non-consecutive images.

[0101] After the static reference base image is determined in step S101, the image containing the target area in the second back kitchen environment image set can be extracted based on the acquired second back kitchen environment image set and the static reference base image, that is, step S102 is executed.

[0102] Regarding step S102: extracting an image of a target area from the acquired second kitchen environment image set according to the static reference base image.

[0103] The specific implementation process of step S102 may be to compare the images in the second back kitchen environment image set with the static reference base image, and extract the images containing the target areas in the images, thereby eliminating the need to identify the objects in the images in the second back kitchen environment image set one by one, and only requiring comparison with the static reference base image, thereby reducing the overhead of computing resources.

[0104] It should be noted that the purpose of this embodiment is to detect safety issues in the kitchen environment to improve food safety. Therefore, target area extraction can be determined in conjunction with relevant objects related to safety issues. For example, if a static reference base image contains no people, when compared with images in the second kitchen environment image set, if there are people, then areas related to people are extracted. For another example, to extract still objects related to safety issues, if a trash can is present in both the static reference base image and the second kitchen environment image set, then the trash can is extracted as a target area to facilitate subsequent detection of the trash can's safety status, such as whether the lid is closed or if trash is overflowing. Of course, if the target image to be extracted is determined based on changes, the static reference base image can be a base image in a safe state. In this embodiment, the static reference base image includes the fewest objects in the first kitchen environment image set, and can include images of still objects in a safe state, such as a closed trash can. Of course, if the trash can is open, this does not affect subsequent target area extraction, type prediction, and safety state identification.

[0105] Regarding step S103: the image of the target area is used as a candidate recognition image to perform type prediction to determine the predicted type of the object in the target area.

[0106] In order to improve the accuracy of the detection result, the target area may be further determined, which may specifically include steps S103 - 11 to S103 - 13 .

[0107] Step S103-11: Determine whether the area ratio of the target area in the static reference base image is greater than or equal to a preset threshold range. For example, if the target area is a reflective point, its area ratio in the static reference base image will differ from the area ratio of the reflective point of a living object's eye in the static reference base image. If the target area is the outline of a mouse, its area ratio in the static reference base image will inevitably be smaller than the area ratio of the target area of ​​a human body in the static reference base image. In other words, to avoid misidentifying light points from other devices as living objects' eyes and thus consuming unnecessary computing resources for identification and other processing, a further threshold range can be used to determine whether type prediction is necessary. The preset threshold range can be a universal, preset empirical value or based on data used by the current camera during the static reference base image determination process, such as data used in iterative optimization of the static reference base image. For example, if a 5% to 7% change area is likely to be a mouse, but the camera is a fisheye lens, there will be some magnification and distortion. Therefore, the preset threshold range can be adjusted, for example to 3% to 6%. Therefore, the preset threshold range can be adjusted in real time according to factors such as the camera's specifications, performance, and installation angle.

[0108] Step S103 - 12 : If yes, the image of the target area is used as a candidate recognition image and input into the type prediction model for type prediction.

[0109] Step S103 - 13 : Determine the predicted type of the object in the target area according to the prediction result.

[0110] The specific identification process of step S103 may include steps S103-21 to S103-22.

[0111] Step S103 - 21 : extracting feature data of the object in the candidate recognition image according to the feature extraction method corresponding to the acquisition time of the candidate recognition image.

[0112] Step S103-22: performing type prediction based on the feature data to obtain the predicted type of the object corresponding to the target area.

[0113] The collection time in step S103-21 can be during the daytime or nighttime periods. Generally, at night, there's a higher probability of animals like cats and mice appearing in the kitchen, so feature extraction can be performed using light reflection. Daytime is typically working time, and the primary focus is on inspecting kitchen staff's behavior and equipment like trash cans. Therefore, local feature extraction can be employed. Of course, whether during the daytime or nighttime periods, the feature extraction methods are not limited to these two, and local feature extraction can also be applied to nighttime feature extraction.

[0114] Therefore, the specific implementation process of step S103-21 may include steps S103-21-1 to S103-21-2.

[0115] Step S103-21-1: When the acquisition time is during the daytime, extract the local feature data of the object in the candidate recognition image according to the local feature extraction method.

[0116] Step S103-21-2: When the acquisition time is during the night time period, the reflective feature data of the object in the candidate recognition image is extracted according to the light reflection feature extraction method.

[0117] Regarding step S104: the image of the target area is input as the image to be detected into the type detection model corresponding to the predicted type for detection, and the safety status of the object in the back kitchen environment is determined. Specifically, when the predicted type of the object in the target area is back kitchen staff, the image of the target area is input as the image to be detected into the first type detection model (detection model of human type) for detection to determine whether the back kitchen staff's attire complies with regulations. If not, a safety status abnormality message is sent to the merchant; and / or, it is determined whether the back kitchen staff has violated regulations. If so, a safety status abnormality message is sent to the merchant. When the object in the target area is a still object, such as a trash can, the image of the target area is input as the image to be detected into the second type detection model (detection model of still object type) for detection to determine whether the still object is in a safe state, for example, whether the trash can is covered, whether the gas stove is turned off, and whether the cutting board is in use. When the object in the target area is a living thing, such as a mouse, cat, dog, cockroach, etc., the image of the target area is determined as the image to be detected and input into the third type of detection model (the detection model of the living thing type) for detection to determine whether the living thing is in a safe state.

[0118] Of course, the examples above, such as still objects, living things, and people, are only examples of common scenarios. Specifically for food safety, kitchen inspections can also include objects that cause safety hazards in the kitchen, such as objects unrelated to the kitchen (such as lighters and power banks that can easily cause fires), the status of the kitchen gas stove, the status of the pots, etc. Anything related to kitchen safety and food safety can be considered for inspection.

[0119] The above is a description of an embodiment of a back kitchen environment detection method provided by the present application. This method embodiment first screens the images in the acquired first back kitchen environment image set to determine a static reference base image, and then compares the static reference base image with the images in the acquired second back kitchen environment image set to extract the image containing the target area, thereby reducing the consumption of computing resources for subsequent target area object type prediction. This is because, on the one hand, there is no need to perform global recognition of the second back kitchen environment image; it is only necessary to compare it with the static reference base image to determine whether the target area exists; on the other hand, the type prediction is only for the extracted target area portion rather than the entire image, so there is no need to consume a large amount of computing resources. Based on the type prediction result of the object object corresponding to the target area, type detection is performed again, thereby not only ensuring the accuracy of the target area object type detection, but also ensuring the accuracy of the safety status identification in the back kitchen environment.

[0120] The above is a detailed description of an embodiment of a back-kitchen environment detection method provided by this application. Corresponding to the aforementioned embodiment of the back-kitchen environment detection method, this application also discloses an embodiment of a back-kitchen environment detection device. See FIG3 . Because the device embodiment is substantially similar to the method embodiment, the description is relatively brief. For relevant details, refer to the description of the method embodiment. The device embodiment described below is merely illustrative.

[0121] As shown in FIG3 , FIG3 is a structural diagram of an embodiment of a back kitchen environment detection device provided by the present application. The embodiment of the device may include a first determination unit 301 to a third determination unit 304 .

[0122] The first determining unit 301 is configured to determine a static reference base image corresponding to a collection device arranged in the kitchen based on a first kitchen environment image set collected by the collection device.

[0123] The extraction unit 302 is configured to extract an image of a target area from the acquired second kitchen environment image set according to the static reference base image.

[0124] The second determining unit 303 is configured to perform type prediction on the image of the target area as a candidate recognition image, and determine a predicted type of the object corresponding to the target area.

[0125] The third determining unit 304 is configured to input the image of the target area as an image to be detected into a type detection model corresponding to the predicted type of the object for detection, so as to determine the safety status of the object in the back kitchen environment.

[0126] The first determining unit 301 in mode 1 may include: an acquiring subunit, a first determining subunit, and a second determining subunit.

[0127] The acquisition subunit is configured to perform object recognition on the images in the first back kitchen environment image set to acquire a recognition image.

[0128] The first determining subunit is configured to determine whether there is a difference image in the recognition images based on a comparison between the recognition images.

[0129] The second determining subunit is configured to determine, when the determination result of the first determining subunit is negative, an image randomly selected from the first kitchen environment image set as the static reference base image.

[0130] The acquisition subunit may include: a region determination subunit and an image acquisition subunit.

[0131] The region determination subunit is configured to perform object edge detection on the images in the first back kitchen environment image set to determine object segmentation regions in the images.

[0132] The image acquisition subunit is configured to perform the object recognition on the segmented area and acquire a recognition image corresponding to the segmented area.

[0133] It further includes: a difference determination subunit and a selection subunit.

[0134] The difference determination subunit is configured to, when the determination of whether there is a difference image with a difference in the recognition image is yes, determine whether the difference is within a difference threshold range determined according to the acquisition device specification and / or acquisition angle.

[0135] The selection subunit is configured to, when the determination result of the difference determination subunit is yes, determine an image randomly selected from the first back kitchen environment image as the static reference base image; and when the determination result of the difference determination subunit is no, determine the image with the least number of objects in the recognition image as the static reference base image corresponding to the acquisition device.

[0136] The second mode of the first determining unit 301 may include: selecting a subunit, identifying a subunit, updating a subunit, and determining a subunit.

[0137] The selection subunit is configured to randomly select an image from the first kitchen environment image set as a candidate static reference base image.

[0138] The recognition subunit is used to perform object recognition on the images in the first back kitchen environment image set.

[0139] The updating subunit is configured to update the candidate static reference base image according to an image in which the number of objects recognized in the first kitchen environment image set is less than the number of objects in the candidate static reference base image.

[0140] The determination subunit is used to determine whether the number of object objects in the candidate static reference base image is the image with the least object objects in the first back kitchen environment image set. If so, the updated candidate static reference base image is determined as the static reference base image.

[0141] The first determining unit 301 may include: an acquiring subunit and a determining subunit.

[0142] The acquisition subunit is used to acquire the first kitchen environment image set of the acquisition device according to the configured acquisition dimension of the kitchen environment image set; wherein the acquisition dimension includes: at least one of: time dimension, food type dimension, and kitchen location dimension.

[0143] The determining subunit is configured to determine a static reference base image corresponding to the acquisition device based on the first kitchen environment image set.

[0144] The second determining unit 303 may include: an area ratio determining subunit, a predicting subunit, and a determining subunit.

[0145] The area ratio determination subunit is configured to determine whether the area ratio between the target region and the static reference base image is greater than or equal to a preset threshold range.

[0146] The prediction subunit is configured to, when a determination result of the area ratio determination subunit is yes, input the image of the target area as a candidate recognition image into a type prediction model for type prediction.

[0147] The determining subunit is configured to determine the predicted type of the object in the target area according to the prediction result.

[0148] The second determining unit 303 may include: an extracting subunit and an acquiring subunit.

[0149] The extraction subunit is used to extract feature data of the object in the candidate recognition image according to the feature extraction method corresponding to the acquisition time of the candidate recognition image; specifically, it may include: when the acquisition time is a daytime period, extracting the local feature data of the object in the candidate recognition image according to the local feature extraction method; when the acquisition time is a nighttime period, extracting the reflective feature data of the object in the candidate recognition image according to the light reflection feature extraction method.

[0150] The acquisition subunit is configured to perform type prediction based on the feature data to obtain a predicted type of the object corresponding to the target area.

[0151] For the content of the above-mentioned device embodiment, reference may be made to step S101 to step S103 in the above-mentioned method embodiment, which will not be described in detail here.

[0152] Based on the above content, the present application also provides a computer storage medium for storing computer programs.

[0153] The program executes the contents of steps S101 to S103 involved in the above-mentioned embodiment of the back kitchen environment detection method.

[0154] Based on the above content, the present application also provides an electronic device, as shown in FIG4 , including a processor 401 to a memory 402 .

[0155] Processor 401.

[0156] The memory 402 is used to store a computer program, which executes the contents of steps S101 to S103 involved in the above-mentioned embodiment of the back kitchen environment detection method.

[0157] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0158] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0159] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0160] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0161] 2. Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] Although the present application is disclosed as above with the preferred embodiments, it is not intended to limit the present application. Any person skilled in the art may make possible goals and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. A method for detecting the kitchen environment, characterized in that, Including: Determine a static reference base image corresponding to the acquisition device according to a first set of kitchen environment images acquired by the acquisition devices arranged in the kitchen; Extract an image of the target area from the acquired second set of kitchen environment images according to the static reference base image; use the image of the target area as a candidate recognition image for type prediction to determine the predicted type of the object in the target area; Input the image of the target area as an image to be detected into a type detection model corresponding to the predicted type for detection to determine the safety status of the object in the kitchen environment.

2. The kitchen environment detection method according to claim 1, wherein The determining a static reference base image corresponding to the acquisition device according to a first set of kitchen environment images acquired by the acquisition devices arranged in the kitchen environment includes: Perform object recognition on the images in the first set of kitchen environment images to obtain recognition images; Determine whether there are different images among the recognition images according to the comparison between the recognition images; If not, determine an image randomly selected from the first set of kitchen environment images as the static reference base image.

3. The kitchen environment detection method according to claim 2, wherein The performing object recognition on the images in the first set of kitchen environment images to obtain recognition images includes: Perform object edge detection on the images in the first set of kitchen environment images to determine the object segmentation areas in the images; Perform object recognition on the segmentation areas to obtain recognition images corresponding to the segmentation areas.

4. The kitchen environment detection method according to claim 2, wherein It also includes: When it is determined that there are different images among the recognition images, determine whether the difference is within a difference threshold range determined according to the acquisition device specifications and / or acquisition angles; If so, determine an image randomly selected from the first set of kitchen environment images as the static reference base image.

5. The kitchen environment detection method according to claim 4, wherein It also includes: When it is determined that the difference is not within the difference threshold range determined according to the acquisition device specifications and / or acquisition angles, determine the image with the least number of objects in the recognition images as the static reference base image corresponding to the acquisition device.

6. The kitchen environment detection method according to claim 1, characterized in that, The determining a static reference base image corresponding to the acquisition device according to a first set of kitchen environment images acquired by the acquisition devices arranged in the kitchen includes: Randomly select an image from the first set of kitchen environment images as a candidate static reference base image; Perform object recognition on the images in the first set of kitchen environment images; Update the candidate static reference base image according to the images in the first set of kitchen environment images in which the number of recognized objects is less than the number of objects in the candidate static reference base image; Determine whether the number of objects in the candidate static reference base image is the image with the least number of objects in the first set of kitchen environment images. If so, determine the updated candidate static reference base image as the static reference base image.

7. The kitchen environment detection method according to claim 1, wherein The using the image of the target area as a candidate recognition image to perform type prediction on the object corresponding to the target area to determine the predicted type of the object in the target area includes: Determine whether the area ratio between the target area and the static reference base image is greater than or equal to a preset threshold range; If so, use the image of the target area as a candidate recognition image and input it into a type prediction model for type prediction; Determine the predicted type of the object in the target area according to the prediction result.

8. The kitchen environment detection method according to claim 1, wherein The using the image of the target area as a candidate recognition image for type prediction and determining the predicted type of the object corresponding to the target area includes: Extract the feature data of the object in the candidate recognition image according to the feature extraction method corresponding to the acquisition time of the candidate recognition image; Perform type prediction according to the feature data to obtain the predicted type of the object corresponding to the target area.

9. The kitchen environment detection method according to claim 8, characterized in that, The extracting the feature data of the object in the candidate recognition image according to the feature extraction method corresponding to the acquisition time of the candidate recognition image includes: When the acquisition time is in the daytime time interval, extract the local feature data of the object in the candidate recognition image according to the local feature extraction method; When the acquisition time is in the nighttime time interval, extract the specular reflection feature data of the object in the candidate recognition image according to the specular reflection feature extraction method.

10. The kitchen environment detection method according to claim 1, wherein, The determining the static reference base image corresponding to the acquisition device according to the first set of images of the back kitchen environment collected by the acquisition device arranged in the back kitchen includes: Obtain the first set of images of the back kitchen environment of the acquisition device according to the acquisition dimension configured for the set of images of the back kitchen environment; wherein, the acquisition dimension includes at least one of a time dimension, a meal type dimension, and a back kitchen location dimension; Determine the static reference base image corresponding to the acquisition device according to the first set of images of the back kitchen environment.

11. A kitchen environment detection device, characterized in that, Includes: A first determination unit for determining the static reference base image corresponding to the acquisition device according to the first set of images of the back kitchen environment collected by the acquisition device arranged in the back kitchen; An extraction unit for extracting the image of the target area from the obtained second set of images of the back kitchen environment according to the static reference base image; A second determination unit for using the image of the target area as a candidate recognition image for type prediction and determining the predicted type of the object corresponding to the target area; A third determination unit for using the image of the target area as a detection image to be input into a type detection model corresponding to the predicted type of the object for detection and determining the safety state of the object in the back kitchen environment.

12. A computer storage medium for storing a computer program; The program executes the back kitchen environment detection method according to any one of claims 1-10 above.

13. An electronic device, comprising: A processor; A memory for storing a computer program, and the program executes the back kitchen environment detection method according to any one of claims 1-10 above.

Images