Method, device and refrigerator for obtaining a clear image

Abstract

The application discloses a clear image acquisition method, device and refrigerator, and the method comprises the following steps: in the time sequence of image shooting by a camera, comparing two adjacent images one by one through a preset comparison strategy until two adjacent images are the same; performing blur detection on one of the two same images, and if the blur is within a preset range, the image is determined as a clear image. Compared with the prior art, the clear image acquisition method of the application first selects the image shot when the food is in a stationary state, and then checks the blur of the image, so that the blur of the image itself caused by dark light or focusing problems is excluded, and a clear image that can be recognized is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of household appliances, in particular to a clear image acquisition method, device and refrigerator. BACKGROUND

[0002] With the development of science and technology, the intelligentization of refrigerator has become an inevitable trend. The so-called intelligent refrigerator is a refrigerator type capable of intelligent control of the refrigerator and intelligent management of food. Specifically, it can automatically change the refrigerator mode, always keep the food in the best storage state, allow users to know the quantity and freshness information of the food in the refrigerator at any time and any place through mobile phones or computers, provide healthy recipes and nutritional taboos for users, and remind users to replenish food regularly.

[0003] To achieve the above functions, it is necessary to install a camera in the refrigerator to take pictures of the food inside the refrigerator. At present, the timing of taking pictures of food is generally after the refrigerator door is closed (i.e. after the user stores food, the variety or quantity of food may change, and the changes need to be obtained in time by taking pictures of food). However, this may cause the food to shake due to the refrigerator door being closed too quickly or too heavily, resulting in blurred food photos that cannot be recognized.

[0004] How to obtain clear and recognizable food images in this case is a problem to be solved at present. SUMMARY

[0005] The present application relates to the field of household appliances, in particular to a clear image acquisition method, device and refrigerator.

[0006] To achieve one of the above-mentioned purposes, an embodiment of the present application provides a clear image acquisition method, characterized in that the method comprises:

[0007] According to the time sequence of the camera taking pictures, the adjacent two pictures are compared one by one through a preset comparison strategy until the two adjacent pictures are the same.

[0008] The blur degree of one of the two same pictures is detected, and if the blur degree is within a preset range, the picture is determined to be a clear picture.

[0009] As a further improvement of the embodiment of the present application, the preset comparison strategy comprises:

[0010] The RGB difference value of each pixel point corresponding to the two pictures is calculated, and if the RGB difference value of each pixel point is within a preset difference value range, the two pictures are determined to be the same, otherwise, the two pictures are determined to be different.

[0011] As a further improvement of an embodiment of the present invention, the “calculating the RGB difference of each pixel in the one-to-one correspondence between the two images” includes:

[0012] Subtract the RGB value of each pixel in one image from the RGB value of the corresponding pixel in the other image to get the RGB difference of each pixel in the two images.

[0013] As a further improvement of an embodiment of the present invention, the RGB difference includes the difference between the values ​​in the R channel, the G channel and the B channel in the RGB color space.

[0014] As a further improvement of an embodiment of the present invention, the “RGB difference of each pixel is within a preset difference range” includes:

[0015] The difference between the values ​​of the R channel, G channel and B channel of each pixel in the RGB color space is within a preset difference range.

[0016] As a further improvement to an embodiment of the present invention, the method of “comparing two adjacent images one by one according to the time sequence of the images captured by the camera using a preset comparison strategy until the two adjacent images are found to be the same” includes:

[0017] Divide the image into N regions. When comparing two adjacent images, compare each corresponding region of the two images one by one. If all N regions of the two images are the same, the two images are determined to be the same and the comparison process ends. Otherwise, the following process is repeated:

[0018] When the first different region M of a group of adjacent images is compared, the comparison of the adjacent images is stopped and the adjacent images are determined to be different images; and the comparison of the next group of adjacent images is continued starting from the region M.

[0019] As a further improvement of an embodiment of the present invention, the “performing blur detection on one of the two identical images” includes:

[0020] Blur detection is performed on one of the two identical images using a Laplacian operator.

[0021] As a further improvement of one embodiment of the present invention, the method further includes:

[0022] After the last group of adjacent images is compared to be different images, the camera is controlled to capture a new image, and the new image is compared with the next image in the last group of adjacent images as the next group of adjacent images.

[0023] To achieve the above-mentioned one of the purposes of the application, an embodiment of the application provides an electronic device, comprising a memory and a processor, the memory stores a computer program capable of running on the processor, and the processor implements the steps in the clear image acquisition method of any one of the above-mentioned methods when executing the program.

[0024] To achieve the above-mentioned one of the purposes of the application, an embodiment of the application provides a refrigerator, which comprises the electronic device as described above.

[0025] Compared with the prior art, the clear image acquisition method of the application first screens out images taken when food is in a stationary state, and then checks the blurriness of the images, so as to exclude the blurring of the images themselves caused by dark light or focusing problems, thereby obtaining clear images that can be identified. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a flowchart of the clear image acquisition method of the application.

[0027] Figure 2 is a schematic diagram of a specific embodiment of the image sub-region of the application.

[0028] Figure 3 is an example of the position of the bottle seat region in the taken image. DETAILED DESCRIPTION

[0029] The application will be described in detail below with reference to the specific embodiments shown in the drawings. However, these embodiments do not limit the application, and the changes made by those skilled in the art in structure, method, or function based on these embodiments are all included in the protection scope of the application.

[0030] In the refrigerator, a yogurt area is provided to centrally monitor the yogurt. The yogurt area is generally provided with a bottle seat, and the yogurt is placed in the bottle seat. In the process of monitoring the yogurt in the bottle seat, image shooting and identification of the yogurt area are required. Since the shooting time is selected after the user closes the door, the food may be shaken due to too fast or too heavy closing of the refrigerator door, so that the food photos taken are blurred and cannot be identified.

[0031] In view of the above-mentioned situation, the application provides a clear image acquisition method, which first screens out images taken when food is in a stationary state, and then checks the blurriness of the images, so as to exclude the blurring of the images themselves caused by dark light or focusing problems, thereby obtaining clear images that can be identified.

[0032] As shown in Figure 1 , the method comprises:

[0033] Step S100: according to the time sequence of the camera taking the images, two adjacent images are compared one by one using a preset comparison strategy until the two adjacent images are found to be the same image.

[0034] The purpose of this step is to filter out images taken when the food is in a static state.

[0035] Assume that there are five images, A, B, C, D, and E, in the order in which the cameras captured them. Then, A and B form the first set of adjacent images, B and C form the second set of adjacent images, C and D form the third set of adjacent images, and so on. During the comparison process, the first set of adjacent images is compared first. If the first set of adjacent images is different, the next set of images is compared until a set of adjacent images is found to be identical, at which point step S200 is performed.

[0036] Furthermore, the preset comparison strategy includes:

[0037] The RGB difference of each pixel in the two images is calculated. If the RGB difference of each pixel is within a preset difference range, the two images are determined to be the same image; otherwise, the two images are determined to be different images.

[0038] The RGB color space is based on the three primary colors R (red), G (green), and B (blue). These colors are superimposed to varying degrees to produce a rich and wide range of colors, hence the name "three-primary color model." Specifically, RGB consists of three channels: R, G, and B. The brightest red + the brightest green + the brightest blue = white; the darkest red + the darkest green + the darkest blue = black; and between the brightest and darkest, red of the same brightness + green of the same brightness + blue of the same brightness = gray. Within any RGB channel, white and black represent the brightness of that color. Therefore, where there is white or grayish-white, the three R, G, and B channels cannot be black, as these three channels are required to form these colors.

[0039] Therefore, the RGB value of each pixel is the value of each pixel in the three channels R, G, and B. The RGB difference of each pixel in the one-to-one correspondence between the two images refers to the subtraction of the RGB values ​​of the pixel at the same position in the two different images, that is, the subtraction of the R channel values, the G channel values, and the B channel values ​​of the pixel at the same position in the RGB color space of the two different images.

[0040] The “RGB difference of each pixel is within a preset difference range” includes:

[0041] The difference between the values ​​of the R channel, G channel and B channel of each pixel in the RGB color space is within a preset difference range.

[0042] In order to speed up the comparison and reduce a lot of repetitive work, in a specific embodiment, the "comparing two adjacent images one by one according to the time sequence of the images captured by the camera using a preset comparison strategy until the two adjacent images are found to be the same image" includes:

[0043] Divide the image into N regions. When comparing two adjacent images, compare each corresponding region of the two images one by one. If all N regions of the two images are the same, the two images are determined to be the same and the comparison process ends. Otherwise, the following process is repeated:

[0044] When the first different region M of a group of adjacent images is compared, the comparison of the adjacent images is stopped and the adjacent images are determined to be different images; and the comparison of the next group of adjacent images is continued starting from the region M.

[0045] For example, Figure 2 As shown, the image is divided into a 3*3 grid, or nine regions, and sorted. When comparing a set of adjacent images, these nine regions are compared one by one according to the sorting order. If these nine regions are identical, the adjacent images are considered identical. Otherwise, when the first different region is found (assuming it is region M), the comparison stops, the adjacent images are considered different, and the comparison of the next set of adjacent images continues. The comparison of the next set of adjacent images begins with region M. Assuming all regions are identical, the comparison order is M, M+1, ..., 9, 1, ..., M-1. Since the food is shaken continuously, this comparison method can reduce the repetitiveness of the previous steps and speed up the comparison.

[0046] Step S200: performing blur detection on one of the two identical images, and if the blur is within a preset range, determining that the image is a clear image.

[0047] The purpose of step S100 is to filter out images taken when the food is stationary, eliminating image blur caused by shaking of the food. When the food is stationary, the image itself may be blurred due to low light or focus issues, affecting the image recognition rate.

[0048] Therefore, this step checks the blurriness of the image in order to eliminate the blur of the image itself caused by dark light or focusing problems, so as to obtain a clear image that can be recognized, or to obtain a clear image with a recognition rate that meets the standard.

[0049] Specifically, the blurriness of one of the two identical images can be detected using a Laplacian operator. That is, the second-order derivative of the image is calculated using the Laplacian operator. If the calculated result is within a preset threshold, the image is determined to be a clear image that can be recognized; otherwise, the image is determined to be blurred.

[0050] It should be noted that there are many ways to obtain the preset threshold, preferably by the following method:

[0051] The Laplace operator is used to calculate the second-order derivatives of images with different blur levels, and then image recognition is performed on these images. According to the recognition rate threshold, images with a recognition rate greater than or equal to the recognition rate threshold are obtained, and the corresponding second-order derivative interval is obtained according to the interval to obtain the preset threshold range.

[0052] Furthermore, the method for obtaining a clear image further includes:

[0053] Step S300: If the blurriness of one of the two identical images is not within a preset range, the image is determined to be a blurred image, and steps S100 and S200 are repeated to continue blur detection on subsequent images to obtain a clear image.

[0054] After it is detected that the image obtained in step S100 is a blurred image, steps S100 and S200 need to be repeated until a clear image that can be recognized is obtained.

[0055] It should be noted that after the last group of adjacent images is found to be different, the camera is controlled to capture a new image, and the new image is compared with the next image in the last group of adjacent images as the next group of adjacent images.

[0056] For example, the front camera takes five images A to E and compares them in chronological order. When it is found that the adjacent images D and E are different images, the camera is controlled to take a new image F, and then images E and F are used as the next set of adjacent images for comparison.

[0057] The method for obtaining a clear image of the present invention filters out blurred images caused by various reasons (for example, blurred images caused by shaking food ingredients, blurred images themselves due to dark light or focusing problems), thereby obtaining a clear image that can be identified.

[0058] In a specific embodiment, a bottle holder area of ​​a refrigerator door is set as a yogurt area, and a fixed camera is installed on the top of the yogurt area. The fixed camera captures the yogurt area at a fixed angle to obtain images of the yogurt area. Since the camera is set on the top of the yogurt area, these captured images will not only include the bottle holder area, but also the surrounding areas (such as Figure 3 As shown), since the main task is to identify the food in the bottle seat area, after the user closes the refrigerator door, the camera is started to capture the image, and the image is uniformly cropped to remove the peripheral area to obtain the image to be detected.

[0059] The image to be detected is divided into a 3*3 nine-square grid, that is, into 9 areas, and sorted. When comparing a group of adjacent images, these 9 areas are compared one by one according to the sorting. If these 9 areas are exactly the same, the adjacent images are determined to be the same image. Otherwise, when the first different area is compared (assuming it is area M), the comparison is stopped, the adjacent images are determined to be different images, and the comparison of the next group of adjacent images is continued. The comparison of the next group of adjacent images starts from the above-mentioned area M. Assuming that all areas are the same, the comparison order is M, M+1,…,9,1,…,M-1.

[0060] After obtaining a pair of adjacent images that are identical, the Laplacian operator is used to perform blur detection on one of the two identical images. This involves calculating the second-order derivative of the image using the Laplacian operator. If the result is within a preset threshold, the image is considered sharp and recognizable; otherwise, it is considered blurred.

[0061] If the blurriness of one of the two identical images is not within the preset threshold range, the previous steps are repeated to continue blur detection on subsequent images to obtain a clear image.

[0062] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, it implements any one of the steps in the above-mentioned method for obtaining a clear image, that is, it implements the steps in any one of the technical solutions in the above-mentioned method for obtaining a clear image.

[0063] The present invention also provides a refrigerator, which includes the electronic device described above.

[0064] It should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation method can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0065] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for obtaining a clear image, characterized in that: The method comprises: Step S100: comparing two adjacent images one by one according to the time sequence of the images captured by the camera using a preset comparison strategy until the two adjacent images are found to be the same image; Step S200: performing a blur detection on one of the two identical images, and if the blur is within a preset range, determining that the image is a clear image; Step S300: If the blurriness of one of the two identical images is not within a preset range, the image is determined to be a blurred image, and steps S100 and S200 are repeated to continue blur detection on subsequent images to obtain a clear image.

2. The method for obtaining a clear image according to claim 1, characterized in that: The preset comparison strategy includes: The RGB difference of each pixel in the two images is calculated. If the RGB difference of each pixel is within a preset difference range, the two images are determined to be the same image; otherwise, the two images are determined to be different images.

3. The method for obtaining a clear image according to claim 2, characterized in that: The "calculating the RGB difference of each pixel in the one-to-one correspondence between two images" includes: Subtract the RGB value of each pixel in one image from the RGB value of the corresponding pixel in the other image to get the RGB difference of each pixel in the two images.

4. The method for obtaining a clear image according to claim 3, wherein: The RGB difference value includes the difference between the values ​​in the R channel, the G channel, and the B channel in the RGB color space.

5. The method for obtaining a clear image according to claim 4, characterized in that: The “RGB difference of each pixel is within a preset difference range” includes: The difference between the values ​​of the R channel, G channel and B channel of each pixel in the RGB color space is within a preset difference range.

6. The method for obtaining a clear image according to claim 2, wherein: The step of “comparing two adjacent images one by one according to the time sequence of the images captured by the camera using a preset comparison strategy until the two adjacent images are found to be the same” includes: Divide the image into N regions. When comparing two adjacent images, compare each corresponding region of the two images one by one. If all N regions of the two images are the same, the two images are determined to be the same and the comparison process ends. Otherwise, the following process is repeated: When the first different region M of a group of adjacent images is compared, the comparison of the adjacent images is stopped and the adjacent images are determined to be different images; and the comparison of the next group of adjacent images is continued starting from the region M.

7. The method for obtaining a clear image according to claim 1, characterized in that: The “performing blur detection on one of the two identical images” includes: Blur detection is performed on one of the two identical images using a Laplacian operator.

8. The method for obtaining a clear image according to claim 1, wherein: The method further comprises: After the last group of adjacent images is compared to be different images, the camera is controlled to capture a new image, and the new image is compared with the next image in the last group of adjacent images as the next group of adjacent images.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor executes the program, the steps in the method for obtaining a clear image according to any one of claims 1 to 8 are implemented.

10. A refrigerator, characterized in that: The refrigerator comprises the electronic device according to claim 9.

Images