Imaging method for articles in drawer and refrigeration equipment

Abstract

The invention discloses an imaging method for articles in a drawer and refrigeration equipment, and the method comprises the steps: obtaining a first image when the drawer is located at a first position, obtaining a second image when the drawer is located at a second position, and combining the distance between the first position and the second position, and fusing the image of the target object in the first image and the second image into a fused image. According to the method, image distortion caused by light reflection is reduced or eliminated, image information under different angles and illumination conditions can be synthesized, clearer and more real object images are finally obtained, the process does not depend on excessive light conditions, a light source does not need to be repeatedly adjusted, real image information of an object can be better restored under different light environments, and the image quality is improved. And the identification accuracy and the fault tolerance of the system are improved, so that the real appearance of the article is restored to the greatest extent.

Description

Technical Field

[0001] This invention relates to the field of imaging technology, and more particularly to an imaging method for a refrigeration device and items inside a drawer. Background Technology

[0002] With technological advancements, the functions of refrigeration equipment (such as refrigerators) have gradually improved. For example, some refrigeration devices can determine the type and freshness of stored food by acquiring images of it. However, for drawers in refrigeration devices with limited space and relatively dim lighting, imaging of items inside is often more limited. This is especially true when food is covered by packaging, as localized reflections from the packaging can cause image distortion, resulting in unclear images. Without clear images, or an inability to accurately discern the contents of the food, further information cannot be assessed, thus affecting the food recognition function of the refrigeration device and leading to a poor user experience. Summary of the Invention

[0003] To address the problem of unclear images of items inside drawers in the prior art, the present invention aims to provide an imaging method and a cooling device for items inside drawers that reduces or eliminates image distortion caused by reflection.

[0004] To achieve one of the above-mentioned objectives, an embodiment of the present invention provides an imaging method for items inside a drawer, comprising the following steps:

[0005] Obtain a first image of the drawer when it is in the first position;

[0006] Obtain a second image of the drawer in its second position;

[0007] By combining the distance between the first position and the second position, the images of the target object in the first image and the second image are fused into a fused image.

[0008] As a further improvement to the present invention, the following steps are also included:

[0009] Establish a mapping relationship between image pixel size and world size;

[0010] The step of fusing the images of the target object in the first image and the second image into a fused image by combining the distance between the first position and the second position includes:

[0011] Based on the distance between the first position and the second position and the mapping relationship, determine the corresponding pixel movement distance on the image;

[0012] Based on the pixel movement distance, the images of the target object in the first image and the second image are fused into a fused image.

[0013] As a further improvement to the present invention, the following steps are also included:

[0014] Get real-time recording;

[0015] When the drawer is in the second position, if a limb is detected in the real-time video and the intersection-over-union ratio of the limb and the target item exceeds the coverage threshold, the starting position and ending position of the limb are recorded.

[0016] The pixel movement distance on the image is determined based on the starting position, the ending position, the distance between the first position and the second position, and the mapping relationship.

[0017] As a further improvement of the present invention, the first position is the closed position when the drawer is in the closed state, and the second position is the moved-out position when the drawer is in the moved-out state.

[0018] As a further improvement of the present invention, obtaining a second image of the drawer in the second position includes:

[0019] Obtain distance information, wherein the distance information is the distance between the current position of the drawer and the closed position;

[0020] Based on the distance information, determine whether the drawer has been moved out and whether it has stopped moving;

[0021] If so, obtain a second image of the drawer when it is in the second position.

[0022] As a further improvement of the present invention, the step of determining whether the drawer has been moved out and stopping its movement based on the distance information includes:

[0023] When the distance information changes and exceeds the distance error threshold, it is determined that the drawer has been moved out.

[0024] When the distance information stops changing but is still greater than the distance error threshold, it is determined that the drawer has been moved out and has stopped moving.

[0025] As a further improvement of the present invention, obtaining a second image of the drawer in the second position includes:

[0026] Identify the target item in the first image;

[0027] When the information of the target item cannot be identified, and the area in the first image where the pixel brightness is greater than the pixel brightness threshold exceeds the reflective area threshold, a second image of the drawer when the drawer is in the second position is obtained.

[0028] As a further improvement of the present invention, the step of fusing the images of the target items in the first image and the second image into a fused image includes:

[0029] Extract the reflective regions in the first image whose pixel brightness is greater than the pixel brightness threshold;

[0030] The reflected area in the second image is replaced with the area that is the same as the reflected area to obtain the fused image.

[0031] As a further improvement of the present invention, the step of fusing the images of the target items in the first image and the second image into a fused image includes:

[0032] Calculate the area of ​​reflective regions in the first image and the second image where the pixel brightness is greater than the pixel brightness threshold;

[0033] Determine the size of the reflective area in each of the first and second images;

[0034] The fused image is obtained by replacing the reflective area in an image with the same reflective area in another image.

[0035] As a further improvement to the present invention, the following steps are also included:

[0036] Identify the target item in the fused image;

[0037] When the information of the target item cannot be identified, and the area in the fused image where the pixel brightness is greater than the pixel brightness threshold exceeds the reflective area threshold, a new second image is acquired and the new fused image is fused.

[0038] To achieve one of the above-mentioned objectives, one embodiment of the present invention provides a refrigeration device, comprising:

[0039] drawer;

[0040] An imaging module is used to acquire images of the inside of the drawer;

[0041] A ranging module, used to obtain the distance the drawer has moved;

[0042] Storage module, used to store computer programs;

[0043] The processing module, when executing the computer program, can implement the steps in the above-described method for imaging items inside a drawer.

[0044] To achieve one of the above-mentioned objectives, one embodiment of the present invention provides a readable storage medium storing a computer program that, when executed by a processing module, can implement the steps in the above-described imaging method for items inside a drawer.

[0045] Compared with existing technologies, the present invention has the following beneficial effects: This method for imaging items inside a drawer acquires images of the drawer at different positions and combines them with distance information between different positions to perform image fusion, thereby reducing or eliminating image distortion caused by reflection and ultimately obtaining a clearer and more realistic image of the item. Regarding distortion caused by reflection on the item's surface, since the degree of reflection varies at different positions, fusing images from different positions can eliminate the adverse effects of reflection. This method does not rely on excessive lighting conditions and does not require repeated adjustments to the light source. It can better restore the true image information of the item under different lighting environments, improving the accuracy of recognition and the system's fault tolerance. Compared with recognition methods that use only a single image, this image fusion method can integrate image information from different angles and lighting conditions, thereby maximizing the restoration of the item's true appearance. It performs particularly well in complex lighting environments, ultimately improving the image's realism and clarity, which in turn facilitates subsequent accurate identification of item information based on the clearer image, improving the user experience. Attached Figure Description

[0046] Figure 1 This is a flowchart of an embodiment of an imaging method for items inside a drawer according to the present invention;

[0047] Figure 2 This is a schematic diagram illustrating the mapping relationship between image pixel size and world size using a checkerboard pattern, according to an embodiment of the present invention.

[0048] Figure 3 This is a schematic diagram of the target item being moved out of the drawer according to an embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram of a target item being moved by a limb according to an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram illustrating the generation of a fused image according to an embodiment of the present invention;

[0051] Figure 6 This is a structural block diagram of electronic components of a refrigeration device according to an embodiment of the present invention;

[0052] Wherein, S1 is the first position; S2 is the second position; S3 is the starting position; S4 is the ending position; P1 is the first image; P2 is the second image; P3 is the fused image; 100 is the cooling device; 10 is the imaging module; 20 is the ranging module; 30 is the storage module; 40 is the processing module; and 50 is the communication bus. Detailed Implementation

[0053] The present invention will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0054] One embodiment of the present invention provides an imaging method and a cooling device for reducing or eliminating image distortion caused by reflections in items inside a drawer.

[0055] The refrigeration equipment in this embodiment can be a refrigerator, freezer, upright refrigerator, wine cabinet, etc. The refrigeration equipment in the following embodiment is described using a refrigerator as an example. The refrigerator includes a cabinet and a door. The door and the cabinet together enclose a refrigeration space. Drawers can be installed in the refrigeration space, and items such as food can be placed in the drawers.

[0056] An imaging module 10 and a ranging module 20 are installed at the drawer. The imaging module 10 is used to acquire images of the inside of the drawer and may include a camera that takes pictures in the direction of the drawer's interior. The ranging module 20 is used to acquire the distance the drawer has moved. The ranging module 20 may include a distance sensor that is installed on the cabinet or drawer to detect the distance between the drawer and the back panel at the rear of the drawer.

[0057] As described in the background art, when capturing images of objects, due to the small space and relatively dim lighting inside drawers, the reflection problem of the packaging bags, such as plastic bags, is particularly obvious. The reflection problem causes image distortion, making it impossible to see the words on the packaging bag or the items inside the packaging bag, which in turn leads to inaccurate target recognition.

[0058] This embodiment proposes an imaging method that combines images from different locations for fusion. This method acquires images from different locations in the drawer and combines the distance information between the two to perform image fusion, which can significantly reduce imaging distortion caused by reflection.

[0059] The following is combined with Figures 1-5 This invention describes an embodiment of an imaging method for items inside a drawer. Although this application provides method operation steps as shown in the following embodiments or flowcharts, the execution order of steps in which there is no necessary causal relationship in logic, based on conventional or non-creative labor, is not limited to the execution order provided in the embodiments of this application.

[0060] Specifically, such as Figure 1 As shown, an imaging method for items inside a drawer according to this embodiment includes the following steps:

[0061] Step S10: Obtain the first image P1 when the drawer is in the first position S1.

[0062] Step S20: Obtain a second image P2 when the drawer is in the second position S2.

[0063] Step S30: Combining the distance between the first position S1 and the second position S2, the images of the target object in the first image P1 and the second image P2 are fused into a fused image P3.

[0064] The distance between the first position S1 and the second position S2 can be obtained by the distance sensor of the ranging module 20, and the first image P1 and the second image P2 can be obtained by the camera of the imaging module 10. The first image P1 and the second image P2 can be the whole image or the cut-out image that only includes the target object.

[0065] The purpose of fusing images from different locations in this embodiment is to address the fact that images obtained from different locations may have defects to varying degrees. Fusion can eliminate these defects. A specific example is the reflective area on the surface of an object. Reflection means that the content in that area is unclear, appearing as only a bright spot. Furthermore, the reflective area of ​​the same object may differ at different distances. Image fusion can effectively remove or reduce these reflective areas, selecting the optimal image portion for fusion to improve the quality of the resulting image.

[0066] By combining the distance between the first position S1 and the second position S2, image fusion can be performed accurately, reducing errors caused by reflections. When the drawer moves, the position of the target item changes. By combining the distance information, the corresponding pixel movement distance on the image can be calculated, ensuring that the image portions of the same target item are correctly aligned when fusion image P3, avoiding inaccurate image overlap due to positional deviations. In other words, fusion only needs to be determined based on distance changes, without needing to change the lighting.

[0067] The image fusion method in this embodiment can integrate image information from different angles to restore the true appearance of an object to the greatest extent, especially performing well in environments with complex lighting and reflective surfaces.

[0068] Wherein, the first position S1 is the closed position when the drawer is in the closed state, and the second position S2 is the moved-out position when the drawer is in the moved-out state.

[0069] When the drawer is closed, the distance d detected by the distance measuring module 20 can be 0, meaning that by d = 0, it can be determined that the drawer is in the first position S1. When the drawer is moved out, the distance d detected by the distance measuring module 20 at the second position S2 can be d1, meaning that by d = d1, it can be determined that the drawer is in the second position S2. Thus, the distance between the first position S1 and the second position S2 is d1. Figure 3As shown in the figure. Here, d1 can be a fixed value or any value greater than a certain specific value d0.

[0070] By explicitly defining the drawer's position state, the system can respond differently in different states, effectively reducing unnecessary imaging processes and saving system resources. For example, images can be acquired only when d = d1, or only when d1 > d0. This method not only improves imaging efficiency but also reduces data processing complexity, optimizing system performance while ensuring image quality.

[0071] The imaging method for items inside a drawer in this embodiment also includes the following steps:

[0072] Step S31: Establish the mapping relationship between image pixel size and world size.

[0073] Mapping relationships can be based on scale, or as... Figure 2 The diagram shows how a checkerboard pattern is used to establish a correlation between image pixel coordinates and world coordinates. This mapping relationship ensures that the pixel movement distance during image fusion corresponds to the actual object movement distance. This allows the system to achieve more accurate object location determination, improving fusion accuracy and overall imaging effect.

[0074] Based on this, step S30 includes:

[0075] S32: Determine the corresponding pixel movement distance on the image based on the distance between the first position S1 and the second position S2 and the mapping relationship;

[0076] S33: Based on the pixel movement distance, merge the images of the target items in the first image P1 and the second image P2 into a fused image P3.

[0077] For example, if one pixel corresponds to 5cm in world coordinates (i.e., 1 pixel: 5cm), then based on the distance d1, the distance of the moved pixels on the image can be determined. Based on this pixel distance, the same target item in the first image P1 and the second image P2 can be associated, such as... Figure 3 As shown, based on the distance d1, it can be determined that target item A has moved to target item A'. In this way, a correspondence is established between target item A in the first image P1 and target item A' in the second image P2, which facilitates image fusion.

[0078] The imaging method for items inside a drawer in this embodiment also includes the following steps:

[0079] Step S41: Acquire real-time recording;

[0080] When the drawer is in the second position S2, if a limb is detected in the real-time video and the intersection-over-interference ratio of the limb and the target item exceeds the coverage threshold, the starting position S3 and the ending position S4 of the limb are recorded.

[0081] Step S42: Determine the corresponding pixel movement distance on the image based on the starting position S3, the ending position S4, the distance between the first position S1 and the second position S2, and the mapping relationship.

[0082] "Limbs" refers to the possibility that a user's hands or other body parts may appear in the camera's field of view when the drawer is opened. This can interfere with the imaging of the items inside the drawer, and the user's limbs may also move the target items. Therefore, this situation is handled by detecting limb targets.

[0083] Specifically, such as Figure 4 As shown, when the drawer is in the second position S2 (e.g., partially open), the real-time video will capture the situation inside the drawer. If a limb is detected during this process, and the intersection-over-union (IOU) ratio between the limb and the target item exceeds a certain preset coverage threshold, it indicates that the limb may be covering or touching the target item. At this time, the real-time video system will record the starting and ending positions S4 of the limb to determine the movement trajectory of the limb.

[0084] Without limbs, target item A should have moved to the position of target item A'. However, due to the influence of limbs, target item A moved to the position of target item A''. Since the drawer is closed at the first position S1, the user cannot extend limbs into it. Therefore, the position of target item A' (x1, y1) is the starting position S3 where the user's limbs are pulled out of the drawer. The position of target item A'''(x2, y2) when the user's limbs are removed is the ending position S4. (x2, y2) - (x1, y1) = (Δx, Δy) is the amount of movement of the item. Combining (Δx, Δy) and... Figure 4 In d1, the position of target item A in the figure can be used to establish a correspondence between the position of target item A and the position of target item A”.

[0085] The purpose of this processing logic is to detect interference when a user's hand or other body parts appear in the camera's view, and to track the movement of the limbs, thereby avoiding erroneous image fusion or object recognition due to limb involvement. For example, if the user's hand moves a target object, the system can update the target object's position based on the recorded start and end points S4, ensuring the accuracy of the fused image P3. By combining limb movement with the drawer's position information, the system can accurately determine whether the object has been moved and its new location, thus ensuring the accuracy of image fusion. This method effectively solves the problem of dynamic interference during the imaging process and improves the system's intelligence level.

[0086] in addition, Figure 3 and Figure 4 The d1 values ​​can be the same or different.

[0087] Further, step S20 includes:

[0088] Step S21: Obtain distance information, wherein the distance information is the distance between the current position of the drawer and the closed position;

[0089] Step S22: Determine whether the drawer has been moved out and stop moving based on the distance information;

[0090] If so, obtain the second image P2 when the drawer is in the second position S2.

[0091] Steps S21 and S22 are used to determine whether the drawer has been moved out and when it has stopped moving. This process greatly improves the accuracy and intelligence of the imaging process. By monitoring the movement status of the drawer in real time, the system can acquire images at the most appropriate time, thereby avoiding image blurring or errors caused by the drawer not being completely moved out or still moving. The system can capture images when the drawer stops moving, further improving the stability and accuracy of imaging and reducing the possibility of error accumulation.

[0092] Additionally, if the determination in step S22 is negative, the drawer may not have moved or may have closed again without stopping. In this case, the second image P2 may not be acquired until the conditions are met again.

[0093] Step S22 further includes:

[0094] Step S221: When the distance information changes and exceeds the distance error threshold, determine that the drawer has been moved out.

[0095] Step S222: When the distance information stops changing and is still greater than the distance error threshold, determine that the drawer has been moved out and has stopped moving.

[0096] Steps S221 and S222 introduce more refined judgment criteria. In particular, when the distance information changes and exceeds the error threshold, the system will determine that the drawer has been moved. In practical applications, the distance sensor may be affected by the environment and minor errors, causing changes in the distance information. By using the distance error threshold to account for the impact of this error, when the change in the distance information reading exceeds the sensor's accuracy, it is considered that the drawer has been moved. This reduces the probability of false triggering of imaging and improves the stability of the system.

[0097] In addition, when the distance information stops, the drawer may close again or stop after opening. By determining that the distance information stops changing and is still greater than the distance error threshold, it is determined that the drawer has moved to a certain open position. At this point, it is determined that the drawer has moved out and stopped moving, which enhances the sensitivity and accuracy of the system. Combining the changes in distance information can more accurately identify the state of the drawer.

[0098] Furthermore, step S20 also includes:

[0099] Step S23: Identify the target item in the first image P1;

[0100] Step S24: When the information of the target item cannot be identified, and the area in the first image P1 where the pixel brightness is greater than the pixel brightness threshold exceeds the reflective area threshold, obtain the second image P2 of the drawer when the drawer is in the second position S2.

[0101] Steps S23 and S24 further address the issue of surface reflection in order to identify the object. Some objects may contain key information in certain areas. If this key information is obscured, even if other areas are clearly visible, it will affect the judgment. Conversely, if some key information is clear, even if other areas are unclear, it can still be identified normally. Therefore, when the object information can be identified in step S23, the second image P2 can be omitted. Only when the information cannot be identified in step S23 is the second image P2 acquired, which improves the efficiency of the judgment.

[0102] In addition, the determination of reflective areas is based on whether the pixel brightness in the first image P1 is greater than the pixel brightness threshold. The more severe the reflection, the larger the pixel value. When it exceeds a certain threshold, such as the pixel brightness threshold, it is identified as reflective.

[0103] The reflective area threshold can be an area parameter or a ratio parameter. When the area in the first image P1 where the pixel brightness is greater than the pixel brightness threshold exceeds the reflective area threshold, it can be considered that there are too many reflective areas, further clarifying that the inability to identify the object is due to reflection. This method can automatically acquire new images under severe reflection conditions and reduce the impact of reflection on recognition through image fusion, thereby improving the realism and clarity of the image. Especially in enclosed environments with complex lighting conditions, such as refrigerator drawers, this method can significantly improve the recognition rate of target objects and reduce erroneous judgments caused by light reflection.

[0104] Furthermore, image fusion such as Figure 5 As shown, by replacing the reflective areas to generate the fused image P3, step S30 specifically includes two implementation methods. In one implementation method, step S30 includes:

[0105] Step S31: Extract the reflective areas in the first image P1 where the pixel brightness is greater than the pixel brightness threshold.

[0106] Step S32: Replace the reflective area in the second image P2 with the same area as the reflective area to obtain the fused image P3.

[0107] In this embodiment, based on a first image P1, a fused image is obtained by replacing the reflective areas of the first image P1 with the same positions in the second image P2 through comparison and replacement operations. This method greatly improves the visual effect of the image, especially when the object surface has strong reflections, effectively reducing the impact of reflective areas on image quality, making the final image more realistic and accurate.

[0108] In another embodiment, step S30 includes:

[0109] Step S33: Calculate the area of ​​reflective regions in the first image P1 and the second image P2 where the pixel brightness is greater than the pixel brightness threshold;

[0110] Step S34: Determine the size of the reflective area of ​​the first image P1 and the second image P2 respectively;

[0111] Step S35: Replace the reflective area in the image with the same reflective area in another image to obtain the fused image P3.

[0112] In this embodiment, by comparing the reflective areas of the first image P1 and the second image P2, and selecting the smaller reflective area for replacement, the system can more efficiently reduce the interference of reflections on the image, ensuring the minimization of reflective areas in the image, thereby preserving the true details of the object to the greatest extent. This method is particularly effective when processing large-area reflective objects, significantly improving image clarity and recognizability, making the identification of target objects more reliable.

[0113] Furthermore, the imaging method for items inside the drawer also includes the following steps:

[0114] Step S50: Identify the target item in the fused image P3;

[0115] When the information of the target item cannot be identified, and the area in the fused image P3 where the pixel brightness is greater than the pixel brightness threshold exceeds the reflective area threshold, the above steps S20 to S30 are re-executed. Specifically, a new second image P2 of the drawer is obtained when the drawer is in the new second position S2. Combining the distance between the first position S1 and the new second position S2, the images of the target item in the first image P1 and the new second image P2 are fused into a new fused image P3.

[0116] Since the drawer doesn't always stop at the same position each time it's opened, the new second position S2 may not be the same as the previous second position S2. This means the position of the reflective area on the target object may change, resulting in a new second image P2 that differs from the previous image. In this way, the system can obtain the optimal image through multiple attempts. This adaptive imaging strategy allows the system to dynamically respond to various lighting changes and object reflection conditions, significantly improving image reliability and the final recognition effect.

[0117] The combination of these detailed technical features and effects ensures that the system can achieve high-quality imaging, improving the accuracy of object recognition and the stability of imaging.

[0118] Compared with the prior art, this embodiment has the following beneficial effects:

[0119] The imaging method for items inside the drawer acquires images of the drawer at different positions and combines them with distance information between these positions to perform image fusion. This reduces or eliminates image distortion caused by reflection, resulting in a clearer and more realistic image of the item. For distortion caused by surface reflection, since the degree of reflection varies at different locations, fusing images from different positions can eliminate the adverse effects of reflection. This method does not rely on excessive lighting conditions and does not require repeated adjustments to the light source. It can effectively reproduce the true image information of the item under different lighting environments, improving recognition accuracy and system fault tolerance. Compared to recognition methods using only a single image, this image fusion method can integrate image information from different angles and lighting conditions, thereby maximizing the reproduction of the item's true appearance. It performs particularly well in complex lighting environments, ultimately improving image realism and clarity. This, in turn, facilitates subsequent accurate identification of item information based on the clearer image, enhancing the user experience.

[0120] In one embodiment, a refrigeration device 100 is provided, and the connection method of the various electronic components within the refrigeration device 100 is as follows: Figure 6 The structural block diagram is shown. The refrigeration device 100 includes an imaging module 10 and a ranging module 20. The imaging module 10 can acquire a first image P1 when the drawer is in a first position S1, and a second image P2 when the drawer is in a second position S2. The ranging module 20 can determine the distance between the first position S1 and the second position S2. Furthermore, the processing module 40 of the refrigeration device 100 can fuse the images of the target item in the first image P1 and the second image P2 into a fused image P3.

[0121] It should be noted that the details of the different functions of these modules in the refrigeration device 100 of this embodiment of the invention are not disclosed. Please refer to the details disclosed in the imaging method for items in a drawer of this embodiment of the invention.

[0122] Those skilled in the art will understand that the structural block diagram is merely an example of the cooling device 100 and does not constitute a limitation on the terminal device of the cooling device 100. It may include more or fewer components than shown in the diagram, or combine certain components, or different components. For example, the cooling device 100 may also include a power module, input / output devices, network access devices, etc.

[0123] The device operating the imaging method for items inside a drawer may also include computing devices such as computers, laptops, PDAs, and cloud servers, as well as, but not limited to, a processing module 40, a storage module 30, and a computer program stored in the storage module 30 and executable on the processing module 40, such as the imaging method program for items inside a drawer described above. When the processing module 40 executes the computer program, it implements the steps in the various embodiments of the imaging method for items inside a drawer described above, for example... Figure 1 The steps are shown.

[0124] The imaging device for the items inside the drawer may also include a communication bus 50, which is used to establish a connection between the imaging module 10, the ranging module 20, the processing module 40 and the storage module 30. The communication bus 50 may include a path for transmitting information between the imaging module 10, the ranging module 20, the processing module 40 and the storage module 30.

[0125] In addition, the present invention also proposes an electronic device, which includes a storage module 30 and a processing module 40. When the processing module 40 executes the computer program, it can implement the steps in the above-mentioned imaging method for items inside a drawer, that is, implement the steps in any of the above-mentioned technical solutions of the imaging method for items inside a drawer.

[0126] The electronic device may be part of the cooling equipment 100, a local terminal device, or part of a cloud server.

[0127] The processing module 40 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processing module 40 is the control center of the refrigeration equipment 100, connecting all parts of the refrigeration equipment 100 via various interfaces and lines.

[0128] The storage module 30 can be used to store the computer programs and / or modules. The processing module 40 implements various functions of the refrigeration device 100 by running or executing the computer programs and / or modules stored in the storage module 30 and calling the data stored in the storage module 30. The storage module 30 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc. In addition, the storage module 30 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0129] For example, the computer program can be divided into one or more modules / units, which are stored in the storage module 30 and executed by the processing module 40 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the refrigeration device 100.

[0130] Furthermore, one embodiment of the present invention provides a readable storage medium storing a computer program, which, when executed by the processing module 40, can implement the steps in the above-described imaging method for items inside a drawer, that is, implement the steps in any of the technical solutions of the above-described imaging method for items inside a drawer.

[0131] If the module integrating the imaging method for the items inside the drawer is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processing module 40, it can implement the steps of the various method embodiments described above.

[0132] The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include any entity or device capable of carrying the computer program code, recording media, U disks, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0133] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0134] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of imaging items within a drawer, the method comprising: The method comprises the following steps: acquiring a first image when the drawer is in a first position; acquiring a second image when the drawer is in a second position; combining the distance between the first position and the second position to fuse the images of the target object in the first image and the second image into a fused image.

2. The method of imaging the contents of a drawer of claim 1, wherein, The method further comprises the following steps: establishing a mapping relationship between image pixel size and world size; combining the distance between the first position and the second position to fuse the images of the target object in the first image and the second image into a fused image comprises: determining the corresponding pixel movement distance on the image according to the distance between the first position and the second position and the mapping relationship; fusing the images of the target object in the first image and the second image into a fused image according to the pixel movement distance.

3. The method of imaging the contents of a drawer of claim 2, wherein, The method further comprises the following steps: acquiring a real-time video; when the drawer is in the second position, if a body is detected in the real-time video and the intersection-over-union of the body and the target object exceeds a coverage threshold, recording the start position and the end position of the body; determining the corresponding pixel movement distance on the image according to the start position, the end position, the distance between the first position and the second position, and the mapping relationship.

4. The method of imaging contents of a drawer of claim 1, wherein, The first position is a closed position in which the drawer is in a closed state, and the second position is a moved-out position in which the drawer is in a moved-out state.

5. The method of imaging the contents of a drawer of claim 4, wherein, The acquiring of the second image when the drawer is in the second position comprises: acquiring distance information, wherein the distance information is the distance between the current position of the drawer and the closed position; determining whether the drawer is moved out and stopped moving according to the distance information; if yes, acquiring the second image when the drawer is in the second position.

6. The method of imaging the contents of a drawer of claim 5, wherein, The determination of whether the drawer is moved out and stopped moving according to the distance information comprises: when the distance information changes and is greater than a distance error threshold, determining that the drawer is moved out; when the distance information stops changing and is still greater than the distance error threshold, determining that the drawer is moved out and stopped moving.

7. The method of imaging contents of a drawer of claim 1, wherein, The acquiring of the second image when the drawer is in the second position comprises: identifying the target object in the first image; when the information of the target object cannot be identified and the area in which the pixel brightness in the first image is greater than a pixel brightness threshold exceeds a reflection area threshold, acquiring the second image of the drawer when the drawer is in the second position.

8. The method of imaging the contents of a drawer of claim 7, wherein, The fusing of the images of the target object in the first image and the second image into a fused image comprises: extracting a reflection area in which the pixel brightness in the first image is greater than the pixel brightness threshold; replacing the reflection area in the second image with the same area in the reflection area to obtain the fused image.

9. The method of imaging the contents of a drawer of claim 7, wherein, The fusing of the images of the target object in the first image and the second image into a fused image comprises: calculating the reflection area in which the pixel brightness in the first image and the second image is greater than the pixel brightness threshold; determining the size of the reflection area in the first image and the second image respectively; The reflection region in the image with a smaller reflection region area is replaced by the same region in another image to obtain the fusion image.

10. The method of imaging the contents of a drawer of claim 7, wherein, Further comprising steps of: identifying the target article in the fusion image; when the information of the target article cannot be identified and the area with pixel brightness greater than the pixel brightness threshold in the fusion image exceeds the reflection region threshold, obtaining a new second image and fusing a new fusion image.

11. A refrigeration appliance characterized in that, comprising: a drawer; an imaging module for obtaining an image inside the drawer; a distance measuring module for obtaining a moving distance of the drawer; a storage module for storing a computer program; a processing module for implementing the steps in the imaging method of the article inside the drawer according to any one of claims 1 to 10 when executing the computer program.

12. A readable storage medium storing a computer program, characterized in that, The computer program is executed by the processing module to implement the steps in the imaging method of the article inside the drawer according to any one of claims 1 to 10.

Images