Article vacancy detection method, refrigeration equipment and storage medium

Abstract

The invention discloses an article vacancy detection method. The method comprises the following steps: acquiring distance data of a target article in a target area and preprocessing the distance data; correspondingly converting the distance data into vacancy data based on a preset conversion relation; and outputting the article vacancy information represented by the vacancy data. By acquiring the distance data and converting the distance data into the vacancy information in the target area, the display number of the articles in the refrigerator can be known more visually, the vacancy condition of the articles can be known more accurately, timely replenishment of workers is facilitated, the replenishment number can be better controlled, and the workload of the workers is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of object vacancy detection in refrigerators, and in particular to an object vacancy detection method, a refrigeration device and a storage medium. Background Art

[0002] The saturation of the merchandise display room in the refrigerator directly affects consumers' shopping experience and merchandise efficiency. Brand owners have high requirements for the display of merchandise in refrigerators. When the front row of merchandise is taken away, the brand owner will ask the terminal sorting staff to replenish the merchandise in time, or move the back row of merchandise forward to ensure that the front row of merchandise is displayed saturated.

[0003] Traditional refrigerator product display monitoring uses cameras to take pictures and directly recognize the image to determine whether there are empty spaces for products. This method is difficult to determine empty spaces when the products in the first row are taken away but there are still products in the back. Specifically, when the products in the first row are taken away and there are still products in the second row, it is difficult to determine the empty spaces for products from the perspective of image recognition. Summary of the invention

[0004] The purpose of the present invention is to provide an article vacancy detection method, refrigeration equipment and storage medium to solve the problem that the method in the prior art of using a camera to shoot the goods displayed in the refrigerator and then identifying whether there are vacant positions of the goods by the naked eye is easily affected by the goods in the back row and it is difficult to accurately judge the vacancy situation.

[0005] To achieve one of the above-mentioned objects of the invention, an embodiment of the present invention provides a method for detecting an empty space of an object, the method comprising: obtaining distance data of a target object in a target area and preprocessing the distance data;

[0006] Based on the preset conversion relationship, the distance data is converted into space data;

[0007] Output the item vacancy information represented by the vacancy data.

[0008] As a further improvement of the present invention, the method further includes: the step of obtaining the distance data of the target object in the target area specifically includes:

[0009] Transmitting a detection signal, and receiving an echo signal of the detection signal;

[0010] The distance between the detection source and the target object is calculated according to the time difference between the detection signal and the echo signal to form a data matrix.

[0011] As a further improvement of the present invention, the method further includes: preprocessing the distance data specifically includes:

[0012] Traversing each value in the data matrix;

[0013] It is determined whether the value is within a preset interval. If the value is smaller than the preset interval or the value is larger than the preset interval, the value is filtered.

[0014] As a further improvement of the present invention, the method further includes: forming a data matrix specifically includes:

[0015] Generate a matrix frame according to the resolution size, and fill each pixel point in the matrix frame with the corresponding distance value detected;

[0016] Based on the size of the target object within the target area, valid data in the matrix frame is extracted to form a data matrix.

[0017] As a further improvement of the present invention, the method further includes: before obtaining the distance data of the target object in the target area, it also includes setting a conversion relationship:

[0018] Dividing the empty positions in the target area according to the size of the target object;

[0019] Calculating the vertical distance between the empty position and the detection source and summarizing to obtain multiple distance values;

[0020] The interval between adjacent distance values ​​is set as the distance interval, and the vacancy data corresponding to each distance interval is established to form a conversion relationship between distance and vacancy.

[0021] As a further improvement of the present invention, the method further includes: further including shooting and acquiring an image of the target area.

[0022] As a further improvement of the present invention, the method further includes: displaying the item space information at a corresponding position on the image.

[0023] The present invention further provides a refrigeration device, wherein the refrigeration device has at least one storage space, part or all of the storage space forms a target area, and a target object is stored in the target area. The refrigeration device further comprises:

[0024] A sensor, the sensor being installed on the cabinet of the refrigeration equipment and being used to collect the distance of the target object;

[0025] A camera, the camera is used to capture an image of the target area;

[0026] A processor, the sensor and the camera are electrically connected to the processor, the processor is provided with a memory, the memory stores a computer program that can be run on the processor, and when the program is executed on the processor, the steps in the object vacancy detection method described in any one of the above items are implemented.

[0027] As a further improvement of the present invention, the refrigeration equipment further includes a display terminal, the display terminal is electrically connected to the processor, and the display terminal is used to receive and display the image taken by the camera and display the item space information on the image.

[0028] The present invention also provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any one of the above-mentioned object empty space detection methods are implemented.

[0029] Compared with the prior art, the present invention has the following beneficial effects: by acquiring distance data and converting it into vacancy information in the target area, the number of items displayed in the refrigerator can be more intuitively understood, and the vacancy status of items can be more accurately known, which helps the staff to replenish stocks in time and can better control the quantity of replenishment, thereby reducing the workload of the staff. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the empty space after the target object of the present invention is taken out from the target area.

[0031] Figure 2 Flow chart of a method for detecting an empty position of an object in one embodiment of the present invention.

[0032] Figure 3 is a flow chart of obtaining distance data in one embodiment of the present invention.

[0033] Figure 4 1 is a flow chart of forming a data matrix and preprocessing in one embodiment of the present invention.

[0034] Figure 5 It is a flow chart for setting the conversion relationship in one embodiment of the present invention.

[0035] Figure 6 It is a schematic diagram of the system structure of a refrigeration device in one embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0037] The present invention provides an article empty space detection method, a refrigeration device and a storage medium. The article empty space detection method can be applied to some refrigeration devices. In one embodiment, a refrigerator is used as an example for explanation. The refrigerator has multiple layers of vertically arranged storage spaces, and articles such as beverages are arranged in each layer of storage space.

[0038] The storage space is the target area detected by the detection method of the present invention, and the objects in the storage space are prevented from being the target objects detected by the detection method of the present invention.

[0039] like Figure 1 As shown, when the target object is taken out from the target area of ​​the storage space, empty spaces will appear in the target area. According to the detection method provided by the present invention, the empty spaces and the number of empty spaces in the target area can be accurately detected.

[0040] It should be noted that the target area is a partial area or the entire area of ​​the storage space. When the volume of the object only occupies a portion of the storage space, the target area is the size of the space occupied by the target object, that is, the partial area of ​​the storage space; when the volume of the object occupies the entire storage space, the target area is the size of the space occupied by the target object, that is, the entire area of ​​the storage space.

[0041] In one embodiment of the present invention, Figure 2 As shown, a method for detecting an empty space of an item is provided, comprising:

[0042] Step S1: Acquire distance data of target objects in a target area and pre-process the distance data.

[0043] The target area is understood to be the area where the target object is displayed. In the embodiment of the refrigerator, the target area is the storage space in the refrigerator, and the target object is the beverage or other objects displayed in the refrigerator.

[0044] The distance data is the distance between the target object and the detection source, and the detection source can be understood as a sensor 1 for detecting the distance.

[0045] In one embodiment, the sensor 1 may be a 3D TOF (Time of Flight) sensor. In other embodiments, the sensor 1 may be an ultrasonic radar, a laser radar, or a structured light sensor.

[0046] Preferably, the present invention uses a 3D TOF (Time of Flight) sensor.

[0047] 3D TOF (Time of Flight) sensor is a depth sensing technology, which can be understood as a sensor that measures the flight time between emitted light and reflected light and directly estimates the depth based on the speed of light. The sensor 1 continuously sends light pulses to the target, then uses the sensor 1 to receive the light fed back from the object and calculates the round-trip flight time of sending and receiving the light pulses, and finally obtains the distance of the target.

[0048] Furthermore, the sensor 1 also converts the distance of the target object by calculating the time difference or phase difference between the emission and reflection of the optical fiber, thereby generating distance data of the target object from the sensor 1, and further obtaining 3D visual information.

[0049] The preprocessing can be understood as filtering out noise and clutter in the distance data to make the data more accurate.

[0050] Step S2: Based on a preset conversion relationship, the distance data is converted into corresponding space data.

[0051] The conversion relationship is the corresponding relationship between the distance of the target object and the empty space of the target object in the target area.

[0052] It should be noted that the display positions of the target objects in the target area are arranged in rows in a direction away from the sensor 1. For example, the first row of target objects in the target area is close to the refrigerator door, that is, close to the sensor 1; the last row of target objects in the target area is located deep in the storage space, that is, away from the refrigerator door and away from the sensor 1.

[0053] In the same straight line direction deep into the storage space, the farther the target object is from the detection source, the more empty spaces there are between the target object and the detection source for quality inspection in this direction. A conversion relationship is formed based on this relationship.

[0054] Step S3: Outputting the item vacancy information represented by the vacancy data.

[0055] The item vacancy information indicates the number of item vacancies in the target area. For example, when the first item in the first row in the target area of ​​the storage space of the refrigerator is empty, the first item in the second row is also empty, and the second item in the first row is empty. Then, according to the distance data obtained by the detection, the vacancy data is converted into two vacancies in the first column and one vacancies in the second column in the target area.

[0056] In this way, by obtaining distance data and converting it into vacancy information in the target area, we can more intuitively understand the number of items on display in the refrigerator and more accurately know the vacancy status of items, which helps staff to replenish stocks in time and better control the quantity of replenishment, reducing the workload of staff.

[0057] like Figure 3 As shown, in one embodiment of the present invention, the obtaining of distance data of a target object in a target area specifically includes:

[0058] Step S1.1: transmitting a detection signal, and receiving an echo signal of the detection signal;

[0059] Step S1.2: Calculate the distance between the detection source and the target object according to the time difference between the detection signal and the echo signal and form a data matrix.

[0060] In one embodiment, the detection signal is an infrared detection light. In another embodiment, the detection signal is an ultrasonic wave.

[0061] The echo signal is the signal returned after the detection signal contacts the target object. The distance of the target object is calculated based on the returned signal and the transmitted signal.

[0062] Furthermore, the phase difference between the returned signal and the transmitted signal can be obtained to obtain 3D visual information, thereby obtaining more accurate distance data.

[0063] The data matrix is ​​a matrix set formed by distance data corresponding to multiple pixel points on the target object in the target area detected by the detection signal. Each distance value in the matrix represents the distance of each pixel on the target object.

[0064] It should be noted that the target area may include multiple target objects, so the distance data in the matrix includes the distances of multiple target objects, including the distance data of target objects at close range and the distance data of target objects at long range.

[0065] The number of data in the data matrix depends on the resolution of the detection source (sensor 1 of the present invention). The smaller the resolution, the less data contained in the matrix, that is, the less distance data representing a target object; conversely, the more data contained.

[0066] like Figure 4 As shown, in one embodiment of the present invention, forming a data matrix specifically includes:

[0067] Step L1: Generate a matrix frame according to the resolution size, and fill the detected corresponding distance value at each pixel point in the matrix frame.

[0068] Each pixel in the matrix box represents a point in the field of view of sensor 1. The target object is detected in real time, and the distance value of the target object at each pixel is measured, and the measured distance value is filled into the matrix box of the corresponding pixel.

[0069] For example, if the resolution of sensor 1 is 32*32, the size of the matrix frame is 32*32, which includes 32*32 pixels, and each pixel represents the distance data returned when the detection signal encounters a target item or other object in the target area.

[0070] Step L2: Based on the size of the target object within the target area, extract valid data in the matrix frame to form a data matrix.

[0071] According to the size of the target object in the detected field of view, the valid data area is determined, and the pixels in the area and their corresponding distance values ​​are extracted to form a data matrix. In this way, invalid data or unimportant data (data of non-target objects) in the field of view can be eliminated.

[0072] In the above scheme, by accurately filling the distance value of each pixel point, the system can more accurately identify and measure the target object, and can also flexibly adjust the size of the matrix frame and the data extraction area according to the resolution of different detection sources (sensor 1 of the present invention) and the size of the target object.

[0073] In one implementation of the present invention, preprocessing the distance data specifically includes:

[0074] Step L3: traverse each value in the data matrix.

[0075] Step L4: Determine whether the value is within a preset interval. If the value is smaller than the preset interval or the value is larger than the preset interval, filter the value.

[0076] For example, the preset interval can be set to [50cm, 300cm] to filter the values ​​in the data matrix. Iterate through each value in the data matrix and filter out the values ​​that are less than 50cm or greater than 300cm and replace them with 0.

[0077] It should be noted that the preset interval can be adjusted according to different application scenarios to make the system more flexible and adaptable.

[0078] In this way, the present invention can exclude values ​​that do not meet the conditions and reduce misjudgment by filtering in a preset interval. The filtering operation reduces unnecessary data processing, filters out noise, and improves the overall operating efficiency of the system.

[0079] like Figure 5 As shown, in one embodiment of the present invention, before obtaining the distance data of the target object in the target area, the method further includes setting a conversion relationship:

[0080] Step K1: dividing the empty positions in the target area according to the size of the target object.

[0081] The target area is divided into spaces in advance according to the size of the target object, so that the space in the target area is arranged in a preset manner to facilitate the storage of the target object.

[0082] Step K2: Calculate the vertical distance between the empty position and the detection source and summarize to obtain multiple distance values.

[0083] In one embodiment, the vertical distance between each vacant position and the detection source (the sensor 1 of the present invention) can be calculated, and the same vertical distances can be combined and aggregated to form a plurality of different distance values.

[0084] In another embodiment, the vertical distance from the detection source (sensor 1 of the present invention) is calculated only for the vacant space in a single straight line direction away from the detection source. For example, the vacant space in the target area is divided into a matrix cell form, i.e., including n rows and m columns, and the distance from the detection source is calculated only for the vacant space in a single column direction.

[0085] It should be noted that the direction of calculation is the distance from the center of the vacancy to the detection source, and because the vacancy has a certain length and width, the calculated distance can be set with a certain error range.

[0086] Step K3: setting the interval between adjacent distance values ​​as the distance interval, and establishing the vacancy data corresponding to each distance interval to form a conversion relationship between distance and vacancy.

[0087] For example, if the calculated and summarized distance values ​​include [150, 200, 250, 300], then the distance intervals are [0, 150], [150, 200], [200, 250], [250, 300]. The corresponding [0, 150] means no vacancy, [150, 200] means one vacancy in the first row, [200, 250] means two vacancy in the first and second rows, [250, 300] means three vacancy in the first to third rows, and so on.

[0088] In this way, by converting distance data into space data, the data processing process is simplified and the computational complexity is reduced. The conversion relationship makes data processing more intuitive and efficient, and speeds up target detection. The conversion relationship can be adjusted according to different application scenarios and target object characteristics, improving the adaptability of the system.

[0089] In one embodiment of the present invention, the method further includes shooting and acquiring an image of the target area, so that the situation inside the refrigerator can be acquired in combination with the shot image, so that the staff can check the real situation.

[0090] Furthermore, the item vacancy information is displayed at the corresponding position on the image. In this way, the vacancy situation is displayed on the image, which is convenient for the staff to check the situation by comparing the data and the image, and can judge whether the data is wrong according to the real situation on the image, so as to quickly feedback the system problem and facilitate emergency maintenance.

[0091] like Figure 6As shown, in one embodiment of the present invention, a refrigeration device is provided, which is used to store cold chain items, and the vacancy status of items in the refrigeration device can be detected by an item vacancy detection method.

[0092] In some embodiments, the refrigeration equipment may include freezers, refrigerators, vending cabinets, etc.

[0093] The refrigeration device has at least one storage space, part or all of which forms a target area, and a target object is stored in the target area. The refrigeration device further includes:

[0094] The sensor 1 is installed on the cabinet of the refrigeration equipment and is used to collect the distance of the target object.

[0095] Specifically, the sensor 1 detects the target object by emitting a detection signal, and receives an echo signal when the signal returns from the target object, and then measures the distance of the target object through the detection signal and the echo signal.

[0096] In one embodiment, a D TOF (Time of Flight) sensor is used. In other embodiments, an ultrasonic radar, a laser radar, or a structured light sensor may also be used.

[0097] The camera 2 is used to capture an image of the target area.

[0098] A processor 3, the sensor 1 and the camera 2 are electrically connected to the processor 3, the processor 3 is provided with a memory 4, the memory 4 stores a computer program that can be run on the processor 3, and when the program is executed on the processor 3, the steps in the object empty space detection method described in any one of the above embodiments are implemented.

[0099] In one embodiment, the sensor 1 and the camera 2 may be installed at positions directly opposite to the target area.

[0100] In another embodiment, the installation position of the sensor 1 may be directly opposite to the target area. The installation position of the camera 2 may be directly above the target area to take a bird's-eye view of the target area.

[0101] In another embodiment, the installation positions of the sensor 1 and the camera 2 can both be located directly above the target area.

[0102] Other embodiments can be installed and set according to the actual situation and will not be described in detail here.

[0103] In one embodiment of the present invention, the refrigeration device further comprises a display terminal 5, wherein the display terminal 5 is electrically connected to the processor 3, and the display terminal 5 is used to receive and display the image captured by the camera 2 and display the item space information on the image.

[0104] Specifically, the display terminal 5 may be a display screen installed on the refrigeration equipment, or a mobile terminal device, such as a mobile phone, or a remote monitoring device, such as a remote host computer.

[0105] The way in which the sensor 1 , the camera 2 and the display terminal 5 are electrically connected to the processor 3 includes wire connection and communication connection.

[0106] In one embodiment of the present invention, a storage medium is further provided, wherein the storage medium stores a computer program, and when the computer program is executed by the processor 3, the steps in the method for detecting an empty position of an object in any one of the above embodiments are implemented.

[0107] To sum up, in order to solve the problem that it is difficult to intuitively monitor the vacancy situation after some of the items stored in refrigeration equipment such as refrigerators are taken away, the present invention provides an item vacancy detection method, which can obtain distance data by detecting the distance between the item and the sensor 1, and convert it into vacancy data through a preset conversion relationship, thereby obtaining the current vacancy situation in the target area, which is convenient for staff to replenish stocks in time.

[0108] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of hardware plus software functional modules. The above-mentioned integrated module implemented in the form of software functional modules can be stored in a computer-readable storage medium. The above-mentioned software functional modules are stored in a storage medium, including a number of instructions for a computer system (which can be a personal computer, a server, or a network system, etc.) or a processor (processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0109] It should be understood that although this specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation mode may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0110] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting an empty space for an object, It is characterized in that include: Acquiring distance data of target objects in a target area and preprocessing the distance data; Based on the preset conversion relationship, the distance data is converted into space data; Output the item vacancy information represented by the vacancy data.

2. The article space detection method according to claim 1, It is characterized in that The obtaining of distance data of target objects in the target area specifically includes: Transmitting a detection signal, and receiving an echo signal of the detection signal; The distance between the detection source and the target object is calculated according to the time difference between the detection signal and the echo signal to form a data matrix.

3. The article empty space detection method according to claim 2, It is characterized in that The distance data preprocessing specifically includes: Traversing each value in the data matrix; It is determined whether the value is within a preset interval. If the value is smaller than the preset interval or the value is larger than the preset interval, the value is filtered.

4. The article empty space detection method according to claim 2, It is characterized in that The formation of the data matrix specifically includes: Generate a matrix frame according to the resolution size, and fill each pixel point in the matrix frame with the corresponding distance value detected; Based on the size of the target object within the target area, valid data in the matrix frame is extracted to form a data matrix.

5. The article empty space detection method according to claim 1, It is characterized in that Before obtaining the distance data of the target object in the target area, the conversion relationship is also set: Dividing the empty positions in the target area according to the size of the target object; Calculating the vertical distance between the empty position and the detection source and summarizing to obtain multiple distance values; The interval between adjacent distance values ​​is set as the distance interval, and the vacancy data corresponding to each distance interval is established to form a conversion relationship between distance and vacancy.

6. The article empty space detection method according to claim 1, It is characterized in that It also includes photographing and acquiring images of the target area.

7. The article empty space detection method according to claim 6, It is characterized in that The item space information is displayed at a corresponding position on the image.

8. A refrigeration device, It is characterized in that The refrigeration device has at least one storage space, part or all of the storage space forms a target area, and the target items are stored in the target area. The refrigeration device also includes: A sensor (1), the sensor (1) being mounted on a cabinet of the refrigeration device and used for collecting the distance of the target object; A camera (2), the camera (2) being used to capture an image of the target area; A processor (3), the sensor (1) and the camera (2) are electrically connected to the processor (3), a memory (4) is provided in the processor (3), a computer program that can be run on the processor (3) is stored in the memory (4), and when the program is executed on the processor (3), the steps in the object empty position detection method according to any one of claims 1 to 7 are implemented.

9. The refrigeration device according to claim 8, It is characterized in that The refrigeration device further comprises a display terminal (5), wherein the display terminal (5) is electrically connected to the processor (3), and the display terminal (5) is used to receive and display the image captured by the camera (2) and to display the item space information on the image.

10. A storage medium, It is characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor (3), the steps of the object empty space detection method as claimed in any one of claims 1 to 7 are implemented.

Images