Vertical air-cooled display cabinet and image processing method thereof

Abstract

The invention relates to the technical field of commodity intelligent identification, and provides a vertical air-cooled display cabinet and an image processing method thereof, through software algorithm adaptation, an image is processed based on a Retinex algorithm, a Gaussian filter in a traditional Retinex algorithm is replaced by a bilateral filter, the obtained image is optimized in a targeted manner, the problem of image fogging is solved, and the image processing efficiency is improved. And by controlling the shooting time of the camera, pictures are shot at different shooting frequencies in a plurality of time sections, and the pick-and-place conditions of articles in the cabinet are obtained only through the camera, so that the production cost is effectively controlled.

Description

Technical Field

[0001] The invention relates to the technical field of intelligent commodity recognition and provides a vertical air-cooled display cabinet and an image processing method thereof. Background Art

[0002] In recent years, the demand for vertical air-cooled display cabinets equipped with camera modules has gradually increased, which makes it convenient for manufacturers to quickly view the display and analyze sales. As a key part of the vertical air-cooled display cabinet, the commodity recognition system is used to obtain the type and quantity information of the commodities purchased on the day, and use the above information to calculate the sales volume of the day. At present, there are two types of solutions for commodity recognition systems. One is to combine camera vision acquisition with gravity recognition of gravity sensing module, and the other is to use camera vision acquisition. The latter has greater advantages in maintenance, service life and cost.

[0003] Currently, there are smart vending cabinets and smart display cabinets on the market that use camera vision acquisition. However, the control method of camera shooting is insufficient. It may not be possible to clearly capture the placement and taking of items in the cabinet. In addition, due to the large temperature difference between the inside and outside of the cabinet, after the cabinet door is opened, the camera installed on the cabinet door is prone to fogging, affecting the clarity of the photo, resulting in a low recognition success rate during use.

[0004] Therefore, how to provide a display cabinet that can clearly capture the placement and taking of items in the cabinet using only camera vision, obtain a clear image by processing the image, and integrate it into the entire recognition system is a technical problem that urgently needs to be solved. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the first purpose of the present invention is to provide an image processing method for a vertical air-cooled display cabinet to remove fog from the image. The second purpose of the present invention is to provide a vertical air-cooled display cabinet that can clearly capture the status of taking and placing items in the cabinet using only a camera.

[0006] In order to solve the first problem of the present invention, the present invention provides an image processing method for a vertical air-cooled display cabinet, characterized in that it includes:

[0007] S1: Get the original image S(i,j) from the original video and original photo, divide the color image into three channels of red, green and blue, extract the pixel values ​​of the three color channels of red, green and blue, and convert the pixel values ​​to the logarithmic domain;

[0008] S2: Decompose the image based on the Retinex algorithm, decompose the image into a reflection image and a brightness image, separate the reflection image,

[0009]

[0010] Where Sc (i, j) is the original image of each channel, R c (i, j) is the reflection image of each channel, L(i, j) is the brightness image, r, g, b are the red, green and blue channels;

[0011] S3: Determine the weight and λ of each pixel value in the filter window;

[0012] S4: Replace the Gaussian filter in the traditional Retinex algorithm with a bilateral filter to correct the brightness component.

[0013]

[0014] Where p∈S(i,j) is the pixel p in the domain S(i,j), σ s is the Gaussian standard deviation of the original image S, σ r is the Gaussian standard deviation in the spatial domain, I i is the horizontal coordinate of the point to be solved, I j is the ordinate with the solution point, * is the convolution symbol;

[0015] S5: Merge the reflection images of each channel and convert the reflection images from the logarithmic domain to the real domain to obtain the output image.

[0016] r(i,j)=logR(i,j)

[0017] Where r(i,j) is the corrected reflected image;

[0018] S6: The output image is linearly stretched and the output is displayed in a corresponding format to obtain an image after defogging.

[0019] Specifically, in step S3, the weight and λ of each pixel value are calculated as follows:

[0020] S3.1: Calculate local contrast,

[0021]

[0022] Where D(i,j) is the local contrast, A max is the maximum gray value, A min is the minimum gray value;

[0023] S3.2: According to the local contrast of each pixel, the membership of low detail area, medium detail area and high detail area is calculated respectively.

[0024] η e =4sigm{α e [D(i,j)-β e ]}×sigm{-α e[D(i,j)-β e ]}

[0025] Where α e is the pixel gray value coefficient, β e is the local contrast coefficient, D(i,j)∈[0,1], e∈{1,2,3}, η1, η2 and η3 are the low-detail region membership, medium-detail region membership and high-detail region membership respectively;

[0026] S3.3: Calculate the weight and λ of each pixel value in the filter window for weight normalization,

[0027]

[0028] In the formula, λ low is the low detail area weight, λ mid is the weight of the medium detail area, λ high Weights the high detail areas.

[0029] Specifically, the λ low =1.5, the λ mid =1.5, the λ high =3.

[0030] In order to solve the second problem of the present invention, the present invention provides a vertical air-cooled display cabinet, characterized in that it includes:

[0031] A cabinet body, wherein a storage cavity is arranged in the cabinet body, and the cabinet body has an opening;

[0032] A cabinet door, which is movably connected to the cabinet body and is used to cover the opening. A Hall switch for detecting the door opening and closing status is provided at the cabinet door;

[0033] A first image acquisition unit, which is arranged inside the cabinet and records an original video of the cabinet door being opened;

[0034] A second image acquisition unit, the second image acquisition unit is arranged at the opening and closing position of the cabinet door, and the second image acquisition unit takes a plurality of original photos of the storage cavity;

[0035] A processor is provided with an image sensor, a digital signal processing chip and a control module, and uses the image processing method described above to process the collected original photos and original videos. The processor is also provided with a data processing module, and the data processing module obtains and filters the time interval between the on and off of the Hall switch.

[0036] Specifically, the second image acquisition unit takes original photos at different shooting frequencies in multiple time segments, and the processor obtains the last photo of each time segment respectively.

[0037] Specifically, the time segment is determined according to the average door opening time T, and includes at least a first time segment and a second time segment, wherein the time of the first time segment is 1 / 2 of the average door opening time T. arrive

[0038] Specifically, the data processing module is provided with a minimum opening time t a , the door opening time interval is less than the minimum door opening time t a The data is invalid data, the data processing module obtains the most recent m time intervals, and then screens out the opening time with the longest n time intervals to obtain mn valid door opening times. The average value is calculated based on the mn valid door opening times to obtain the average door opening time T.

[0039] Specifically, in the first time segment, each t b One photo is taken every t seconds. c Take a photo per second, the t b >t c .

[0040] Specifically, the cameras of the first image acquisition unit and the second image acquisition unit are provided with a hydrophobic nano-ion plating film.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. The present invention processes images based on the Retinex algorithm through software algorithm adaptation, and replaces the Gaussian filter in the traditional Retinex algorithm with a bilateral filter to correct the brightness component, optimize the acquired image in a targeted manner, solve the problem of image fogging, improve the clarity of the image, and control the production cost.

[0043] 2. The present invention controls the shooting time of the camera, takes photos at different shooting frequencies in multiple time segments, and obtains the placement status of items in the cabinet only through the camera. There is no need to install additional equipment such as gyroscopes, thereby saving production costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of the present invention;

[0045] Figure 2 It is a schematic diagram of the door structure;

[0046] Figure 3 This is a schematic diagram of the cabinet structure. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of some embodiments of the present invention clearer, the technical solutions of some embodiments of the present invention will be clearly and completely described below in conjunction with the drawings of some embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. And in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Unless otherwise defined, the technical or scientific terms used in the present invention shall have the usual meanings understood by persons with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "include" or "comprise" and the like mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" and the like are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0049] The present invention provides a vertical air-cooled display cabinet, comprising a cabinet body and a cabinet door movably connected to the cabinet body, wherein a storage cavity for storing commodities is arranged in the cabinet body, the storage cavity is a box body with an opening, and the door body is arranged at the opening to cover the opening. At least one first image acquisition unit 1 is installed in the cabinet body, and the first image acquisition unit 1 is a camera with a recording function, which can be arranged at the top or bottom of the box body or at the four corners of the inner side of the box body, and photographs the taking of commodities in the storage cavity from the inner side of the storage cavity. At least two second image acquisition units 2 are arranged at the opening and closing of the cabinet door, and the second image acquisition unit 2 is a camera with a photo taking function, and the second image acquisition unit 2 forms a certain angle with the cabinet door, and photographs the taking of commodities in the storage cavity from the outer side of the storage cavity.

[0050] The refrigerator also includes a controller, a memory, and an image acquisition interface. The controller includes a processor, a communication interface, and a communication bus. The controller is used to control the operation and operation of the above-mentioned devices, as well as the communication and cooperation between the internal components, and the external and internal data processing functions. The image acquisition interface is used to transmit the images and videos collected by the first image acquisition unit 1 and the second image acquisition unit 2 to the processor. The processor is provided with an image sensor, a digital signal processing chip, and a control module. The optical image generated by the scene through the lens is projected onto the surface of the image sensor, converted into an electrical signal, converted into a digital image signal after A / D (digital-to-analog conversion), and then processed in a digital signal processing chip (DSP), and finally output through the control module.

[0051] In order to optimize the photos and videos collected by the first image acquisition unit 1 and the second image acquisition unit 2 in a targeted manner and solve the problem of poor image clarity caused by lens fogging, the acquired images are optimized in a targeted manner through software algorithm adaptation. The present invention replaces the Gaussian filter in the traditional Retinex algorithm with a bilateral filter, and uses the improved Retinex algorithm with edge-preserving bilateral filtering to perform image enhancement processing.

[0052] Retinex theory holds that the color and brightness of objects perceived by humans are the result of the interaction between light and matter. The color and brightness of objects perceived by the human eye are determined by the reflective properties of the object's surface, and have nothing to do with the spectral properties projected to the human eye. The Retinex image enhancement algorithm removes the influence of the illumination component in the original image and solves the reflective component that reflects the essential color of the object, thus achieving the purpose of image enhancement.

[0053] Retinex theory believes that an image can be expressed as the product of the reflection component R(i,j) and the illumination component L(i,j). The object's ability to reflect the three primary colors of red, green, and blue determines the color of the object observed by the human eye, so we get the following formula:

[0054] S(i,j)=R(i,j)·L(i,j)

[0055] Where S(i,j) is the original image, which represents the reflected light image visible to the naked eye; R(i,j) is the reflected image, that is, the internal attribute of the image; L(i,j) is the brightness image, and the dynamic transformation range of the pixels in the image is directly constrained by it; (i,j) is the spatial two-dimensional coordinate of the pixel point.

[0056] The bilateral filtering algorithm is a nonlinear filtering algorithm based on local information of an image. Its principle is to multiply two Gaussian functions, one of which is related to the spatial distance, and the other is related to the grayscale distance. It has the advantages of maintaining edges, reducing noise, and smoothing. When processing each pixel value, the intensity of the pixel is represented by the weighted average of the pixels around it. The most important point is that the bilateral filtering takes into account the Euclidean distance of the pixel (while the ordinary Gaussian low-pass filter only considers the influence of the position on the central pixel), and also takes into account the radiation difference. These two weights are considered when calculating the central pixel. The closer the pixel value, the greater the weight. The two weight kernels of spatial proximity and grayscale similarity are nonlinearly combined, which not only takes into account the spatial relationship between the central pixel and the surrounding pixels, but also takes into account the similarity of the grayscale values ​​between adjacent pixels, so as to achieve the effect of adaptive filtering.

[0057] The improved Retinex algorithm steps are as follows

[0058] S1: Obtain the original image S(i,j) from the original video and original photo, divide the color image into three channels of red, green and blue, extract the pixel values ​​of the three color channels of red, green and blue, and convert the pixel values ​​to the logarithmic domain.

[0059] Operations in the logarithmic domain are closer to the visual characteristics of the human eye and can convert multiplication operations into simple addition operations.

[0060] S2: Decompose the image based on the Retinex algorithm, decompose the image into a reflection image and a brightness image, separate the reflection image,

[0061]

[0062] Where S c (i, j) is the original image of each channel, R c (i, j) is the reflection image of each channel, L(i, j) is the brightness image, and r, g, and b are the red, green, and blue channels.

[0063] S3: Determine the weight and λ of each pixel value in the filter window.

[0064] S3.1: Calculate local contrast,

[0065]

[0066] Where D(i,j) is the local contrast, A max is the maximum gray value, A min is the minimum gray value.

[0067] S3.2: According to the local contrast of each pixel, the membership of the low detail area, medium detail area and high detail area is calculated respectively. None of the three areas of low, medium and high detail can be accurately distinguished. Therefore, the local contrast of each pixel is needed to calculate the size of the membership of the three areas.

[0068] η e =4sigm{α e [D(i,j)-β e ]}×sigm{-α e [D(i,j)-β e ]}

[0069] In the formula, α e is the pixel gray value coefficient, β e is the local contrast coefficient, D(i,j)∈[0,1], e∈{1,2,3}, η1, η2 and η3 are the low-detail region membership, medium-detail region membership and high-detail region membership respectively.

[0070] After a large number of experiments, it was found that when α1=α3=20, α2=30, β1=0, β2=0.8, β3=1, better effects were achieved.

[0071] S3.3: The membership degree calculated by the formula is used as the weight to obtain the weight and λ of each pixel value. The weight and λ of each pixel value are used for weight normalization.

[0072]

[0073] In the formula, λ low is the low detail area weight, λ mid is the weight of the medium detail area, λ high is the weight of the high-detail area. It is found through experiments that when λ low =λ mid =1.5,λ high When =3, image enhancement can achieve a relatively ideal effect.

[0074] S4: Replace the Gaussian filter in the traditional Retinex algorithm with a bilateral filter to correct the brightness component.

[0075]

[0076] Where p∈S(i,j) is the pixel p in the domain S(i,j), σ s is the Gaussian standard deviation of the original image S, σ r is the Gaussian standard deviation in the spatial domain, I i is the horizontal coordinate of the point to be solved, I j is the ordinate with the solution point, and * is the convolution symbol.

[0077] S5: Merge the reflection images of each channel and convert the reflection images from the logarithmic domain to the real domain to obtain the output image.

[0078] r(i,j)=logR(i,j)

[0079] Where r(i,j) is the corrected reflected image;

[0080] S6: The output image is linearly stretched and the output is displayed in a corresponding format to obtain an image after defogging.

[0081] A Hall switch 3 is provided at the cabinet door. The Hall switch 3 is provided at the bottom or top of the cabinet door to detect the switch status of the cabinet door. The Hall switch 3 can output two digital quantities, 0 for conduction and 1 for interruption. A data processing module is also provided in the processor. The Hall switch 3 is connected to the processor signal. The data processing module processes the digital quantity and records the time interval when the digital quantity changes from 0 to 1 and then to 0, that is, the time interval when the cabinet door is opened.

[0082] The first image acquisition unit 1 and the second image acquisition unit 2 are controlled by the processor to shoot and record. The first image acquisition unit 1 records the original video of the cabinet door being open. After the cabinet door is opened, the first image acquisition unit 1 starts recording, and stops recording after the cabinet door is closed. The second image acquisition unit 2 takes original photos at different shooting frequencies in multiple time segments in the state where the cabinet door is open, and obtains the last photo of each time segment respectively.

[0083] The data processing module records, counts and filters the time interval t of the cabinet door opening, and based on a certain number of time intervals, the data processing module obtains the most recent m time intervals to calculate the average door opening time T, and sets the time segment based on the average door opening time T. The m time intervals can be 10 to 100 times. Considering that the cabinet door may be opened by mistake or staggered, the cabinet door may be quickly closed after it is opened, and no actual purchase behavior occurs. The time for the above-mentioned rapid door opening and closing is an invalid door opening time, which will significantly shorten the average door opening time T, so the above-mentioned time interval needs to be screened out from the statistical data. Therefore, the minimum opening time t is set. a , the data processing module will open the door time interval t less than the minimum opening time t a Data filtering, t a It can be 1-3 seconds. Considering that some customers may greatly extend the time interval t of cabinet door opening due to choice difficulties or forgetting to close the cabinet door, in order to reduce the impact of the above situation on the average door opening time T, the opening time with the longest n time intervals is screened out from the obtained time intervals. The number of n is set according to the number of m counted. For example, when m is 10, n can be 1-3. For example, when m is 100, n can be 10-20. Get mn valid door opening times, and calculate the average value based on the mn valid door opening times to get the average door opening time T.

[0084] There are at least two time segments. In different time segments, the second image acquisition unit 2 takes original photos at different frequencies, and the processor obtains the last photo of each time segment. The time segment includes at least a first time segment and a second time segment. The first time segment is the average door opening time T. arrive During this time period, customers mainly select commodities, and the second image acquisition unit 2 takes pictures at a lower frequency every t b The second time segment is a segment where the door opening time is greater than the door opening time. In this time segment, the second image acquisition unit 2 takes a photo at a higher frequency every t cA photo is taken every 0.5-2 seconds, for example. Further, a third time segment can be set. The time of the third time segment is much longer than the average door opening time, for example, 5-10 times the average door opening time. There may be a situation where the cabinet door is forgotten to be closed. In this time segment, a photo is taken every t at a low frequency. d Take a photo in seconds.

[0085] The cameras of the first image acquisition unit 1 and the second image acquisition unit 2 are provided with a hydrophobic nano-ion plating film to reduce fog at the cameras.

[0086] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. An image processing method for a vertical air-cooled display cabinet, characterized in that: include: S1: Get the original image S(i,j) from the original video and original photo, divide the color image into three channels of red, green and blue, extract the pixel values ​​of the three color channels of red, green and blue, and convert the pixel values ​​to the logarithmic domain; S2: Decompose the image based on the Retinex algorithm, decompose the image into a reflection image and a brightness image, separate the reflection image, Where S c (i, j) is the original image of each channel, R c (i, j) is the reflection image of each channel, L(i, j) is the brightness image, r, g, b are the red, green and blue channels; S3: Determine the weight and λ of each pixel value in the filter window; S4: Replace the Gaussian filter in the traditional Retinex algorithm with a bilateral filter to correct the brightness component. Where p∈S(i,j) is the pixel p in the domain S(i,j), σ s is the Gaussian standard deviation of the original image S, σ r is the Gaussian standard deviation in the spatial domain, I i is the horizontal coordinate of the point to be solved, I j is the ordinate with the solution point, * is the convolution symbol; S5: Merge the reflection images of each channel and convert the reflection images from the logarithmic domain to the real domain to obtain the output image. r(i,j)=log R(i,j) Where r(i,j) is the corrected reflected image; S6: The output image is linearly stretched and the output is displayed in a corresponding format to obtain an image after defogging.

2. The image processing method of the vertical air-cooled display cabinet according to claim 1, characterized in that: In step S3, the weight and λ of each pixel value are calculated as follows, S3.1: Calculate local contrast, Where D(i, j) is the local contrast, A max is the maximum gray value, A min is the minimum gray value; S3.2: According to the local contrast of each pixel, the membership of low detail area, medium detail area and high detail area is calculated respectively. or e =4sigm{α e [D(i,j)-β e ]}×sigm{-α e [D(i,j)-β e ]} Where α e is the pixel gray value coefficient, β e is the local contrast coefficient, D(i, j)∈[0,1], e∈{1, 2, 3}, η1, η2 and η3 are the low-detail region membership, medium-detail region membership and high-detail region membership respectively; S3.3: Calculate the weight and λ of each pixel value in the filter window for weight normalization, In the formula, λ low is the low detail area weight, λ mid is the weight of the medium detail area, λ high Weights the high detail areas.

3. The image processing method of the vertical air-cooled display cabinet according to claim 2, characterized in that: The lambda low =1.5, the λ mid =1.5, the λ high =3.

4. A vertical air-cooled display cabinet, characterized in that: include: A cabinet body, wherein a storage cavity is arranged in the cabinet body, and the cabinet body has an opening; A cabinet door, which is movably connected to the cabinet body and is used to cover the opening. A Hall switch for detecting the door opening and closing status is provided at the cabinet door; A first image acquisition unit, which is arranged inside the cabinet and records an original video of the cabinet door being opened; A second image acquisition unit, the second image acquisition unit is arranged at the opening and closing position of the cabinet door, and the second image acquisition unit takes a plurality of original photos of the storage cavity; A processor is provided with an image sensor, a digital signal processing chip and a control module, and uses the image processing method according to any one of claims 1 to 3 to process the collected original photos and original videos. The processor is also provided with a data processing module, and the data processing module obtains and filters the time interval between the on and off of the Hall switch.

5. The vertical air-cooled display cabinet according to claim 4, characterized in that: The second image acquisition unit takes original photos at different shooting frequencies in multiple time segments, and the processor obtains the last photo of each time segment respectively.

6. The vertical air-cooled display cabinet according to claim 5, characterized in that: The time segment is determined according to the average door opening time T, and at least includes a first time segment and a second time segment, wherein the time of the first time segment is the average door opening time T. arrive 7. The vertical air-cooled display cabinet according to claim 6, characterized in that: The data processing module is provided with a minimum opening time t a , the door opening time interval is less than the minimum door opening time t a The data is invalid data, the data processing module obtains the most recent m time intervals, and then screens out the opening time with the longest n time intervals to obtain mn valid door opening times. The average value is calculated based on the mn valid door opening times to obtain the average door opening time T.

8. The vertical air-cooled display cabinet according to claim 6, characterized in that: In the first time segment, each t b One photo is taken every t seconds. c Take a photo per second, the t b >t c .

9. The vertical air-cooled display cabinet according to claim 4, characterized in that: The cameras of the first image acquisition unit and the second image acquisition unit are provided with a hydrophobic nano-ion plating film.

Images