Intelligent Cabinet, Article Identification Method, Electronic Device, Medium and Product

Abstract

Embodiments of the present disclosure disclose an intelligent cabinet, an item recognition method, an electronic device, a medium, and a product. The intelligent cabinet includes: a first camera group including at least one first camera; N other camera groups, each other camera group including at least one camera, and the focal lengths of the cameras in different camera groups are different, where N is an integer greater than or equal to 2; wherein, the focal length of the first camera is less than the focal lengths of the cameras in the N other camera groups, and the first camera group and the N other camera groups are located on the same plane inside the intelligent cabinet. This technical solution has low cost and can improve the accuracy of item recognition while enhancing the image processing speed.

Description

Technical Field

[0001] The present disclosure relates to the technical field of intelligent devices, and particularly relates to an intelligent cabinet, an article recognition method, an electronic device, a medium and a product. Background Art

[0002] In view of the increasing labor cost, more and more intelligent cabinets have emerged on the market to realize the automatic vending function. Different from the traditional mechanical vending machine, the existing intelligent cabinets have the characteristics of light weight, simple structure, low price and friendly interaction. At present, the intelligent cabinet mainly realizes the automatic vending function by identifying the taken-out articles through the article recognition technology. A good article recognition technology can improve the recognition rate of the backend algorithm, reduce the loss of goods, reduce the cost of manual intervention and improve the user experience.

[0003] At present, the intelligent cabinet mainly identifies the taken-out articles through the following two methods: One method is as Figure 1A shown. A camera 11 is placed on each layer of the shelf of the intelligent cabinet 1. The camera 11 is installed at the top of each layer of the shelf, and the shooting direction is perpendicular to the ground. By analyzing the images captured by each camera 11, solutions such as classifying the bottle caps, detecting the number of bottle caps, and comparing the number before and after taking away the articles are realized, and then the taken-away articles are detected. Another method is as Figure 1B shown. Four cameras 12 are placed at the four corners of the cabinet door frame of the intelligent cabinet 1. The lens direction of the camera 12 is the diagonal of the position where the camera 12 is located. This method can capture the articles entering and leaving the cabinet 360 degrees. The image of the taken-away article can be obtained from the camera in one of the four directions, and image processing is performed on the article image to identify the taken-away article. Summary of the Invention

[0004] Embodiments of the present disclosure provide an intelligent cabinet, an article recognition method, a device, an electronic device, a medium and a product.

[0005] In a first aspect, an intelligent cabinet is provided in the embodiments of the present disclosure.

[0006] Specifically, the intelligent cabinet includes:

[0007] A first camera group, including at least one first camera;

[0008] N other camera groups, each other camera group includes at least one camera, the focal lengths of the cameras in the first other camera group are greater than the focal lengths of the cameras in the second other camera group, the first other camera group and the second other camera group are any two camera groups in the N other camera groups, and N is an integer greater than or equal to 2;

[0009] Among them, the focal length of the first camera is less than the focal lengths of the cameras in each of the N other camera groups, and the first camera group and the N other camera groups are located on the same plane inside the intelligent cabinet.

[0010] In a second aspect, an article recognition method is provided in an embodiment of the present disclosure, which is applied to any one of the above intelligent cabinets. The method includes:

[0011] Obtain a first sequence of images captured by the first camera within a preset time period;

[0012] Based on the first sequence of images, determine the distance level where the target distance between the moved target article and the first camera is located. The distance level includes N levels, and N is an integer greater than or equal to 2;

[0013] Based on the correspondence between the N levels and the N other camera groups, obtain a second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located within the preset time period;

[0014] Identify the target article based on the second sequence of images.

[0015] In a third aspect, an article recognition device is provided in an embodiment of the present disclosure. Among them, it includes:

[0016] A first acquisition module configured to acquire a first sequence of images captured by the first camera within a preset time period;

[0017] A determination module configured to determine, based on the first sequence of images, the distance level where the target distance between the moved target article and the first camera is located. The distance level includes N levels, and N is an integer greater than or equal to 2;

[0018] A second acquisition module configured to obtain, based on the correspondence between the N levels and the N other camera groups, a second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located within the preset time period;

[0019] An identification module configured to identify the target article based on the second sequence of images.

[0020] In a fourth aspect, an electronic device is provided in an embodiment of the present disclosure, including a memory, a processor, and a computer program stored on the memory. Among them, the processor executes the computer program to implement any one of the above methods.

[0021] In a fifth aspect, a computer-readable storage medium is provided in an embodiment of the present disclosure, on which computer instructions are stored. When the computer instructions are executed by a processor, any one of the above methods is implemented.

[0022] In a sixth aspect, an embodiment of the present disclosure provides a computer program product, which includes computer instructions that, when executed by a processor, implement any of the above methods.

[0023] For the above technical solution, it only needs to set at least three groups of cameras with different focal lengths on a plane, without setting more groups of cameras at multiple angles or according to the number of shelves. The cost is relatively low. Moreover, by using a combination of cameras with multiple different focal lengths, accurate image acquisition can be performed on items at different distances. In this way, for a target item to be moved at different distances, only the image with clearer imaging can be selected for item recognition, which improves the accuracy of item recognition while enhancing the image processing speed.

[0024] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In conjunction with the drawings, through the following detailed description of non-limiting embodiments, other features, objects, and advantages of the present disclosure will become more apparent. In the drawings:

[0026] Figure 1A A schematic structural diagram of an intelligent cabinet in the prior art is shown;

[0027] Figure 1B A schematic structural diagram of an intelligent cabinet in the prior art is shown;

[0028] Figure 2A A schematic structural block diagram of an intelligent cabinet according to an embodiment of the present disclosure is shown;

[0029] Figure 2B A schematic structural block diagram of a central control system according to an embodiment of the present disclosure is shown;

[0030] Figure 2C A schematic structural block diagram of a control board according to an embodiment of the present disclosure is shown;

[0031] Figure 2D A schematic structural block diagram of a power management module according to an embodiment of the present disclosure is shown;

[0032] Figure 3 A schematic structural diagram of an intelligent cabinet according to an embodiment of the present disclosure is shown;

[0033] Figure 4 A schematic flowchart of an item recognition method according to an embodiment of the present disclosure is shown;

[0034] Figure 5Schematic flowchart of an article recognition method according to an embodiment of the present disclosure;

[0035] Figure 6 Schematic flowchart of an article recognition method according to an embodiment of the present disclosure;

[0036] Figure 7 Schematic flowchart of an article recognition method according to an embodiment of the present disclosure;

[0037] Figure 8 Schematic flowchart of an article recognition method according to an embodiment of the present disclosure;

[0038] Figure 9 Schematic flowchart of an article recognition method according to an embodiment of the present disclosure;

[0039] Figure 10 Schematic block diagram of an article recognition device according to an embodiment of the present disclosure;

[0040] Figure 11 Schematic block diagram of an electronic device according to an embodiment of the present disclosure;

[0041] Figure 12 Schematic diagram of a computer system suitable for implementing the article recognition method according to an embodiment of the present disclosure. Detailed implementation manners

[0042] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for clarity, parts irrelevant to the description of the exemplary embodiments are omitted in the drawings.

[0043] In the present disclosure, it should be understood that terms such as "including" or "having" are intended to indicate the presence of features, numbers, steps, actions, components, parts, or combinations thereof disclosed in this specification, and do not exclude the possibility of the presence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0044] In the present disclosure, it should be understood that the orientation or positional relationships indicated by terms such as "upper", "lower", "vertical", "horizontal", "inner", "outer", "top", "bottom", etc. are based on the orientation or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention.

[0045] In the present disclosure, it is to be understood that the meaning of "a plurality" is two or more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0046] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0047] As mentioned above, in view of the rising labor costs, more and more smart cabinets are appearing on the market to realize automatic vending functions. Different from traditional mechanical vending machines, existing smart cabinets are light in weight, simple in structure, cheap, and interactive. At present, smart cabinets can realize automatic vending functions by identifying the items taken out through object recognition technology. Good object recognition technology can improve the recognition rate of back-end algorithms, reduce cargo damage, reduce the cost of manual intervention, and improve user experience.

[0048] At present, smart cabinets automatically identify the items taken out mainly in the following two ways: one is Figure 1A As shown, a camera 11 is placed on each shelf of the smart cabinet 1. The camera 11 is installed on the top of each shelf, and the shooting direction is vertical to the ground. By analyzing the images captured by each camera 11, bottle cap classification, detection of the number of bottle caps, and comparison of the number of items before and after taking away are realized, and then the taken items are detected. This method requires cameras to be installed on each layer. The camera 11 is relatively close to the photographed objects, and the shooting range is large. An ultra-wide-angle lens needs to be used, which leads to huge costs. In addition, the method of counting bottle caps cannot distinguish different varieties in the same series. Another method is as follows Figure 1B As shown, four cameras 12 are placed at the four corners of the cabinet door frame of the smart cabinet 1, and the lens of the camera 12 is facing the diagonal of the position of the camera 12. In this way, the items entering and leaving the cabinet can be captured at 360 degrees, and the image of the taken item can be obtained from the camera in one of the four directions, and the image of the item is processed to identify the taken item; in this way, the camera with a fixed angle on the door frame is often a fixed-focus camera, and the imaging effect of objects far away from the focal length is not good on the camera, and more details cannot be identified, which brings workload to the later image processing; the cameras installed in the four corners are difficult to hide due to the angle problem, which affects the appearance; at the same time, since the camera installation angle has four dimensions, it will inevitably capture the characteristic information of the person picking up the goods, which is easy to cause the other party to be disgusted.

[0049] In view of the above deficiencies, the present disclosure proposes an intelligent cabinet, which is provided with a first camera group including at least one first camera and N other camera groups each including at least one camera; wherein, the focal length of the first camera is less than the focal lengths of the cameras in the N other camera groups, and the first camera group and the N camera groups are located on the same plane inside the intelligent cabinet; in this way, the distance of the moved target item from the plane where the camera groups are located can be determined according to the image captured by the first camera with the smallest focal length (i.e., the largest viewing angle range), and the image captured by the camera in the camera group that can clearly capture the details of the target item can be selectively taken for item recognition according to the distance, so that images with clearer imaging can be selected for item recognition for target items at different distances, improving the image processing speed and the accuracy of item recognition at the same time.

[0050] The intelligent cabinet provided in the embodiments of the present application may be a temperature control cabinet with a refrigeration function, such as a refrigerator cabinet, a freezer cabinet, a refrigerator, a wine cabinet, a cosmetic preservation cabinet, etc., or a temperature control cabinet with a heating function, such as a warming cabinet, a heating cabinet, a hot drink cabinet, etc. Of course, it may also be a common intelligent cabinet without a temperature control function, such as a snack cabinet, a small commodity cabinet, etc. The embodiments of the present application do not impose any restrictions on the specific type of the intelligent cabinet.

[0051] Exemplarily, Figure 2A Fig. shows a schematic structural block diagram of an intelligent cabinet according to an embodiment of the present disclosure. The intelligent cabinet may be a temperature control cabinet with a heating or refrigeration function, such as Figure 2A As shown, the intelligent cabinet 2 may include a compressor 21, a condenser 22, a throttling element 23, and an evaporator 24. Among them, the compressor 21, the condenser 22, the throttling element 23, and the evaporator 24 are connected by pipes filled with refrigerant to form a closed pipeline, constituting a refrigeration system or a heating system capable of circulating refrigerant.

[0052] Among them, the compressor refers to a driven fluid machine used to lift low-pressure refrigerant to high-pressure refrigerant. The compressor can suck in low-temperature and low-pressure gaseous refrigerant, compress the refrigerant by driving a piston through the operation of the motor, and then discharge high-temperature and high-pressure gaseous refrigerant to provide power for the refrigeration cycle. The compressor may include a reciprocating compressor, a screw compressor, a rotary compressor, a scroll compressor, a centrifugal compressor, etc. The embodiments of the present application do not impose any restrictions on the specific type of the compressor.

[0053] A condenser refers to a heat exchanger used to enable heat exchange between the refrigerant in the condenser and the air outside the condenser to achieve heat release. Specifically, the condenser may include a relatively long pipe for accommodating the refrigerant, which is usually made of a metal material with strong heat conduction performance such as copper, and the pipe is usually coiled into a spiral shape. Additionally, to improve the heat exchange efficiency of the condenser, heat sinks with excellent heat conduction performance can be provided on the pipe to increase the heat dissipation area, thereby accelerating the speed of heat exchange and improving the heat exchange efficiency. It is also possible to set a blower or fan matching the condenser to accelerate the flow rate of the air around the condenser, thereby accelerating the speed of heat exchange and improving the heat exchange efficiency.

[0054] The throttling element is used to throttle the liquid refrigerant at normal temperature and high pressure to become a gaseous refrigerant at low temperature and low pressure. The throttling element can also be referred to as a throttling element or a regulating valve, and the throttling element can include an expansion valve, a capillary tube, etc. Additionally, the throttling element can also control the flow rate of the refrigerant flowing through the throttling element to prevent the flow rate of the refrigerant flowing through the throttling element from being too large or too small. Among them, if the flow rate of the refrigerant flowing through the throttling element is too large, it will cause the refrigerant flowing out of the throttling element to still include liquid refrigerant, and the liquid refrigerant entering the compressor will cause liquid hammer, damaging the compressor; if the flow rate of the refrigerant flowing through the throttling element is too small, it will cause too little refrigerant to enter the compressor, reducing the working efficiency of the compressor.

[0055] An evaporator refers to a heat exchanger used to enable heat exchange between the refrigerant in the evaporator and the air outside the condenser to achieve heat absorption. Specifically, the evaporator may include a relatively long pipe for accommodating the refrigerant, which is usually made of a metal material with strong heat conduction performance such as copper, and the pipe is usually coiled into a spiral shape. Additionally, to improve the heat exchange efficiency of the condenser, heat sinks with excellent heat conduction performance can be provided on the pipe to increase the heat dissipation area, thereby accelerating the speed of heat exchange and improving the heat exchange efficiency. In some embodiments, a blower or fan matching the evaporator can also be set, and the air generated by the blower or fan is used to accelerate the flow rate of the air around the evaporator, thereby accelerating the speed of heat exchange and improving the heat exchange efficiency.

[0056] The refrigerant can also be called a refrigerant, a coolant, or a refrigerant fluid, and it refers to the medium substance that completes energy conversion in a refrigeration system or a heating system. The refrigerant is usually a substance that easily undergoes reversible phase changes (such as absorbing heat to become a gas and releasing heat to become a liquid). Through reversible phase changes, the refrigerant can transfer heat. Specifically, when the gaseous refrigerant is pressurized, it releases heat and becomes a liquid, and when the high-pressure liquid is depressurized to become a gas, it will absorb heat. The refrigerant can include ammonia, air, water, brine, Freon (which can also be called chlorofluorocarbon, chlorofluorocarbon compound), etc. Among them, Freon can include trichlorofluoromethane, dichlorofluoromethane, trifluoromethane, tetrafluoroethane, dichlorotrifluoroethane, etc.

[0057] When the intelligent cabinet is a temperature-controlled cabinet with a refrigeration function, the low-temperature and low-pressure gaseous refrigerant flows from the evaporator into the compressor. The compressor compresses the low-temperature and low-pressure gaseous refrigerant and makes the high-temperature and high-pressure gaseous refrigerant flow into the condenser; the high-temperature and high-pressure gaseous refrigerant exchanges heat with the air outside the condenser, so that the high-temperature and high-pressure gaseous refrigerant is cooled into a normal-temperature and high-pressure liquid refrigerant in the condenser. Then, the normal-temperature and high-pressure liquid refrigerant flows into the throttling element. The throttling element throttles the normal-temperature and high-pressure liquid refrigerant, so that the refrigerant flowing out of the throttling element is transformed into a low-temperature and low-pressure liquid refrigerant; the low-temperature and low-pressure liquid refrigerant flows into the evaporator. The low-temperature and low-pressure liquid refrigerant exchanges heat with the air outside the evaporator, and the low-temperature and low-pressure liquid refrigerant evaporates into a low-temperature and low-pressure gaseous refrigerant to absorb heat. Among them, the air outside the evaporator can be introduced into the storage area of the intelligent cabinet, and the air outside the condenser can be introduced outside the intelligent cabinet, so as to realize transporting the heat in the storage area of the intelligent cabinet to the outside of the intelligent cabinet and refrigerating the storage area of the intelligent cabinet.

[0058] When the intelligent cabinet is a temperature-controlled cabinet with a heating function, the low-temperature and low-pressure gaseous refrigerant flows from the condenser into the compressor. The compressor compresses the low-temperature and low-pressure gaseous refrigerant and makes the high-temperature and high-pressure gaseous refrigerant flow into the evaporator; the high-temperature and high-pressure gaseous refrigerant exchanges heat with the air outside the evaporator, so that the high-temperature and high-pressure gaseous refrigerant is cooled into a normal-temperature and high-pressure liquid refrigerant in the evaporator. Then, the normal-temperature and high-pressure liquid refrigerant flows into the throttling element. The throttling element throttles the normal-temperature and high-pressure liquid refrigerant, so that the refrigerant flowing out of the throttling element is transformed into a low-temperature and low-pressure liquid refrigerant; the low-temperature and low-pressure liquid refrigerant flows into the condenser. The low-temperature and low-pressure liquid refrigerant exchanges heat with the air outside the condenser, and the low-temperature and low-pressure liquid refrigerant evaporates into a low-temperature and low-pressure gaseous refrigerant to absorb heat. Among them, the air outside the evaporator can be introduced into the storage area of the intelligent cabinet, and the air outside the condenser can be introduced outside the intelligent cabinet, so as to realize transporting the heat outside the intelligent cabinet to the storage area of the intelligent cabinet and heating the storage area of the intelligent cabinet.

[0059] In an embodiment of the present application, the intelligent cabinet includes a cabinet body and a door body, and a central control system, a control board and a power management module are arranged on the cabinet body.

[0060] In an embodiment of the present application, Figure 2B A schematic structural block diagram of the central control system according to an embodiment of the present disclosure is shown, as Figure 2B shown, the central control system 25 includes a processor 251, a random access memory 252, a flash memory 253, a wireless local area network Bluetooth module 254, a cellular communication module 255, a gyroscope 256, a microphone 257, a speaker 258, and a camera 259.

[0061] The processor 251 may include one or more processing units. For example, the processor 251 may include an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a memory, a video codec, a digital signal processor, a baseband processor, and / or a neural network processor. Among them, different processing units may be independent devices or integrated in one or more processors.

[0062] Among them, the image signal processor is used to process the data fed back by the camera. For example, when taking a photo, the shutter is opened, and the light passes through the lens and is transmitted to the camera sensor. The optical signal is converted into an electrical signal, and the camera sensor transmits the electrical signal to the image signal processor for processing and converts it into an image visible to the naked eye. The image signal processor can also optimize the algorithm for image noise, brightness, and skin color. The image signal processor can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the image signal processor may be provided in the camera.

[0063] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, the digital signal processor can be used to perform Fourier transform on the frequency point energy, etc.

[0064] The video codec is used to compress or decompress digital videos. The smart cabinet can support one or more video codecs. In this way, the smart cabinet can play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0065] The neural network computing processor, by referring to the biological neural network structure, such as referring to the transmission mode between human brain neurons, can quickly process the input information and can also continuously self-learn. Through the neural network computing processor, applications such as intelligent cognition of the smart cabinet can be realized, such as image recognition, face recognition, speech recognition, text understanding, etc.

[0066] In some embodiments, the processor may include one or more interfaces. The interfaces may include an integrated circuit interface, an integrated circuit built-in audio interface, a pulse code modulation interface, a universal asynchronous receiver / transmitter interface, a mobile industry processor interface, a general-purpose input / output interface, a subscriber identity module interface, and / or a universal serial bus interface, etc.

[0067] The random access memory 252 can be used to store computer-executable program code, and the executable program code includes instructions and data. The processor 251 executes various functional applications and data processing of the smart cabinet by running the instructions stored in the random access memory 252. The random access memory 252 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, the image playback function, etc.). The data storage area can store the data created during the use of the smart cabinet (such as audio data, image data, etc.).

[0068] The flash memory 253 can be used to expand the storage capacity of the smart cabinet. The flash memory 253 can communicate with the processor 251 through the flash memory interface to implement the data storage function. For example, files such as music and videos are saved in the flash memory.

[0069] The processor 251, the random access memory 252, and the flash memory 253 can form a minimum system to provide a system operating environment.

[0070] The wireless local area network Bluetooth module 254 can provide wireless communication solutions applied to the smart cabinet, including wireless local area network, Bluetooth, global navigation satellite system, frequency modulation, near field communication technology, infrared technology, etc. The wireless local area network Bluetooth module 254 can be one or more devices integrating at least one communication processing module. The wireless local area network Bluetooth module 254 receives electromagnetic waves via the antenna, frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 251. The wireless local area network Bluetooth module 254 can also receive the signal to be sent from the processor 251, frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna and radiate it out. In an embodiment of the present application, communication can be carried out with the user's terminal through the wireless local area network Bluetooth module.

[0071] The cellular communication module 255 can provide wireless communication solutions applied to the smart cabinet, including 2G / 3G / 4G / 5G, etc. The cellular communication module 255 can include at least one filter, switch, power amplifier, low-noise amplifier, etc. The cellular communication module 255 can receive electromagnetic waves through the antenna, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The cellular communication module 255 can also amplify the signal modulated by the modulation and demodulation processor, convert it into electromagnetic waves through the antenna and radiate it out. In some embodiments, at least some functional modules of the cellular communication module 255 can be disposed in the processor 251. In some embodiments, at least some functional modules of the cellular communication module 255 and at least some modules of the processor 251 can be disposed in the same device. In an embodiment of the present application, communication can be carried out with the cloud server of the maintenance service provider of the smart cabinet through the cellular communication module 255.

[0072] Through the wireless local area network Bluetooth module 254 and the cellular communication module 255, the intelligent cabinet can communicate with the network and other devices through wireless communication technologies. The wireless communication technologies may include Global System for Mobile Communications, General Packet Radio Service, Code Division Multiple Access, Wideband Code Division Multiple Access, Time Division Code Division Multiple Access, Long Term Evolution, etc.

[0073] The gyroscope 256 can be used to determine the real-time attitude of the cabinet door of the intelligent cabinet.

[0074] The microphone 257, also known as the "microphone" or "transmitter", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak close to the microphone 257 with their mouth to input the sound signal into the microphone 257. The intelligent cabinet can be provided with at least one microphone 257. In some other embodiments, the intelligent cabinet can be provided with two microphones 257, which can not only collect sound signals but also achieve a noise reduction function. In some other embodiments, the intelligent cabinet can also be provided with three, four or more microphones 257 to collect sound signals, reduce noise, identify the sound source, and achieve functions such as directional recording. In one embodiment of the present application, the sound during the operation of the intelligent cabinet can be collected through the microphone 257.

[0075] The speaker 258, also known as the "loudspeaker", is used to convert audio electrical signals into sound signals. The intelligent cabinet can play music, play prompt voices, or play alarm messages through the speaker 258.

[0076] The camera 259 is used to capture static images or videos. An object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge-coupled device or a complementary metal-oxide-semiconductor phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the image signal processor to convert it into a digital image signal. The image signal processor outputs the digital image signal to the digital signal processor for processing. The digital signal processor converts the digital image signal into an image signal in a standard format such as RGB or YUV. In some embodiments, the intelligent cabinet can include one or more cameras 259. In one embodiment of the present application, the camera 259 can have the function of heating itself to ensure that its lens does not fog up.

[0077] In one embodiment of the present application, Figure 2C shows a schematic structural block diagram of a control board according to an embodiment of the present disclosure, as Figure 2CAs shown, the control board 26 includes a power interface 261, a communication interface 262, a metering chip 263, a micro-control unit chip 264, a real-time clock chip, a lamp switch interface 265, a temperature control switch interface 266, an evaporation fan interface 267, a compressor interface 268, a condenser fan interface 269, and a temperature sensor interface 260.

[0078] Among them, the power interface 261 is used to connect to a power management module, and the power management module supplies power to the control board through this power interface; the communication interface 262 is used to enable the control board 26 to communicate with a central control system 25 or other modules; the metering chip 263, that is, an electric quantity sensor, can obtain voltage data, current data, real-time power data, and average power data through the metering chip 263. The time of the micro-control unit chip 264 can be maintained through the real-time clock chip. The control signal of the lamp switch of the intelligent cabinet can be received through the lamp switch interface 265. The control signal of the temperature control switch of the intelligent cabinet can be received through the temperature control switch interface 266. An evaporation fan control signal can be sent to the evaporation fan of the intelligent cabinet through the evaporation fan interface 267 to control the operation of the evaporation fan. A compressor control signal can be sent to the compressor of the intelligent cabinet through the compressor interface 268 to control the operation of the compressor. A condenser fan control signal can be sent to the condenser fan of the intelligent cabinet through the condenser fan interface 269 to control the operation of the condenser fan. Temperature sensor data collected by one or more temperature sensors can be received through the temperature sensor interface 260 to facilitate determining the temperature values at one or more positions of the intelligent cabinet.

[0079] In an embodiment of the present application, Figure 2D The schematic structural block diagram of a power management module according to an embodiment of the present disclosure is shown, as Figure 2D shown, the power management module 27 includes an AC-DC conversion module 271, a charging management module 272, and a battery 273. The power management module 27 is used to supply power to the main board and the control board, and perform charge and discharge management on the battery. The power management module 27 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 27 can also be provided in a processor.

[0080] In an embodiment of the present application, the intelligent cabinet further includes a display screen. The intelligent cabinet realizes the display function through a graphics processor, a display screen, an application processor, etc. The graphics processor is a microprocessor for image processing, and is connected to the display screen and the application processor. The graphics processor is used to perform mathematical and geometric calculations and is used for graphics rendering. The processor may include one or more graphics processors, which execute program instructions to generate or change display information.

[0081] The display screen is used to display images, videos, etc. The display screen includes a display panel. The display panel can adopt a liquid crystal display screen, an organic light-emitting diode, an active matrix organic light-emitting diode or an active matrix organic light-emitting diode, a flexible light-emitting diode, a quantum dot light-emitting diode, etc. In some embodiments, the smart cabinet can include one or more display screens.

[0082] It can be understood that the structure schematically shown in the embodiments of the present application does not constitute a specific limitation on the smart cabinet. In other embodiments of the present application, the smart cabinet may include more or fewer components than those shown, or combine certain components, or split certain components, or have different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0083] The details of the embodiments of the present disclosure will be introduced in detail through specific examples below.

[0084] The present disclosure provides a smart cabinet, which includes a first camera group and N other camera groups, where N is an integer greater than or equal to 2; the first camera group includes at least one first camera, and each of the N other camera groups includes at least one camera. By way of example, Figure 3 The structural schematic diagram of a smart cabinet according to an embodiment of the present disclosure is shown, as Figure 3 shown, the smart cabinet 3 includes a first camera group 301 and N = 2 other camera groups, namely one other camera group 302 and another other camera group 303. The first camera group 301 may include one first camera, and both the one other camera group 302 and the another other camera group 303 may also each include one camera. Of course, in other embodiments, it may be that the first camera group 301 includes two or more first cameras, the one other camera group 302 includes two or more cameras, and the another other camera group 303 also includes two or more cameras, and so on. Specifically, the number of cameras in each group can be determined according to the shooting areas of the respective cameras and the item-taking area of the smart cabinet, as long as it is ensured that the number of cameras in each group can enable the cameras in that group to comprehensively capture the entire item-taking area of the smart cabinet. It should be noted here that each camera in each camera group corresponds to a specific shooting area, and each camera in each camera group can capture images within its corresponding shooting area.

[0085] In this embodiment, the focal length of the first camera in the first camera group 301 is less than the focal lengths of the cameras in each of the N other camera groups. The focal lengths of the cameras in the first other camera group are greater than the focal lengths of the cameras in the second other camera group. The first other camera group and the second other camera group are any two of the N other camera groups. That is, among any two of the N other camera groups, the focal lengths of the cameras in one other camera group are necessarily greater than the focal lengths of the cameras in the other other camera group. The focal lengths of the cameras in different other camera groups among the N other camera groups are different. For example, as Figure 3 shown, when N = 2, the focal lengths of the cameras in one other camera group 302, i.e., the first other camera group, are greater than the focal lengths of the cameras in the other other camera group 303, i.e., the second other camera group. When N = 3, the N other camera groups include other camera group 1, other camera group 2, and other camera group 3. The focal lengths of the cameras in other camera group 1 are greater than the focal lengths of the cameras in other camera group 2, and the focal lengths of the cameras in this other camera group 2 are greater than the focal lengths of the cameras in this other camera group 3.

[0086] In this embodiment, the first camera group and the N camera groups are on the same plane inside the smart cabinet. For example, they can be on the top surface, the bottom surface, or the side surface inside the smart cabinet, etc.

[0087] In this embodiment, sequence images or videos can be continuously captured by the cameras in the first camera group and the N other camera groups for image analysis to identify the moved items. Specifically, first, the distance of the moved item from the plane where the camera is located can be determined based on the image captured by the first camera with the smallest focal length. Then, the images captured by the cameras in the camera group that can clearly capture the details of the target item can be selectively taken according to the distance for item recognition. In this way, images with clearer imaging can be selected for item recognition of target items at different distances, improving the item recognition accuracy while enhancing the image processing speed.

[0088] In this embodiment, only at least three camera groups with different focal lengths need to be set on one plane, without setting more camera groups at multiple angles or according to the number of shelves. The cost is relatively low. Moreover, by using a combination of cameras with multiple different focal lengths, accurate image acquisition can be performed on items at different distances. In this way, for the moved target items at different distances, only the images with clearer imaging can be selected for item recognition, improving the item recognition accuracy while enhancing the image processing speed.

[0089] In an embodiment of the present disclosure, when N is 2, the N other camera groups include 2 other camera groups, i.e., the second camera group and the third camera group. For example, as Figure 3As shown, the second camera group can be another camera group 302, and the third camera group can be another camera group 303. The first camera group 301 includes a first camera, the second camera group includes a second camera, and the third camera group includes a third camera. The focal length of the first camera in the first camera group is less than the focal length of the second camera in the second camera group, and the focal length of the second camera in the second camera group is less than the focal length of the third camera in the third camera group. By way of example, the first camera 301 can be a wide-angle camera for viewing a larger range, having the largest field of view angle and the smallest focal length; the third camera can be a narrow field of view camera (also referred to as a short focal length camera) for capturing the characteristics of distant objects in a specific area, having the largest focal length; the focal length of the second camera is in the middle (also referred to as a medium focal length camera), and the field of view angle is also in the middle, for supplementing the deficiencies of the first camera and the third camera.

[0090] In this embodiment, sequence images or videos can be continuously captured by the cameras in the first camera group, the second camera group, and the third camera group for image analysis to identify the moved items. Specifically, first, the distance of the moved item from the plane where the camera is located can be determined based on the image captured by the first camera with the smallest focal length. Then, according to the distance, an image captured by the second camera with a medium focal length that can clearly capture the details of nearby items can be selectively taken, or an image captured by the third camera with the largest focal length that can clearly capture the details of distant items can be selectively taken for item identification. When the distance is far, an image captured by the third camera with the largest focal length that can clearly capture the details of distant items is selected for item identification. When the distance is near, an image captured by the second camera with a medium focal length that can clearly capture the details of nearby items is selected for item identification. In this way, for items at different distances, only the images with clearer imaging can be selectively taken for item identification, improving the image processing speed and the accuracy of item identification at the same time.

[0091] In an embodiment of the present disclosure, the first camera group and the N other camera groups are located at the top inside the smart cabinet.

[0092] By way of example, as Figure 3 shown, the first camera in the first camera group 301 and two other camera groups, namely one other camera group 302 and another other camera group 303, are located at the top inside the smart cabinet 3; in this way, it is possible to avoid capturing the characteristic information of the pick-up personnel, ensuring user privacy and improving the user experience.

[0093] In an embodiment of the present disclosure, the intelligent cabinet may further include an image processor, which is connected to the first camera group and the N camera groups, and is configured to obtain the image information captured by each camera in the first camera group and the N camera groups, and perform image analysis and processing based on the image information to identify the moved items. The specific analysis process may be: obtaining a first sequence of images captured by the first camera within a preset time period; determining, based on the first sequence of images, the distance level where the target distance between the moved target item and the first camera is located; obtaining, based on the corresponding relationship between the distance level and the N other camera groups, the second sequence of images captured by the cameras in the other camera groups corresponding to the distance level where the target distance is located within the preset time period, and identifying the target item based on the second sequence of images.

[0094] In this embodiment, the preset time period may be a time period within any range of 12s to 18s during the working time of the first camera. For example, the preset time period may be 15s during the working time of the first camera; the first sequence of images may be a sequence of images arranged according to the shooting time of the first camera. The first camera may continuously capture images in the corresponding area inside the intelligent cabinet and transmit them to the image processor in order according to the shooting time. The image processor may determine the distance level between the moved target item and the first camera based on the first sequence of images captured by the first camera. The distance level includes N levels, and the N levels respectively correspond to the N camera groups. The corresponding rule may be that the farther the distance level is, the larger the focal length of the cameras in the corresponding camera group is. In this way, based on the corresponding relationship between the distance level and the N other camera groups, the second sequence of images captured by the cameras in the other camera groups corresponding to the distance level where the target distance is located within the preset time period can be obtained, and the target item can be identified based on the second sequence of images.

[0095] In this embodiment, an image processor may be provided inside the intelligent cabinet to select clearer images for item recognition for items at different distances, and automatic item recognition can be performed, which improves the item recognition accuracy while enhancing the image processing speed.

[0096] The present disclosure also provides an item recognition method. Figure 4 A flowchart showing an item recognition method according to an embodiment of the present disclosure is shown. This method is applied to any of the above intelligent cabinets, as Figure 4 shown. The item recognition method includes the following steps:

[0097] In step S401, a first sequence of images captured by the first camera within a preset time period is obtained.

[0098] In an embodiment of the present disclosure, the first camera can continuously capture images of a corresponding area inside the intelligent cabinet when the intelligent cabinet is working, so that a first sequence of images within a preset time period can be obtained. For example, if the preset time period is 15s, timing can start from when the intelligent cabinet starts working, and an acquisition operation is performed every 15s to obtain the first sequence of images captured by the first camera within the previous 15s before the current moment.

[0099] In step S402, based on the first sequence of images, determine the distance level where the target distance between the moved target item and the first camera is located.

[0100] In an embodiment of the present disclosure, image analysis can be performed on the first sequence of images to determine whether an item has been moved. The movement includes two cases: taking out and putting back. If an item has been moved, then determine that the item is the target item. The first sequence of images can be analyzed to determine the distance level between the moved target item and the first camera. There can be N distance levels, and the N distance levels respectively correspond to the above N camera groups. The corresponding rule can be that the farther the distance level, the larger the focal length of the camera in the corresponding camera group.

[0101] In step S403, based on the correspondence between the N levels and the N other camera groups, obtain the second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located within the preset time period.

[0102] In an embodiment of the present disclosure, the corresponding rule of the correspondence can be that the farther the distance level, the larger the focal length of the camera in the corresponding camera group, and the N levels correspond one-to-one with the N other camera groups.

[0103] In an embodiment of the present disclosure, based on this correspondence, the other camera group corresponding to the distance level where the target distance is located can be directly obtained, and then the second sequence of images captured by the cameras in the corresponding other camera group within the preset time period can be obtained. In this way, the farther the moved target item is from the plane where the camera is located, the images captured by the camera with a larger focal length are obtained, and the target item farther away is clearer when captured by the camera with a larger focal length.

[0104] In step S404, identify the target item based on the second sequence of images.

[0105] In an embodiment of the present disclosure, if the distance level where the target object to be moved is located relative to the first camera is determined based on the first sequence of images, it indicates that the target object has been moved within the preset time period. Therefore, the target object must be present in the second sequence of images captured by the cameras in the other camera group within the preset time period. Moreover, since the second sequence of images is captured by the cameras in the other camera group corresponding to the distance level where the target distance is located, an image that can clearly display the target object can be obtained in this way. Thus, the target object can be accurately identified, improving the accuracy of object identification. Additionally, object identification is only based on the second sequence of images captured by the corresponding cameras here, reducing the workload by 1 / N during subsequent object identification and enhancing the image processing speed.

[0106] Before object identification in this embodiment, the distance level of the target object to be moved can be determined through the first sequence of images captured by the first camera, and then the second sequence of images captured by the camera that can clearly capture the target object is selected based on this distance level. In this way, the target object can be accurately identified, improving the accuracy of object identification. Additionally, object identification is only based on the second sequence of images captured by the corresponding cameras here, reducing the workload by 1 / N during subsequent object identification and enhancing the image processing speed.

[0107] In an embodiment of the present disclosure, when N is equal to 2, the distance levels include a long-distance level and a short-distance level, and the N other camera groups include a second camera group and a third camera group; as Figure 5 shown, step S403 in the above object identification method, that is, the step of obtaining the second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located based on the correspondence between the N levels and the N other camera groups, may include the following steps S4031 and S4032.

[0108] In step S4031, in response to the distance level where the target distance is located being the short-distance level, obtain the second sequence of images captured by the second camera of the second camera group corresponding to the short-distance level within the preset time period;

[0109] In step S4032, in response to the distance level where the target distance is located being the long-distance level, obtain the second sequence of images captured by the third camera of the third camera group corresponding to the long-distance level within the preset time period;

[0110] Wherein, the focal length of the second camera in the second camera group is less than the focal length of the third camera in the third camera group.

[0111] In this embodiment, if the distance level between the target item and the first camera is the close distance level, it indicates that the target item is near the plane where the camera is located. Since the focal length of the second camera is less than that of the third camera, the second camera can capture the target item nearby more clearly. Therefore, the second sequence image of the target item captured by the second camera can be selected for item recognition. In this way, the target item can be accurately recognized.

[0112] In an embodiment of the present disclosure, if the distance level between the target item and the first camera is the long distance level, it indicates that the target item is far from the plane where the camera is located. Since the focal length of the second camera is less than that of the third camera, the third camera can capture the target item far away more clearly. Therefore, the third sequence image of the target item captured by the third camera can be selected for item recognition. In this way, the target item can be accurately recognized.

[0113] In this embodiment, when the distance is close, the second sequence image captured by the second camera with a medium focal length that can clearly capture the details of nearby items can be selected for item recognition. When the distance is far, the second sequence image captured by the third camera with a long focal length that can clearly capture the details of distant items can be selected for item recognition. In this way, for target items at different distances, only the images with clearer imaging can be selected for item recognition, which improves the image processing speed and the accuracy of item recognition at the same time.

[0114] In an embodiment of the present disclosure, as Figure 6 shown, step S401 in the above item recognition method, that is, the step of obtaining the first sequence image captured by the first camera within a preset time period, may further include the following steps:

[0115] In step S4011, in response to detecting that the cabinet door of the smart cabinet is in an open state, obtain the first sequence image captured by the first camera within a preset time period, and the preset time period starts from the moment when the cabinet door of the smart cabinet is detected to be opened and ends after counting a preset duration.

[0116] In this embodiment, the real-time attitude of the cabinet door of the smart cabinet can be detected by a gyroscope. In this way, when it is detected that the cabinet door of the smart cabinet is in an open state, the first camera is started to take pictures, and the first sequence image captured by the first camera within a preset time period is obtained. In this way, unnecessary shooting can be avoided, shooting resources can be saved, and power can be conserved.

[0117] In this embodiment, the preset time period starts from the moment when the cabinet door of the smart cabinet is detected to be opened and ends after counting for a preset duration. The preset duration can be about 15s. Research has found that users usually complete the process of opening the cabinet door of the smart cabinet to pick up and place items within 15s. Therefore, it is possible to start shooting from the moment when the cabinet door of the smart cabinet is detected to be opened to obtain a sequence of images within these 15s.

[0118] In an embodiment of the present disclosure, as Figure 7 shown, step S402 in the above item recognition method, that is, the step of determining the distance level where the target distance between the moved target item and the first camera is located based on the first sequence of images, may further include the following steps:

[0119] In step S4021, input the first sequence of images into a preset classification model, execute the classification model, and output the distance level where the target distance between the target item and the first camera is located.

[0120] In this embodiment, the distance level where the target distance between the moved target item and the first camera is located can be distinguished by the classification model. The classification model can be pre-stored in the processor of the smart cabinet or can be trained by the processor in the smart cabinet itself, and there is no limitation here.

[0121] In this embodiment, the step of obtaining the preset classification model may include: obtaining sample data, where the sample data includes sample sequence images when a sample item is moved and the sample distance level of the sample item from the camera that shoots the sample sequence images; training an initial classification model based on the sample data to obtain the preset classification model. Among them, the training process may be to input the sample sequence images into the initial classification model, compare the obtained output result with the sample distance level corresponding to the sample sequence images, determine whether the output result is correct, and then calculate the correct rate of the output result (the correct rate of the output result is the number of sample data with correct output results / the total number of all sample data); if the correct rate of the output result is lower than a preset correct rate threshold, continuously modify the parameters of the initial classification model until the correct rate of the output result is greater than or equal to the preset correct rate threshold to obtain the preset classification model.

[0122] In this embodiment, the preset classification model is used to determine the distance level where the target distance between the moved target item and the first camera is located, and the determination result is more accurate.

[0123] In an embodiment of the present disclosure, in the above item recognition method, the cameras in the other camera group include at least two cameras. The step of obtaining a part of the second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located in step 403 may include the following steps A1 and A2:

[0124] In step A1, obtain a target camera in the corresponding other camera group whose captured object-taking area overlaps with the object-taking area captured by the first camera;

[0125] In step A2, obtain the second sequence of images captured by the target camera within the preset time period.

[0126] In this embodiment, assume that the object-taking area captured by the first camera is area a, the corresponding camera group includes two cameras, namely camera 1 and camera 2. The object-taking area captured by camera 1 is area a, and the object-taking area captured by camera 2 is area b. Then the target camera in the corresponding other camera group whose captured object-taking area overlaps with the object-taking area captured by the first camera is camera 1. Since camera 1 and the first camera capture the same object-taking area, if the first camera captures that the target item is moved in this object-taking area, then camera 1 can also capture the moved target item in this object-taking area. Thus, by obtaining the second sequence of images captured by camera 1, the target item can be identified based on this.

[0127] In this embodiment, when installing the cameras of each camera group, it is necessary to set the object-taking areas captured by each camera, and the object-taking areas captured by each camera can be stored. In this way, the target camera can be directly determined according to the pre-stored object-taking areas captured by each camera.

[0128] In this embodiment, the second sequence of images captured by the target camera in the corresponding other camera group can be obtained and analyzed, rather than obtaining and analyzing the second sequence of images of all the target cameras in the corresponding other camera group, reducing unnecessary image analysis and improving the image processing speed.

[0129] In an embodiment of the present disclosure, in the above item recognition method, the method may further include the following steps A3 and A4, and the above step A2 may be implemented as the following step A21:

[0130] In step A3, in response to at least two target cameras being included in the corresponding other camera group, based on the first sequence of images, determine the target area of the target item in the object-taking area captured by the first camera;

[0131] In step A4, based on the target area, determine, from the at least two target cameras, a target camera whose captured object-taking area includes the target area.

[0132] In step A21, obtain a second sequence of images captured by the one target camera within the preset time period.

[0133] In this embodiment, the focal length of the first camera is the smallest. When the user's object-taking area is relatively large, one first camera can capture the largest object-taking area. The focal lengths of the cameras in the other camera groups are less than that of the first camera. In this case, it is possible that the object-taking area captured by the first camera includes the object-taking areas captured by two or more cameras in the corresponding other camera groups. At this time, the second sequence of images captured by these two or more target cameras can be comprehensively analyzed for target object recognition. However, in order to reduce unnecessary image analysis, first, based on the first sequence of images, determine the target area of the target object in the object-taking area captured by the first camera, and then determine, from the at least two target cameras, a target camera whose captured object-taking area includes the target area; this one target camera must have captured the target object, and thus only the second sequence of images captured by this one target camera needs to be analyzed for images.

[0134] For example, assume that the target area where the user moves the target object is within area a. The object-taking area captured by the first camera is area a and area b. The object-taking area captured by camera 1 in the other camera group is area a, and the object-taking area captured by camera 2 in the other camera group is area b. At this time, the target cameras whose captured object-taking areas overlap with the object-taking area captured by the first camera in the corresponding other camera group are camera 1 and camera 2. Obtain area a of the target object in the object-taking area captured by the first camera. At this time, the second sequence of images captured by camera 1 in the other camera group within the preset time period can be obtained for target object recognition.

[0135] The implementation process of the above object recognition method is introduced in detail through the following embodiments.

[0136] Embodiment 1:

[0137] In this embodiment, a first camera group and N = 2 other camera groups, namely a second camera group and a third camera group, are provided in the intelligent cabinet. The first camera group includes one first camera, the second camera group includes one second camera, and the third camera group includes one third camera; wherein, the focal length of the first camera in the first camera group is less than the focal length of the second camera in the second camera group, and the focal length of the second camera in the second camera group is less than the focal length of the third camera in the third camera group. Figure 8A schematic flowchart of an article recognition method according to an embodiment of the present disclosure is shown. As Figure 8 shown, the article recognition method may include the following steps S801 to S805.

[0138] In step S801, in response to detecting that the cabinet door of the smart cabinet is in an open state, a first sequence of images captured by the first camera within a preset time period is obtained. The preset time period starts from the moment when the cabinet door of the smart cabinet is detected to be opened and ends after a preset duration of timing.

[0139] In step S802, the first sequence of images is input into a preset classification model, the classification model is executed, and the distance level where the target distance between the target article and the first camera is located is output. The distance level includes two levels.

[0140] In step S803, in response to the distance level where the target distance is located being the close distance level, a second sequence of images captured by the second camera of the second camera group corresponding to the close distance level within the preset time period is obtained.

[0141] In step S804, in response to the distance level where the target distance is located being the far distance level, a second sequence of images captured by the third camera of the third camera group corresponding to the far distance level within the preset time period is obtained.

[0142] In step S805, the target article is recognized based on the second sequence of images.

[0143] Before performing article recognition in this embodiment, the distance level of the target article being moved can be determined through the first sequence of images captured by the first camera, and then the second sequence of images captured by the camera that can clearly capture the target article is selected through this distance level, so that the target article can be accurately recognized, improving the accuracy of article recognition. Moreover, here the article recognition is only based on the second sequence of images captured by the corresponding camera, reducing the workload by 1 / 2 during subsequent article recognition and improving the image processing speed.

[0144] Embodiment 2

[0145] In this embodiment, a first camera group and N = 2 other camera groups, namely a second camera group and a third camera group, are provided in the smart cabinet. The first camera group includes one first camera, the second camera group includes one second camera, and the third camera group includes two third cameras. Among them, the focal length of the first camera in the first camera group is less than the focal length of the second camera in the second camera group, and the focal length of the second camera in the second camera group is less than the focal length of the third camera in the third camera group. Figure 9The flowchart shows a method for identifying an item according to an embodiment of the present disclosure. As Figure 9 shown, the method for identifying an item may include the following steps S901 to S905.

[0146] In step S901, in response to detecting that the cabinet door of the smart cabinet is in an open state, a first sequence of images captured by the first camera within a preset time period is obtained. The preset time period starts from the moment when the cabinet door of the smart cabinet is detected to be opened and ends after counting for a preset duration.

[0147] In step S902, the first sequence of images is input into a preset classification model, the classification model is executed, and the distance level where the target distance between the target item and the first camera is located is output. The distance level includes two levels.

[0148] In step S903, in response to the distance level where the target distance is located being the close distance level, a second sequence of images captured by the second camera of the second camera group corresponding to the close distance level within the preset time period is obtained.

[0149] In step S904, in response to the distance level where the target distance is located being the long distance level, a target camera whose pick-up area captured by the third camera group overlaps with the pick-up area captured by the first camera is obtained.

[0150] In step S905, in response to the third camera group including at least two target cameras, based on the first sequence of images, the target area of the target item in the pick-up area captured by the first camera is determined.

[0151] In step S906, based on the target area, one target camera whose pick-up area includes the target area is determined from the at least two target cameras.

[0152] In step S907, a second sequence of images captured by the one target camera within the preset time period is obtained.

[0153] In step S908, the target item is identified based on the second sequence of images.

[0154] Before item recognition in this embodiment, the distance level of the target item being moved can be determined from the first sequence of images captured by the first camera. Then, based on this distance level and the item-taking areas of the cameras within the corresponding camera group for this distance level, the second sequence of images captured by the target camera that can clearly capture the target item is selected. In this way, the target item can be accurately recognized, improving the accuracy of item recognition. Moreover, item recognition is only based on the second sequence of images captured by the corresponding camera, reducing the workload during subsequent item recognition and enhancing the image processing speed.

[0155] The following is an embodiment of the apparatus of the present disclosure, which can be used to implement the method embodiment of the present disclosure.

[0156] Figure 10 It is a structural block diagram of an item recognition apparatus according to an embodiment of the present disclosure. This apparatus can be implemented as part or all of an electronic device through software, hardware, or a combination of both. The item recognition apparatus includes:

[0157] A first acquisition module 1001, configured to acquire a first sequence of images captured by the first camera within a preset time period;

[0158] A determination module 1002, configured to determine, based on the first sequence of images, the distance level in which the target distance between the target item being moved and the first camera is located. The distance level includes N levels, and N is an integer greater than or equal to 2;

[0159] A second acquisition module 1003, configured to acquire a second sequence of images captured by the cameras in the other camera group corresponding to the distance level in which the target distance is located within the preset time period, based on the correspondence between the N levels and N other camera groups;

[0160] An identification module 1004, configured to identify the target item based on the second sequence of images.

[0161] In one implementation of the present disclosure, N is equal to 2, the distance level includes a long-distance level and a short-distance level, and the N other camera groups include a second camera group and a third camera group; the second acquisition module 1003 is configured to:

[0162] In response to the distance level in which the target distance is located being the short-distance level, acquire the second sequence of images captured by the second camera of the second camera group corresponding to the short-distance level within the preset time period;

[0163] In response to the distance level in which the target distance is located being the long-distance level, acquire the second sequence of images captured by the third camera of the third camera group corresponding to the long-distance level within the preset time period;

[0164] Among them, the focal length of the second camera in the second camera group is less than the focal length of the third camera in the third camera group.

[0165] In one implementation of the present disclosure, the first acquisition module 1001 is configured to:

[0166] In response to detecting that the cabinet door of the smart cabinet is in an open state, acquire a first sequence of images captured by the first camera within a preset time period, where the preset time period starts from the moment when the cabinet door of the smart cabinet is detected to be opened and ends after counting for a preset duration.

[0167] In one implementation of the present disclosure, the determination module 1002 is configured to:

[0168] Input the first sequence of images into a preset classification model, execute the classification model, and output the distance level where the target distance between the target item and the first camera is located.

[0169] In one implementation of the present disclosure, the cameras in the corresponding other camera group include at least two cameras, and a part of the second acquisition module 1003 that acquires the second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located within the preset time period is configured to:

[0170] Acquire a target camera in the corresponding other camera group whose picked-up area overlaps with the picked-up area captured by the first camera;

[0171] Acquire the second sequence of images captured by the target camera within the preset time period.

[0172] In one implementation of the present disclosure, the method can also be configured to:

[0173] In response to there being at least two target cameras in the corresponding other camera group, based on the first sequence of images, determine the target area of the target item in the picked-up area captured by the first camera;

[0174] Based on the target area, determine one target camera from the at least two target cameras whose picked-up area includes the target area;

[0175] The acquisition of the second sequence of images captured by the target camera within the preset time period can be configured to:

[0176] Acquire the second sequence of images captured by the one target camera within the preset time period.

[0177] In this embodiment, the article recognition device corresponds to the above-mentioned article recognition method. For specific details, reference can be made to the description of the article recognition method above, which will not be elaborated here.

[0178] The present disclosure also discloses an electronic device. Figure 11 The schematic structural block diagram of an electronic device according to an embodiment of the present disclosure is shown, as Figure 11 shown, the electronic device 1100 includes a memory 1101 and a processor 1102; wherein the memory 1101 is used to store one or more computer instructions, and wherein, the one or more computer instructions are executed by the processor 1102 to implement the following method steps:

[0179] Obtain a first sequence of images captured by a first camera within a preset time period;

[0180] Based on the first sequence of images, determine the distance level where the target distance between the moved target article and the first camera is located. The distance level includes N levels, and N is an integer greater than or equal to 2;

[0181] Based on the corresponding relationship between the N levels and N other camera groups, obtain a second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located within the preset time period;

[0182] Identify the target article based on the second sequence of images.

[0183] In an implementation manner of the present disclosure, N is equal to 2, the distance level includes a long-distance level and a short-distance level, and the N other camera groups include a second camera group and a third camera group; the step of obtaining a second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located based on the corresponding relationship between the N levels and the N other camera groups includes:

[0184] In response to the distance level where the target distance is located being the short-distance level, obtain a second sequence of images captured by the second camera of the second camera group within the preset time period;

[0185] In response to the distance level where the target distance is located being the long-distance level, obtain a second sequence of images captured by the third camera of the third camera group within the preset time period;

[0186] Wherein, the focal length of the second camera in the second camera group is less than the focal length of the third camera in the third camera group.

[0187] In one implementation of the present disclosure, the obtaining of the first sequence of images captured by the first camera within a preset time period includes:

[0188] In response to detecting that the cabinet door of the intelligent cabinet is in an open state, obtain the first sequence of images captured by the first camera within a preset time period, where the preset time period starts from the moment when the cabinet door of the intelligent cabinet is detected to be opened and ends after counting for a preset duration.

[0189] In one implementation of the present disclosure, the determining, based on the first sequence of images, the distance level where the target distance between the moved target item and the first camera is located includes:

[0190] Input the first sequence of images into a preset classification model, execute the classification model, and output the distance level where the target distance between the target item and the first camera is located.

[0191] In one implementation of the present disclosure, the cameras in the corresponding other camera group include at least two cameras, and the obtaining of the second sequence of images captured by the cameras in the corresponding other camera group at the distance level where the target distance is located within the preset time period includes:

[0192] Obtain the target camera in the corresponding other camera group whose captured item-taking area overlaps with the item-taking area captured by the first camera;

[0193] Obtain the second sequence of images captured by the target camera within the preset time period.

[0194] In one implementation of the present disclosure, the method further includes:

[0195] In response to there being at least two target cameras in the corresponding other camera group, based on the first sequence of images, determine the target area of the target item in the item-taking area captured by the first camera;

[0196] Based on the target area, determine one target camera from the at least two target cameras whose captured item-taking area includes the target area;

[0197] The obtaining of the second sequence of images captured by the target camera within the preset time period includes:

[0198] Obtain the second sequence of images captured by the one target camera within the preset time period.

[0199] Figure 12 It is a schematic structural diagram of a computer system suitable for implementing the item recognition method according to an embodiment of the present disclosure. As Figure 12As shown, computer system 1200 includes a processing unit 1201, which can perform various processes in the above embodiments according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage section 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the system 1200 are also stored. The processing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0200] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1212, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed so that a computer program read therefrom can be installed into the storage section 1208 as needed. Among them, the processing unit 1201 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0201] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, the program segment, or the part of code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0202] The units or modules involved in the embodiments described in the present disclosure can be implemented in software or in hardware. The described units or modules can also be provided in a processor, and the names of these units or modules do not constitute a limitation to the units or modules themselves in some cases.

[0203] As another aspect, the present disclosure also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the device in the above-described embodiments; or it may exist alone and be a computer-readable storage medium not assembled into the device. The computer-readable storage medium stores one or more programs, and the one or more programs are used by one or more processors to execute the methods described in the present disclosure.

[0204] In addition, the present disclosure also provides a computer program product, in which a computer program is stored. When the computer program is executed by a processor, the processor can at least implement the methods provided in the foregoing embodiments.

[0205] The above description is only the preferred embodiments of the present disclosure and the description of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

Claims

1. An intelligent cabinet, characterized in that, it includes: A first camera group, including at least one first camera; N other camera groups, each other camera group includes at least one camera, the focal lengths of the cameras in the first other camera group are greater than the focal lengths of the cameras in the second other camera group, the first other camera group and the second other camera group are any two camera groups among the N other camera groups, and N is an integer greater than or equal to 2; wherein, the focal length of the first camera is less than the focal lengths of the cameras in the N other camera groups, and the first camera group and the N other camera groups are located on the same plane inside the intelligent cabinet; The intelligent cabinet further includes: An image processor, connected to the first camera group and the N other camera groups, for acquiring a first sequence of images captured by the first camera within a preset time period; based on the first sequence of images, determining the distance level where the target distance between the moved target item and the first camera is located; based on the correspondence between the distance level and the N other camera groups, acquiring a second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located within the preset time period, and identifying the target item based on the second sequence of images; The cameras in the corresponding other camera group include at least two cameras. When the image processor is used to acquire the second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located within the preset time period, the image processor is further used for: Acquiring a target camera in the corresponding other camera group whose captured item-taking area overlaps with the item-taking area captured by the first camera; Acquiring the second sequence of images captured by the target camera within the preset time period.

2. The intelligent cabinet according to claim 1, characterized in that, N is 2, and the N other camera groups include a second camera group and a third camera group; wherein, the focal length of the second camera in the second camera group is less than the focal length of the third camera in the third camera group.

3. The intelligent cabinet according to claim 1, characterized in that, The first camera group and the N other camera groups are located at the top inside the intelligent cabinet.

4. An item identification method, characterized in that, applied to the intelligent cabinet according to any one of claims 1 to 3, the method includes: Acquiring a first sequence of images captured by the first camera within a preset time period; Based on the first sequence of images, determining the distance level where the target distance between the moved target item and the first camera is located, the distance level includes N levels, and N is an integer greater than or equal to 2; Based on the correspondence between the N levels and the N other camera groups, acquiring a second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located within the preset time period; Identifying the target item based on the second sequence of images; The cameras in the corresponding other camera groups include at least two cameras, and obtaining the second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located within the preset time period includes: Obtaining a target camera in the corresponding other camera group whose captured object area overlaps with the object area captured by the first camera; Obtaining the second sequence of images captured by the target camera within the preset time period.

5. The method according to claim 4, wherein, N is equal to 2, the distance levels include a long-distance level and a short-distance level, and the N other camera groups include a second camera group and a third camera group; based on the correspondence between the N levels and the N other camera groups, obtaining the second sequence of images captured by the cameras in the other camera group corresponding to the distance level where the target distance is located within the preset time period includes: In response to the distance level where the target distance is located being the short-distance level, obtaining the second sequence of images captured by the second camera of the second camera group corresponding to the short-distance level within the preset time period; In response to the distance level where the target distance is located being the long-distance level, obtaining the second sequence of images captured by the third camera of the third camera group corresponding to the long-distance level within the preset time period; wherein, the focal length of the second camera in the second camera group is less than the focal length of the third camera in the third camera group.

6. The method according to claim 4, wherein, Obtaining the first sequence of images captured by the first camera within the preset time period includes: In response to detecting that the cabinet door of the smart cabinet is in an open state, obtaining the first sequence of images captured by the first camera within the preset time period, and the preset time period starts from the moment when the cabinet door of the smart cabinet is detected to be opened and ends after counting a preset duration.

7. The method according to claim 4, wherein, Based on the first sequence of images, determining the distance level where the target distance between the moved target item and the first camera is located includes: Inputting the first sequence of images into a preset classification model, executing the classification model, and outputting the distance level where the target distance between the target item and the first camera is located.

8. The method according to claim 4, wherein, The method further includes: In response to there being at least two target cameras in the corresponding other camera group, based on the first sequence of images, determining the target area of the target item in the object area captured by the first camera; Based on the target area, determining one target camera from the at least two target cameras whose captured object area includes the target area; The obtaining the second sequence of images captured by the target camera within the preset time period includes: Obtaining the second sequence of images captured by the one target camera within the preset time period.

9. An electronic device, wherein, including a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the method according to any one of claims 4-8.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that, when the computer instructions are executed by a processor, the method according to any one of claims 4-8 is implemented.

11. A computer program product comprising computer instructions, characterized in that, when the computer instructions are executed by a processor, the method according to any one of claims 4-8 is implemented.

Images