Article identification method, apparatus, device, and medium

Abstract

Embodiments of the present disclosure disclose an article identification method, device, equipment and medium, the method comprising: acquiring at least two to-be-identified images collected by a display cabinet at a target time from different image collection angles respectively, and determining a target image region in each to-be-identified image; performing image identification on the target image region to obtain to-be-identified article information and to-be-identified posture position information; acquiring a simulated collection image according to the to-be-identified article information and the to-be-identified posture position information, and acquiring a matching degree of each to-be-identified image and the corresponding simulated collection image; in response to the matching degree of each to-be-identified image and the corresponding simulated collection image belonging to a target matching degree interval, determining a target article to be removed from the display cabinet or moved into the display cabinet according to the to-be-identified article information. The scheme can ensure that the accuracy of the information of the determined article to be removed from the display cabinet or moved into the display cabinet is relatively high, and the user experience is improved.

Description

Technical Field

[0001] This disclosure relates to the field of control technology, specifically to an article identification method, device, equipment, and medium. Background Technology

[0002] In recent years, to facilitate user access to product information, merchants and businesses have often placed items in display cases, simultaneously storing and showcasing goods. When a user needs an item from the display case, they can open it and retrieve it. In this scenario, the display case can identify the item being removed and upload the results. This allows other devices or systems, such as servers or cloud platforms, to use the uploaded identification data to determine the user or item removed from the display case for statistical purposes. Summary of the Invention

[0003] This disclosure provides an article identification method, apparatus, device, and medium.

[0004] Firstly, this disclosure provides an item identification method.

[0005] Specifically, object identification methods include:

[0006] Acquire at least two images to be identified from different image acquisition angles captured by the display case at a target time, and determine the target image region in each image to be identified. The target image region includes the user's hand and the item to be identified located in the target region. The target region includes at least a part of the item entrance / exit of the display case.

[0007] Image recognition is performed on the target image region in at least two images to be recognized in order to obtain information on the object to be recognized and the position information of the object to be recognized. The information on the object to be recognized is used to indicate the type and quantity of the object to be recognized, and the position information of the object to be recognized is used to indicate the position and posture of the user's hand and the position and posture of the object to be recognized.

[0008] Based on the information of the object to be identified and the posture and position information to be identified, obtain the simulated acquisition image corresponding to each image to be identified, and obtain the matching degree between each image to be identified and the corresponding simulated acquisition image. The image acquisition angle of the simulated acquisition image is the same as the image acquisition angle of the corresponding image to be identified.

[0009] In response to the fact that the matching degree between each image to be identified and the corresponding simulated acquired image is within the target matching degree range, the target item to be identified is determined to be moved out of or into the display case based on the information of the item to be identified.

[0010] In one implementation of this disclosure, a simulated image corresponding to each image to be identified is obtained based on the information of the object to be identified and the posture and position information to be identified, including:

[0011] Construct a 3D model of the target based on the information of the object to be identified and its posture and position.

[0012] Based on the target 3D model, obtain the simulated acquisition image corresponding to each image to be identified.

[0013] In one implementation of this disclosure, before determining the target image region in each image to be recognized, the method further includes:

[0014] Obtain an ambient background image corresponding to the image acquisition angle of each image to be identified. The ambient background image includes at least a portion of the item entrance / exit of the display case, but does not include the user's hand or any of the items to be identified.

[0015] Identify the target image region in each image to be identified, including:

[0016] Each image to be identified is compared with the background image to determine the target image region in each image to be identified.

[0017] In one implementation of this disclosure, the ambient background image corresponding to the image acquisition angle is acquired by an image acquisition device on the display cabinet corresponding to the image acquisition angle within a preset time period after the cabinet door is unlocked.

[0018] In one implementation of this disclosure, a simulated image corresponding to each image to be identified is obtained based on the information of the object to be identified and the posture and position information to be identified, including:

[0019] Based on the information of the object to be identified, the posture and position information of the object to be identified, and the background image of the environment, a simulated image is acquired for each image to be identified.

[0020] In one implementation of this disclosure, image recognition is performed on target image regions in at least two images to be recognized to obtain information about the object to be recognized and the pose and position information to be recognized, including:

[0021] User hand recognition is performed on the target image region in at least two images to be recognized, and the target object image region containing only the object to be recognized is determined based on the hand recognition result.

[0022] Identify the target object image region to obtain information about the object to be identified;

[0023] Image recognition is performed on the target image region in at least two images to be recognized in order to obtain overlapping occlusion information. The overlapping occlusion information is used to indicate the positional relationship between the object to be recognized, the user's hand, and the image acquisition device used to acquire the image to be recognized in the corresponding image to be recognized.

[0024] The pose and position information of the object to be identified is obtained based on the information of overlapping occlusion relationships and the information of the object to be identified.

[0025] In one implementation of this disclosure, the method further includes:

[0026] In response to the fact that the matching degree between any image to be identified and the corresponding simulated acquired image does not fall within the target matching degree range, the information on the corrected items to be identified is obtained, and the number of items to be identified indicated by the corrected information is less than the number of items to be identified indicated by the corrected information.

[0027] Based on the corrected information of the object to be identified and the posture and position information to be identified, obtain the corrected simulated acquisition image corresponding to each image to be identified, and obtain the matching degree between each corrected simulated acquisition image and the corresponding simulated acquisition image. The image acquisition angle of the corrected simulated acquisition image is the same as the image acquisition angle of the corresponding image to be identified.

[0028] In response to the fact that the matching degree between each image to be identified and the corresponding corrected simulated acquisition image is within the target matching degree range, the target item to be removed from or moved into the display case is determined based on the corrected information of the item to be identified.

[0029] Secondly, this disclosure provides an item identification device.

[0030] Specifically, the item recognition device includes:

[0031] The image acquisition module is configured to acquire at least two images to be identified from different image acquisition angles at a target time, and to determine the target image region in each image to be identified. The target image region includes the user's hand and the item to be identified located in the target region. The target region includes at least a portion of the item entrance / exit of the display case.

[0032] The image recognition module is configured to perform image recognition on target image regions in at least two images to be recognized in order to obtain information on the object to be recognized and information on the posture and position to be recognized. The information on the object to be recognized is used to indicate the type and quantity of the object to be recognized, and the information on the posture and position to be recognized is used to indicate the position and posture of the user's hand and the position and posture of the object to be recognized.

[0033] The image simulation module is configured to acquire a simulated acquisition image corresponding to each image to be identified based on the information of the object to be identified and the posture and position information to be identified, and to acquire the matching degree between each image to be identified and the corresponding simulated acquisition image. The image acquisition angle of the simulated acquisition image is the same as the image acquisition angle of the corresponding image to be identified.

[0034] The item recognition module is configured to determine the target item to be moved out of or into the display case based on the item information, in response to the fact that the matching degree between each image to be recognized and the corresponding simulated acquisition image both fall within the target matching degree range.

[0035] Thirdly, this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method as described in the first aspect or any implementation thereof.

[0036] Fourthly, this disclosure provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the method as described in the first aspect or any implementation thereof.

[0037] Fifthly, this disclosure provides a computer program product including computer instructions that, when executed by a processor, implement the method as described in the first aspect or any implementation thereof.

[0038] The technical solutions provided in this disclosure may have the following beneficial effects:

[0039] In the above technical solution, at least two images to be identified are acquired from different image acquisition angles at a target time from the display case, and a target image region is determined in each image. This target image region includes the user's hand at the item entrance / exit of the display case and the item to be identified, i.e., the user's hand and items that may be moved into or out of the display case by the user's hand. Image recognition is performed on the target image region in the at least two images to obtain information about the item to be identified and information about the posture and position of the item to be identified. The information about the item to be identified indicates the type and quantity of the item, i.e., the type and quantity of items that may be moved into or out of the display case by the user's hand. The information about the posture and position of the item to be identified indicates the position and posture of the user's hand and the position and posture of the item to be identified, i.e., the position and posture of the item when it is moved into or out of the display case by the user's hand. The system determines the position and orientation of the item in three-dimensional space. Based on the information of the item to be identified and its orientation, a simulated image is acquired for each image to be identified. The matching degree between each image to be identified and its corresponding simulated image is calculated. The image acquisition angle of the simulated image is the same as that of the corresponding image to be identified. This means the simulated image is estimated based on previously acquired information about the item to be identified and its orientation. If the previously acquired information is accurate, the matching degree between the image to be identified and its corresponding simulated image is also high, falling within the target matching degree range. Therefore, in response to the matching degree of each image to be identified falling within the target matching degree range, the system determines the target item to be moved out of or into the display case based on the item information. This ensures a high accuracy rate in identifying the items to be moved out of or into the display case, facilitating appropriate settlement and improving the user experience.

[0040] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0041] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:

[0042] Figure 1 A schematic structural block diagram of a display cabinet according to an embodiment of the present disclosure is shown.

[0043] Figure 2 A schematic structural block diagram of a motherboard according to an embodiment of the present disclosure is shown.

[0044] Figure 3 A schematic structural block diagram of a control panel according to an embodiment of the present disclosure is shown.

[0045] Figure 4 A schematic structural block diagram of a power management module according to an embodiment of the present disclosure is shown.

[0046] Figure 5 A flowchart of an article identification method according to an embodiment of the present disclosure is shown.

[0047] Figure 6 A schematic structural diagram of a display cabinet according to an embodiment of the present disclosure is shown.

[0048] Figure 7 A schematic top view of a display case according to one embodiment of the present disclosure is shown.

[0049] Figure 8 A schematic structural block diagram of an article identification device according to an embodiment of the present disclosure is shown.

[0050] Figure 9 A schematic structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0051] Figure 10 This is a schematic diagram of the structure of a computer system suitable for implementing an article identification method according to an embodiment of the present disclosure. Detailed Implementation

[0052] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement them. Furthermore, for clarity, portions unrelated to the description of the exemplary embodiments have been omitted from the drawings.

[0053] In this disclosure, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, figures, steps, behaviors, components, parts or combinations thereof disclosed in this specification, and do not preclude the possibility of the presence or addition of one or more other features, figures, steps, behaviors, components, parts or combinations thereof.

[0054] It should also be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0055] As mentioned above, with the development of technology and the improvement of people's living standards, businesses no longer simply place goods on shelves. Instead, to facilitate users' understanding of product information, they can place items in display cases, achieving simultaneous storage and display of goods. When users need to remove items from the display cases, they can open the cases themselves and perform the corresponding operations.

[0056] In recent years, the number of display cases put into operation has gradually increased. During the use of display cases, merchants or enterprises generally need to identify the items in the display cases to determine the items that have been moved out of or into the display cases. Based on the above information, statistics can be compiled. Based on the statistical results, the quantity and types of remaining items in the display cases can be known. In addition, further settlement can be carried out based on the statistical results.

[0057] In order to ensure accurate identification of items being removed from or moved into the display case, multiple cameras installed on the display case can be used to capture multiple images from different angles after the display case is unlocked. The captured images are then subjected to image recognition, and the information of the items being removed from or moved into the display case is determined based on the image recognition results.

[0058] However, during implementation, the applicant discovered that due to the rapid increase in the types and quantities of items stored in the display cases, and the significant differences in the shapes and sizes of different types of items, users' hands might obscure some smaller items in the captured images. Furthermore, when users move multiple items out of or into the display cases at once, these items might overlap, causing mutual occlusion in the captured images. Consequently, the accuracy of image recognition based on a single captured image to determine the items moved out of or into the display cases was poor.

[0059] In view of the above-mentioned shortcomings, in one embodiment of this disclosure, an item recognition method is proposed. This method acquires at least two images of a display case taken from different image acquisition angles at a target time, and determines a target image region in each image. This target image region includes a user's hand located at the item entrance / exit of the display case, as well as the item to be recognized, i.e., the user's hand and items that may be moved into or out of the display case by the user's hand. Image recognition is performed on the target image region in the at least two images to obtain information about the item to be recognized and information about the posture and position of the item to be recognized. The information about the item to be recognized indicates the type and quantity of the item to be recognized, i.e., the type and quantity of items that may be moved into or out of the display case by the user's hand. The information about the posture and position of the item to be recognized indicates the position and posture of the user's hand and the position and posture of the item to be recognized, i.e., the position and posture of the item when moved into or out of the display case by the user's hand. The system determines the position and posture of items being moved into or out of the display case in three-dimensional space. Based on the information of the item to be identified and its posture, a simulated image is acquired for each image to be identified. The matching degree between each image to be identified and its corresponding simulated image is calculated. The image acquisition angle of the simulated image is the same as that of the corresponding image to be identified; that is, the simulated image is estimated based on the previously acquired information of the item to be identified and its posture. If the previously acquired information is accurate, the matching degree between the image to be identified and its corresponding simulated image is also high, falling within the target matching degree range. Therefore, in response to the matching degree of each image to be identified falling within the target matching degree range, the system determines the target item to be moved out of or into the display case based on the item information. This ensures a high accuracy rate in identifying the items moved out of or into the display case, facilitating appropriate calculations and improving the user experience.

[0060] The item identification method provided in this application embodiment can be applied to display cabinets, which can have a temperature control function. The temperature control function can be a cooling function, such as a refrigerated display cabinet, a frozen display cabinet, a refrigerator, a wine cabinet, a cosmetic preservation cabinet, etc.; the temperature control function can also be a heating function, such as a warm cabinet, a heated display cabinet, a hot beverage cabinet, etc. This application embodiment does not limit the specific type of display cabinet.

[0061] For example, Figure 1 A schematic structural block diagram of a display cabinet according to an embodiment of the present disclosure is shown, such as... Figure 1As shown, the display cabinet 100 may include a compressor 11, a condenser 12, a throttling element 13, and an evaporator 14. The compressor 11, condenser 12, throttling element 13, and evaporator 14 are connected by pipes filled with refrigerant to form a closed pipeline, which constitutes a refrigeration system or heating system capable of circulating refrigerant.

[0062] The compressor refers to a driven fluid machine used to upgrade low-pressure refrigerant to high-pressure refrigerant. The compressor can draw in low-temperature, low-pressure gaseous refrigerant, and after the refrigerant is compressed by the piston driven by the motor, it discharges high-temperature, high-pressure gaseous refrigerant to provide power for the refrigeration cycle. The compressor can include reciprocating compressors, screw compressors, rotary compressors, scroll compressors and centrifugal compressors, etc. The specific type of compressor is not limited in the embodiments of this application.

[0063] A condenser is a heat exchanger used to exchange heat between the refrigerant inside the condenser and the air outside, thereby releasing heat. Specifically, a condenser may include a long pipe for containing the refrigerant, typically made of a metal with high thermal conductivity such as copper, and often coiled into a spiral shape. Furthermore, to improve the heat exchange efficiency of the condenser, heat sinks with excellent thermal conductivity can be installed on the pipes to increase the heat dissipation area, thereby accelerating the heat exchange rate and improving efficiency. Additionally, installing a fan or blower matched to the condenser can increase the airflow around it, further accelerating heat exchange and improving efficiency.

[0064] A throttling element is used to reduce the pressure of room-temperature, high-pressure liquid refrigerant, transforming it into a low-temperature, low-pressure gaseous refrigerant. This element can also be called a throttling device or a regulating valve, and may include expansion valves, capillary tubes, etc. Furthermore, the throttling element controls the flow rate of the refrigerant passing through it, preventing excessive or insufficient flow. If the flow rate is too high, the refrigerant exiting the element will still contain liquid refrigerant, which can cause liquid slugging and damage to the compressor. Conversely, if the flow rate is too low, insufficient refrigerant will enter the compressor, reducing its efficiency.

[0065] An evaporator is a heat exchanger used to exchange heat between the refrigerant inside the evaporator and the air outside the condenser, thereby achieving heat absorption. Specifically, an evaporator may include a long pipe for containing the refrigerant. This pipe is typically made of a metal with high thermal conductivity, such as copper, and is often coiled into a spiral shape. Furthermore, to improve the heat exchange efficiency of the condenser, heat sinks with excellent thermal conductivity can be installed on the pipe to increase the heat dissipation area, thereby accelerating the heat exchange rate and improving efficiency. Additionally, a fan or blower matched to the evaporator can be used to increase the airflow around the evaporator, further accelerating heat exchange and improving efficiency.

[0066] Refrigerant, also known as coolant or refrigerant fluid, refers to the medium through which energy conversion is completed in a refrigeration or heating system. Refrigerants are typically substances that readily undergo reversible phase changes (e.g., absorbing heat to become a gas and releasing heat to become a liquid). Through these reversible phase changes, refrigerants can transfer heat. Specifically, a gaseous refrigerant releases heat and becomes a liquid when pressurized, and absorbs heat when the high-pressure liquid is depressurized and becomes a gas. Refrigerants can include ammonia, air, water, brine, and Freon (also known as chlorofluorocarbons or fluorocarbons), among which Freon can include chlorofluoromethane, dichlorofluoromethane, trifluoromethane, tetrafluoroethane, and dichlorofluoroethane.

[0067] When the display case is equipped with a refrigeration function, low-temperature, low-pressure gaseous refrigerant flows from the evaporator into the compressor. The compressor compresses the low-temperature, low-pressure gaseous refrigerant, causing the high-temperature, high-pressure gaseous refrigerant to flow into the condenser. The high-temperature, high-pressure gaseous refrigerant exchanges heat with the outside air through the condenser, cooling it into a normal-temperature, high-pressure liquid refrigerant. This liquid refrigerant then flows into a throttling element, which restricts the flow, causing the refrigerant exiting the element to become a low-temperature, low-pressure liquid refrigerant. This low-temperature, low-pressure liquid refrigerant flows into the evaporator, where it exchanges heat with the outside air, evaporating and vaporizing into a low-temperature, low-pressure gaseous refrigerant to absorb heat. In this system, outside air can be introduced into the storage area of ​​the display case through the evaporator, and outside air can be introduced into the outside of the display case through the condenser, thereby transferring the heat in the storage area of ​​the display case to the outside of the display case and cooling the storage area.

[0068] When the display case is equipped with a heating function, low-temperature, low-pressure vaporous refrigerant flows from the condenser into the compressor. The compressor compresses the low-temperature, low-pressure vaporous refrigerant, causing the high-temperature, high-pressure vaporous refrigerant to flow into the evaporator. The high-temperature, high-pressure vaporous refrigerant exchanges heat with the outside air through the evaporator, cooling it into a normal-temperature, high-pressure liquid refrigerant. This liquid refrigerant then flows into a throttling element, which throttles the flow, causing the refrigerant exiting the element to become a low-temperature, low-pressure liquid refrigerant. This low-temperature, low-pressure liquid refrigerant flows into the condenser, where it exchanges heat with the outside air, evaporating and vaporizing into a low-temperature, low-pressure gaseous refrigerant to absorb heat. The system allows outside air to be introduced into the storage area of ​​the display case, and outside air to be introduced into the outside of the display case, thereby transferring heat from outside the display case to the storage area to heat the storage area.

[0069] In one embodiment of this application, the display cabinet includes a cabinet body and a cabinet door, wherein a control board and a power management module may be installed in the cabinet body, and a main board may be installed in the cabinet door.

[0070] In one embodiment of this application, Figure 2 A schematic structural block diagram of a motherboard according to an embodiment of the present disclosure is shown, such as... Figure 2 As shown, the motherboard 200 includes a processor 201, random access memory 202, flash memory 203, wireless LAN Bluetooth module 204, gyroscope 205, pressure sensor 206, microphone 207, speaker 208, camera 209, and cellular communication module 210.

[0071] A processor may include one or more processing units, such as an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a memory, a video codec, a digital signal processor, a baseband processor, and / or a neural network processor. The different processing units may be independent devices or integrated into one or more processors.

[0072] The image signal processor (Image Signal Processor) processes data fed back from the camera. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then transmitted to the Image Signal Processor for processing, transforming it into a visible image. The Image Signal Processor can also perform algorithmic optimizations on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the Image Signal Processor can be integrated into the camera itself.

[0073] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, DSPs can be used to perform Fourier transforms on frequency energy.

[0074] Video codecs are used to compress or decompress digital video. A display case can support one or more video codecs. This allows the display case to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0075] Neural network computing processors, by drawing inspiration from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, can rapidly process input information and continuously learn on their own. These processors can enable applications such as intelligent cognition in display cases, including image recognition, facial recognition, speech recognition, and text understanding.

[0076] In some embodiments, the processor may include one or more interfaces. Interfaces may include integrated circuit interfaces, integrated circuit built-in audio interfaces, pulse code modulation interfaces, universal asynchronous transceiver interfaces, mobile industry processor interfaces, universal input / output interfaces, user identity module interfaces, and / or universal serial bus interfaces, etc.

[0077] Random access memory 202 can be used to store computer executable program code, which includes instructions and data. Processor 201 executes various functional applications and data processing of the display case by running the instructions stored in random access memory 202. Random access memory 202 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the display case (such as audio data, image data, etc.).

[0078] Flash memory 203 can be used to expand the storage capacity of the display case. Flash memory 203 can communicate with processor 201 via the flash memory interface to achieve data storage functionality. For example, it can store music, video, and other files in the flash memory.

[0079] A minimal system can be constructed using processor 201, random access memory 202, and flash memory 203 to provide the system operating environment.

[0080] The Bluetooth LAN module 204 can provide wireless communication solutions for display cases, including wireless LAN, Bluetooth, GPS, FM, NNHF, and infrared technologies. The Bluetooth LAN module 204 can be one or more devices integrating at least one communication processing module. The Bluetooth LAN module 204 receives electromagnetic waves via an antenna, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to the processor 201. The Bluetooth LAN module 204 can also receive signals to be transmitted from the processor 201, perform frequency modulation and amplification, and then convert them into electromagnetic waves for radiation via the antenna. In one embodiment of this application, the Bluetooth LAN module can communicate with a user's terminal.

[0081] The cellular communication module 210 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use in display cases. The cellular communication module 210 may include at least one filter, switch, power amplifier, low-noise amplifier, etc. The cellular communication module 210 can receive electromagnetic waves via an antenna, filter and amplify the received electromagnetic waves, and transmit them to a modem processor for demodulation. The cellular communication module 210 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna. In some embodiments, at least some functional modules of the cellular communication module 210 may be housed in the processor 201. In some embodiments, at least some functional modules of the cellular communication module 210 and at least some modules of the processor 201 may be housed in the same device. In one embodiment of this application, the cellular communication module 210 can communicate with the cloud server of the display case operator.

[0082] Through the Bluetooth wireless LAN module 204 and the cellular communication module 210, the display case can communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSMA), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband CDMA, Time Division CDMA, and Long Term Evolution (LTE).

[0083] The gyroscope 205 can be used to determine the real-time attitude of the display case doors.

[0084] The pressure sensor 206 is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor 206 can be disposed on the display screen. There are many types of pressure sensors 206, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive materials. When a force is applied to the pressure sensor 206, the capacitance between the electrodes changes, and the pressure intensity is determined based on the change in capacitance. When a touch operation is applied to the display screen, the intensity of the touch operation is identified by the pressure sensor 206, and the touch position can also be calculated based on the identification signal from the pressure sensor 206. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the beverage selection app icon, a command to view specific beverage information is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the beverage selection app icon, a command to purchase a beverage is executed.

[0085] Microphone 207, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 207, inputting the sound signal into microphone 207. A display case can be equipped with at least one microphone 207. In some embodiments, the display case can be equipped with two microphones 207, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, the display case can be equipped with three, four, or more microphones 207, enabling sound signal collection, noise reduction, sound source identification, and directional recording, among other functions. In one embodiment of this application, the microphone 207 can be used to collect the sound of the display case during operation.

[0086] Speaker 208, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The display case can play music or announcements via speaker 208.

[0087] Camera 209 is used to capture images, including still images and moving images (i.e., video). An object is projected onto a photosensitive element by an optical image generated through a lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an image signal processor (ISS) for conversion into a digital image signal. The ISS outputs the digital image signal to a digital signal processor (DSP) for further processing. The DSP converts the digital image signal into image signals in standard formats such as RGB and YUV. In some embodiments, the display case may include one or more cameras 209. In one embodiment of this application, the camera 209 may have a self-heating function to ensure that its lens does not fog up.

[0088] In one embodiment of this application, Figure 3 A schematic structural block diagram of a control panel according to an embodiment of the present disclosure is shown, such as... Figure 3 As shown, the control board 300 includes a power input interface 301, a power output interface 302, a metering chip 303, a microcontroller chip 304, a real-time clock chip, a light switch interface 305, a temperature control switch interface 306, an evaporator fan interface 307, a compressor interface 308, a condenser fan interface 309, a temperature sensor interface 310, a communication interface 311, and a power interface 312.

[0089] The metering chip 303, also known as the power sensor, acquires voltage, current, real-time power, and average power data. A real-time clock chip maintains the time for the microcontroller chip 304. The light switch interface 305 receives control signals from the display case's light switch. The temperature control switch interface 306 receives control signals from the display case's temperature control switch. The evaporator fan interface 307 sends control signals to the evaporator fan to control its operation. The compressor interface 308 sends control signals to the compressor to control its operation. The condenser fan interface 309 sends control signals to the condenser fan to control its operation. The temperature sensor interface 310 receives temperature data from one or more temperature sensors to determine the temperature at one or more locations within the display case.

[0090] In one embodiment of this application, Figure 4 A schematic structural block diagram of a power management module according to an embodiment of the present disclosure is shown, such as... Figure 4As shown, the power management module 400 includes an AC-to-DC conversion module 401, a charging management module 402, and a battery 403. The power management module 400 supplies power to the motherboard and control board and manages the charging and discharging of the battery. The power management module 400 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 400 may also be located within the processor.

[0091] In one embodiment of this application, the display case further includes a display screen. The display case implements its display function through a graphics processor, a display screen, and an application processor. The graphics processor is a microprocessor for image processing, connected to the display screen and the application processor. The graphics processor is used to perform mathematical and geometric calculations and for graphics rendering. The processor may include one or more graphics processors that execute program instructions to generate or modify display information.

[0092] The display screen is used to display still images, videos, etc. The display screen includes a display panel. The display panel can be a liquid crystal display, an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode, a quantum dot light-emitting diode, etc. In some embodiments, the display case may include one or more display screens.

[0093] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the display cabinet. In other embodiments of this application, the display cabinet may include more or fewer components than illustrated, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. For example, by combining different components, the display cabinet in the embodiments of this application may be any of the following: a retail cabinet, a heated cabinet, a refrigerated cabinet, a freezer, a combination cabinet, or a display case.

[0094] Figure 5 The flowchart illustrates an article identification method according to an embodiment of the present disclosure. This method can be applied to a server or a terminal device. The server may include a data processing server such as a cloud server or a distributed server, and the terminal device may include a smart mobile communication terminal, a tablet computer, a laptop computer, a desktop computer, a wearable device, etc.

[0095] like Figure 5 As shown, the item identification method includes the following steps S101-S104:

[0096] In step S101, at least two images to be identified are acquired from different image acquisition angles at the target time by the display case, and the target image region in each image to be identified is determined.

[0097] The target image area includes the user's hand and the item to be identified located in the target area, and the target area includes at least a portion of the item entrance / exit of the display case.

[0098] In one embodiment of this disclosure, acquiring the image to be identified can be achieved by receiving an image to be identified sent by an image acquisition device matched with the display case, by reading a pre-stored image to be identified, or by receiving an image to be identified sent by other devices or systems. The image to be identified can be a static image, a dynamic image, or a video.

[0099] In one embodiment of this disclosure, the item entry / exit of the display case can be understood as follows: items stored in the display case can only be moved out of the display case through the item entry / exit, and items outside the display case can only be moved into the display case through the item entry / exit. The item entry / exit can be closed or opened through the cabinet door of the display case.

[0100] It should be noted that, provided that at least a portion of the item entrance / exit of the display case is included, the image to be identified may also include images of all or part of the display case, or images of all or part of the items in the display case.

[0101] In one embodiment of this disclosure, the image acquisition device to be recognized, matched with the display case, can be an image acquisition device independent of the display case and communicating with it via wired or wireless means; alternatively, the image acquisition device to be recognized, matched with the display case, can also be an image acquisition device installed on the cabinet body or door of the display case. For example, the image acquisition device to be recognized can be installed on the top or bottom of the door, on the side wall of the door near the cabinet body, on the side wall of the door away from the cabinet body, or on the door handle. This application does not limit the specific installation location of the image acquisition device to be recognized. The display case may include one or more image acquisition devices to be recognized, and this application does not limit the specific number of image acquisition devices to be recognized.

[0102] For example, Figure 6 A schematic structural diagram of a display cabinet according to an embodiment of the present disclosure is shown. Figure 7 A schematic top view of a display case according to one embodiment of the present disclosure is shown, such as... Figure 6 as well as Figure 7 As shown, the display case includes a cabinet body 501, a cabinet door 502, and an image acquisition device 503 to be recognized, wherein the image acquisition device 503 to be recognized is connected to the top surface of the inner wall of the cabinet body 501.

[0103] The cabinet 501 includes a display area 511 and an item entrance / exit 521. The display area 511 is connected to the outside of the cabinet 501 through the item entrance / exit 521. The display area 511 is used to accommodate items stored in the display cabinet. Items in the display area 511 can be moved out of the display cabinet through the item entrance / exit 521, or items can be moved into the display area 511 through the item entrance / exit 521.

[0104] The cabinet doors can be rotatably connected to the cabinet body, or they can be slidably connected to the cabinet body. Alternatively, the cabinet doors can be connected to the cabinet body by folding. It should be noted that this application does not specifically limit the way the cabinet doors are connected to the cabinet body. For ease of understanding, this is just a general description. Figure 6 as well as Figure 7 The following explanation uses the rotating connection between cabinet door 502 and cabinet body 501 as an example. Cabinet door 502 is used to open or close the item access entrance 521.

[0105] The image acquisition device 503 can be used to acquire at least one of the following: images of all display areas 511, images of a portion of display areas 511, and images of the item entrance / exit 521, to obtain the image to be identified. It should be noted that the display case may include only one image acquisition device 503, or it may include multiple image acquisition devices 503 that acquire images from different directions. This disclosure does not impose a specific limit on the number of image acquisition devices 503 to be identified.

[0106] In one embodiment of this disclosure, determining the target image region in each image to be identified can be understood as determining at least a portion of the target region containing the item entrance / exit of the display case in each image to be identified, and performing image recognition on the target region. The result of the image recognition can be used to indicate whether the target region contains a user's hand and an item to be identified, and the target region is determined to be a target image region based on the result of the image recognition. Alternatively, image recognition can be performed on each image to be identified, and the result of the image recognition can be used to indicate the position of the user's hand and the item to be identified in the image to be identified. When the result of the image recognition determines that the user's hand and the item to be identified are located in the corresponding target region, the corresponding target region is determined to be a target image region.

[0107] In step S102, image recognition is performed on the target image region in at least two images to be identified in order to obtain information about the object to be identified and the posture and position information to be identified.

[0108] Among them, the information on the object to be identified is used to indicate the type and quantity of the object to be identified, and the information on the posture and position to be identified is used to indicate the position and posture of the user's hand and the position and posture of the object to be identified.

[0109] In one embodiment of this disclosure, the type of the item to be identified can be understood as the item name, item price, item storage number, etc.

[0110] In one embodiment of this disclosure, the position of the user's hand can be understood as the position coordinates of at least one point on the surface of the user's hand or the center of the user's hand in three-dimensional space, wherein the position coordinates can be taken as the pole of a certain point on the display case.

[0111] In one embodiment of this disclosure, the posture of the user's hand can be understood as the angle between at least one axis of the user's hand (e.g., the central axis of the palm, or the central axis of the knuckle of any finger closest to the palm, etc.) and at least one axis of the display case (e.g., the central axis of the display case, or an axis parallel to any edge of the display case, etc.).

[0112] In one embodiment of this disclosure, the position of the item to be identified can be understood as the position coordinates of at least one point on the surface of the item to be identified or the center of the item to be identified in three-dimensional space, wherein the position coordinates can be taken as the pole of a certain point on the display case.

[0113] In one embodiment of this disclosure, the orientation of the object to be identified can be understood as the angle between at least one axis of the object to be identified (e.g., the central axis of the object to be identified, or an axis parallel to any face of the object to be identified) and at least one axis of the display case (e.g., the central axis of the display case, an axis parallel to any edge of the display case).

[0114] In step S103, the simulated acquisition image corresponding to each image to be identified is obtained based on the information of the object to be identified and the posture and position information to be identified, and the matching degree between each image to be identified and the corresponding simulated acquisition image is obtained.

[0115] In this case, the image acquisition angle of the simulated image is the same as the image acquisition angle of the corresponding image to be identified.

[0116] In one embodiment of this disclosure, the simulated acquisition image corresponding to each image to be identified is obtained based on the information of the object to be identified and the posture and position information to be identified. This can be understood as substituting the information of the object to be identified and the posture and position information to be identified into a pre-acquired algorithm to perform calculations and obtain the corresponding simulated acquisition image; or it can be understood as taking the information of the object to be identified and the posture and position information to be identified as inputs into a pre-acquired simulated acquisition image model to obtain the simulated acquisition image output by the simulated acquisition image model.

[0117] In one embodiment of this disclosure, the matching degree between the image to be identified and the corresponding simulated acquired image can be understood as indicating the degree of similarity between the image to be identified and the corresponding simulated acquired image. The more similar the two are, the greater the matching degree can be; or, the more similar the two are, the smaller the matching degree can be.

[0118] In step S104, in response to the fact that the matching degree of each image to be identified and the corresponding simulated acquisition image both belong to the target matching degree range, the target item to be identified is determined to be moved out of or into the display case based on the information of the item to be identified.

[0119] In one embodiment of this disclosure, the matching degree between the image to be identified and the corresponding simulated acquisition image both fall within the target matching degree range, which can be understood as the similarity between the two meeting the requirements, that is, the situation in the simulated acquisition images corresponding to the multiple images to be identified is close enough to the real situation.

[0120] In the technical solution provided in this disclosure, at least two images to be identified are acquired from different image acquisition angles at a target time from the display case, and a target image region is determined in each image to be identified. This target image region includes the user's hand at the item entrance / exit of the display case and the item to be identified, i.e., the user's hand and items that may be moved into or out of the display case by the user's hand. Image recognition is performed on the target image region in the at least two images to obtain information about the item to be identified and information about the posture and position of the item to be identified. The information about the item to be identified indicates the type and quantity of the item to be identified, i.e., the type and quantity of items that may be moved into or out of the display case by the user's hand. The information about the posture and position of the item to be identified indicates the position and posture of the user's hand and the position and posture of the item to be identified, i.e., when the item is moved into or out of the display case by the user's hand. The system determines the position and orientation of the two objects in three-dimensional space. Based on the information of the object to be identified and its orientation, a simulated image is acquired for each image to be identified. The matching degree between each image to be identified and its corresponding simulated image is then calculated. The image acquisition angle of the simulated image is the same as that of the corresponding image to be identified. This means the simulated image is estimated based on previously acquired information about the object to be identified and its orientation. If the previously acquired information is accurate, the matching degree between the image to be identified and its corresponding simulated image is also high, falling within the target matching degree range. Therefore, in response to the matching degree of each image to be identified falling within the target matching degree range, the system determines the target object to be moved out of or into the display case based on the object to be identified information. This ensures a high accuracy rate in identifying the items to be moved out of or into the display case, facilitating appropriate settlement and improving the user experience.

[0121] In one implementation of this disclosure, a simulated image corresponding to each image to be identified is obtained based on the information of the object to be identified and the posture and position information to be identified, including:

[0122] Construct a 3D model of the target based on the information of the object to be identified and its posture and position.

[0123] Based on the target 3D model, obtain the simulated acquisition image corresponding to each image to be identified.

[0124] In one embodiment of this disclosure, obtaining the simulated acquisition image corresponding to each image to be identified based on the target three-dimensional model can be understood as performing two-dimensional image acquisition on the target three-dimensional model from the image acquisition angle corresponding to each image to be identified, so as to obtain the simulated acquisition image corresponding to each image to be identified.

[0125] In the technical solution provided in this disclosure, by constructing a target 3D model based on the information of the object to be identified and the posture and position information to be identified, and by obtaining the simulated acquisition image corresponding to each image to be identified based on the target 3D model, the difficulty of obtaining the simulated acquisition image corresponding to each image to be identified can be reduced, the efficiency of obtaining the simulated acquisition image corresponding to each image to be identified can be improved, and the user experience can be improved.

[0126] In one implementation of this disclosure, before determining the target image region in each image to be recognized, the method further includes:

[0127] Obtain an ambient background image corresponding to the image acquisition angle of each image to be identified. The ambient background image includes at least a portion of the item entrance / exit of the display case, but does not include the user's hand or any of the items to be identified.

[0128] Identify the target image region in each image to be identified, including:

[0129] Each image to be identified is compared with the background image to determine the target image region in each image to be identified.

[0130] In one embodiment of this disclosure, the ambient background image corresponding to the image acquisition angle can be understood as the image acquired at the corresponding image acquisition angle when the user's hand and the item to be identified are not in the area corresponding to the item entrance / exit of the display case.

[0131] In one embodiment of this disclosure, acquiring the environmental background image corresponding to the image acquisition angle of each image to be identified can be understood as reading the environmental background image corresponding to the image acquisition angle of each image to be identified that was stored in advance, or it can be obtained by the image acquisition device corresponding to the image acquisition angle of each image to be identified when the environmental background image acquisition conditions are met.

[0132] For example, the ambient background image corresponding to the image acquisition angle is acquired by an image acquisition device on the display case corresponding to the image acquisition angle within a preset time period after the cabinet door is unlocked. This preset time period can be less than or equal to 1 second.

[0133] In the technical solution provided in this disclosure, by acquiring an environmental background image corresponding to the image acquisition angle of each image to be identified, and comparing each image to be identified with the environmental background image to determine the target image region in each image to be identified, the difficulty of determining the target image region in each image to be identified can be reduced, the amount of computation can be reduced, the efficiency of determining the target image region in each image to be identified can be improved, and the user experience can be improved.

[0134] In one implementation of this disclosure, a simulated image corresponding to each image to be identified is obtained based on the information of the object to be identified and the posture and position information to be identified, including:

[0135] Based on the information of the object to be identified, the posture and position information of the object to be identified, and the background image of the environment, a simulated image is acquired for each image to be identified.

[0136] In one embodiment of this disclosure, a simulated acquisition image corresponding to each image to be identified is obtained based on the information of the object to be identified, the posture and position information to be identified, and the environmental background image. This can be understood as adding image information obtained based on the information of the object to be identified and the posture and position information to the environmental background image to obtain the corresponding simulated acquisition image. For example, a target 3D model can be constructed based on the information of the object to be identified and the posture and position information to be identified. Two-dimensional images of the target 3D model are then acquired from the image acquisition angle corresponding to each image to be identified, and the acquired images are added to the environmental background image to obtain the corresponding simulated acquisition image.

[0137] In the technical solution provided in this disclosure, by acquiring simulated images corresponding to each image to be identified based on the information of the item to be identified, the posture and position information to be identified, and the background image of the environment, the similarity between the acquired simulated images and the real environment can be improved, and the influence of environmental factors on the matching degree between the subsequently acquired simulated images and the images to be identified can be reduced. This helps to improve the accuracy of subsequently determining the information of items moved out of or into the display case.

[0138] In one implementation of this disclosure, image recognition is performed on target image regions in at least two images to be recognized to obtain information about the object to be recognized and the pose and position information to be recognized, including:

[0139] User hand recognition is performed on the target image region in at least two images to be recognized, and the target object image region containing only the object to be recognized is determined based on the hand recognition result.

[0140] Identify the target object image region to obtain information about the object to be identified;

[0141] Image recognition is performed on the target image region in at least two images to be recognized in order to obtain overlapping occlusion information. The overlapping occlusion information is used to indicate the positional relationship between the object to be recognized, the user's hand, and the image acquisition device used to acquire the image to be recognized in the corresponding image to be recognized.

[0142] The pose and position information of the object to be identified is obtained based on the information of overlapping occlusion relationships and the information of the object to be identified.

[0143] In one embodiment of this disclosure, the overlapping occlusion relationship information can be understood as indicating the front-to-back relationship between the object to be identified and the user's hand when viewed from the perspective of the image acquisition device used to acquire the image to be identified; that is, when viewed from this perspective, whether the object to be identified obstructs the user's hand or the user's hand obstructs the object to be identified. Furthermore, the overlapping occlusion relationship information can also be used to indicate the area or proportion of the occlusion between the object to be identified and the user's hand.

[0144] In the technical solution provided in this disclosure, by performing user hand recognition on the target image region in at least two images to be recognized, and determining the target object image region that contains only the object to be recognized based on the hand recognition result, recognizing the target object image region to obtain the object information to be recognized, and performing image recognition on the target image region in at least two images to be recognized to obtain overlapping occlusion relationship information, and obtaining the pose position information to be recognized based on the overlapping occlusion relationship information and the object information to be recognized, the difficulty of obtaining the pose position information to be recognized can be reduced and the accuracy of the obtained pose position information to be recognized can be improved.

[0145] In one implementation of this disclosure, the method further includes:

[0146] In response to the fact that the matching degree between any image to be identified and the corresponding simulated acquired image does not fall within the target matching degree range, the information on the corrected items to be identified is obtained, and the number of items to be identified indicated by the corrected information is less than the number of items to be identified indicated by the corrected information.

[0147] Based on the corrected information of the object to be identified and the posture and position information to be identified, obtain the corrected simulated acquisition image corresponding to each image to be identified, and obtain the matching degree between each corrected simulated acquisition image and the corresponding simulated acquisition image. The image acquisition angle of the corrected simulated acquisition image is the same as the image acquisition angle of the corresponding image to be identified.

[0148] In response to the fact that the matching degree between each image to be identified and the corresponding corrected simulated acquisition image is within the target matching degree range, the target item to be removed from or moved into the display case is determined based on the corrected information of the item to be identified.

[0149] In the technical solution provided in this disclosure, considering that when the matching degree between any image to be identified and the corresponding simulated acquisition image does not fall within the target matching degree range, the reason for this situation is often that the number of items to be identified indicated by the item information is greater than the actual number of items to be identified. Therefore, in response to the matching degree between any image to be identified and the corresponding simulated acquisition image not falling within the target matching degree range, corrected item information is obtained, so that the number of items to be identified indicated by the corrected item information is less than the actual number of items to be identified. Based on the corrected item information and the target posture and position information, a corrected simulated acquisition image corresponding to each image to be identified is obtained, and the matching degree between each corrected simulated acquisition image and the corresponding simulated acquisition image is obtained. In response to the matching degree between each image to be identified and the corresponding corrected simulated acquisition image falling within the target matching degree range, the target items to be moved out of or into the display case are determined based on the corrected item information. This ensures that the accuracy of the information of the items to be moved out of or into the display case is high, so as to perform the corresponding settlement and improve the user experience.

[0150] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein.

[0151] Figure 8 A schematic structural block diagram of an article identification device according to an embodiment of the present disclosure is shown. This article identification device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 8 As shown, the item recognition device 600 includes:

[0152] The image acquisition module 601 is configured to acquire at least two images to be identified from different image acquisition angles at a target time, and to determine the target image region in each image to be identified. The target image region includes the user's hand and the item to be identified located in the target region. The target region includes at least a part of the item entrance / exit of the display case.

[0153] The image recognition module 602 is configured to perform image recognition on target image regions in at least two images to be recognized in order to obtain information on the object to be recognized and information on the posture and position to be recognized. The information on the object to be recognized is used to indicate the type and quantity of the object to be recognized, and the information on the posture and position to be recognized is used to indicate the position and posture of the user's hand and the position and posture of the object to be recognized.

[0154] The image simulation module 603 is configured to acquire a simulated acquisition image corresponding to each image to be identified based on the information of the object to be identified and the posture and position information to be identified, and to acquire the matching degree between each image to be identified and the corresponding simulated acquisition image. The image acquisition angle of the simulated acquisition image is the same as the image acquisition angle of the corresponding image to be identified.

[0155] The item recognition module 604 is configured to determine the target item to be moved out of or into the display case based on the item information in response to the fact that the matching degree between each image to be recognized and the corresponding simulated acquisition image both fall within the target matching degree range.

[0156] The above technical solution acquires at least two images of the display case taken from different image acquisition angles at a target time, and determines the target image region in each image. This target image region includes the user's hand at the item entrance / exit of the display case and the item to be identified, i.e., the user's hand and items that may be moved into or out of the display case by the user's hand. Image recognition is performed on the target image region in the at least two images to obtain information about the item to be identified and the positional information of the item to be identified. The information about the item to be identified indicates the type and quantity of the item, i.e., the type and quantity of items that may be moved into or out of the display case by the user's hand. The positional information of the item to be identified indicates the position and posture of the user's hand and the position and posture of the item to be identified, i.e., the position and posture of the item when it is moved into or out of the display case by the user's hand. The system identifies the position and orientation of the item in three-dimensional space. Based on the information of the item to be identified and its orientation, a simulated image is acquired for each image to be identified. The matching degree between each image to be identified and its corresponding simulated image is calculated. The acquisition angle of the simulated image is the same as that of the corresponding image to be identified. This means the simulated image is estimated based on previously acquired information about the item to be identified and its orientation. If the previously acquired information is accurate, the matching degree between the image to be identified and its corresponding simulated image will also be high, falling within the target matching degree range. Therefore, in response to the matching degree of each image to be identified falling within the target matching degree range, the system determines the target item to be moved out of or into the display case based on the item information. This ensures a high accuracy rate in identifying the items to be moved out of or into the display case, facilitating appropriate settlement and improving the user experience.

[0157] This disclosure also discloses an electronic device, Figure 9 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0158] like Figure 9As shown, the electronic device includes a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to embodiments of the present disclosure.

[0159] Figure 10 A schematic diagram of the structure of a computer system suitable for implementing an article identification method according to an embodiment of the present disclosure is shown.

[0160] like Figure 10 As shown, the computer system includes a processing unit that can execute various methods described above based on a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer system. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0161] The following components are connected to the I / O interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard disks, etc.; and communication sections including network interface cards such as LAN cards and modems. The communication section performs communication processes via a network such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as needed. The processing unit can be implemented as a CPU, GPU, TPU, FPGA, NPU, etc.

[0162] In particular, according to embodiments of this disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing the methods described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium.

[0163] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0164] The units or modules described in the embodiments of this disclosure can be implemented in software or programmable hardware. The described units or modules can also be located in a processor, and the names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0165] In another aspect, this disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system described above; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to perform the methods described in this disclosure.

[0166] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. An item recognition method, characterized in that, The method includes: At least two images to be identified are acquired from different image acquisition angles at a target time from the display case, and a target image region is determined in each image to be identified. The target image region includes the user's hand and the item to be identified located in the target region. The target region includes at least a part of the item entrance / exit of the display case. Image recognition is performed on the target image region in the at least two images to be identified in order to obtain information on the object to be identified and information on the posture and position to be identified. The information on the object to be identified is used to indicate the type and quantity of the object to be identified, and the information on the posture and position to be identified is used to indicate the position and posture of the user's hand and the position and posture of the object to be identified. Based on the information of the object to be identified and the posture and position information of the object to be identified, a simulated acquisition image corresponding to each image to be identified is obtained, and the matching degree between each image to be identified and the corresponding simulated acquisition image is obtained. The image acquisition angle of the simulated acquisition image is the same as the image acquisition angle of the corresponding image to be identified. In response to the fact that the matching degree between each image to be identified and the corresponding simulated acquired image is within the target matching degree range, the target item to be identified is determined to be moved out of or into the display case based on the information of the item to be identified.

2. The item identification method according to claim 1, characterized in that, The step of obtaining the simulated acquisition image corresponding to each image to be identified based on the information of the object to be identified and the posture and position information to be identified includes: A 3D model of the target is constructed based on the information of the object to be identified and the posture and position information of the object to be identified. Based on the target 3D model, obtain the simulated acquisition image corresponding to each image to be identified.

3. The item identification method according to claim 1, characterized in that, Before determining the target image region in each image to be identified, the method further includes: Obtain an environmental background image corresponding to the image acquisition angle of each image to be identified. The environmental background image includes at least a portion of the item entrance / exit of the display case, and does not include either the user's hand or the item to be identified. The process of determining the target image region in each image to be identified includes: Each image to be identified is compared with the background image to determine the target image region in each image to be identified.

4. The item identification method according to claim 3, characterized in that, The ambient background image corresponding to the image acquisition angle is acquired by the image acquisition device on the display cabinet within a preset time period after the cabinet door is unlocked, and the image acquisition angle corresponds to the image acquisition angle.

5. The item identification method according to claim 3, characterized in that, The step of obtaining the simulated acquisition image corresponding to each image to be identified based on the information of the object to be identified and the posture and position information to be identified includes: Based on the information of the object to be identified, the posture and position information of the object to be identified, and the background image of the environment, a simulated acquisition image corresponding to each image to be identified is obtained.

6. The item identification method according to claim 1, characterized in that, The step of performing image recognition on the target image region in the at least two images to be recognized, to obtain information about the object to be recognized and the pose and position information to be recognized, includes: User hand recognition is performed on the target image region in the at least two images to be recognized, and the target image region containing only the item to be recognized is determined based on the hand recognition result; The target item image region is identified to obtain information about the item to be identified; Image recognition is performed on the target image regions in the at least two images to be recognized to obtain overlapping occlusion information. The overlapping occlusion information is used to indicate the positional relationship between the object to be recognized, the user's hand, and the image acquisition device used to acquire the image to be recognized in the corresponding image to be recognized. The pose and position information to be identified is obtained based on the overlapping occlusion relationship information and the information of the object to be identified.

7. The item identification method according to claim 1, characterized in that, The method further includes: In response to the fact that the matching degree between any image to be identified and the corresponding simulated acquired image does not belong to the target matching degree range, the corrected item information is obtained, wherein the number of items to be identified indicated by the corrected item information is less than the number of items to be identified indicated by the item information. Based on the corrected object information and the object posture and position information, a corrected simulated acquisition image corresponding to each image to be identified is obtained, and the matching degree between each corrected simulated acquisition image and the corresponding simulated acquisition image is obtained. The image acquisition angle of the corrected simulated acquisition image is the same as the image acquisition angle of the corresponding image to be identified. In response to the fact that the matching degree between each image to be identified and the corresponding corrected simulated acquisition image both fall within the target matching degree range, the target item to be removed from or moved into the display case is determined based on the corrected item information.

8. An item identification device, characterized in that, The device includes: The image acquisition module is configured to acquire at least two images to be identified from different image acquisition angles captured by the display case at a target time, and to determine the target image region in each image to be identified. The target image region includes the user's hand and the item to be identified located in the target region. The target region includes at least a portion of the item entrance / exit of the display case. The image recognition module is configured to perform image recognition on the target image region in the at least two images to be recognized, so as to obtain information on the object to be recognized and information on the posture and position to be recognized. The information on the object to be recognized is used to indicate the type and quantity of the object to be recognized, and the information on the posture and position to be recognized is used to indicate the position and posture of the user's hand and the position and posture of the object to be recognized. The image simulation module is configured to acquire a simulated acquisition image corresponding to each image to be identified based on the information of the object to be identified and the posture and position information to be identified, and to acquire the matching degree between each image to be identified and the corresponding simulated acquisition image, wherein the image acquisition angle of the simulated acquisition image is the same as the image acquisition angle of the corresponding image to be identified. The item recognition module is configured to determine, based on the item information, whether a target item is moved out of or into the display case, in response to each image to be recognized and the corresponding simulated acquisition image both falling within the target matching range.

9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method of any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by a processor, the computer instructions implement the method of any one of claims 1-7.

Images