Intelligent cabinet display recognition methods, systems, equipment and computer-readable media

Abstract

This application discloses a method, system, device, and computer-readable medium for recognizing smart cabinet displays. The method includes: acquiring a source image of the smart cabinet display to be recognized; using a position detector to identify the position information of all cabinets, shelf layers, and goods in the source image, generating corresponding cabinet bounding boxes, shelf layer bounding boxes, and goods bounding boxes; using an SKU detector to identify the specified SKU position information and SKU category attributes, generating SKU bounding boxes; and calculating the overlap between the cabinet bounding boxes, shelf layer bounding boxes, goods bounding boxes, and SKU bounding boxes to perform smart cabinet display recognition. This application, by introducing a position detector, avoids interference caused by tilted camera angles, multiple smart cabinets side-by-side in the image, SKUs stacked outside the smart cabinet, and reflective imaging from cabinet doors. By employing separate detection methods using position detectors and SKU detectors, the modeling efficiency and overall recognition accuracy are improved.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method, system, device and computer-readable medium for intelligent cabinet layout recognition. Background Technology

[0002] In the fast-moving consumer goods (FMCG) sector, the demand for digital transformation of smart vending machines is becoming increasingly strong. Since the display of branded smart vending machines will affect the company's final performance, it is necessary to grasp the display information in a timely manner. The traditional approach is for brands to send store visitors to the brand's terminals to manually calculate the display area, or to take photos with their mobile phones and send them to the server backend for manual calculation.

[0003] With the development of image recognition technology, image-based target detection methods can significantly improve the efficiency of shelf layout calculation and greatly enhance the digital capabilities of smart cabinet terminals, addressing the aforementioned issues. However, introducing this method also brings new problems: since the camera is installed on the cabinet door, the images acquired as the door opening angle changes contain multiple smart cabinets of the same or different brands. This can lead to issues such as other goods piled outside the smart cabinet or reflections from the glass door after it is opened, thus interfering with recognition accuracy. Furthermore, existing image target detection methods recognize both location and attribute information together. When switching between multiple brands, the annotation efficiency is greatly reduced, resulting in a long modeling cycle and difficulty in guaranteeing recognition accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a method, system, device and computer-readable medium for intelligent cabinet display recognition, so as to solve the problems of low recognition efficiency and inability to guarantee recognition accuracy when using existing image target detection technology for intelligent cabinet display recognition.

[0005] To achieve the above objectives, this application provides a smart cabinet layout recognition method, comprising:

[0006] Acquire the source image of the smart cabinet layout to be identified, use the position detector to identify the position information of all cabinets, shelf layers and products in the source image, and generate the corresponding cabinet bounding boxes, shelf layer bounding boxes and product bounding boxes;

[0007] The SKU detector is used to identify the location information and category attributes of the specified SKUs, and then the SKU bounding boxes are generated.

[0008] Based on the cabinet boundary frame, shelf layer boundary frame, product boundary frame, and SKU boundary frame, the overlap between each boundary frame is calculated to perform intelligent cabinet layout recognition.

[0009] Furthermore, the calculation of the overlap between the bounding boxes includes:

[0010] Calculate the first overlap between the product bounding box and the SKU bounding box. When the first overlap is greater than a first preset value, delete the corresponding product bounding box and merge the remaining product bounding box with all SKU bounding boxes to obtain the merged bounding box.

[0011] Furthermore, the calculation of the overlap between the bounding boxes also includes:

[0012] The smart cabinet with the largest area in the cabinet boundary frame is taken as the target cabinet. The second overlap degree between the boundary frames of all shelf layers and the cabinet boundary frame of the target cabinet is calculated. When the second overlap degree is greater than the second preset value, the corresponding shelf layer boundary frame is determined to belong to the target cabinet. The total number of shelf layers of the target cabinet is calculated.

[0013] Furthermore, the calculation of the overlap between the bounding boxes also includes:

[0014] Calculate the third overlap between the shelf layer bounding box and the merged bounding box of the target cabinet. When the third overlap is greater than the third preset value, determine that the merged bounding box belongs to the current shelf layer. Continue until all merged bounding boxes are matched to the corresponding shelf layer positions, determine the product position and category, SKU position and category in each shelf layer, and generate the intelligent cabinet layout recognition result.

[0015] This application also provides an intelligent cabinet display recognition system, including:

[0016] The position detector recognition unit is used to acquire the source image of the smart cabinet layout to be recognized, and uses the position detector to recognize the position information of all cabinets, shelf layers and products in the source image, and generates the corresponding cabinet bounding boxes, shelf layer bounding boxes and product bounding boxes.

[0017] The SKU detector identification unit is used to identify the specified SKU location information and SKU category attributes using the SKU detector, and generate the SKU bounding box.

[0018] The intelligent cabinet layout recognition unit is used to calculate the overlap between the cabinet boundary frame, shelf layer boundary frame, product boundary frame and SKU boundary frame to perform intelligent cabinet layout recognition.

[0019] Furthermore, the intelligent cabinet layout recognition unit is also used for:

[0020] Calculate the first overlap between the product bounding box and the SKU bounding box. When the first overlap is greater than a first preset value, delete the corresponding product bounding box and merge the remaining product bounding box with all SKU bounding boxes to obtain the merged bounding box.

[0021] Furthermore, the intelligent cabinet layout recognition unit is also used for:

[0022] The smart cabinet with the largest area in the cabinet boundary frame is taken as the target cabinet. The second overlap degree between the boundary frames of all shelf layers and the cabinet boundary frame of the target cabinet is calculated. When the second overlap degree is greater than the second preset value, the corresponding shelf layer boundary frame is determined to belong to the target cabinet. The total number of shelf layers of the target cabinet is calculated.

[0023] Furthermore, the intelligent cabinet layout recognition unit is also used for:

[0024] Calculate the third overlap between the shelf layer bounding box and the merged bounding box of the target cabinet. When the third overlap is greater than the third preset value, determine that the merged bounding box belongs to the current shelf layer. Continue until all merged bounding boxes are matched to the corresponding shelf layer positions, determine the product position and category, SKU position and category in each shelf layer, and generate the intelligent cabinet layout recognition result.

[0025] This application also provides a terminal device, including:

[0026] One or more processors;

[0027] A memory, coupled to the processor, for storing one or more programs;

[0028] When the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent cabinet layout recognition method as described in any of the preceding claims.

[0029] This application also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the intelligent cabinet layout recognition method as described in any of the preceding claims.

[0030] Compared to existing technologies, the advantages of this application are as follows:

[0031] This application discloses a method, system, terminal device, and storage medium for intelligent cabinet display recognition. The method includes: acquiring a source image of the intelligent cabinet display to be recognized; using a position detector to identify the position information of all cabinets, shelf layers, and goods in the source image, and generating corresponding cabinet bounding boxes, shelf layer bounding boxes, and goods bounding boxes; using an SKU detector to identify the specified SKU position information and SKU category attributes, and generating SKU bounding boxes; and calculating the overlap between the cabinet bounding boxes, shelf layer bounding boxes, goods bounding boxes, and SKU bounding boxes to perform intelligent cabinet display recognition.

[0032] This application introduces a position detector to avoid interference caused by tilted camera angles, multiple smart cabinets side-by-side in the image, stacked SKUs outside the smart cabinet, and reflective imaging from the cabinet door. By using a dual-detector approach with both a position detector and an SKU detector to separate position and category attributes, the efficiency of modeling can be effectively improved. At the same time, it avoids the missed detections that occur in traditional single-detection modes, thereby improving the overall recognition accuracy. Attached Figure Description

[0033] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram illustrating the detection principle of the intelligent cabinet layout recognition method provided in a certain embodiment of this application;

[0035] Figure 2 This is a flowchart illustrating a smart cabinet layout recognition method provided in a certain embodiment of this application;

[0036] Figure 3 This is a schematic image of a smart cabinet display photograph provided in one embodiment of this application;

[0037] Figure 4 This is a schematic diagram of the structure of an intelligent cabinet layout recognition system provided in a certain embodiment of this application;

[0038] Figure 5 This is a schematic diagram of the structure of a terminal device provided in a certain embodiment of this application. Detailed Implementation

[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0040] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0041] It should be understood that the terminology used in this application specification is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this application specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0042] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0043] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.

[0044] It should be noted that existing smart cabinet display recognition methods are often affected by photographic conditions, such as multiple smart cabinets side-by-side, SKUs stacked outside the smart cabinets, and reflective imaging from the cabinet doors, which ultimately affect recognition accuracy and result in long modeling cycles. Therefore, this embodiment aims to provide a smart cabinet display recognition method that improves modeling efficiency and overall recognition accuracy by employing both position detectors and SKU detectors for separate detection. The detection principle of this application's method is as follows: Figure 1 As shown. By Figure 1 As can be seen, after acquiring the source image of the smart cabinet display captured by the camera, it undergoes image preprocessing, and then is detected by a position detector and an SKU detector respectively. The position detector can detect the position of the cabinet and the shelf layer, while the SKU detector is used to merge the SKU and product bounding boxes. Finally, the detection results of the two are matched to calculate the product position and category information, SKU position and category information of the shelf layer in the cabinet, thus completing the entire smart cabinet display recognition process.

[0045] Please see Figure 2 This application provides a method for recognizing the layout of smart cabinets in one embodiment. For example... Figure 2 As shown, the intelligent cabinet layout recognition method includes steps S10 to S30. The specific steps are as follows:

[0046] S10. Obtain the source image of the smart cabinet layout to be identified, use the position detector to identify the position information of all cabinets, shelf layers and goods in the source image, and generate the corresponding cabinet bounding boxes, shelf layer bounding boxes and goods bounding boxes.

[0047] S20. Use the SKU detector to identify the specified SKU location information and SKU category attributes, and generate SKU bounding boxes;

[0048] S30. Based on the cabinet boundary frame, shelf layer boundary frame, product boundary frame and SKU boundary frame, calculate the overlap between each boundary frame to perform intelligent cabinet layout recognition.

[0049] To facilitate understanding, this embodiment first describes the structure of the smart cabinet involved. Typically, a smart cabinet includes a cabinet body, cabinet doors, shelf layers, cameras, and a computing unit. To allow users to clearly see the types of goods on the shelf layers, the cabinet doors are usually made of transparent glass. The shelf layers are used to display different categories of goods. The camera is usually installed above the cabinet body to capture images of the changes in the smart cabinet's layout when the user operates it. The computing unit typically contains a processor with built-in algorithm models to calculate relevant data based on the current layout of the smart cabinet, ultimately enabling interaction with the user.

[0050] Based on the above intelligent cabinet structure Figure 3 A source image illustration of the smart cabinet layout to be identified is provided. For example... Figure 3 As shown, the image contains smart cabinet B, smart cabinet A, and the reflective portion of the glass door of smart cabinet A. This situation is usually caused by interference from multiple smart cabinets side-by-side in the image, SKUs stacked outside the smart cabinets, and reflective imaging from the cabinet doors, which affects the recognition accuracy. The following will explain... Figure 3 Taking the acquisition of the source image of the smart cabinet layout to be identified as an example, the layout recognition process of this application will be explained as follows:

[0051] 1) First, the camera is triggered to take a picture based on the door opening signal. After preprocessing, the source image for shelf recognition is obtained, i.e. Figure 3 The image shown.

[0052] 2) Use a position detector to identify the position information of all cabinets, shelf layers and goods in the source image, and generate the corresponding cabinet bounding boxes, shelf layer bounding boxes and goods bounding boxes; the position detector is set in the computing unit to identify the position information of all cabinets, shelf layers and goods in the source image, and can also identify the corresponding category attribute information.

[0053] 3) Use the SKU detector to identify the specified SKU location information and SKU category attributes, and generate SKU bounding boxes.

[0054] It should be noted that when constructing the location detector and the SKU detector, it is preferable to use the same source image. The image obtained by the actual smart cabinet door opening signal, after preprocessing, is used as the source image library for the location detector and the SKU detector.

[0055] Specifically, in the actual training of the location detector, training samples are first obtained, namely, the top left and bottom right corner boxes of the cabinets, shelf layers, and products in the cabinets in the labeled source images. At this time, the category attributes of the SKUs are not distinguished, and they are set as three categories: cabinet category, shelf layer category, and product category. Such images and labeled information are used as input to the target detector to train a location detector, which can identify and locate all cabinets, all shelf layers, and all products in the smart cabinet.

[0056] Similarly, when training the SKU detector, it is necessary to label the SKUs specified by the brand in the image. At this time, we only focus on and label the top left and bottom right corners of the SKUs specified by the brand, as well as the category attributes of the SKUs. Using such images and labeling information as input to the target detector, we can train an SKU target detector to realize the detection and recognition of the location and category attributes of the specified SKUs in the smart cabinet image.

[0057] In training both the location detector and the SKU detector, a target detector must first be trained. This target detector can be the currently mainstream transformer target detector, which offers better recognition performance and efficiency compared to traditional target detectors based on CNN or RNN neural network models, and is therefore a preferred choice. It is important to emphasize that the specific target detector used can be selected based on actual needs; this embodiment only provides a preferred method and does not constitute a limitation on the type of target detector.

[0058] 4) Based on the cabinet boundary frame, shelf layer boundary frame, product boundary frame and SKU boundary frame, calculate the overlap between each boundary frame to perform intelligent cabinet layout recognition.

[0059] In this step, based on the bounding boxes obtained in steps 2) and 3), the overlap of the bounding box areas is mainly used as the primary means to achieve accurate identification of the smart cabinet layout.

[0060] Specifically, calculating the overlap between bounding boxes includes the following cases:

[0061] 4.1) Calculate the first overlap between the product bounding box and the SKU bounding box. When the first overlap is greater than the first preset value, delete the corresponding product bounding box and merge the remaining product bounding box with all SKU bounding boxes to obtain the merged bounding box.

[0062] Understandably, the main purpose of this step is to merge SKU bounding boxes and product bounding boxes. This is achieved by iterating through and calculating the area overlap between each SKU and product bounding box (the first overlap). Then, it's determined whether this first overlap exceeds a preset value. If so, the product attribute information in the current product bounding box is modified to use SKU attribute information. In other words, when the overlap between the SKU and product bounding boxes reaches a certain value, SKU attribute information is considered to replace product attribute information for product recognition. After modifying the product attribute information to SKU attribute information, the current product bounding box is deleted, and the remaining product bounding boxes are merged with all SKU bounding boxes to obtain a merged bounding box.

[0063] 4.2) Take the smart cabinet with the largest area in the cabinet boundary frame as the target cabinet, calculate the second overlap between the boundary frames of all shelf layers and the cabinet boundary frame of the target cabinet, and when the second overlap is greater than the second preset value, determine that the corresponding shelf layer boundary frame belongs to the target cabinet, and calculate the total number of shelf layers of the target cabinet.

[0064] In step 2), the cabinet bounding box is obtained. Since there may be multiple cabinets in the source image, in order to reduce interference, the area of ​​all cabinet bounding boxes is calculated here, that is, the area of ​​all cabinets is calculated, and then the cabinet bounding box with the largest area is selected as the position of the smart cabinet corresponding to the current camera in the image. In other words, the smart cabinet corresponding to the cabinet bounding box with the largest proportion is selected as the target cabinet.

[0065] Once the target cabinet is selected, the next step is to determine the shelf layer details, including layer position and height. In this step, based on the target cabinet's position, the second overlap between the bounding boxes of all shelf layers and the target cabinet's bounding box is calculated. When the second overlap is greater than a second preset value, the current shelf layer is considered to belong to the target cabinet. After calculating all shelf layers using this method, the number of shelf layers in the target cabinet can be determined. After calculating all shelf layers, combined with the height information of the shelf layers in the image, the total number of shelf layers in the cabinet and the position information of each layer can be calculated.

[0066] 4.3) Calculate the third overlap between the shelf layer bounding box and the merged bounding box of the target cabinet. When the third overlap is greater than the third preset value, determine that the merged bounding box belongs to the current shelf layer. Continue until all merged bounding boxes are matched to the corresponding shelf layer positions, determine the product position and category, SKU position and category in each shelf layer, and generate the intelligent cabinet layout recognition result.

[0067] Finally, in step 4.3), the product location and category information, as well as the SKU location and category information, of each shelf layer in the smart cabinet are mainly calculated. First, the area overlap ratio (third overlap ratio) of the bounding box and the merged bounding box of each shelf layer is calculated. Then, it is determined whether the third overlap ratio is greater than a third preset value. If so, the merged bounding box is considered to belong to the current shelf layer. This process continues until all merged bounding boxes have been calculated, at which point all merged bounding boxes are matched to their corresponding shelf layer locations. Then, based on the product information and SKU information in the merged bounding boxes, the product location and category, as well as the SKU location and category information, of each shelf layer are determined, generating the final recognition result and completing the smart cabinet layout recognition task.

[0068] In summary, the smart cabinet layout recognition method provided in this application can avoid interference caused by multiple smart cabinets side by side in the image, stacked SKUs outside the smart cabinet, and reflective imaging of the cabinet door due to the tilt of the camera. By using a dual detector method of position detector and SKU detector to separate position and category attributes, the efficiency of modeling can be effectively improved. At the same time, it can avoid the missed detection of traditional single detection, thereby improving the overall recognition accuracy.

[0069] Please see Figure 4 One embodiment of this application also provides an intelligent cabinet display recognition system, including:

[0070] The position detector recognition unit 01 is used to acquire the source image of the smart cabinet layout to be recognized, and to use the position detector to recognize the position information of all cabinets, shelf layers and goods in the source image, and generate the corresponding cabinet bounding boxes, shelf layer bounding boxes and goods bounding boxes.

[0071] The SKU detector identification unit 02 is used to identify the specified SKU location information and SKU category attributes using the SKU detector, and generate the SKU bounding box.

[0072] The intelligent cabinet layout recognition unit 03 is used to calculate the overlap between the cabinet boundary frame, shelf layer boundary frame, product boundary frame and SKU boundary frame for intelligent cabinet layout recognition.

[0073] In one embodiment, the intelligent cabinet layout recognition unit 03 is further used for:

[0074] Calculate the first overlap between the product bounding box and the SKU bounding box. When the first overlap is greater than a first preset value, delete the corresponding product bounding box and merge the remaining product bounding box with all SKU bounding boxes to obtain the merged bounding box.

[0075] In one embodiment, the intelligent cabinet layout recognition unit 03 is further used for:

[0076] The smart cabinet with the largest area in the cabinet boundary frame is taken as the target cabinet. The second overlap degree between the boundary frames of all shelf layers and the cabinet boundary frame of the target cabinet is calculated. When the second overlap degree is greater than the second preset value, the corresponding shelf layer boundary frame is determined to belong to the target cabinet. The total number of shelf layers of the target cabinet is calculated.

[0077] In one embodiment, the intelligent cabinet layout recognition unit 03 is further used for:

[0078] Calculate the third overlap between the shelf layer bounding box and the merged bounding box of the target cabinet. When the third overlap is greater than the third preset value, determine that the merged bounding box belongs to the current shelf layer. Continue until all merged bounding boxes are matched to the corresponding shelf layer positions, determine the product position and category, SKU position and category in each shelf layer, and generate the intelligent cabinet layout recognition result.

[0079] Please see Figure 5 One embodiment of this application provides a terminal device, including:

[0080] One or more processors;

[0081] A memory, coupled to the processor, for storing one or more programs;

[0082] When the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent cabinet layout recognition method as described above.

[0083] The processor controls the overall operation of the terminal device to complete all or part of the steps of the intelligent cabinet display identification method described above. The memory stores various types of data to support the operation of the terminal device. This data may include, for example, instructions for any application or method used to operate on the terminal device, as well as application-related data. The memory can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0084] In an exemplary embodiment, the terminal device may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the intelligent cabinet layout recognition method as described in any of the above embodiments and achieve the same technical effect as the above method.

[0085] In another exemplary embodiment, a computer-readable medium including a computer program is also provided. When executed by a processor, the computer program implements the steps of the smart cabinet display recognition method as described in any of the above embodiments. For example, the computer-readable medium may be the aforementioned memory including the computer program, which can be executed by a processor of a terminal device to complete the smart cabinet display recognition method as described in any of the above embodiments and achieve the same technical effects as the aforementioned method.

[0086] It should be understood that the computer-readable medium in the embodiments of this application may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. More specific examples of computer-readable storage media include at least (a non-exhaustive list) the following: an electrical connection (electronic device) having one or more wirings, a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM). Furthermore, the computer-readable storage medium may even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for recognizing the layout of an intelligent cabinet, characterized in that, include: Acquire the source image of the smart cabinet layout to be identified, use the position detector to identify the position information of all cabinets, shelf layers and products in the source image, and generate the corresponding cabinet bounding boxes, shelf layer bounding boxes and product bounding boxes; The SKU detector is used to identify the location information and category attributes of the specified SKUs, and then the SKU bounding boxes are generated. Based on the cabinet boundary frame, shelf layer boundary frame, product boundary frame and SKU boundary frame, calculate the overlap between each boundary frame to perform intelligent cabinet layout recognition. The calculation of the overlap between each bounding box includes calculating the third overlap between the shelf layer bounding box of the target cabinet and the merged bounding box. When the third overlap is greater than a third preset value, the merged bounding box is determined to belong to the current shelf layer. This process continues until all merged bounding boxes are matched to the corresponding shelf layer positions. Then, the product position and category, SKU position and category in each shelf layer are determined, and a smart cabinet layout recognition result is generated. The merged bounding box is the set of bounding boxes formed by merging the product bounding box and the SKU bounding box after redundancy removal. The shelf layer bounding box of the target cabinet is the bounding box corresponding to the shelf layer of the target cabinet after the target cabinet is determined from all cabinets.

2. The intelligent cabinet layout recognition method according to claim 1, characterized in that, The calculation of the overlap between each bounding box includes: Calculate the first overlap between the product bounding box and the SKU bounding box. When the first overlap is greater than a first preset value, delete the corresponding product bounding box and merge the remaining product bounding box with all SKU bounding boxes to obtain the merged bounding box.

3. The intelligent cabinet layout recognition method according to claim 2, characterized in that, The calculation of the overlap between the bounding boxes also includes: The smart cabinet with the largest area in the cabinet boundary frame is taken as the target cabinet. The second overlap degree between the boundary frames of all shelf layers and the cabinet boundary frame of the target cabinet is calculated. When the second overlap degree is greater than the second preset value, the corresponding shelf layer boundary frame is determined to belong to the target cabinet. The total number of shelf layers of the target cabinet is calculated.

4. An intelligent cabinet display recognition system, characterized in that, include: The position detector recognition unit is used to acquire the source image of the smart cabinet layout to be recognized, and uses the position detector to recognize the position information of all cabinets, shelf layers and products in the source image, and generates the corresponding cabinet bounding boxes, shelf layer bounding boxes and product bounding boxes. The SKU detector identification unit is used to identify the specified SKU location information and SKU category attributes using the SKU detector, and generate the SKU bounding box. The intelligent cabinet layout recognition unit is used to calculate the overlap between the cabinet boundary frame, shelf layer boundary frame, product boundary frame and SKU boundary frame to perform intelligent cabinet layout recognition. The calculation of the overlap between each bounding box includes calculating the third overlap between the shelf layer bounding box of the target cabinet and the merged bounding box. When the third overlap is greater than a third preset value, the merged bounding box is determined to belong to the current shelf layer. This process continues until all merged bounding boxes are matched to the corresponding shelf layer positions. Then, the product position and category, SKU position and category in each shelf layer are determined, and a smart cabinet layout recognition result is generated. The merged bounding box is the set of bounding boxes formed by merging the product bounding box and the SKU bounding box after redundancy removal. The shelf layer bounding box of the target cabinet is the bounding box corresponding to the shelf layer of the target cabinet after the target cabinet is determined from all cabinets.

5. The intelligent cabinet layout recognition system according to claim 4, characterized in that, The intelligent cabinet layout recognition unit is also used for: Calculate the first overlap between the product bounding box and the SKU bounding box. When the first overlap is greater than a first preset value, delete the corresponding product bounding box and merge the remaining product bounding box with all SKU bounding boxes to obtain the merged bounding box.

6. The intelligent cabinet layout recognition system according to claim 5, characterized in that, The intelligent cabinet layout recognition unit is also used for: The smart cabinet with the largest area in the cabinet boundary frame is taken as the target cabinet. The second overlap degree between the boundary frames of all shelf layers and the cabinet boundary frame of the target cabinet is calculated. When the second overlap degree is greater than the second preset value, the corresponding shelf layer boundary frame is determined to belong to the target cabinet. The total number of shelf layers of the target cabinet is calculated.

7. A terminal device, characterized in that, include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent cabinet layout recognition method as described in any one of claims 1-3.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the intelligent cabinet layout recognition method as described in any one of claims 1-3.

Images