Container switch status identification method, device and electronic equipment

Abstract

This application provides a method, apparatus, and electronic device for identifying the switch status of a container. The method includes: acquiring a container image and performing tag recognition to determine the physical state of paired tags; and determining the container switch status based on the physical state of the paired tags. This method can achieve automatic identification of the container switch status, reduce the installation complexity of automatic container switch identification, and reduce implementation costs.

Description

Technical Field

[0001] This application relates to the field of video surveillance technology, and in particular to a method, apparatus and electronic device for identifying the switch status of a container. Background Technology

[0002] For cargo containers such as boxes, cabinets, or express delivery lockers (hereinafter referred to as containers), it is usually necessary to be able to automatically identify their open or closed status, that is, to automatically identify whether the container door is open or closed.

[0003] Currently, the main solution for automatically identifying the on / off status of containers is to install independent sensors on the containers and connect them to a network. This method of automatically identifying the on / off status of containers by acquiring sensor signals increases the complexity of container installation and has a high implementation cost. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus and electronic device for identifying the switch status of a container.

[0005] Specifically, this application is implemented through the following technical solution:

[0006] According to a first aspect of the embodiments of this application, a method for identifying the open / closed state of a container is provided. The container is a double-door or multi-door container. For any door of the container, two paired tags are provided at designated positions on the door and another door paired with the door. The method includes:

[0007] Acquire container images and perform label recognition to determine the physical state of paired labels;

[0008] The container's on / off state is determined based on the physical state of the paired tags.

[0009] According to a second aspect of the embodiments of this application, a container opening / closing status identification device is provided, wherein the container is a double-door or multi-door container, and for any door of the container, two paired tags are provided at designated positions on the door and another door paired with the door, the device comprising:

[0010] Image sensor, used to acquire images of the container;

[0011] The processor is used to identify tags and determine the physical state of paired tags; based on the physical state of paired tags, it determines the container's on / off state.

[0012] According to a third aspect of the present application, an electronic device is provided, including a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor being configured to execute the machine-executable instructions to implement the method provided in the first aspect.

[0013] According to a fourth aspect of the embodiments of this application, a machine-readable storage medium is provided, wherein machine-executable instructions are stored therein, and when the machine-executable instructions are executed by a processor, the method provided in the first aspect is implemented.

[0014] The technical solution provided in this application can bring at least the following beneficial effects:

[0015] By setting paired tags on the doors of the container and identifying the tags in the acquired container images, the physical state of the paired tags can be determined. Then, the container's open / closed state can be determined based on the physical state of the paired tags. This achieves automatic identification of the container's open / closed state without the need to install independent sensors, reducing the installation complexity and implementation cost of automatic container open / closed identification. Attached Figure Description

[0016] Figure 1 This is a schematic flowchart illustrating a container switch state recognition method according to an exemplary embodiment of this application;

[0017] Figure 2A and Figure 2B This is a schematic diagram of the label shown in an embodiment of this application;

[0018] Figure 3 This is a schematic diagram of the structure of a double-door container switch status recognition system, as shown in an exemplary embodiment of this application.

[0019] Figure 4A This is a schematic diagram illustrating a label symmetrical side distance condition according to an exemplary embodiment of this application;

[0020] Figure 4B This is a schematic diagram illustrating a label symmetrical side length difference condition according to an exemplary embodiment of this application;

[0021] Figure 5 This is a schematic diagram illustrating an exemplary embodiment of the present application of a process for recognizing the switch status of a double-door container;

[0022] Figure 6A This is a schematic diagram illustrating an exemplary embodiment of this application in which any label in an image does not exist;

[0023] Figure 6B This is a schematic diagram illustrating an exemplary embodiment of this application, showing that the symmetrical sides of a label in an image are not parallel;

[0024] Figure 6C This is a schematic diagram illustrating an exemplary embodiment of this application where the label distance in an image exceeds a threshold;

[0025] Figure 6DThis is a schematic diagram illustrating an exemplary embodiment of this application where the difference in the symmetrical side length of a label in an image exceeds a threshold.

[0026] Figure 7 This is a schematic diagram illustrating a cabinet door state switching according to an exemplary embodiment of this application;

[0027] Figure 8 This is a schematic diagram of the structure of a container switch status recognition device shown in an exemplary embodiment of this application;

[0028] Figure 9 This is a schematic diagram of the hardware structure of an electronic device illustrated in an exemplary embodiment of this application. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0030] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0031] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0032] It should be noted that, unless otherwise specified, the containers mentioned in this application embodiment are all double-door or multi-door containers.

[0033] In this embodiment of the application, in order to achieve automatic identification of the container's open / closed state (i.e., whether the container's door is open or closed), for any door of the container, two paired labels (these two labels can be called a pair of paired labels) can be set at a designated location on the door and another door paired with the door, so as to automatically identify the container's open / closed state based on the label status in the obtained container image.

[0034] For example, for any door, the other door paired with that door refers to the door adjacent to that door when the container door is closed.

[0035] For example, any given door can be paired with multiple other doors. For instance, for a container with three doors (which can be opened and closed by relative translation), the middle door can be paired with the door on the left and the door on the right.

[0036] In this embodiment of the application, based on the above-mentioned label settings, the container's open / closed state can be automatically identified according to the different states of the paired labels when the container is in an open state (also known as an open door state) and a closed state (also known as a closed door state).

[0037] Please see Figure 1 This is a flowchart illustrating a container switch state recognition method provided in an embodiment of this application. Figure 1 As shown, the container switch state identification method may include the following steps:

[0038] For ease of description and understanding, the following description uses a double-door container as an example. However, it should be understood that in the embodiments of this application, the container is not limited to a double-door container, but may also include multi-door containers, double-door or multi-door boxes, express delivery lockers, etc.

[0039] Step S100: Obtain the container image and perform label recognition to determine the physical state of the paired labels.

[0040] In this embodiment of the application, an image acquisition device can be used to acquire images of a double-door container and its doors, and the acquired images can be used to identify the labels and determine the status of the paired labels.

[0041] For example, label recognition can be achieved using a pre-trained label recognition model (such as a neural network model for label recognition). During the training process of the label recognition model, common label training samples can be used to train the label recognition model using deep learning. With the labels as the foreground, the label recognition model identifies the labels, and then the physical state of the paired labels is determined.

[0042] For example, image segmentation and other techniques can be used to identify the location and region of the labels set on the cabinet door in the image, and then determine the physical state of the paired labels.

[0043] For example, the physical state of a pair of tags may include whether the tag exists (i.e. whether the tag is recognized) and the positional relationship between the two tags in the pair, such as the distance between the tags, the relative relationship between specified parts of the tags, etc.

[0044] Step S110: Determine the container switch status based on the physical state of the paired tag.

[0045] In this embodiment of the application, it is considered that after the label is set, the physical state of the paired label will usually have obvious differences between the open and closed states of the container.

[0046] For example, the distance between paired labels when the container door is closed is usually significantly smaller than the distance between paired labels when the container door is open.

[0047] For example, when the container door is open, some of the paired tags may not be recognized.

[0048] Taking the relative sliding of container doors as an example, assuming the label is placed at the container door partition (i.e., the adjacent position of the two paired doors when the doors are closed), the label placed on the rear door may not be recognized when the container door is open because it is blocked by the front door.

[0049] Taking a container door as an example, where the door opens and closes by rotating around a hinge, if the door is open and the rotation angle is large, the label on the door may not be identifiable.

[0050] Accordingly, once the physical state of the paired tag is determined, the container's on / off state can be determined based on the physical state of the paired tag.

[0051] For example, the status of the matched tag can be used to determine whether the container door is closed, based on whether the physical status of the matched tag is consistent with that of the matched tag when the door is closed.

[0052] It can be seen that, in Figure 1 In the method shown, paired tags are set on the doors of the container and the physical state of the paired tags is determined by tag recognition of the acquired container image. Then, the container opening and closing state can be determined based on the physical state of the paired tags. Automatic identification of container opening and closing state is achieved without the need to install independent sensors, which reduces the installation complexity of automatic container opening and closing identification and reduces the implementation cost.

[0053] In some embodiments, determining the container opening / closing state based on the physical state of the paired tag may include:

[0054] For any pair of paired tags, based on the determined physical state of the paired tags, determine whether the physical state of the paired tags satisfies the container closing condition; the container closing condition includes the conditions that the physical state of the paired tags must satisfy when the container is in a closed state.

[0055] The container is determined to be in a closed state if the physical states of all paired tags of the container meet the container closing conditions.

[0056] For example, for a double-door container, a pair of matching labels can be placed on the container door.

[0057] Once the physical state of the pairing label set on the container door is determined, it can be determined whether the physical state of the pairing label meets the container closing conditions.

[0058] For example, container closure conditions may include conditions that the physical state of the paired tag must meet when the container is in a closed state.

[0059] For example, the container closing condition can be determined based on the physical state of the tag in the container closed state. That is, the physical state of the tag paired with the container closed state can be set as needed, and the container opening and closing state can be automatically identified based on the physical state of the paired tag and the physical state of the subsequently identified paired tags.

[0060] In one example, the container closing condition includes both tags in the paired tags being identified, and one or more of the following conditions:

[0061] The symmetrical sides of the two tags in a pair are parallel;

[0062] The distance between the symmetrical edges of two tags in a pair is less than or equal to the first threshold.

[0063] The difference in the side length of the symmetrical side of the two labels in the pair is less than or equal to the second threshold.

[0064] For example, when setting pairing tags on a container, you can ensure that the container is within the acquisition range of the image acquisition device and that the container is in a closed state, so that the pairing tags can be acquired.

[0065] In addition, when setting matching labels, for example by pasting, printing or hanging them on cabinet doors, both labels can have at least one straight edge (or chord) of equal length, and the two labels are mirror images of each other about the straight edge as an axis of symmetry (this straight edge can be called the symmetrical edge, and the symmetrical edges in the two labels are parallel to each other).

[0066] It should be noted that the above-mentioned equal lengths may include completely equal lengths, or unequal lengths, but the length difference is within an acceptable error range.

[0067] Similarly, the above-mentioned two symmetrical sides being parallel can include the lines containing the two symmetrical sides being parallel, or the included angle between the lines containing the two symmetrical sides being within an acceptable range.

[0068] Furthermore, after the paired tags are set, a distance threshold for identifying the container's open / closed state can be set based on the distance between the paired tags when the container is closed (which can be indicated by the distance between symmetrical sides).

[0069] Accordingly, during the automatic container status recognition process, it can be determined whether both tags in the paired tags have been recognized.

[0070] When both tags are identified, the state of the paired tags can be determined to meet the container closing condition based on one or more of the following conditions: whether the symmetrical sides of the two tags in the paired tags are parallel, whether the distance between the symmetrical sides of the two tags in the paired tags is less than or equal to a preset distance threshold (which can be called the first threshold), and whether the difference in the side length of the symmetrical sides of the two tags in the paired tags is less than or equal to a preset side length difference threshold (which can be called the second threshold).

[0071] Taking all the above conditions as an example, the state of the paired label can be determined to meet the container closing condition if both labels in the paired label are identified, the symmetrical sides of the two labels in the paired label are parallel, the distance between the symmetrical sides of the two labels in the paired label is less than or equal to the first threshold, and the difference in the side length of the symmetrical sides of the two labels in the paired label is less than or equal to the second threshold.

[0072] In some embodiments, determining the container opening / closing state based on the physical state of the paired tag may include:

[0073] For any pair of paired tags, based on the determined physical state of the paired tags, determine whether the physical state of the paired tags meets the container opening conditions; the container opening conditions include the conditions that the physical state of the paired tags must meet when the container is in the open state.

[0074] The container is determined to be in an open state if the physical state of any pair of paired tags of the container satisfies the container opening condition.

[0075] For example, once the physical state of the pairing label set on the container door is determined, it can be determined whether the state of the pairing label meets the container opening conditions based on the determined physical state of the pairing label.

[0076] For example, container opening conditions may include conditions that the physical state of the paired tags must meet when the container is in an open state.

[0077] For example, the container opening condition can be determined based on the physical state of the tag when the container is open. That is, the physical state of the paired tag when the container is open can be set as needed, and the container opening and closing state can be automatically identified based on the physical state of the paired tag and the physical state of the subsequently identified paired tags.

[0078] In one example, the container opening condition may include at least one label in the paired labels not being recognized;

[0079] or,

[0080] Both tags in a pair are identified, but the symmetrical sides of the two tags are not parallel, the distance between the symmetrical sides of the two tags is greater than a first threshold, and / or the difference in the side length of the symmetrical sides of the two tags is greater than a second threshold.

[0081] For example, when setting pairing tags on a container, you can ensure that the container is within the acquisition range of the image acquisition device and that the container is in a closed state, so that the pairing tags can be acquired.

[0082] In addition, when setting matching labels, for example by pasting, printing or hanging them on cabinet doors, both labels can have at least one straight edge (or chord) of equal length, and the two labels are mirror images of each other about the straight edge as an axis of symmetry (this straight edge can be called the symmetrical edge, and the symmetrical edges in the two labels are parallel to each other).

[0083] It should be noted that the above-mentioned equal lengths may include completely equal lengths, or unequal lengths, but the length difference is within an acceptable error range.

[0084] Similarly, the above-mentioned two symmetrical sides being parallel can include the lines containing the two symmetrical sides being parallel, or the included angle between the lines containing the two symmetrical sides being within an acceptable range.

[0085] Furthermore, after the paired tags are set, a distance threshold for identifying the container's open / closed state can be set based on the distance between the paired tags when the container is open (which can be indicated by the distance between symmetrical sides).

[0086] Accordingly, during the automatic container status recognition process, it can be determined whether both tags in the paired tags have been recognized.

[0087] If at least one tag in a tag pair is not identified, it can be determined that the physical state of the paired tags satisfies the container opening condition.

[0088] It should be noted that, in order to reduce the probability of misidentification of the container's open / closed state due to one or both tags being temporarily obscured in a pair of paired tags, the duration of the situation can be counted if at least one tag in any pair of paired tags is not identified. If the duration exceeds a preset time, the physical state of the paired tags is determined to meet the container's opening conditions.

[0089] For example, when both tags in a pairing tag are identified, the physical state of the pairing tag can be determined to meet the container opening conditions based on one or more of the following conditions: whether the symmetrical sides of the two tags in the pairing tag are parallel, whether the distance between the symmetrical sides of the two tags in the pairing tag is less than or equal to a preset distance threshold (such as the first threshold mentioned above), and whether the difference in the side length of the symmetrical sides of the two tags in the pairing tag is less than or equal to a preset side length difference threshold (such as the second threshold mentioned above).

[0090] For example, if both tags in a pair of tags are identified, but the symmetrical sides of the two tags in the pair of tags are not parallel, the distance between the symmetrical sides of the two tags in the pair of tags is greater than a first threshold, and / or the difference in the side length of the symmetrical sides of the two tags in the pair of tags is greater than a second threshold, the physical state of the pair of tags can be determined to meet the container opening condition.

[0091] In one example, if there is only one straight edge in a label, then that straight edge is the symmetrical edge between the two labels in the pair.

[0092] If there is more than one straight edge in a label, the straight edge in the label with a preset identifier is the symmetrical edge of the two labels in the paired label.

[0093] For example, if the label has only one straight edge, such as a label like... Figure 2A As shown (the label is in semi-circular form), the straight edge (such as...) can be... Figure 2A The chord of the middle label serves as the symmetrical side of the two labels in the paired label.

[0094] When the number of straight edges in a label exceeds one, for example, a label like... Figure 2B As shown (the labels are in triangle form), the symmetrical sides of the two labels in a pair can be marked using preset identifiers. For example, the symmetrical sides can be marked by specifying a color or pattern.

[0095] For example, if the matching label set on the door of a double-door container meets the container closing conditions, it can be determined that the double-door container is in a closed state.

[0096] It should be noted that for multi-door containers, the container is considered closed only if each pair of paired tags on the container meets the container's closing conditions.

[0097] In one example, determining the container's on / off state based on the physical state of the paired tags may also include:

[0098] If the container was in a closed state in the previous state, and the state of any pair of paired tags does not meet the container's closed condition, then the container is determined to be in an open state.

[0099] For example, if the state of any pair of paired tags determined in the manner described above does not meet the container closing condition when the container is in a closed state, the container state can be updated to an open state.

[0100] As an example, before determining the container to be in an open state, if the container's previous state was closed and the physical state of any pair of paired tags does not satisfy the container's closed condition, the following may also be included:

[0101] Determine whether the duration for which the physical state of any pair of paired tags does not meet the container closing conditions has reached a preset time threshold.

[0102] If the duration for which the physical state of any pair of paired tags does not meet the container closing condition does not reach a preset time threshold, the container is determined to be in a closed state.

[0103] If the duration for which the physical state of any pair of paired tags does not meet the container closing condition reaches a preset time threshold, the operation of determining the container to be in the open state described above will be executed.

[0104] For example, considering that when the container is in a closed state, the state of the paired tags determined by tag recognition of the acquired container image may not meet the container closure condition due to reasons such as occlusion.

[0105] For example, when pedestrians move in front of a container, they may obstruct the labels on the container door. Consequently, the image acquisition device may not be able to identify one or both of the paired labels in the container image.

[0106] Accordingly, in order to improve the accuracy of container switch state recognition, when the previous state of the container is the container closed state, and it is determined in the above manner that the physical state of any pair of paired tags does not meet the container closing condition (which can be called the state of the paired tags does not meet the container closing condition), the duration of the physical state of the paired tags not meeting the container closing condition can be statistically analyzed, and it can be determined whether the duration of the physical state of the paired tags not meeting the container closing condition reaches a preset time threshold.

[0107] It should be noted that when multiple pairs of paired tags are set on the container, if the physical state of any pair of paired tags does not meet the container closing condition, it can be determined that the physical state of the paired tags does not meet the container closing condition.

[0108] If the duration for which the physical state of the paired tag does not meet the container closing conditions reaches a preset time threshold, the container can be determined to be in an open state.

[0109] If the duration for which the physical state of the paired tag does not meet the container closing condition does not reach a preset time threshold, the container state can be kept closed.

[0110] For example, the above-mentioned statistics on the duration during which the physical state of the paired tag does not meet the container closing condition can be achieved by counting the number of frames in the identified image.

[0111] For example, if the container was in a closed state in the previous state and the physical state of the paired tag does not meet the container closing condition, the container state can be kept closed first, and the paired tag state can be identified in the next frame image. The number of frames that are continuously identified as having paired tags whose physical state does not meet the container closing condition can be counted. When the number of frames reaches a preset frame threshold, the duration for which the physical state of the paired tag does not meet the container closing condition can be determined.

[0112] In some embodiments, before performing tag recognition and determining the physical state of the configured tags after obtaining a container image, the process may further include:

[0113] The acquired container image is input into the trained container state recognition model to obtain the state recognition result of the container state recognition model;

[0114] If the state recognition result is unknown, the above operation of tag recognition and determining the physical state of the paired tags will be performed.

[0115] If the state recognition result is a known state, the container's on / off state is determined based on the state recognition result; wherein, the known state includes the container being in an open state or the container being in a closed state.

[0116] For example, in order to improve the efficiency of container switch state recognition, a neural network model (which can be called a container state recognition model) can be pre-trained for container switch state recognition.

[0117] This container state recognition model can identify labels on input container images and determine the container's on / off state based on the identified labels.

[0118] For example, during the training process of the container state recognition model, training samples in the closed state of the container (which may include container images with different label settings) and training samples in the open state of the container (which may include container images with different label settings and different opening methods) can be collected, and the collected training samples can be used to train the container state recognition model so that the container state recognition model can learn the container open / closed state under different label states.

[0119] Accordingly, once a container image is obtained, it can be input into a trained container state recognition model to obtain the state recognition result of the container state recognition model.

[0120] For example, the state recognition results of the container state recognition model can include known states and unknown states.

[0121] Known states can include whether the container is open or closed; unknown states are those where it is uncertain whether the container is open or closed.

[0122] If the state identification result is a known state, the container's on / off state can be determined based on the state identification result.

[0123] If the state recognition result is unknown, the tag recognition can be performed in the manner described in the above embodiments to determine the physical state of the paired tag, and then the container opening / closing state can be determined based on the physical state of the paired tag.

[0124] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, the technical solutions provided in the embodiments of this application are described below with reference to specific examples.

[0125] This application provides a solution for recognizing the open / closed status of double-door or multi-door containers using video images, which can be applied to industry scenarios such as stores and warehouses.

[0126] For example, the containers mentioned above may include, but are not limited to, shipping containers, refrigerated cabinets, or vending machines.

[0127] In this embodiment, taking the recognition of the open / closed status of a double-door container as an example, its system structure diagram can be as follows: Figure 3 As shown, where:

[0128] 1. Image acquisition device: used to acquire images of double-door containers and their doors;

[0129] 2. Double-door container: A container consisting of two doors, including sliding and rotating containers;

[0130] 3. Label: A physical marker used by the processing module to determine the cabinet door status;

[0131] It should be noted that, Figure 3 The example shown is merely one instance of a label format, and not a limitation thereof.

[0132] 4. Processing module: Determines the cabinet door status based on the images acquired by the image acquisition device.

[0133] The label design in the embodiments of this application will be described below.

[0134] The labels are used by the system to help determine the status of the cabinet doors. For a double-door container, two labels are required, i.e., a pair of labels, which are installed (pasted, printed, hung, etc.) on the cabinet door partition (i.e., the adjacent positions of the two paired cabinet doors when the cabinet doors are closed).

[0135] For example, one symmetrical side of the label can be as parallel and coincident as possible with the edge of the partition, as illustrated in the diagram below. Figure 2A and Figure 2B As shown.

[0136] In an optional embodiment, the labels need to meet the following conditions in terms of shape: (1) Both labels have at least one straight edge (or chord) of equal length; (2) The two labels are mirror images of each other with the straight edge in (1) as the axis of symmetry, which is called the symmetrical edge; (3) If there are straight lines on other sides besides the symmetrical edge, the symmetrical edge can be marked with a specific color near the symmetrical edge.

[0137] It should be noted that for cabinet doors made of transparent material, the label can be installed inside or outside the cabinet door.

[0138] The following describes the process for identifying the open / closed status of a double-door container based on the above system architecture.

[0139] like Figure 5 As shown, the process for recognizing the open / closed status of a double-door container can include:

[0140] 3.3.1 Labels shall be installed on both doors of the double-door container as described above;

[0141] 3.3.2. Acquire images of the double-door container doors using an image acquisition device;

[0142] 3.3.3. Analyze the image through the processing module;

[0143] 3.3.4 The container status can include three states: S0: Unknown; S1: Container door closed; S2: Container door open;

[0144] 3.3.5. The initial state of the processing module when it starts is S0 (for example, no tag has been recognized yet).

[0145] 3.3.6 The processing module analyzes the acquired images, and the specific logic is as follows;

[0146] 3.3.6.1. Use image segmentation and other techniques to identify the position and region of two labels in the image, which are denoted as label1 and label2.

[0147] It should be noted that, in this embodiment of the application, considering that containers have a variety of shapes and materials in real-world scenarios, if the container opening and closing status is identified by directly using deep learning methods to identify container doors, a large amount of training material would be required, and it would still be difficult to cover all situations. Therefore, by setting labels on container doors and using deep learning methods to identify the status of paired labels to determine the container opening and closing status, the identification object category is singular, and there is no need for overly complex reasoning modules, which greatly reduces the complexity of implementation.

[0148] Furthermore, since the label is usually much smaller than the container door, even if the image acquisition device cannot capture the entire container or container door, the user can flexibly adjust the label position according to the angle of the image acquisition device. Thus, even if the image acquisition device cannot capture the entire container or container door, it can still capture the complete label, thereby achieving automatic recognition of the opening and closing of the container door.

[0149] 3.3.6.2. Obtain all straight edges based on the pixel information segmented by the label. If there is only one straight edge, it is a symmetrical edge; otherwise, the symmetrical edge can be determined based on the specific markers described in 3.2.

[0150] 3.3.6.3 If the physical state of the paired tags meets the following conditions (i.e., the container closing conditions mentioned above, also referred to as the cabinet door closing conditions in this embodiment), then the cabinet door is determined to be closed: C0, both tags exist simultaneously; C1, the symmetrical sides of the two tags are parallel; C2, the distance D between the symmetrical sides of the two tags is less than or equal to the set threshold th1 (e.g., ...). Figure 4A As shown, i.e., the first threshold mentioned above); C3, the difference in the symmetrical side lengths of the two labels |L1-L2| is less than or equal to the set threshold th2 (e.g. Figure 4B As shown, i.e., the second threshold mentioned above).

[0151] 3.3.6.4 When the physical state of the paired tag meets the cabinet door closing condition, update the cabinet state to S1; otherwise, maintain the current cabinet state (S0 or S2) and continue to judge the next frame image.

[0152] 3.3.6.5 If the container is in state S1, but the physical state of the paired label in the image does not meet the door closing condition within a time period t (i.e. the preset time threshold mentioned above), then the container state is set to S2; otherwise, the state remains unchanged.

[0153] 3.3.6.6 Identify the next frame image and repeat steps 3.3.6.1-3.3.6.4.

[0154] It should be noted that the container status can be manually set. For example, if the container moves out of the image acquisition device's acquisition range and then moves back in, the container status can be manually reset to S0.

[0155] For example, for a double-door container, instances where one door is open or both doors are open simultaneously, causing the label status to not meet the door closure condition, can include: Figure 6A , Figure 6B , Figure 6C as well as Figure 6D The situation is shown below. Where:

[0156] Figure 6A This applies if one of the tags in a pair does not exist. Figure 6B This is for cases where the symmetrical sides of the paired labels are not parallel; Figure 6C This applies to cases where the distance between the symmetrical edges in the paired labels is too large (i.e., exceeds the threshold th1 mentioned above); Figure 6D This refers to the case where the difference in the side length of the symmetrical side in the paired label is too large (i.e., it exceeds the threshold th2 mentioned above).

[0157] For example, the state machine in the above container opening / closing status recognition process can be as follows: Figure 7 As shown.

[0158] like Figure 7 As shown, when the container status is S0, if it is determined that the physical status of the paired tag meets the container door closing condition, the container status can be updated to S1; otherwise, it remains S0.

[0159] It should be noted that the determination conditions for the unknown state and the open state overlap. For example, if no tag is detected, the container state may be either unknown or open. Therefore, if the physical state of the paired tag does not meet the conditions for the container door to be closed, the container can remain in the S0 state if it is determined that the physical state of the paired tag does not meet the conditions for the container door to be closed.

[0160] If the physical state of the container is S1, and it is determined that the physical state of the paired tag does not meet the door closing condition, and the duration exceeds time t, then the container state can be updated to S2; otherwise, it remains S1.

[0161] If the physical state of the container is S2 and the physical state of the paired tag meets the door closing condition, the container state can be updated to S1; otherwise, it remains S2.

[0162] The method provided in this application has been described above. The apparatus provided in this application is described below:

[0163] Please see Figure 8 This is a schematic diagram of the structure of a container switch status identification device provided in an embodiment of this application, as shown below. Figure 8 As shown, the container switch status identification device may include:

[0164] Image sensor 810 is used to acquire images of the container;

[0165] The processor 820 is used to perform tag identification, determine the physical state of paired tags, and determine the container's on / off state based on the physical state of paired tags.

[0166] In some embodiments, the processor 820 determines the container switch state based on the physical state of the paired tag, including:

[0167] For any pair of paired tags, based on the determined physical state of the paired tags, determine whether the physical state of the paired tags meets the container closing conditions; the container closing conditions include the conditions that the physical state of the paired tags must meet when the container is in a closed state.

[0168] The container is determined to be in a closed state if the status of all paired tags of the container meets the container closing condition.

[0169] In some embodiments, the container closing condition includes both tags in the paired tags being identified, and one or more of the following conditions:

[0170] The symmetrical sides of the two tags in a pair are parallel;

[0171] The distance between the symmetrical edges of two tags in a pair is less than or equal to the first threshold.

[0172] The difference in the side length of the symmetrical side of the two labels in the pair is less than or equal to the second threshold.

[0173] In some embodiments, the processor 820 determines the container switch state based on the physical state of the paired tag, including:

[0174] For any pair of paired tags, based on the determined physical state of the paired tags, determine whether the physical state of the paired tags meets the container opening conditions; the container opening conditions include the conditions that the physical state of the paired tags must meet when the container is in the open state.

[0175] The container is determined to be in an open state if the physical state of any of its paired tags meets the container opening conditions.

[0176] In some embodiments, the container opening condition includes at least one of the paired tags not being recognized;

[0177] or,

[0178] Both tags in a pair are identified, but the symmetrical sides of the two tags are not parallel, the distance between the symmetrical sides of the two tags is greater than a first threshold, and / or the difference in the side length of the symmetrical sides of the two tags is greater than a second threshold.

[0179] In some embodiments, when there is only one straight edge in the label, the straight edge is the symmetrical edge of the two labels in the paired label;

[0180] If there is more than one straight edge in a label, the straight edge in the label with a preset identifier is the symmetrical edge of the two labels in the paired label.

[0181] In some embodiments, the processor 820 determines the container switch state based on the physical state of the paired tag, further including:

[0182] If the container was in a closed state in the previous state, and the physical state of any pair of paired tags does not meet the container's closed condition, then the container is determined to be in an open state.

[0183] In some embodiments, before the processor 820 determines that the container is in an open state, if the container was previously in a closed state and the physical state of any pair of paired tags does not meet the container closing condition, the processor further includes:

[0184] Determine whether the duration for which the physical state of any pair of paired tags does not meet the container closing conditions has reached a preset time threshold.

[0185] If the duration for which the physical state of any pair of paired tags does not meet the container closing condition does not reach a preset time threshold, the container is determined to be in a closed state.

[0186] If the duration for which the physical state of any pair of paired tags does not meet the container closing condition reaches a preset time threshold, the operation of determining the container to be in the open state is executed.

[0187] In some embodiments, before the processor 820 performs tag recognition and determines the physical state of the configured tag when the image sensor 810 acquires an image of the container, it may further include:

[0188] The acquired container image is input into the trained container state recognition model to obtain the state recognition result of the container state recognition model;

[0189] If the state identification result is unknown, the operation of performing tag identification and determining the physical state of the configured tag will be performed.

[0190] If the state identification result is a known state, the container switch state is determined based on the state identification result; wherein, the known state includes the container being in an open state or the container being in a closed state.

[0191] This application provides an electronic device including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the container switch state recognition method described above.

[0192] Please see Figure 9 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device may include a processor 901 and a memory 902 storing machine-executable instructions. The processor 901 and the memory 902 can communicate via a system bus 903. Furthermore, by reading and executing the machine-executable instructions in the memory 902 corresponding to the container switch state identification logic, the processor 901 can execute the container switch state identification method described above.

[0193] The memory 902 mentioned in this document can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0194] In some embodiments, a machine-readable storage medium, such as Figure 9 The memory 902 in the device stores machine-executable instructions, which, when executed by a processor, implement the container switch state identification method described above. For example, the storage medium may be ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0195] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0196] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for identifying the on / off state of a container, characterized in that, The container is a double-door or multi-door container. For any door of the container, two matching tags are placed at designated locations on that door and another door paired with it. The method includes: The container image is acquired and the label is identified to determine the physical state of the paired labels. The physical state of the paired labels includes whether the labels exist and the positional relationship between the two labels in the pair. Determine the container's open / closed state based on the physical state of the paired tags; Determining the container's open / closed state based on the physical state of the paired tags includes: For any pair of paired tags, based on the determined physical state of the paired tags, determine whether the physical state of the paired tags meets the container closing conditions; the container closing conditions include the conditions that the physical state of the paired tags must meet when the container is in a closed state. The container is determined to be in a closed state if the physical states of all paired tags of the container meet the container closing conditions. The container closing conditions include both tags in the paired tags being identified, and one or more of the following conditions: The symmetrical sides of the two tags in a pair are parallel; The distance between the symmetrical edges of two tags in a pair is less than or equal to the first threshold. The difference in the side length of the symmetrical side of the two paired labels is less than or equal to the second threshold. In the case where there is only one straight edge in the label, the straight edge is the symmetrical edge of the two labels in the pair; If there is more than one straight edge in a label, the straight edge in the label with a preset identifier is the symmetrical edge of the two labels in the paired label.

2. The method according to claim 1, characterized in that, Determining the container's open / closed state based on the physical state of the paired tags includes: For any pair of paired tags, based on the determined physical state of the paired tags, determine whether the physical state of the paired tags meets the container opening conditions; the container opening conditions include the conditions that the physical state of the paired tags must meet when the container is in the open state. The container is determined to be in an open state if the physical state of any pair of paired tags of the container satisfies the container opening condition.

3. The method according to claim 2, characterized in that, The container opening condition includes that at least one of the paired tags is not recognized; or, Both tags in a pair are identified, but the symmetrical sides of the two tags are not parallel, and / or the distance between the symmetrical sides of the two tags is greater than a first threshold, and / or the difference in the side length of the symmetrical sides of the two tags is greater than a second threshold.

4. The method according to claim 1, characterized in that, Determining the container's open / closed state based on the physical state of the paired tags further includes: If the container was in a closed state in the previous state, and the physical state of any pair of paired tags does not meet the container's closed condition, then the container is determined to be in an open state.

5. The method according to claim 4, characterized in that, If the container was previously in a closed state and the physical state of any pair of paired tags does not meet the container's closed condition, the process before determining the container to be in an open state further includes: Determine whether the duration for which the physical state of any pair of paired tags does not meet the container closing conditions has reached a preset time threshold. If the duration for which the physical state of any pair of paired tags does not meet the container closing condition does not reach a preset time threshold, the container is determined to be in a closed state. If the duration for which the physical state of any pair of paired tags does not meet the container closing condition reaches a preset time threshold, the operation of determining the container to be in the open state is executed.

6. The method according to any one of claims 1-5, characterized in that, Before performing tag recognition and determining the physical state of the configured tags after obtaining the container image, the process also includes: The acquired container image is input into the trained container state recognition model to obtain the state recognition result of the container state recognition model; If the state recognition result is unknown, the operation of performing tag recognition and determining the physical state of the paired tags will be performed. If the state identification result is a known state, the container switch state is determined based on the state identification result; wherein, the known state includes the container being in an open state or the container being in a closed state.

7. A container switch status recognition device, characterized in that, The container is a double-door or multi-door container. For each door of the container, two matching tags are provided at designated locations on the door and another door paired with it. The device includes: Image sensor, used to acquire images of the container; The processor is used to identify tags and determine the physical state of paired tags; based on the physical state of paired tags, it determines the container's open / closed state; wherein, the physical state of paired tags includes whether the tags exist and the positional relationship between the two tags in the paired tags. The processor determines the container's on / off state based on the physical state of the paired tag, including: For any pair of paired tags, based on the determined physical state of the paired tags, determine whether the physical state of the paired tags meets the container closing conditions; the container closing conditions include the conditions that the physical state of the paired tags must meet when the container is in a closed state. The container is determined to be in a closed state if the physical states of all paired tags of the container meet the container closing conditions. The container closing conditions include both tags in the paired tags being identified, and one or more of the following conditions: The symmetrical sides of the two tags in a pair are parallel; The distance between the symmetrical edges of two tags in a pair is less than or equal to the first threshold. The difference in the side length of the symmetrical side of the two paired labels is less than or equal to the second threshold. In the case where there is only one straight edge in the label, the straight edge is the symmetrical edge of the two labels in the pair; If there is more than one straight edge in a label, the straight edge in the label with a preset identifier is the symmetrical edge of the two labels in the paired label.

8. The apparatus according to claim 7, characterized in that, The processor determines the container's on / off state based on the physical state of the paired tag, including: For any pair of paired tags, based on the determined physical state of the paired tags, determine whether the physical state of the paired tags meets the container opening conditions; the container opening conditions include the conditions that the physical state of the paired tags must meet when the container is in the open state. If the physical state of any of the paired tags of the container meets the container opening conditions, the container is determined to be in an open state. The container opening condition includes that at least one of the paired tags is not recognized; or, Both tags in the pair are identified, but the symmetrical sides of the two tags in the pair are not parallel, and / or the distance between the symmetrical sides of the two tags in the pair is greater than the first threshold, and / or the difference in the side length of the symmetrical sides of the two tags in the pair is greater than the second threshold. The processor determines the container's on / off state based on the physical state of the paired tag, and further includes: If the container was in a closed state in the previous state, and the physical state of any pair of paired tags does not meet the container closing condition, then the container is determined to be in an open state. Specifically, before the processor determines the container to be in an open state, if the container's previous state was closed and the physical state of any pair of paired tags does not meet the container's closed condition, the processor further includes: Determine whether the duration for which the physical state of any pair of paired tags does not meet the container closing conditions has reached a preset time threshold. If the duration for which the physical state of any pair of paired tags does not meet the container closing condition does not reach a preset time threshold, the container is determined to be in a closed state. If the duration for which the physical state of any pair of paired tags does not meet the container closing condition reaches a preset time threshold, the operation of determining the container to be in an open state is executed. And / or, Before the processor performs tag recognition and determines the physical state of the configured tag after the image sensor acquires an image of the container, the process further includes: The acquired container image is input into the trained container state recognition model to obtain the state recognition result of the container state recognition model; If the state identification result is unknown, the operation of performing tag identification and determining the physical state of the configured tag will be performed. If the state identification result is a known state, the container switch state is determined based on the state identification result; wherein, the known state includes the container being in an open state or the container being in a closed state.

9. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the method as described in any one of claims 1-6.

Images