Method, apparatus, terminal and computer-readable storage medium for identifying unpacking status

Abstract

This invention provides a method, apparatus, terminal, and computer-readable storage medium for identifying the unpacking status. The method involves acquiring a video frame to be identified; using a detection model to identify the state of the box within the video frame, obtaining probability values ​​for each preset first state category; determining the probability values ​​corresponding to each preset second state category based on the correspondence between each preset first state category and each preset second state category, and the probability values ​​of each preset first state category corresponding to the box; and selecting the preset second state category with the highest probability value as the box's state identification result. This application effectively avoids switching between state categories in the identification result by mapping fine-grained classification levels to coarse-grained classification levels, thus improving the accuracy of package detection status.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to an unpacking status recognition method, apparatus, terminal, and computer-readable storage medium. Background Technology

[0002] With the development of the express delivery and logistics industry, the corresponding parcel security inspection system has also become more sophisticated. To better assist logistics companies in supervising the opening and inspection of parcels, it is necessary to identify the status of the opened parcels. However, existing identification methods are prone to issues such as abrupt changes in the opening status and unsmooth opening, leading to false alarms and frequent alerts on the platform. Summary of the Invention

[0003] The main technical problem solved by this invention is to provide a method, device, terminal and computer-readable storage medium for identifying the unpacking status, thereby solving the problem of low accuracy in package detection status in the prior art.

[0004] To solve the above-mentioned technical problems, the first technical solution adopted by the present invention is: to provide an unpacking status recognition method, the unpacking status recognition method comprising:

[0005] Acquire the video frame to be identified; the video frame to be identified contains a box.

[0006] The detection model identifies the state of the box contained in the video frame to be identified, and obtains the probability value of the box's open state as each preset first state category.

[0007] Based on the correspondence between each preset first state category and each preset second state category, and the probability value of each preset first state category corresponding to the box, the probability value corresponding to each preset second state category is determined; the number of preset second state categories is less than the number of preset first state categories.

[0008] The preset second state category corresponding to the probability value with the largest value is selected as the state recognition result of the box.

[0009] The training methods for the recognition and detection model include:

[0010] Obtain the training dataset, which includes multiple sample images containing boxes; the sample images are associated with the annotation information of the boxes, including the annotation probability values ​​of the box's open state for each preset state;

[0011] The box state in the sample image is identified by the recognition and detection model, and the predicted probability value of the box opening state is obtained as each preset state.

[0012] The recognition and detection model is iteratively trained based on the error between the predicted probability value and the labeled probability value corresponding to each preset state.

[0013] The training methods for the recognition and detection model also include:

[0014] Perform data augmentation on the training dataset.

[0015] This includes data augmentation of the training dataset, including:

[0016] Based on the annotation information corresponding to each sample image in the training dataset, the sample images are classified to obtain multiple state categories corresponding to the training dataset;

[0017] The weight of each state category is obtained based on the number of sample images corresponding to each state category;

[0018] The image weight corresponding to the sample image is determined based on the state category to which the sample image belongs, the weight of the state category, and the number of state categories.

[0019] If an image weight is less than a preset value, the sample image corresponding to that weight is copied and added to the training dataset.

[0020] This includes data augmentation of the training dataset, including:

[0021] The predicted state and recognition confidence of the box are obtained by detecting the state of the target contained in the sample images in the training dataset through the recognition detection model.

[0022] If the error between the labeling information of the box and the predicted state is less than the pre-threshold, the identification confidence of the box is compared with the pre-set confidence.

[0023] If the recognition confidence of the box is less than the preset confidence, the sample image corresponding to the recognition confidence is copied and added to the training dataset.

[0024] The identification and detection model is iteratively trained based on the error between the predicted probability value and the labeled probability value corresponding to each preset state, including:

[0025] Based on the hyperparameters and the predicted probability value corresponding to the preset state output by the recognition and detection model, the error value corresponding to the preset state is determined.

[0026] The recognition and detection model is trained iteratively by summing the error values ​​corresponding to each preset state.

[0027] The unpacking status recognition method also includes:

[0028] A mapping table is pre-constructed, which includes the mapping relationship between each preset first state category and each preset second state category.

[0029] To solve the above-mentioned technical problems, the second technical solution adopted by the present invention is: to provide an unpacking status identification device, the unpacking status identification device comprising:

[0030] The acquisition module is used to acquire the video frame to be identified; the video frame to be identified contains the box.

[0031] The processing module is used to identify the state of the box contained in the video frame to be identified through the identification and detection model, and obtain the probability value of the box's open state as each preset first state category.

[0032] The analysis module is used to determine the probability value corresponding to each preset second state category based on the correspondence between each preset first state category and each preset second state category and the probability value of each preset first state category corresponding to the box; the number of preset second state categories is less than the number of preset first state categories.

[0033] The determination module is used to select the preset second state category corresponding to the probability value with the largest value as the state recognition result of the box.

[0034] To solve the above-mentioned technical problems, the third technical solution adopted by the present invention is to provide a terminal, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. The processor is used to execute program data to implement the steps in the above-mentioned unpacking status identification method.

[0035] To solve the above-mentioned technical problems, the fourth technical solution adopted by the present invention is to provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps in the above-mentioned unpacking status identification method.

[0036] The beneficial effects of this invention are as follows: Unlike existing technologies, this invention provides a method, apparatus, terminal, and computer-readable storage medium for identifying the unpacking state. The method involves acquiring a video frame to be identified; the video frame contains a box; a detection model identifies the state of the box in the video frame to be identified, obtaining probability values ​​for each preset first state category; based on the correspondence between each preset first state category and each preset second state category, and the probability values ​​of each preset first state category corresponding to the box, a probability value corresponding to each preset second state category is determined; the number of preset second state categories is less than the number of preset first state categories; the preset second state category corresponding to the highest probability value is selected as the state identification result of the box. In this application, the state of the box is divided into fine-grained categories, and the probability value of the box in the video frame belonging to each preset first state category is detected. Then, each preset first state category is mapped to each preset second state category in coarse-grained division. The state recognition result of the box in the video frame is determined according to the probability value corresponding to each preset second state category. By mapping the fine-grained division level to the coarse-grained division level, the back-and-forth switching of state categories in the recognition result is effectively avoided, and the accuracy of package detection state is improved. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating the unpacking status recognition method provided by the present invention;

[0039] Figure 2 yes Figure 1 A flowchart illustrating a specific embodiment of step S2 in the provided unpacking status recognition method;

[0040] Figure 3 This is a diagram illustrating the proportion of sample images corresponding to each labeled category in the training dataset;

[0041] Figure 4 yes Figure 2 A flowchart illustrating a specific embodiment of step S21 in the provided unpacking status recognition method;

[0042] Figure 5 This is a schematic diagram of the framework of an embodiment of the unpacking status recognition device provided by the present invention;

[0043] Figure 6This is a schematic diagram of the framework of an embodiment of the terminal provided by the present invention;

[0044] Figure 7 A schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present invention. Detailed Implementation

[0045] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0046] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0047] In this article, the term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "more" in this article means two or more objects.

[0048] To enable those skilled in the art to better understand the technical solution of the present invention, the following describes in further detail a method for identifying the unpacking status provided by the present invention in conjunction with the accompanying drawings and specific embodiments.

[0049] Please see Figure 1 , Figure 1 This is a flowchart illustrating the unpacking status recognition method provided by the present invention. This embodiment provides an unpacking status recognition method applicable to the recognition of the unpacking status of various three-dimensional boxes, such as packages, cabinets, and containers. In this embodiment, a rectangular package is used as an example for illustration. The unpacking status recognition method includes the following steps.

[0050] S1: Obtain the video frame to be identified; the video frame to be identified contains a box.

[0051] Specifically, a video stream is acquired by capturing still images of the same scene using an image acquisition device. Three different frame extraction intervals (30s, 60s, and 120s) are set according to the length and quality of the video stream to perform frame extraction processing, avoiding excessive similarity between video frames and preventing unnecessary computational loss. Each extracted video frame is then used as a separate video frame to be identified.

[0052] S2: The detection model identifies the state of the box contained in the video frame to be identified, and obtains the probability value of the box's open state as each preset first state category.

[0053] S3: Based on the correspondence between each preset first state category and each preset second state category and the probability value of each preset first state category corresponding to the box, determine the probability value corresponding to each preset second state category; the number of preset second state categories is less than the number of preset first state categories.

[0054] S4: Select the preset second state category corresponding to the probability value with the largest value as the state recognition result of the box.

[0055] In one embodiment, the training method for the identification and detection model in step S2 includes the following steps.

[0056] If the opening and closing states of a box are simply defined as "closed" and "open," the closed state is relatively clear, but it's difficult to determine when the box is open and the size of the opening. Furthermore, the transition from closed to open or vice versa can be too abrupt. Therefore, when labeling sample images in the training dataset, fine-grained opening rules are needed to make the package opening process smoother.

[0057] Based on the specific structure of the box and the number of hinges required for identification, the open state of the box is subdivided into at least three categories. In this embodiment, taking a rectangular package as an example, the open state of the box is subdivided into five categories. According to the number of hinges that are opened, the open state categories are defined as closed, fully open, 1 / 4 open, 1 / 2 open, and 3 / 4 open.

[0058] Specifically, when all hinges of the cabinet are closed, meaning the opening angle of all hinges is less than a preset angle, the cabinet is considered to be in a closed state. When one hinge's opening angle is greater than or equal to the preset angle, and the other three hinges' opening angles are less than the preset angle, the cabinet is considered to be in a 1 / 4 open state. When two hinges' opening angles are greater than or equal to the preset angle, and the other two hinges' opening angles are less than the preset angle, the cabinet is considered to be in a 1 / 2 open state. When three hinges' opening angles are greater than or equal to the preset angle, and the remaining hinge's opening angle is less than the preset angle, the cabinet is considered to be in a 3 / 4 open state. When all hinges' opening angles are greater than the preset angle, the cabinet is considered to be in a fully open state. The opening angle is the minimum angle at which the hinge rotates to the cabinet's sealing plane. The preset angle can be 90°, 60°, 75°, etc., and can be set according to actual needs. Specifically, if a hinge is missing or too small, it is confirmed to be in the open state. Invisible hinges are assumed to be in the closed state.

[0059] In another embodiment, when the box to be identified is a triangular prism with three hinges, the open state of the box is subdivided into four categories. Based on the number of hinges that are open, the open state of the box is defined as closed, fully open, 1 / 3 open, and 2 / 3 open.

[0060] By using the above rules, sample images containing boxes in the training dataset are "labeled" to determine the actual open state of the boxes.

[0061] Please see Figure 2 , Figure 2 yes Figure 1 A flowchart illustrating a specific embodiment of step S2 in the provided unpacking status recognition method.

[0062] S21: Obtain the training dataset, which includes multiple sample images containing boxes; the sample images are associated with the annotation information of the boxes, including the annotation probability values ​​of the box's open state for each preset state.

[0063] When the actual state of the box in the sample image is closed, the annotation information corresponding to the sample image is: the annotation probability of the closed state is 100%, and the annotation probability values ​​corresponding to each of the other preset states are 0.

[0064] Please see Figure 3 , Figure 3 This is a diagram showing the proportion of sample images corresponding to each labeled category in the training dataset.

[0065] Using the fine-grained opening-box rules described above can lead to a situation where the number of sample images corresponding to certain preset states is too small, resulting in class imbalance. For example... Figure 3 As shown, the number of sample images corresponding to the preset state of "off" accounts for 75% of all sample images, while the number of sample images for other preset states is relatively small. Therefore, it is inevitable that the error rate of state recognition of the box or the prediction confidence will be high when there are too few sample images corresponding to the preset states.

[0066] Data augmentation can be performed on the training dataset.

[0067] In one embodiment, the number of sample images corresponding to preset states can be increased to alleviate this problem. The sample images are classified based on the annotation information corresponding to each sample image in the training dataset to obtain multiple state categories corresponding to the training dataset.

[0068] Please see Figure 4 , Figure 4 yes Figure 2 A flowchart illustrating a specific embodiment of step S21 in the provided unpacking status recognition method.

[0069] S211: Based on the number of sample images corresponding to each state category, obtain the weight of each state category.

[0070] Specifically, before training, the sample images in the training dataset are clustered to obtain N sample images corresponding to each state category. c The weight of each state category is determined based on the number of sample images corresponding to each state category.

[0071] S212: Determine the image weight corresponding to the sample image based on the state category to which the sample image belongs, the weight of the state category, and the number of state categories.

[0072] Specifically, the image weight W corresponding to each sample image in the training dataset is calculated based on the following formula. img .

[0073]

[0074] In the formula: W img Represents the image weights; C represents the total number of state categories; n c Represents the number of each category; map c This represents the mAP of class c in each iteration.

[0075] S213: In response to an image weight being less than a preset value, the sample image corresponding to the image weight is copied and added to the training dataset.

[0076] Specifically, the weights of each image are compared with preset values ​​to determine whether the sample image is a high-frequency or low-frequency sample image. Based on the principle of sampling fewer high-frequency categories and more low-frequency categories, the goal is to train the recognition and detection network with as many sample images as possible that are difficult to identify correctly, thereby improving the recognition accuracy of the recognition and detection model.

[0077] If the image weight is less than the preset value, the sample image corresponding to the image weight is determined to be a low-frequency sample image, and the sample image is copied and added to the training dataset again.

[0078] In another embodiment, a detection model is used to detect the state of targets contained in sample images in the training dataset, obtaining the predicted state and recognition confidence of the box. If the error between the box's annotation information and the predicted state is less than a pre-threshold, the recognition confidence of the box is compared with a pre-set confidence level. If the recognition confidence of the box is less than the pre-set confidence level, the sample image corresponding to the recognition confidence level is copied and added to the training dataset. The pre-set confidence level can be set according to actual conditions, for example, it can be set to 0.7.

[0079] S22: The box state in the sample image is identified by the recognition and detection model, and the predicted probability value of the box opening state is obtained as each preset state.

[0080] Specifically, the state of the box in each sample image is identified by the recognition and detection model, and the predicted probability value of each preset state corresponding to the box in each sample image is obtained.

[0081] S23: Iteratively train the recognition and detection model based on the error between the predicted probability value and the labeled probability value corresponding to each preset state.

[0082] Specifically, based on the hyperparameters and the predicted probability values ​​corresponding to the preset states output by the recognition and detection model, the error values ​​corresponding to the preset states are determined. The recognition and detection model is then iteratively trained based on the sum of the error values ​​corresponding to each preset state. Since the different states of the box are independent of each other, the softmax function is used in the output layer.

[0083] In one embodiment, the error value Loss corresponding to each preset state is calculated based on the following formula.

[0084]

[0085]

[0086] In the formula: y′ c This represents the predicted probability value for class C; y c This represents the labeled probability value predicted as category C. α and γ represent hyperparameters. Hyperparameter α is used to increase the weight of the error corresponding to the low-proportion preset state; hyperparameter γ is used to decrease the weight of the error value of the sample image corresponding to the recognition confidence level higher than the preset confidence level.

[0087] In one embodiment, a mapping table is constructed prior to step S3.

[0088] While fine-grained box-opening rules are clear enough to prevent misjudgments in the annotation information of sample images, there is still a certain prediction error between the preset first state categories detected by the recognition and detection model. This causes the box recognition result to switch back and forth between the two preset first state categories. Therefore, by setting coarse-grained box-opening rules, the switching of recognition results is avoided, making the box-opening process smoother. Coarse-grained box-opening rules set the box's open state to multiple preset second state categories. The total number of preset second state categories is less than the total number of preset first state categories.

[0089] In one embodiment, the preset second state category includes three states: package closed, package in progress, and package open.

[0090] A mapping table is pre-constructed, which includes the mapping relationship between each preset first state category and each preset second state category.

[0091] In one embodiment, a first preset state category of "closed" and a 1 / 4 open state correspond to a second preset state category of "package closed"; a first preset state category of "1 / 2 open" corresponds to a second preset state category of "package opening"; and a first preset state category of "3 / 4 open" and a fully open state correspond to a second preset state category of "package open". The specific correspondence can be set according to actual conditions and is not further restricted. The correspondence between coarse-grained and fine-grained unpacking rules can be adjusted autonomously based on different application scenarios without retraining the detection model, thus saving costs.

[0092] Specifically, based on the correspondence between each preset first state category and a preset second state category, and the probability value of each preset first state category corresponding to the box, the probability value corresponding to each preset second state category is calculated. The preset second state category corresponding to the highest probability value is selected as the state recognition result of the box.

[0093] For example, the probability of the box being in a closed state is determined by summing the probability values ​​of the first preset state category (closed) and the second preset state category (1 / 4 open) in the video frame to be identified; the probability of the box being in a partially open state is determined by summing the probability values ​​of the first preset state category (1 / 2 open) in the video frame to be identified; and the probability of the box being in a fully open state is determined by summing the probability values ​​of the first preset state category (fully open) and the second preset state category (3 / 4 open) in the video frame to be identified. Based on the probability values ​​corresponding to each preset second state category, the platform outputs the preset second state category with the highest probability value as the box's state recognition result.

[0094] The unpacking status recognition method provided in this embodiment acquires a video frame to be recognized; the video frame contains a box; a recognition detection model identifies the status of the box in the video frame to obtain the probability value of the box's unpacking status belonging to each preset first status category; based on the correspondence between each preset first status category and each preset second status category, and the probability value of each preset first status category corresponding to the box, the probability value corresponding to each preset second status category is determined; the number of preset second status categories is less than the number of preset first status categories; the preset second status category corresponding to the highest probability value is selected as the box's status recognition result. In this application, the box's status is divided into fine-grained categories, and the probability value of the box in the video frame belonging to each preset first status category is detected. Then, each preset first status category is mapped to each preset second status category in a coarse-grained division. The box's status recognition result in the video frame is determined according to the probability value corresponding to each preset second status category. By mapping the fine-grained division level to the coarse-grained division level, the back-and-forth switching of status categories in the recognition result is effectively avoided, improving the accuracy of package detection status.

[0095] Please see, Figure 5 This is a schematic diagram of the framework of an embodiment of the unpacking status recognition device provided by the present invention. This embodiment provides an unpacking status recognition device 60, which includes an acquisition module 61, a processing module 62, an analysis module 63, and a determination module 64.

[0096] The acquisition module 61 is used to acquire the video frame to be identified; the video frame to be identified contains the box.

[0097] The processing module 62 is used to identify the state of the box contained in the video frame to be identified through the identification detection model, and obtain the probability value of the box's open state as each preset first state category.

[0098] The analysis module 63 is used to determine the probability value corresponding to each preset second state category based on the correspondence between each preset first state category and each preset second state category and the probability value of each preset first state category corresponding to the box; the number of preset second state categories is less than the number of preset first state categories.

[0099] The determination module 64 is used to select the preset second state category corresponding to the probability value with the largest value as the state recognition result of the box.

[0100] The unpacking status recognition device provided in this embodiment performs fine-grained classification of the box's status, detects the probability value of the box in the video frame belonging to each preset first status category, and then maps each preset first status category to each preset second status category in a coarse-grained classification. Based on the probability value corresponding to each preset second status category, the status recognition result of the box in the video frame is determined. By mapping the fine-grained classification level to the coarse-grained classification level, the back-and-forth switching of status categories in the recognition result is effectively avoided, and the accuracy of package detection status is improved.

[0101] Please see Figure 6 , Figure 6 This is a schematic diagram of a terminal embodiment provided by the present invention. The terminal 80 includes a memory 81 and a processor 82 coupled to each other. The processor 82 is used to execute program instructions stored in the memory 81 to implement the steps of any of the above-described embodiments of the unpacking state recognition method. In a specific implementation scenario, the terminal 80 may include, but is not limited to, a microcomputer or a server. In addition, the terminal 80 may also include mobile devices such as laptops and tablets, which are not limited here.

[0102] Specifically, processor 82 controls itself and memory 81 to implement the steps of any of the above-described embodiments of the unpacking state identification method. Processor 82 can also be referred to as a CPU (Central Processing Unit). Processor 82 may be an integrated circuit chip with signal processing capabilities. Processor 82 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 82 can be implemented using integrated circuit chips.

[0103] Please see Figure 7 , Figure 7 This is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present invention. The computer-readable storage medium 90 stores program instructions 901 that can be executed by a processor. The program instructions 901 are used to implement the steps of any of the above embodiments of the unpacking state recognition method.

[0104] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0105] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0107] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0108] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0109] The above are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for recognizing unpacking status, characterized in that, The unpacking status recognition method includes: Acquire a video frame to be identified; the video frame to be identified contains a box. The detection model identifies the state of the box contained in the video frame to be identified, and obtains the probability value of the box's open state being each preset first state category; the preset first state category includes at least one of closed state, 1 / 4 open state, 1 / 2 open state, 3 / 4 open state and fully open state. Based on the correspondence between each preset first state category and each preset second state category, and the probability value of each preset first state category corresponding to the box, the probability value corresponding to each preset second state category is determined; the number of types of preset second state categories is less than the number of types of preset first state categories; the preset second state category includes at least one of package closed state, package opening state, and package open state; The preset second state category corresponding to the probability value with the largest value is selected as the state recognition result of the box.

2. The unpacking status recognition method according to claim 1, characterized in that, The training method for the recognition and detection model includes: Obtain a training dataset, which includes multiple sample images containing the box; the sample images are associated with annotation information of the box, and the annotation information includes the annotation probability value of the box's open state being each preset state; The recognition and detection model is used to identify the state of the box in the sample image to obtain the predicted probability value of the box's open state as each of the preset states; The recognition and detection model is iteratively trained based on the error between the predicted probability value and the labeled probability value corresponding to each preset state.

3. The unpacking status recognition method according to claim 2, characterized in that, The training method for the recognition and detection model also includes: Data augmentation processing is performed on the training dataset.

4. The unpacking status recognition method according to claim 3, characterized in that, The data augmentation process performed on the training dataset includes: Based on the annotation information corresponding to each sample image in the training dataset, the sample images are classified to obtain multiple state categories corresponding to the training dataset; The weight of each state category is obtained based on the number of sample images corresponding to each state category; The image weight corresponding to the sample image is determined based on the state category to which the sample image belongs, the weight of the state category, and the number of state categories. If the image weight is less than a preset value, the sample image corresponding to the image weight is copied and added to the training dataset.

5. The unpacking status recognition method according to claim 3, characterized in that, The data augmentation process performed on the training dataset includes: The recognition and detection model is used to detect the state of the box contained in the sample images in the training dataset, so as to obtain the predicted state and recognition confidence of the box; If the error between the labeling information of the box and the predicted state is less than a pre-threshold, the identification confidence of the box is compared with a pre-set confidence. If the recognition confidence of the box is less than the preset confidence, the sample image corresponding to the recognition confidence is copied and added to the training dataset.

6. The unpacking status recognition method according to claim 2, characterized in that, The iterative training of the recognition and detection model based on the error value between the predicted probability value and the labeled probability value corresponding to each preset state includes: Based on the hyperparameters and the predicted probability value corresponding to the preset state output by the recognition and detection model, the error value corresponding to the preset state is determined. The recognition and detection model is iteratively trained based on the summation of the error values ​​corresponding to each preset state.

7. The unpacking status identification method according to claim 1, characterized in that, The unpacking status recognition method further includes: A mapping table is pre-constructed, which includes the mapping relationship between each of the preset first state categories and each of the preset second state categories.

8. An unpacking status recognition device, characterized in that, The unpacking status recognition device includes: The acquisition module is used to acquire a video frame to be identified; the video frame to be identified contains a box. The processing module is used to identify the state of the box contained in the video frame to be identified by an identification detection model, and to obtain the probability value of the box's open state being each preset first state category; the preset first state category includes at least one of closed state, 1 / 4 open state, 1 / 2 open state, 3 / 4 open state and fully open state; The analysis module is used to determine the probability value corresponding to each preset second state category based on the correspondence between each preset first state category and each preset second state category and the probability value of each preset first state category corresponding to the box; the number of types of preset second state categories is less than the number of types of preset first state categories; the preset second state category includes at least one of package closed state, package in the process of opening state and package open state; The determination module is used to select the preset second state category corresponding to the probability value with the largest value as the state recognition result of the box.

9. A terminal, characterized in that, The terminal includes a memory, a processor, and a computer program stored in the memory and running on the processor, the processor being used to execute program data to implement the steps in the unpacking status identification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the unpacking status identification method as described in any one of claims 1 to 7.

Images