Icing and frosting state detection method, controller and storage medium

Abstract

The invention relates to the technical field of refrigeration equipment, particularly provides an icing and frosting state detection method, a controller and a storage medium, and aims to solve the problem of how to improve the efficiency and accuracy of detecting the storage location of a refrigeration house at low cost. In order to achieve the purpose, the method provided by the invention comprises the steps of obtaining a real-time storage location image of a current refrigeration storage location; based on a pre-trained icing and frosting state detection model, determining a detection result of a corresponding icing and frosting state according to the real-time storage location image; wherein the icing and frosting state detection model comprises a decision tree model, the decision tree model comprises nodes, the nodes comprise leaf nodes, and category marks of the leaf nodes are used for determining a detection result of the icing and frosting state of the refrigeration storage location. According to the method and the device, whether the refrigeration storage location is in the icing and frosting state or not can be automatically detected through the decision tree model in the icing and frosting state detection model, so that the labor cost is reduced, and the detection accuracy and the detection efficiency of the icing and frosting state are improved.

Description

Technical Field

[0001] The present application relates to the technical field of refrigeration equipment, and in particular to an icing and frosting state detection method, a controller and a storage medium. Background Art

[0002] Cold storage is a refrigeration equipment used to create an environment with a different temperature or humidity from the outdoor environment, and store food, liquids, chemicals, medicines or vaccines at a constant temperature and humidity. Large cold storage, in particular, is usually kept at an ultra-low temperature, that is, around minus 80 degrees. At this temperature, if items need to be stored and retrieved, it is no longer feasible to do so manually. The current mainstream solution is to use automated robotic arms to take and place items. However, as the number of times items are taken and placed increases, it is inevitable that water vapor will enter. When water vapor condenses on the cold storage space, frost or ice will occur. When frost and ice are severe, the robotic arm will collide with the condensed frost and ice when taking and placing items, causing items to fall, damage to the robotic arm, or deformation of the shelf.

[0003] The current detection method is usually to check whether the cold storage locations are frozen or frosted by taking photos regularly and checking manually. However, large cold storages usually have tens of thousands of locations. Not only is the workload of taking photos huge, but the manual inspection and identification of photos not only consumes a lot of manpower, but also cannot detect the locations in the state of ice and frost in time, which is inefficient and has low accuracy.

[0004] Accordingly, a new solution is needed in the art to solve the above problems. Summary of the invention

[0005] In order to overcome the above-mentioned defects, the present application is proposed to solve or at least partially solve the technical problem of how to improve the efficiency and accuracy of detecting cold storage locations with low labor costs.

[0006] In a first aspect, a method for detecting an icing or frosting state is provided, the method comprising:

[0007] Get the real-time image of the current cold storage location;

[0008] Based on a pre-trained icing and frosting state detection model, a detection result of the icing and frosting state corresponding to the current cold storage location is determined according to the real-time storage location image;

[0009] Among them, the icing and frosting state detection model includes at least one decision tree model, the decision tree model includes at least one node, the node includes at least one leaf node, and the category label of the leaf node is used to determine the detection result of the icing and frosting state of the cold storage location.

[0010] In one technical solution of the above method,

[0011] The step of determining the icing and frosting state corresponding to the current cold storage location according to the real-time storage location image includes:

[0012] According to the real-time storage location image, determine the attribute number to be tested and the attribute value of the attribute number to be tested, wherein the attribute number to be tested is the number identifying the image feature of the real-time storage location image, and the attribute value is the feature value of the image feature; wherein the attribute number to be tested is used to determine the attribute value to be tested corresponding to the node during the decision tree traversal process; and the attribute value of the attribute number to be tested is used to determine the node to be traversed during the decision tree traversal process;

[0013] Generate a storage location image data set to be tested according to the attribute number to be tested and the attribute value corresponding to the attribute number to be tested;

[0014] Based on the image data set of the storage location to be tested, calling at least one of the decision tree models to obtain a category label of at least one target leaf node, wherein the category label includes an icing or frosting state or a non-icing or frosting state;

[0015] The detection result of the icing and frosting state is determined according to the category label of the at least one target leaf node.

[0016] In one technical solution of the above method,

[0017] Each of the nodes of the decision tree model is a dictionary containing an attribute number and a corresponding attribute value;

[0018] Each of the decision tree models obtains a category label of a target leaf node according to the following steps, including:

[0019] For each iteration in the traversal process of the decision tree model, based on the root node of the decision tree model, all child nodes of the root node are obtained;

[0020] According to the attribute number corresponding to the root node, determining the attribute number to be tested in the storage location image data set to be tested;

[0021] For each of the child nodes, based on the attribute value of the child node, determining whether the attribute value corresponding to the attribute number to be tested meets a preset condition;

[0022] If not, proceed to the next child node judgment;

[0023] If so, taking the child node as the root node and entering the next iteration of the decision tree model traversal process;

[0024] When the child node is a leaf node, the iteration is stopped, the leaf node is used as the target leaf node, and the category label of the target leaf node is returned.

[0025] In one technical solution of the above method,

[0026] The method further comprises obtaining the icing and frosting state detection model according to the following steps:

[0027] Acquire an annotated storage location image for training, wherein the annotated storage location image includes an annotated label of an icing or frosting state or an annotated label of an unicing or frosting state;

[0028] According to the annotated storage location image, determine the attribute number and the corresponding attribute value for training, wherein the attribute number for training is a number that identifies the image feature of the annotated storage location image, and the attribute value is a feature value of the image feature; wherein the attribute number for training is used to generate a node in the decision tree generation process; and the attribute value of the attribute number for training is used to determine the position of each node in the decision tree generation process;

[0029] Obtaining a training data set according to the annotated labels of the annotated storage location images and the attribute numbers and corresponding attribute values ​​used for training;

[0030] Based on a decision tree algorithm, at least one decision tree model is generated according to the training data set to obtain the icing and frost detection model.

[0031] In one technical solution of the above method,

[0032] The training data set includes an attribute list, and the attribute list includes all attribute numbers used for training in the training data set;

[0033] Generating at least one decision tree model according to the training data set includes:

[0034] For each iteration of each decision tree model generation process, determining, according to the training data set, an information gain value corresponding to each attribute number for training in the attribute list, and determining the attribute number for training corresponding to the maximum information gain value as the first attribute number;

[0035] Determine a root node of a decision tree according to the first attribute number;

[0036] Based on the root node, determining the child nodes of the root node according to each attribute value of the first attribute number;

[0037] adjusting the attribute list according to the first attribute number, and adjusting the training data set according to the adjusted attribute list;

[0038] Taking the child node as a root node, and using the adjusted attribute list and the training data set to perform a next iteration to determine a child node of the child node;

[0039] When the iteration cutoff condition is met, the iteration is stopped to obtain the final decision tree model.

[0040] In one technical solution of the above method,

[0041] The determining, based on the root node and according to each attribute value of the first attribute number, the child nodes of the root node includes:

[0042] For each attribute value corresponding to the first attribute number, determining a target storage location image from the training data set, the target storage location image being a storage location image after the first attribute number is annotated with the attribute value;

[0043] Obtaining a first training subset according to the attribute values ​​of all attribute numbers used for training of the target storage location image;

[0044] Determine, according to the first training subset, an information gain value corresponding to each attribute number for training in the attribute list, and determine an attribute number for training corresponding to a maximum information gain value as a second attribute number;

[0045] The child node corresponding to the attribute value of the root node is determined according to the second attribute number.

[0046] In one technical solution of the above method,

[0047] When the iteration cutoff condition is met, stopping the iteration includes:

[0048] Obtaining the attribute value of the attribute number for training corresponding to the root node;

[0049] Determine whether all the annotated storage location images corresponding to the attribute value are annotated labels in an icing or frosting state or annotated labels in an unicing or frosting state;

[0050] If so, determining that the iteration cutoff condition is satisfied;

[0051] The child nodes of the root node are determined as leaf nodes, and the category labels of the leaf nodes are determined according to the label tags, and the iteration is stopped.

[0052] In one technical solution of the above method,

[0053] The method further comprises determining the attribute number and the corresponding attribute value according to the following steps, including:

[0054] Performing balanced grayscale conversion and binarization on the storage location image to obtain a binarized storage location image corresponding to the storage location image;

[0055] Based on a grid of a preset size, the preset position of the binary storage location image is evenly divided into a plurality of grid images, and each of the grid images is numbered;

[0056] Determining the attribute number according to the numbering result of the grid image;

[0057] The attribute value corresponding to the attribute number is determined according to the white pixel ratio of each of the grid images.

[0058] In a second aspect, a controller is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any one of the technical solutions of the above-mentioned icing and frosting state detection method is implemented.

[0059] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the method described in any one of the technical solutions of the above-mentioned icing and frosting state detection method.

[0060] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:

[0061] In the technical solution for implementing the icing and frosting state detection method provided by the present application, the present application obtains a real-time storage location image of the current cold storage location; based on a pre-trained icing and frosting state detection model, the detection result of the icing and frosting state corresponding to the current cold storage location is determined according to the real-time storage location image; wherein the icing and frosting state detection model includes at least one decision tree model, the decision tree model includes at least one node, the node includes at least one leaf node, and the category label of the leaf node is used to determine the detection result of the icing and frosting state of the cold storage location. Through the above configuration, the present application can realize automatic detection of whether the cold storage location is in an icing and frosting state through at least one decision tree model in the icing and frosting state detection model, thereby reducing labor costs. At the same time, the leaf nodes are determined by traversing the decision tree model according to the features in the image of the current storage location, and then the icing and frosting state corresponding to the storage location image is determined, thereby improving the intelligence level of detecting the icing and frosting state of the storage location image, thereby improving the detection accuracy and efficiency of the icing and frosting state, and avoiding the problem of damage to the robotic arm or falling of items caused by the collision of the robotic arm with condensed frost and ice when taking objects in the cold storage location. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The disclosure of the present application will become easier to understand with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. Among them:

[0063] Figure 1 It is a schematic flow chart of the main steps of a method for detecting an icing and frosting state according to an embodiment of the present application;

[0064] Figure 2 This is a schematic diagram of a cold storage location in an embodiment of the present application;

[0065] Figure 3 is a schematic diagram of a mechanical arm for picking up and placing objects in an embodiment of the present application;

[0066] Figure 4 is a schematic diagram of the relative positions of the installation method of the mechanical arm on the column in the embodiment of the present application;

[0067] Figure 5 is a detailed structural diagram of the mechanical arm in the embodiment of the present application;

[0068] Figure 6 is a schematic diagram of an actual camera of an industrial camera in an embodiment of the present application;

[0069] Figure 7 is a schematic diagram of a storage location image containing a label in an unfrozen or frosted state after being marked in an embodiment of the present application;

[0070] Figure 8is a schematic diagram of a storage location image with a label indicating an ice and frosting state after being marked in an embodiment of the present application;

[0071] Fig. 9 It is a schematic diagram of a binarized storage location image after being equally divided into grids of a preset size in an embodiment of the present application;

[0072] Fig.10 is a schematic diagram of a binary storage location image after selecting a preset position in an embodiment of the present application;

[0073] Fig.11 is a schematic diagram of a binary storage location image in which each grid image is numbered in an embodiment of the present application;

[0074] Fig.12 is a schematic diagram of the main structure of a controller according to an embodiment of the present application;

[0075] Reference numerals:

[0076] 01: cold storage location; 02: robotic arm; 03: column; 04: industrial camera; 05: light source; 06 shovel plate; 11: memory; 12: processor. DETAILED DESCRIPTION

[0077] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0078] In the description of the present application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, memory, and may also include software parts, such as program code, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B or A and B. The singular terms "one" and "the" may also include plural forms.

[0079] See attached Figure 1 , Figure 1FIG. 1 is a schematic flow chart of the main steps of a method for detecting icing and frosting conditions according to an embodiment of the present application. Figure 1 As shown, the icing and frosting state detection method in the embodiment of the present application mainly includes the following steps S101 to S102.

[0080] Step S101: Acquire a real-time location image of the current cold storage location.

[0081] In this embodiment, the cold storage may refer to a warehouse set to ultra-low temperature for storing items, wherein ultra-low temperature generally refers to about minus 80 degrees Celsius. A cold storage location refers to a storage area in a cold storage, which is generally used to store items. In the prior art, cold storage generally uses an automated robotic arm to take and place items in the location.

[0082] In an application scenario according to an embodiment of the present application, the storage location in the cold storage can refer to the attached Figure 2 , Figure 2 This is a schematic diagram of a cold storage location in an embodiment of the present application. Figure 3 As shown, attached Figure 3 It is a schematic diagram of a robotic arm for picking up and placing objects in an embodiment of the present application.

[0083] In this embodiment, please refer to the attached Figure 4 , Figure 4 Schematic diagram of the relative position of the installation method of the mechanical arm on the column in the embodiment of the present application. Figure 4 As shown in the figure, the robot arm is installed on the column in front of the cold storage location. Through relative movement with the column, it moves to the position in front of the corresponding storage location and completes the picking and placing of items by controlling the shovel plate.

[0084] In one implementation, a light source and an industrial camera may be installed on a robotic arm, the light source may be used to provide appropriate lighting conditions for the industrial camera, and the industrial camera may be used to capture a real-time location image of the current cold storage location.

[0085] In this embodiment, please refer to the attached Figure 5 , attached Figure 5 Detailed structural diagram of the mechanical arm in the embodiment of the present application. Figure 5 As shown, the mechanical arm may include a shovel plate for picking up and placing items, a light source installed under the shovel plate, and an industrial camera. The light source may be installed on both sides of the industrial camera to provide lighting conditions for the industrial camera.

[0086] In some embodiments, the industrial camera can refer to the attached Figure 6 , attached Figure 6The industrial camera in the embodiment of the present application can be a 10-megapixel black-and-white camera, and the light source can be a surface light with an intensity of 6000-8000 candela per square meter (cd / m 2 ).

[0087] In one embodiment, when the robotic arm picks up and places items, a storage location image of the current cold storage location can be taken in real time to detect the ice and frost state of the current cold storage location to reduce the collision between the robotic arm and the condensed frost or ice on the storage location.

[0088] Step S102: Based on the pre-trained icing and frosting state detection model, the detection result of the icing and frosting state corresponding to the current cold storage location is determined according to the real-time storage location image.

[0089] In this embodiment, the icing and frosting state detection model includes at least one decision tree model, the decision tree model includes at least one node, the node includes at least one leaf node, and the category label of the leaf node is used to determine the detection result of the icing and frosting state of the cold storage location.

[0090] In one implementation, the real-time storage location images may be processed first to obtain a storage location image data set to be tested, and then based on the storage location image data set to be tested, a decision tree model in an icing and frosting state detection model may be called to obtain an icing and frosting state detection result.

[0091] In one implementation, step S102 may further include steps S1021 to S1024:

[0092] Step S1021: Determine the attribute number to be tested and the attribute value of the attribute number to be tested according to the real-time storage location image.

[0093] Step S1022: Generate a storage location image data set to be tested according to the attribute number to be tested and the attribute value corresponding to the attribute number to be tested.

[0094] Step S1023: Based on the storage location image dataset to be tested, call at least one decision tree model to obtain a category label of at least one target leaf node.

[0095] Step S1024: Determine the detection result of the icing and frosting state according to the category label of at least one target leaf node.

[0096] In this embodiment, the attribute number to be tested is the number of the image feature that identifies the real-time storage location image, and the attribute value is the characteristic value of the image feature. The attribute number to be tested is used to determine the attribute value to be tested corresponding to the node during the decision tree traversal process, and the attribute value of the attribute number to be tested is used to determine the node to be traversed during the decision tree traversal process.

[0097] In this embodiment, the category mark of each target leaf node may include an icing or frosting state or an unicing or frosting state. In some other embodiments, other marks may be used to represent the icing or frosting state and the unicing or frosting state, such as marking the icing or frosting state as "1" and marking the unicing or frosting state as "0".

[0098] In an application scenario according to the present embodiment, the icing and frosting state model may include three decision tree models Tree1, Tree2 and Tree3, the attribute lists corresponding to each decision tree model may be different, and the attribute lists may include the attribute numbers of all nodes of the decision tree model. Then, in step S1021, the attribute number to be tested and the attribute value of the attribute number to be tested corresponding to the real-time storage location image may be determined according to the attribute list of each decision tree model.

[0099] Then step S1024 may specifically include: selecting the category label result1 of the target leaf node obtained from the decision tree model Tree1, the category label result2 of the target leaf node obtained from the decision tree model Tree2, and the category label result3 of the target leaf node obtained from the decision tree model Tree3 according to the majority principle as the final detection result of the icing and frosting state. For example, "1" represents the icing and frosting state, and "0" represents the non-icing and frosting state. If result1 is "1", result2 is "1", and result3 is "0", "1" is selected, that is, the icing and frosting state is the final detection result of the icing and frosting state.

[0100] In one implementation, each node of the decision tree model may be a dictionary including an attribute number and a corresponding attribute value.

[0101] In this embodiment, each decision tree model can obtain the category label of the target leaf node according to the following steps, including steps S201 to S206:

[0102] Step S201: for each iteration in the traversal process of the decision tree model, based on the root node of the decision tree model, all child nodes of the root node are obtained.

[0103] Step S202: Determine the attribute number to be tested in the storage location image data set to be tested according to the attribute number corresponding to the root node.

[0104] Step S203: for each child node, based on the attribute value of the child node, determine whether the attribute value corresponding to the attribute number to be tested meets the preset condition. If so, proceed to step S205; if not, proceed to step S204.

[0105] Step S204: determine the next child node.

[0106] In this implementation, it is necessary to traverse each child node of the root node until a child node whose attribute value is equal to the attribute value corresponding to the attribute number to be tested is found.

[0107] Step S205: taking the child node as the root node and entering the next iteration of the decision tree model traversal process.

[0108] Step S206: When the child node is a leaf node, the iteration is stopped, the leaf node is used as the target leaf node, and the category label of the target leaf node is returned.

[0109] In an application scenario according to an embodiment of the present application, a decision tree model calling method result_Prediction(tree, entries, data) can be established, where tree is used to represent the trained decision tree model called, entries is used to represent the attribute list of the called decision tree model, and data is used to represent the image data set of the warehouse location to be analyzed.

[0110] The calling method of each decision tree model may include the following steps S301 to S304:

[0111] Step S301: Obtain the root node of the decision tree model tree;

[0112] Step S302: Obtain the dictionary of all child nodes of the root node to obtain the attribute values ​​of all child nodes;

[0113] Step S303: obtaining the index value of the attribute number of the root node in the attribute list entries, and determining the attribute number to be tested corresponding to the index value in the data storage location image data set to be tested;

[0114] Step S304: traverse each child node of the root node. If the attribute value of the child node is equal to the attribute value of the attribute number to be tested, use the child node as the root node to recursively call this calling method until the child node is a leaf node, and return the category label of the final leaf node.

[0115] In one implementation, the present embodiment may further include obtaining an icing and frosting state detection model according to the following steps S401 to S404:

[0116] Step S401: Obtain annotated storage location images for training.

[0117] In this embodiment, the labeled storage location image includes a label for the frozen or frosted state or a label for the unfrozen or frosted state. Figure 7 and attached Figure 8 , attached Figure 7 : is a schematic diagram of a storage location image containing a label of an unfrozen or frosted state after marking in an embodiment of the present application, Figure 8 It is a schematic diagram of a storage location image with a label indicating ice and frosting status after annotation in an embodiment of the present application.

[0118] Step S402: Determine the attribute number and the corresponding attribute value used for training according to the labeled storage location image.

[0119] In this embodiment, the attribute number used for training is the number of the image feature of the labeled storage location image, and the attribute value is the feature value of the image feature. The attribute number used for training is used to generate nodes in the decision tree generation process; the attribute value of the attribute number used for training is used to determine the position of each node in the decision tree generation process.

[0120] In one implementation, step S402 may further include steps S4021 to S4024:

[0121] Step S4021: performing balanced grayscale conversion and binarization on the storage location image to obtain a binarized storage location image corresponding to the storage location image.

[0122] In this embodiment, the binary storage location image refers to an image with only two gray levels, that is, the gray value of any pixel in the image is 0 or 255, representing black and white respectively.

[0123] Step S4022: Based on a grid of a preset size, the preset position of the binary storage location image is equally divided into a plurality of grid images, and each grid image is numbered.

[0124] In one embodiment, please refer to the attached Fig. 9 , attached Fig. 9 is a schematic diagram of a binarized storage location image after being equally divided into grids of a preset size in an embodiment of the present application. Fig. 9 As shown, the binary storage location image can be evenly divided using a grid with a preset size of 100 mm×100 mm.

[0125] In one embodiment, please refer to the attached Fig.10 , attached Fig.10 is a schematic diagram of a binary storage location image after selecting a preset position in an embodiment of the present application. Fig.10 As shown in Figure 2, frost or ice condensed on the storage location will mostly occur at the edge of the storage location, that is, Fig.10 The position marked by the rectangular box in the middle can be the edge position of the binary storage location image (i.e. Fig.10The position marked by the rectangular box in the figure is used as the preset position to obtain the averaged grid image of the edge position of the binary storage location image.

[0126] In this embodiment, please refer to the attached Fig.11 , attached Fig.11 is a schematic diagram of a binary storage location image in which each grid image is numbered in an embodiment of the present application. Fig.11 As shown, the grid images cut out from the grid can be numbered (such as Fig.11 1, 2, 3, 4, 5, 6, 7, 8, 9, 10.......) shown in the figure, and a grid image numbered 1-4000 is obtained.

[0127] Step S4023: Determine the attribute number according to the numbering result of the grid image.

[0128] In this embodiment, the number of each grid image may be used as the image feature of the storage location image, thereby determining a plurality of attribute numbers.

[0129] Step S4024: Determine the attribute value corresponding to the attribute number according to the white pixel ratio of each grid image.

[0130] In this embodiment, the white pixel ratio of each grid image is determined according to the RGB value of each pixel in each grid image. The white pixel ratio of each grid image can be determined using the following formula: white pixel ratio = number of white pixels / total number of pixels * 100.

[0131] In this implementation, the white pixel ratio of each grid image may be directly used as the attribute value of the attribute number corresponding to the grid image.

[0132] Step S403: obtaining a training data set according to the annotated labels of the annotated storage location images, the attribute numbers used for training, and the corresponding attribute values.

[0133] In an application scenario according to this embodiment, step S403 may generate a training data set as shown in Table 1. As shown in Table 1, the training data set includes multiple storage location image data, for example, one storage location image data may include all data in the same row corresponding to the storage location image data ID 1.

[0134] Table 1. Label type table of storage location image data in training dataset

[0135] Among them, "0" in the label type column is used to represent the label of "non-frozen or non-frosted state", and "1" is used to represent the label of "frozen or non-frosted state".

[0136] In one implementation, to prevent data overfitting, a random forest algorithm can be used for model training. That is, data corresponding to 400 storage location image IDs are randomly selected from the training data set three times to generate data sets dataset1, dataset2, and dataset3, and three decision tree models tree1, tree2, and tree3 are obtained by training the three generated data sets. Among them, the random forest algorithm (Random Forest Algorithm) is an ensemble learning algorithm, which belongs to the ensemble learning method of the bootstrap aggregation (Bagging) strategy.

[0137] Step S404: Based on the decision tree algorithm, at least one decision tree model is generated according to the training data set to obtain an ice and frost detection model.

[0138] In this embodiment, the training data set may include an attribute list, and the attribute list may include all attribute numbers used for training in the training data set. For example, the attribute list may be {a1, a2, a3, a4, a5, a6, ..., a4000}, where a1-a4000 are all attribute numbers, which are used to represent the number of each grid image into which the binary storage location image is divided.

[0139] In one implementation, step S404 may further include steps S4041 to S4046:

[0140] Step S4041: For each iteration of each decision tree model generation process, determine the information gain value corresponding to each attribute number used for training in the attribute list according to the training data set, and determine the attribute number used for training corresponding to the maximum information gain value as the first attribute number.

[0141] In this embodiment, the information gain value is an indicator to measure the reduction of information brought by a certain feature when dividing a data set, and is often used in decision tree algorithms to select split points so as to divide the data set into purer subsets.

[0142] Step S4042: Determine the root node of the decision tree according to the first attribute number.

[0143] In this implementation, the root node may be a dictionary whose attribute number is the first attribute number and whose attribute value is 0.

[0144] Step S4043: Based on the root node, determine the child nodes of the root node according to each attribute value of the first attribute number.

[0145] Step S4044: adjusting the attribute list according to the first attribute number, and adjusting the training data set according to the adjusted attribute list.

[0146] In this embodiment, step S4044 may specifically include: removing the first attribute number from the attribute list, and according to the attribute list with the first attribute number removed, removing all attribute values ​​corresponding to the first attribute number in the training data set to ensure that each attribute number is used only once.

[0147] Step S4045: Take the child node as the root node, and use the adjusted attribute list and training data set to perform the next iteration to determine the child node of the child node.

[0148] Step S4046: When the iteration cutoff condition is met, the iteration is stopped to obtain the final decision tree model.

[0149] In one implementation, step S4043 may further include steps S40431 to S40434:

[0150] Step S40431: For each attribute value corresponding to the first attribute number, determine a target storage location image from the training data set.

[0151] In this embodiment, the target storage location image is a storage location image in which the first attribute number is annotated with the attribute value.

[0152] Step S40432: Obtain a first training subset according to the attribute values ​​of all attribute numbers used for training of the target storage location image.

[0153] Step S40433: according to the first training subset, determine the information gain value corresponding to each attribute number used for training in the attribute list, and determine the attribute number used for training corresponding to the maximum information gain value as the second attribute number.

[0154] Step S40434: Determine the child node corresponding to the attribute value of the root node according to the second attribute number.

[0155] As an example, the first attribute number can be set to a1, and the attribute values ​​corresponding to the first attribute number can include 10 and 20. First, all the storage location image data with the attribute value of attribute number a1 of 10 can be determined in the training data set, and the first training subset can be generated. The information gain values ​​of all attribute numbers except attribute number a1 in the first training subset are determined, and the attribute number aN corresponding to the maximum information gain value, that is, the second attribute number, is determined. Then, a child node of the root node can be determined as a dictionary of [aN, 10], where aN is the attribute number of the child node, and 10 is the attribute value of the child node. Repeat the above steps to obtain all child nodes of the root node.

[0156] In one implementation, step S4046 may further include steps S40461 to S40464:

[0157] Step S40461: Obtain the attribute value of the attribute number used for training corresponding to the root node.

[0158] Step S40462: Determine whether the annotation labels of all annotated storage location images corresponding to the attribute values ​​are all annotation labels of frozen or frosted states or annotation labels of non-frozen or non-frosted states; if so, proceed to step S40463; if not, the iteration cutoff condition is not met.

[0159] Step S40463: Determine whether the iteration cutoff condition is satisfied.

[0160] Step S40464: determine the child nodes of the root node as leaf nodes, determine the category labels of the leaf nodes according to the annotation labels, and stop iteration.

[0161] In an application scenario according to an embodiment of the present application, a decision tree generation function may be created to create a decision tree model. The decision tree generation function may include the following steps:

[0162] Step S501: Determine the attribute number ai that is most suitable for splitting according to the attribute number ai corresponding to the maximum information gain value in the attribute list in the training data set.

[0163] Step S502: Create a dictionary dic[ <ai>, <0> ] as the root node of the tree, where the key of the dictionary is the attribute number ai and the value is 0.

[0164] Step S503: Delete the attribute number ai from the attribute list.

[0165] Step S504: Obtain all attribute values ​​corresponding to the attribute number ai from the training data set to generate a data set temp_Dataset.

[0166] Step S505: perform a deduplication operation on the data set temp_Dataset to obtain a data set unique_temp_Dataset.

[0167] Step S506: Obtain the annotation label of each storage location image corresponding to the attribute number ai.

[0168] Step S507: Count the most frequently appearing annotation tags and their times. If the annotation tags are not of the same category, proceed to step S508. If they are of the same category, proceed to step S509.

[0169] Step S508: For each value of the data set unique_temp_Dataset, recursively call the decision tree generation function.

[0170] Step S509: Generate a leaf node as a child node of the node corresponding to the attribute number ai, and use the label with the largest number of occurrences as the classification label of the leaf node.

[0171] In this embodiment, if the decision tree generation function is generate_tree(dataSet, entries, used_entries), where dataSet is the input training data set, entries is used to represent the attribute list, and used_entries is used to represent the used attribute number in the attribute list. Then the decision tree generation function is recursively called, and the following recursive formula can be used to return the dictionary of all tree nodes:

[0172] dic[ ， <value>]=generate_tree(splitDataSet(dataSet, i, value), entries, used_entries)

[0173] The splitDataSet(dataSet, i, value) function is used to return the attribute values ​​corresponding to other attribute numbers corresponding to the storage location image with attribute number i and attribute value value in the dataSet data set. In this embodiment, dataSet is a training data set.

[0174] In this embodiment, the three data sets dataset1, dataset2, and dataset3 can be respectively trained by calling the decision tree generation function to generate decision tree models tree1, tree2, and tree3.

[0175] In one implementation, step S4041 may further include the following steps S40411 to S40413 to determine the information gain value corresponding to each attribute number in the training data set:

[0176] Step S40411: For each attribute number, obtain all attribute values ​​corresponding to the attribute number in the training data set.

[0177] Step S40412: After deduplication is performed on all attribute values, for each attribute value, the attribute values ​​of other attribute numbers of the binary storage location image corresponding to each attribute value in the training data set are obtained to obtain other attribute value data sets.

[0178] Step S40413: determine the conditional probability of other attribute value data sets with respect to the training data set, and determine the information gain value of the attribute number according to the conditional probability.

[0179] In an application scenario according to an embodiment of the present application, the information gain value corresponding to each attribute number in the training data set can be trained by the following steps:

[0180] Step S601: Get each attribute number in the current traversal attribute list A={a1, a2, a3, a4, a5, ...}, assuming that the current attribute number is a1.

[0181] Step S602: fetch all attribute values ​​corresponding to the current attribute number a1 to generate a data subset subdataset.

[0182] Step S603: remove duplicate values ​​in the data subset subdataset to obtain the data subset unique_subdataset.

[0183] Step S604: loop through each attribute value of the data subset unique_subdataset, assuming that the current attribute value is value.

[0184] Step S605: Create an empty list ret_dataset.

[0185] Step S606: store the attribute value of the attribute number a1 in the training data set, which is equal to the attribute value of other attribute numbers in the storage location image data of value, in the list ret_dataset.

[0186] In this embodiment, a splitDataSet(dataSet, entityIndex, value) function can be constructed to return the attribute value of the attribute number entityIndex in the dataSet data set, which is equal to the attribute value of other attribute numbers in the storage location image data of value. For example, when the data set dataSet is {[11,22],[10,41],[11,9]}, after calling the splitDataSet(dataSet,0,11) function, since the data with a value of 11 on the 0th attribute column is [11,22] and [11,9], after removing the value on the 0th attribute column, the remaining data is

[22] and [9], so the data [

[22] , [9]] will be returned.

[0187] In this implementation, when dataSet is a training data set, the splitDataSet(dataSet, a1, value) function may be called to implement step S606.

[0188] Step S607: until all attribute values ​​of the data subset unique_subdataset are traversed, the conditional probability prob_ret is calculated according to the number of attribute values ​​in the list ret_dataset.

[0189] In this embodiment, the conditional probability prob_ret can be calculated according to the following formula:

[0190] prob_ret=total amount of attribute value data in ret_dataset / total amount of attribute value data in training data set.

[0191] Step 608: Calculate the conditional entropy new_shannon according to the conditional probability prob_ret.

[0192] In this implementation, the conditional entropy new_shannon can be calculated according to the following formula:

[0193] new_shannon=prob_ret*get_Shonnon(ret_dataset)

[0194] Among them, get_Shonnon(ret_dataset) is used to calculate the Shannon entropy of the list ret_dataset.

[0195] Step S609: Determine the information gain value info_gain corresponding to the attribute number a1 according to the conditional entropy new_shannon.

[0196] In this implementation, the information gain value info_gain may be calculated according to the following formula:

[0197] info_gain=get_Shonnon(dataset1)-new_shannon

[0198] Step S610: until each attribute number in the attribute list A={a1, a2, a3, a4, a5, ...} is traversed, and the information gain value corresponding to each attribute number is obtained.

[0199] In an application scenario according to an embodiment of the present application, the splitDataSet(dataSet, entityIndex, value) function in step S606 may further include steps S701 to S704:

[0200] Step S701: traverse each storage location image data in the input data set dataSet, and determine whether the attribute value of the attribute number entityIndex in each storage location image data is equal to value. If yes, proceed to step S702.

[0201] Step S702: Create a new list reduced_dataset for storing other data except the values ​​on the specified attribute column.

[0202] Step S703: Use the slicing operation to obtain other attribute values ​​in the current storage location image data.

[0203] In this embodiment, the slice operation data[:entityIndex] can be used to obtain the attribute values ​​of other attribute numbers before the attribute number entityIndex in the current storage location image data, and add them to the list reduced_dataset. Then the slice operation data[ientityIndex+1:] is used to obtain the attribute values ​​of other attribute numbers after the attribute number entityIndex in the current storage location image data, and also add them to the list reduced_dataset, and then the list reduced_dataset is added to the list ret_dataset.

[0204] Step S704: After the traversal is completed, the final list ret_dataset is obtained

[0205] In one implementation, the get_Shonnon(dataset) function in step S608 and step D609, assuming that dataset is a data set input to the function, may further include steps S801 to S802:

[0206] Step S801: Traverse all the annotation labels in the dataset and the number of occurrences of each type of annotation label, and store them in the dictionary endityDic1.

[0207] In this embodiment, the marking labels include marking labels for iced and frosted states and marking labels for non-iced and non-frosted states.

[0208] In this implementation, the Shannon entropy may be initialized to 0 first.

[0209] Step S802: traverse each annotation tag in the dictionary endityDic1, calculate the probability of each annotation tag appearing, and calculate the Shannon entropy shannon according to the probability.

[0210] In this implementation, the Shannon entropy can be calculated according to the following formula:

[0211] H(x)=-∑(p(x)×log2(p(x)))

[0212] Among them, H(X) is used to represent the Shannon entropy of the dataset dataset, x is used to represent the annotation label, which may include the annotation label of ice and frost and the annotation label of non-ice and frost, and p(x) is used to represent the probability of the appearance of the annotation label x in all the storage location image data in the dataset dataset.

[0213] Based on the method described in the above steps S101 to S102, the present application can realize the automatic detection of whether the cold storage location is in the state of ice and frosting through at least one decision tree model in the ice and frosting state detection model, and use machine judgment to replace manual judgment, saving a lot of manpower and reducing labor costs. At the same time, according to the features in the image of the current storage location, the decision tree model is traversed to determine the leaf nodes, and then the ice and frosting state corresponding to the storage location image is determined, which improves the intelligence level of detecting the ice and frosting state of the storage location image, quantifies the manual experience, and uses the decision tree algorithm to automatically obtain the feature points of the storage location image, thereby improving the detection accuracy and efficiency of the ice and frosting state, and avoiding the collision of the robotic arm with the condensed frost and ice when taking objects from the cold storage location, which may cause the robotic arm to be damaged or the items to fall.

[0214] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in this application, and therefore will also fall within the scope of protection of this application.

[0215] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0216] Another aspect of the present application also provides a computer-readable storage medium.

[0217] In an embodiment of a computer-readable storage medium according to the present application, the computer-readable storage medium may be configured to store a program for executing the icing and frosting state detection method of the above method embodiment, and the program may be loaded and run by a processor to implement the above icing and frosting state detection method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

[0218] Another aspect of the present application also provides a controller.

[0219] In an embodiment of a controller according to the present application, the controller may include at least one processor; and a memory connected to the at least one processor in communication; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method for detecting the icing and frosting state described in any of the above embodiments is implemented. The controller described in the present application may include but is not limited to mobile phones, tablet computers, desktop computers, laptops, handheld computers, notebook computers, vehicle-mounted devices, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), augmented reality (AR) and virtual reality (VR) devices, etc., and the embodiments of the present application are not limited to this. Please refer to the attached Fig.12 , Fig.12 FIG. 4 exemplarily shows that the memory 11 and the processor 12 are communicatively connected via a bus.

[0220] So far, the technical solution of the present application has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.< / value> < / ai>

Claims

1. A method for detecting icing and frosting conditions, characterized in that: The method comprises: Get the real-time image of the current cold storage location; Based on a pre-trained icing and frosting state detection model, a detection result of the icing and frosting state corresponding to the current cold storage location is determined according to the real-time storage location image; Among them, the icing and frosting state detection model includes at least one decision tree model, the decision tree model includes at least one node, the node includes at least one leaf node, and the category label of the leaf node is used to determine the detection result of the icing and frosting state of the cold storage location.

2. The icing and frosting state detection method according to claim 1, characterized in that: The step of determining the icing and frosting state corresponding to the current cold storage location according to the real-time storage location image includes: According to the real-time storage location image, determine the attribute number to be tested and the attribute value of the attribute number to be tested, wherein the attribute number to be tested is the number identifying the image feature of the real-time storage location image, and the attribute value is the feature value of the image feature; wherein the attribute number to be tested is used to determine the attribute value to be tested corresponding to the node during the decision tree traversal process; and the attribute value of the attribute number to be tested is used to determine the node to be traversed during the decision tree traversal process; Generate a storage location image data set to be tested according to the attribute number to be tested and the attribute value corresponding to the attribute number to be tested; Based on the image data set of the storage location to be tested, calling at least one of the decision tree models to obtain a category label of at least one target leaf node, wherein the category label includes an icing or frosting state or a non-icing or frosting state; The detection result of the icing and frosting state is determined according to the category label of the at least one target leaf node.

3. The icing and frosting state detection method according to claim 2, characterized in that: Each of the nodes of the decision tree model is a dictionary containing an attribute number and a corresponding attribute value; Each of the decision tree models obtains a category label of a target leaf node according to the following steps, including: For each iteration in the traversal process of the decision tree model, based on the root node of the decision tree model, all child nodes of the root node are obtained; According to the attribute number corresponding to the root node, determining the attribute number to be tested in the storage location image data set to be tested; For each of the child nodes, based on the attribute value of the child node, determining whether the attribute value corresponding to the attribute number to be tested meets a preset condition; If not, proceed to the next child node judgment; If so, taking the child node as the root node and entering the next iteration of the decision tree model traversal process; When the child node is a leaf node, the iteration is stopped, the leaf node is used as the target leaf node, and the category label of the target leaf node is returned.

4. The icing and frosting state detection method according to claim 1, characterized in that: The method further comprises obtaining the icing and frosting state detection model according to the following steps: Acquire an annotated storage location image for training, wherein the annotated storage location image includes an annotated label of an icing or frosting state or an annotated label of an unicing or frosting state; According to the annotated storage location image, determine the attribute number and the corresponding attribute value for training, wherein the attribute number for training is a number that identifies the image feature of the annotated storage location image, and the attribute value is a feature value of the image feature; wherein the attribute number for training is used to generate a node in the decision tree generation process; and the attribute value of the attribute number for training is used to determine the position of each node in the decision tree generation process; Obtaining a training data set according to the annotated labels of the annotated storage location images and the attribute numbers and corresponding attribute values ​​used for training; Based on a decision tree algorithm, at least one decision tree model is generated according to the training data set to obtain the icing and frost detection model.

5. The icing and frosting state detection method according to claim 4, characterized in that: The training data set includes an attribute list, and the attribute list includes all attribute numbers used for training in the training data set; Generating at least one decision tree model according to the training data set includes: For each iteration of each decision tree model generation process, determining, according to the training data set, an information gain value corresponding to each attribute number for training in the attribute list, and determining the attribute number for training corresponding to the maximum information gain value as the first attribute number; Determine a root node of a decision tree according to the first attribute number; Based on the root node, determining the child nodes of the root node according to each attribute value of the first attribute number; adjusting the attribute list according to the first attribute number, and adjusting the training data set according to the adjusted attribute list; Taking the child node as a root node, and using the adjusted attribute list and the training data set to perform a next iteration to determine a child node of the child node; When the iteration cutoff condition is met, the iteration is stopped to obtain the final decision tree model.

6. The icing and frosting state detection method according to claim 5, characterized in that: The determining, based on the root node and according to each attribute value of the first attribute number, the child nodes of the root node includes: For each attribute value corresponding to the first attribute number, determining a target storage location image from the training data set, the target storage location image being a storage location image after the first attribute number is annotated with the attribute value; Obtaining a first training subset according to the attribute values ​​of all attribute numbers used for training of the target storage location image; Determine, according to the first training subset, an information gain value corresponding to each attribute number for training in the attribute list, and determine an attribute number for training corresponding to a maximum information gain value as a second attribute number; The child node corresponding to the attribute value of the root node is determined according to the second attribute number.

7. The icing and frosting state detection method according to claim 5, characterized in that: When the iteration cutoff condition is met, stopping the iteration includes: Obtaining the attribute value of the attribute number for training corresponding to the root node; Determine whether all the annotated storage location images corresponding to the attribute value are annotated labels in an icing or frosting state or annotated labels in an unicing or frosting state; If so, determining that the iteration cutoff condition is satisfied; The child nodes of the root node are determined as leaf nodes, and the category labels of the leaf nodes are determined according to the label tags, and the iteration is stopped.

8. The icing and frosting state detection method according to claim 2 or 4, characterized in that: The method further comprises determining the attribute number and the corresponding attribute value according to the following steps, including: Performing balanced grayscale conversion and binarization on the storage location image to obtain a binarized storage location image corresponding to the storage location image; Based on a grid of a preset size, the preset position of the binary storage location image is evenly divided into a plurality of grid images, and each of the grid images is numbered; Determining the attribute number according to the numbering result of the grid image; The attribute value corresponding to the attribute number is determined according to the white pixel ratio of each of the grid images.

9. A controller, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; Wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the icing and frosting state detection method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the icing and frosting state detection method according to any one of claims 1 to 8.

Images