Method and device for detecting border crossing of goods stored in storage location

Abstract

The invention provides a storage location stored goods border crossing detection method and device, and relates to the field of goods border crossing detection, and the method comprises the steps: obtaining a goods image of a storage location area, and the goods image comprises a goods grid line; identifying cargo grid lines in the cargo image, marking the cargo grid lines in the cargo image through a first pixel value, marking a non-cargo grid line area in the cargo image through a second pixel value, and obtaining a first image; and comparing the pixel value of each position on the cargo grid line in the first image with the pixel value of the corresponding position on the cargo grid line in the second image, if the pixel values of the corresponding positions are all the same, determining that the condition that the cargo crosses the boundary does not exist, and if the pixel values of the corresponding positions are different, determining that the condition that the cargo crosses the boundary exists. By adopting the cargo boundary crossing detection method provided by the invention, the defects of a traditional target detection algorithm can be avoided, the situation of misjudgment is reduced, and the accuracy of cargo boundary crossing detection is improved.

Description

Technical Field

[0001] The present application relates to the technical field of cargo cross-border detection, and in particular to a method and device for detecting cross-border storage of cargo. Background Art

[0002] At present, most of the storage location storage goods out-of-bounds detection methods based on surveillance cameras are implemented using traditional image segmentation algorithms and target detection algorithms. Among them, the image segmentation algorithm is used to segment the image in which the goods are stored. The traditional target detection algorithm detects the goods in the segmented image, marks the goods with a detection frame, and then compares the detection frame with the goods grid line to determine whether the goods have crossed the boundary.

[0003] Traditional object detection algorithms require that the marked detection frame is highly fitted to the edge of the goods. If the detection frame is marked too large and does not approach the edge of the goods, it is easy to cause misjudgment when comparing the detection frame with the grid line of the goods. Summary of the invention

[0004] The present application provides a method and device for detecting cross-boundary storage of goods in a warehouse. The method for detecting cross-boundary storage of goods described in the present application can avoid the drawbacks of traditional target detection algorithms, reduce misjudgments, and improve the accuracy of cross-boundary storage of goods.

[0005] In a first aspect, the present application provides a method for detecting cross-boundary storage of goods at a storage location, comprising:

[0006] Acquire a cargo image of a storage area, wherein the cargo image includes cargo grid lines;

[0007] Identify the cargo grid lines in the cargo image, and mark the cargo grid lines in the cargo image by a first pixel value, and mark the non-cargo grid line area in the cargo image by a second pixel value, to obtain a first image;

[0008] The pixel values ​​at each position on the cargo grid line in the first image are compared with the pixel values ​​at the corresponding positions on the cargo grid line in the second image. If the pixel values ​​at the corresponding positions are all the same, it is determined that there is no cargo crossing the boundary. If the pixel values ​​at the corresponding positions are different, it is determined that there is cargo crossing the boundary. The second image is an image of the storage area without cargo stored therein, and the cargo grid line in the second image is marked by the first pixel value.

[0009] The present application marks the cargo grid lines in the cargo image so that the pixel values ​​of the cargo grid lines are different from the pixel values ​​of the non-cargo grid lines, and then compares the pixel values ​​of the cargo grid lines in the cargo image with the pixel values ​​of the cargo grid lines in the second image to determine whether there is any cargo out of bounds in the storage area, wherein the second image is an image when no cargo is stored in the storage area and the cargo grid lines in the second image are also marked by the first pixel values. The method of the present application does not need to mark a detection frame for the cargo, which can avoid the drawbacks of the traditional target detection algorithm, reduce the occurrence of misjudgments, and improve the accuracy of cargo out-of-bounds detection.

[0010] Based on the first aspect, in a possible implementation, the cargo image is input into a neural network model to obtain the first image, the cargo grid lines in the first image are marked by the first pixel values, and the non-cargo grid line areas in the first image are marked by the second pixel values; wherein the neural network model includes an encoder and a decoder, the encoder includes a deep separable convolution module and a compact split attention CSA module, the deep separable convolution module is used to extract spatial dimension features and channel dimension features in the cargo image, and the decoder includes the compact split attention CSA module, and the compact split attention CSA module is used to extract the interaction information between the spatial dimension features and the channel dimension features of the feature map input into the compact split attention CSA module.

[0011] The deep separable convolution module greatly reduces the number of parameters in the neural network model by splitting and extracting the spatial dimension features and the channel dimension features, making the neural network more lightweight. Compared with the traditional attention module, for example, the SE (squeeze excitation) attention module only pays attention to the channel information, and the CBAM (convolutional block attention module) attention module pays attention to both channel features and spatial features, but pays attention to the channel features first and then the spatial features, and there is no interactive fusion of channel features and spatial features. In this application scheme, the CSA module helps to extract the interactive information in the channel dimension and the spatial dimension, and obtain more detailed and global features.

[0012] Based on the first aspect, in a possible implementation, the encoder includes multiple depth-wise separable convolution modules, and the multiple depth-wise separable convolution modules use jump connections.

[0013] Multiple depth-wise separable convolutional modules use skip connections to extract the features of the input image as comprehensively as possible.

[0014] Based on the first aspect, in a possible implementation, the compact split attention CSA module extracts features of the feature map input into the compact split attention CSA module through multiple paths based on the attention mechanism, and the multiple paths use different convolution operations on multiple different channel dimensions.

[0015] By grouping channels and using different convolution kernel sizes in multiple branches to extract features, more detailed features can be extracted.

[0016] Based on the first aspect, in a possible implementation, the compact split attention CSA module performs a convolution operation on the image input into the compact split attention CSA module to obtain a first feature map;

[0017] Splitting the first feature map into two feature maps according to the channel dimension, and obtaining a second feature map on the first channel dimension and a third feature map on the second channel dimension respectively;

[0018] First path: performing a convolution operation on the second feature map to obtain a fourth feature map on the first channel dimension;

[0019] The second path: performing a convolution operation on the third feature map, then summing the feature map obtained after the convolution with the fourth feature map on the first channel dimension and performing a convolution operation again to obtain a fifth feature map on the second channel dimension;

[0020] Calculate the attention weight parameters corresponding to the fourth feature map on the first channel dimension and the fifth feature map on the second channel dimension respectively;

[0021] According to the corresponding attention weight parameter, performing weighted summation on the fourth feature map and the fifth feature map to obtain a sixth feature map;

[0022] After performing a convolution operation on the sixth feature map, the convolution operation is performed and the image input into the compact split attention CSA module is summed to obtain an output feature map.

[0023] First, the feature map is divided into two groups according to the channel dimension, and features are extracted in two channel dimensions to obtain feature maps of two channel dimensions. Then, the attention weight parameters corresponding to the feature maps of the two channel dimensions are calculated, which helps to extract the interaction information in the channel dimension and the spatial dimension and obtain more detailed and global features.

[0024] Based on the first aspect, in a possible implementation, the encoder includes multiple feature extraction layers, a first feature extraction layer among the multiple feature extraction layers includes the multiple depth-separable convolution modules and a pooling layer, a last feature extraction layer among the multiple feature extraction layers includes the compact split attention CSA module, and other feature extraction layers among the multiple feature extraction layers include the compact split attention CSA module and the pooling layer;

[0025] The decoder includes at least multiple feature restoration layers, each of the multiple feature restoration layers includes an upsampling module, a connection module and the compact split attention CSA module, and the connection module is used to respectively connect the multiple feature extraction layers in the encoder with the multiple feature restoration layers in the decoder.

[0026] Based on the first aspect, in a possible implementation, the loss function of the neural network model is a weighted sum of a first function Dice Loss and a second function Focal Loss.

[0027] Using the weighted sum of the first function Dice Loss and the second function Focal Loss as the loss function of the neural network model can make the neural network model converge faster. Combining the Focal Loss function with the Dice Loss function as the total loss function can overcome the disadvantage of Dice Loss' inaccurate prediction of small targets, making the detection results of the neural network model more accurate.

[0028] In a second aspect, the present application provides a device for detecting a storage location for goods crossing a boundary, comprising:

[0029] An acquisition module, used to acquire a cargo image of a storage area, wherein the cargo image includes cargo grid lines;

[0030] a recognition module, configured to recognize the cargo grid lines in the cargo image, mark the cargo grid lines in the cargo image by a first pixel value, and mark a non-cargo grid line area in the cargo image by a second pixel value, so as to obtain a first image;

[0031] A determination module is used to compare the pixel values ​​of each position on the cargo grid line in the first image with the pixel values ​​of the corresponding positions on the cargo grid line in the second image. If the pixel values ​​of the corresponding positions are all the same, it is determined that there is no cargo crossing the boundary; if the pixel values ​​of the corresponding positions are different, it is determined that there is cargo crossing the boundary; wherein, the second image is an image of the storage area without cargo stored, and the cargo grid line in the second image is marked by the first pixel value.

[0032] Based on the second aspect, in a possible implementation, the recognition module is used to: identify the cargo grid lines in the cargo image through a neural network model to obtain the first image, the cargo grid lines in the first image are marked by the first pixel values, and the non-cargo grid line areas in the first image are marked by the second pixel values; wherein the neural network model includes an encoder and a decoder, the encoder includes a deep separable convolution module and a compact split attention CSA module, the deep separable convolution module is used to extract spatial dimension features and channel dimension features in the cargo image, and the decoder includes the compact split attention CSA module, and the compact split attention CSA module is used to extract the interactive information between the spatial dimension features and the channel dimension features of the feature map input into the compact split attention CSA module.

[0033] Based on the second aspect, in a possible implementation, the encoder includes multiple depth-wise separable convolution modules, and the multiple depth-wise separable convolution modules use jump connections.

[0034] Based on the second aspect, in a possible implementation, the compact split attention CSA module extracts features of the feature map input into the compact split attention CSA module through multiple paths based on the attention mechanism, and the multiple paths use different convolution operations on multiple different channel dimensions.

[0035] Based on the second aspect, in a possible implementation, the identification module is used to:

[0036] Performing a convolution operation on the image input into the compact split attention CSA module through the compact split attention CSA module to obtain a first feature map;

[0037] The first feature map is split into two feature maps according to the channel dimension by the compact split attention CSA module, and a second feature map on the first channel dimension and a third feature map on the second channel dimension are obtained respectively;

[0038] First path: performing a convolution operation on the second feature map through the compact split attention CSA module to obtain a fourth feature map on the first channel dimension;

[0039] The second path: performing a convolution operation on the third feature map through the compact split attention CSA module, and then summing the feature map obtained after the convolution with the fourth feature map on the first channel dimension and performing a convolution operation again to obtain a fifth feature map on the second channel dimension;

[0040] The attention weight parameters corresponding to the fourth feature map on the first channel dimension and the fifth feature map on the second channel dimension are calculated respectively by the compact split attention CSA module;

[0041] Performing weighted summation on the fourth feature map and the fifth feature map according to the corresponding attention weight parameter through the compact split attention CSA module to obtain a sixth feature map;

[0042] The compact split attention CSA module performs a convolution operation on the sixth feature map and then sums it with the image input to the compact split attention CSA module to obtain an output feature map.

[0043] Based on the second aspect, in a possible implementation, the encoder includes multiple feature extraction layers, a first feature extraction layer among the multiple feature extraction layers includes the multiple depth-separable convolution modules and a pooling layer, a last feature extraction layer among the multiple feature extraction layers includes the compact split attention CSA module, and other feature extraction layers among the multiple feature extraction layers include the compact split attention CSA module and the pooling layer;

[0044] The decoder includes at least multiple feature restoration layers, each of the multiple feature restoration layers includes an upsampling module, a connection module and the compact split attention CSA module, and the connection module is used to respectively connect the multiple feature extraction layers in the encoder with the multiple feature restoration layers in the decoder.

[0045] Based on the second aspect, in a possible implementation, the loss function of the neural network model is a weighted sum of a first function Dice Loss and a second function Focal Loss.

[0046] Each functional module in the second aspect is used to implement the method described in the first aspect and any possible implementation method of the first aspect.

[0047] In a third aspect, an embodiment of the present application provides a server, comprising a memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the method described in the above-mentioned first aspect and any possible implementation method of the first aspect.

[0048] In a fourth aspect, an embodiment of the present application provides a computer storage medium, comprising program instructions, which, when executed by the controller, causes the controller to execute the method described in the first aspect and any possible implementation of the first aspect.

[0049] In a fifth aspect, the present application provides a computer program product, including program instructions, and when the computer program product is executed by a controller, the controller is used to execute the method described in the first aspect and any possible implementation of the first aspect. The computer program product can be a software installation package, and when it is necessary to use the method provided by any possible design of the first aspect, the computer program product can be downloaded and executed on the controller to implement the method described in the first aspect and any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0051] Figure 1 A schematic diagram of a system architecture provided for this application;

[0052] Figure 2 A flowchart of a method for detecting cross-boundary storage of goods provided in this application;

[0053] Figure 3A An example diagram provided for this application;

[0054] Figure 3B An example diagram provided for this application;

[0055] Figure 4 A schematic diagram of the structure of a neural network model provided for this application;

[0056] Figure 5 A schematic diagram of the structure of the CSA module provided for this application;

[0057] Figure 6 A is an example of an image when no goods are stored in the storage area;

[0058] Figure 6 B is for Figure 6 An example diagram of a second image obtained by identifying image A;

[0059] Figure 7 A schematic diagram of an output image provided for this application;

[0060] Figure 8 A schematic diagram of the structure of a storage location storage goods crossing boundary detection device provided in this application;

[0061] Fig. 9 A schematic diagram of the structure of a server provided for this application. DETAILED DESCRIPTION

[0062] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0063] It should be noted that the terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0064] It should be noted that when used in this specification and the appended claims, the terms "comprises" and any of their variations are intended to cover non-exclusive inclusions. For example, a system, product or device including a series of units / devices is not limited to the listed units / devices, but may optionally include units / devices not listed, or may also optionally include other units / devices inherent to these products or devices.

[0065] It will also be understood that the term “if” may be interpreted as “when” or “upon” or “in response to determining” or “in response to detecting” or “in the event of…” depending on the context.

[0066] This application provides a system, see Figure 1 , Figure 1 A schematic diagram of a system architecture provided in this application. The system involves at least one image acquisition device, a switch and a server.

[0067] The image acquisition device may be, for example, a camera, an image sensor, etc. The image acquisition device is used to acquire images or videos of the storage area, wherein the storage area is divided into cargo grid lines, and cargo grid lines may be stored in the cargo grid lines. The image acquisition device is also used to send the acquired images or videos of the storage area to the server.

[0068] The switch is used to send the images or videos collected by the image collection device to the server.

[0069] The server is used to receive the image or video sent by the image acquisition device through the switch, and process the image or video to determine whether the goods stored in the cargo grid line have crossed the boundary. If it is determined that the goods have crossed the boundary, the position of the goods can be adjusted by manual handling to store the goods in the cargo grid line, or the position of the goods can be adjusted by an automated guided vehicle (AGV) to make the goods located in the cargo grid line.

[0070] Based on the above system architecture, this application provides a method for detecting cross-border storage of goods in a warehouse, which can be applied to a server. Figure 2 , Figure 2 A flow chart of a method for detecting cross-boundary storage of goods in a storage location provided in the present application, the method includes but is not limited to the description of the following content.

[0071] S101. Acquire a cargo image of a storage area, wherein the cargo image includes cargo grid lines.

[0072] The storage area is divided into cargo grid lines, and cargo can be stored in each grid of the cargo grid lines. The server obtains a cargo image of the storage area, wherein the cargo image includes the cargo grid lines. Optionally, the server obtains the cargo image of the storage area from an image acquisition device.

[0073] S102: recognizing cargo grid lines in the cargo image, marking the cargo grid lines in the cargo image by using a first pixel value, and marking non-cargo grid line areas in the cargo image by using a second pixel value, to obtain a first image.

[0074] The cargo grid lines in the cargo image are identified, the cargo grid lines (areas) are marked by the first pixel value, and the non-cargo grid line areas are marked by the second pixel value, and the first image is obtained, and the first pixel value is different from the second pixel value. It can be understood that if the first image is displayed, the cargo grid lines (areas) in the first image are one color, and the non-cargo grid line areas are another color.

[0075] Among them, some of the cargo grid lines in the storage area may have cargo stored in them, some may not, all of them may have cargo stored in them, or none of them may have cargo stored in them. Therefore, the non-cargo grid line area in the first image includes one or more of the area where cargo is located and the area where empty grids are located.

[0076] For example, Figure 3A and Figure 3B As shown, Figure 3A and Figure 3B An example diagram provided for this application. Figure 3Ais an example of a cargo image, where cargo is stored in grids 1, 2, 4, 6, 7, and 9, and cargo is not stored in grids 3, 5, and 8. Figure 3A The cargo image shown is identified and obtained Figure 3B The example diagram shown, Figure 3B In the example, the cargo grid line area is marked by the first pixel value, which is in Figure 3B The non-cargo grid line area (including the grid area where the cargo is located and the area where the empty grid is located) is marked by the second pixel value. Figure 3B It should be noted that since the goods in grid 9 press part of the upper edge of the grid line, the upper edge of the grid line of grid 9 is not completely marked. Figure 3B Here, the numbers 1 to 9 in the grid are for the convenience of description, and the marks added in the figure may not exist in actual applications. Of course, they may exist in actual applications, and this application does not limit them.

[0077] In a specific implementation, this step can be achieved through a neural network model. The cargo image is input into the neural network model. The neural network model marks the cargo grid lines in the cargo image with a first pixel value and marks the non-cargo grid line area with a second pixel value to obtain a first image.

[0078] This application provides a neural network model, for example, see Figure 4 The structural diagram shown in the figure shows that the neural network model is based on the improvement of the classic encoder and decoder structure of the U-Net neural network. The improvements include: using a 3*3 depth-separable convolution module to replace the original convolution operation, adding a compact split attention (CSA) module before downsampling, and adding a CSA module after the upsampling jump connection. The neural network model is introduced below.

[0079] The neural network model adopts an encoder structure (left branch) and a decoder structure (right branch). The encoder is used to gradually extract the features of the input image by downsampling, gradually reduce the size of the feature map, and reduce the resolution. The decoder is used to gradually restore the size of the feature map by upsampling to improve the resolution. In the encoder part, three 3*3 depthwise separable convolution modules (i.e., ConvBlock in the figure) are first used. The depthwise separable convolution module is used to split the spatial dimension and channel dimension of the input image and extract low-level semantic information from the input image. The depthwise separable convolution module adopts a depthwise separable convolution operation. Its structure is shown in the dotted box. First, a 3*3 depthwise convolution (DWise) is performed, then activated by the activation function ReLU, and then normalized (batch normal, BN) to obtain the spatial features of the input image; then a 1*1 convolution operation is used to extract the features of one channel, and then activated by the activation function ReLU, and then normalized BN to obtain the channel features of the input image.

[0080] Three 3*3 depth-separable convolutional modules use skip connections, such as Figure 4 As shown in , the feature map output by the first depth-wise separable convolution module is input into the second depth-wise separable convolution module, and then the feature map output by the second depth-wise separable convolution module is combined with the feature map output by the first depth-wise separable convolution module and input into the third depth-wise separable convolution module. The third depth-wise separable convolution module is used to extract features from the feature map obtained after the combination.

[0081] In the present application, a deep separable convolution module is used to replace the original convolution operation. The deep separable convolution module greatly reduces the number of parameters in the neural network model by splitting and extracting the spatial dimension features and the channel dimension features, making the neural network more lightweight. For example, for a color (RGB) image, the spatial dimension is 3*3, and there are three channels of R, G, and B. When the original convolution operation is used to process the image, the spatial dimension features and the channel dimension features are extracted together, and 3*3*3=27 parameters need to be calculated; the deep separable convolution module of the present application is used to split and extract the spatial dimension features and the channel dimension features, and 3*3*1+1*1*3=12 parameters need to be calculated, where 3*3*1 means spatial processing of 1 channel, and 1*1*3 means 1*1 spatial processing of 3 channels. Therefore, the deep separable convolution module of the present application can reduce the number of parameters. In addition, the first depth-wise separable convolution module and the second depth-wise separable convolution module adopt the idea of ​​residual block short connection and are connected by skip connection, which can extract the features of the input image as comprehensively as possible.

[0082] After the three 3*3 depth-wise separable convolution modules, the maximum pooling layer MaxPooling is connected. MaxPooling is used to downsample the feature map output by the third depth-wise separable convolution module.

[0083] The CSA module (Block) extracts the features of the feature map of the input CSA module through multiple paths based on the attention mechanism, where multiple paths use different convolution operations on multiple different channel dimensions. Each CSA module is followed by a 2*2 maximum pooling layer MaxPooling with a stride of 2. Each CSA and each MaxPooling is used to perform downsampling operations to extract higher semantic information, while reducing the size of the feature map and reducing the resolution. The encoder includes multiple downsampling operations to gradually extract the features of the input image. Each time a downsampling operation is performed, the feature map is reduced by half, such as Figure 4 In the figure, the feature map size output by the third depth-wise separable convolution module is 1 / 2 of the original image, the feature map size output by the first CSA module after extracting features is 1 / 4 of the original image, the feature map size output by the second CSA module after extracting features is 1 / 8 of the original image, the feature map size output by the third CSA module after extracting features is 1 / 16 of the original image, and the feature map size output by the fourth CSA module after extracting features is 1 / 32 of the original image.

[0084] The decoder part includes multiple feature restoration layers, each of which includes an upsampling module, a connection module and a CSA module. The decoder gradually restores the size of the image through upsampling operations. Among them, the connection module is used to connect multiple feature extraction layers in the encoder with multiple feature restoration layers in the decoder respectively, such as Figure 4 As shown in . Among them, the multiple feature extraction layers in the encoder include a first feature extraction layer, a second feature extraction layer, a third feature extraction layer, a fourth feature extraction layer, and a fifth feature extraction layer. The first feature extraction layer includes three 3*3 depth-separable convolution modules and a maximum pooling layer. The second feature extraction layer, the third feature extraction layer, and the fourth feature extraction layer all include a CSA module and a maximum pooling layer. The fifth feature extraction layer includes a CSA module.

[0085] Specific connection such as Figure 4As shown, the connection module connects the feature map output by the upsampling module in the decoder with the feature map of the corresponding feature extraction layer in the encoder. Specifically, the feature map output by the upsampling module in the first feature restoration layer is connected with the feature map output by the CSA module in the fourth feature extraction layer, wherein the feature map output by the upsampling module in the first feature restoration layer is the same as the feature map output by the CSA module in the fourth feature extraction layer, and both are 1 / 16 of the original image; the feature map output by the upsampling module in the second feature restoration layer is connected with the feature map output by the CSA module in the third feature extraction layer, wherein the feature map output by the upsampling module in the second feature restoration layer is connected with the feature map output by the CSA module in the third feature extraction layer. The feature maps output by the A module have the same size, which is 1 / 8 of the original image; the feature map output by the upsampling module in the third feature restoration layer is connected with the feature map output by the CSA module in the second feature extraction layer, wherein the feature map output by the upsampling module in the third feature restoration layer is the same size as the feature map output by the CSA module in the second feature extraction layer, and both are 1 / 4 of the original image; the feature map output by the upsampling module in the fourth feature restoration layer is connected with the feature map output by the CSA module in the first feature extraction layer, wherein the feature map output by the upsampling module in the fourth feature restoration layer is the same size as the feature map output by the CSA module in the first feature extraction layer, and both are 1 / 2 of the original image. The meaning of connection is to combine the feature maps of the two. For example, the feature map output by the upsampling module in the first feature restoration layer is combined with the feature map output by the CSA module in the fourth feature extraction layer, and then the combined feature map is input into the CSA module of the first feature restoration layer; the feature map output by the upsampling module in the second feature restoration layer is combined with the feature map output by the CSA module in the third feature extraction layer, and then the combined feature map is input into the CSA module of the second feature restoration layer; the feature map output by the upsampling module in the third feature restoration layer is combined with the feature map output by the CSA module in the second feature extraction layer, and then the combined feature map is input into the CSA module of the third feature restoration layer; the feature map output by the upsampling module in the fourth feature restoration layer is combined with the feature map output by the CSA module in the first feature extraction layer, and then the combined feature map is input into the CSA module of the fourth feature restoration layer.

[0086] The decoder part also includes 1*1 convolution operations and activation functions (such as Figure 4 Sigmoid / SoftMax in ), 1*1 convolution operation and activation function are used to output feature maps.

[0087] The following is an introduction to the CSA module, which uses the grouping / splitting idea to group large channels to extract features. Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of the CSA module provided in this application.

[0088] The CSA module extracts features by dividing into two groups (two branch paths). 1) First, the CSA module performs a 1*1 convolution operation on the image or feature map input to the CSA module to obtain the first feature map. If the number of channels of the input image is C, then after the 1*1 convolution operation, the number of channels of the first feature map obtained is c'.

[0089] 2) Group / split the first feature map after the 1*1 convolution operation into two feature maps according to the channel dimension to obtain a second feature map on the first channel dimension and a third feature map on the second channel dimension. For example, if the number of channels of the first feature map is 1 to 10, a total of 10 channels, and the first feature map is divided according to the channel dimension, the channel dimension of the second feature map (first channel dimension) can be 1 to 5, a total of 5 channel dimensions, and the channel dimension of the third feature map (second channel dimension) can be 6 to 10, a total of 5 channel dimensions.

[0090] 3) The first path: perform a 3*3 convolution operation on the second feature map to obtain the fourth feature map. The feature dimension of the fourth feature map is the same as that of the second feature map, which is c' / 2. Figure 5 In , the fourth feature map is represented by F1.

[0091] 4) The second path: perform a 3*3 convolution operation on the third feature map, then sum the feature map obtained after the convolution with the fourth feature map, and then perform a 3*3 convolution operation again to obtain the fifth feature map. The feature dimension of the fifth feature map is the same as that of the third feature map, both of which are c' / 2; Figure 5 In , the fifth feature map is represented by F2.

[0092] 5) Calculate the attention weight parameters corresponding to the fourth feature map and the fifth feature map respectively. Figure 5 As shown, the method for calculating the attention weight parameters corresponding to the fourth feature map and the fifth feature map is: the fourth feature map and the fifth feature map are concatenated and summed based on the channel dimension, and the channel dimension of the feature map obtained after the summation is c', wherein the formula for summing the fourth feature map F1 and the fifth feature map F2 is:

[0093]

[0094] Among them, Xi represents the input image, Fi(Xi) represents the application of F function to the input image Xi to obtain the feature map on the branch path, and H, W, and C are the height, width, and channel of the input image respectively.

[0095] Then, the feature map after the splicing, merging and summing is globally pooled to collect global spatial features. The channel dimension of the feature map obtained after global pooling is c'. The global pooling formula is as follows:

[0096]

[0097] α and β represent the point-by-point spatial information of the height and width of the feature map, respectively. Represents the feature map of (α, β) on the c-th channel.

[0098] The feature maps after global pooling are divided into 2 groups, each group includes some features, and the channel dimension of each group of feature maps is c'; 1*1 convolution operation is performed on the two groups of feature maps respectively, and then through normalization BN and activation function ReLU, two groups of feature maps with channel dimension c' are still obtained; then 1*1 convolution operation and normalization function SoftMax are performed again to obtain the attention weight parameter a1 corresponding to the fourth feature map and the attention weight parameter a2 corresponding to the fifth feature map respectively.

[0099] 6) According to the corresponding attention weight parameter, the fourth feature map on the first channel dimension and the fifth feature map on the second channel dimension are weighted summed to obtain the sixth feature map, where the feature map calculation formula on the cth channel is:

[0100]

[0101] a i (c) represents the attention weight parameter on the cth channel.

[0102] Here a i (c) is a (soft) assignment weight, calculated as follows:

[0103]

[0104] Represents the attention weight of the global spatial information S on the c-th channel and is quantized using two 1x1Conv and BatchNorm, ReLU activations.

[0105] 7) After convolution operation on the sixth feature map, sum it with the image input to the compact split attention CSA module to obtain the output feature map. Here, the CSA module uses the standard residual architecture (ResNet), and the output feature map Y is calculated using skip connection: Y = V + X, X is the input image or feature map, and V is the feature map obtained after convolution operation on the sixth feature map.

[0106] In the above two branch paths: one branch path uses a 1*1 convolution and a 3*3 convolution to obtain a feature map F1 on a channel dimension, and one branch path uses a 1*1 convolution and two 3*3 convolutions, and the feature map on the first branch path is added to the second branch path, that is, the feature map on the first channel dimension is added to the second channel dimension. By grouping the channels and using different convolution kernel sizes in the two branches to extract features, more detailed features can be extracted. First, the feature map is divided into two groups according to the channel dimension, and features are extracted in two channel dimensions to obtain feature maps of two channel dimensions. Then, the attention weight parameters corresponding to the feature maps of the two channel dimensions are calculated, which helps to extract the interactive information in the channel dimension and the spatial dimension, and obtain more detailed and global features.

[0107] Commonly used attention modules, for example, the SE (squeeze excitation) attention module only pays attention to channel information. Although the CBAM (convolutional block attention module) attention module pays attention to both channel features and spatial features, it pays attention to channel features first and then to spatial features, and there is no interactive fusion of channel features and spatial features.

[0108] The above introduces the basic architecture of the neural network model, which also includes a loss function. The loss function of the neural network model in this application is the weighted sum of the first function Dice Loss and the second function Focal Loss, that is:

[0109] L total =α D L Dice +β F L Focal ,α D +β F =1

[0110] Among them, α D , β F are the weights of the first function Dice Loss and the second function Focal Loss, α D , β F The specific value can be set by the user according to actual conditions.

[0111] in,

[0112]

[0113] The value range of the first function Dice Loss is [0,1], where X represents the cargo image input into the neural network model, and Y represents the image output by the above neural network model structure.

[0114] Since Dice Loss is very unfavorable for the prediction of small targets, due to the small area of ​​small targets, such as the prediction results at the pixel level, once the prediction results of small targets are wrong, it will cause a large fluctuation in the coefficients of the Dice Loss function, resulting in large gradient changes and unstable training. Therefore, this application combines Dice Loss and Focal Loss to design an overall loss function.

[0115] Focal Loss is based on binary cross entropy. It is a dynamically scaled cross entropy loss. Through a dynamic scaling factor, the weight of easily distinguishable samples can be dynamically reduced during training, thereby quickly focusing on samples that are difficult to distinguish (may be positive samples or negative samples, but they are all helpful for training the network). For positive samples, the loss of samples with large prediction probabilities (simple samples) becomes smaller, while the loss of samples with small prediction probabilities (difficult examples) becomes larger, thereby increasing the focus on difficult examples. The formula for Focal Loss is as follows:

[0116]

[0117] Among them, γ=2, α=0.75, y is the label, corresponding to 0, 1 in the binary classification; the value range of p is 0~1, which is the probability predicted by the neural network model that it belongs to the cargo grid line.

[0118] The above neural network model is pre-trained, and the training process is roughly as follows: obtain multiple positive samples and multiple negative samples, the positive samples include cargo images in which the cargo in the storage area has not crossed the boundary, and the negative samples include cargo images in which the cargo in the storage area has crossed the boundary, and label the images through software, the labels include marking the cargo grid line areas in the positive samples and negative samples with the first pixel value, and marking the non-cargo grid line areas with the second pixel value, and train the neural network model based on the positive samples, negative samples and labels, so as to obtain the trained neural network model. The trained neural network model can be used to identify cargo images.

[0119] S103. Compare the pixel values ​​of each position on the cargo grid line in the first image with the pixel values ​​of the corresponding position on the cargo grid line in the second image, where the second image is an image of the storage area without cargo stored, and the cargo grid line in the second image is marked by the first pixel value.

[0120] The second image is an image when no goods are stored in the storage area. The second image includes a goods grid line, and the goods grid line area in the second image is marked with a first pixel value. Here, the non-goods grid line area in the second image can be marked with a second pixel value, or with a first pixel value, or with other pixel values. Because the present application only cares about the pixel values ​​of the goods grid line in the second image, and does not care about the pixel values ​​of the non-goods grid line area, the non-goods grid line area can be marked with any pixel value.

[0121] The second image is pre-stored in the server. The second image is obtained by identifying and marking an image collected when no goods are stored in the storage area. The identification and marking include: marking the goods grid line area in the collected image with the first pixel value. In one implementation, first, an image of the storage area when no goods are stored is obtained, and then the image is input into the above-mentioned trained neural network model for identification to obtain the second image. Figure 6 A and Figure 6 As shown in B, Figure 6 A is an example of an image when no goods are stored in the storage area. Figure 6 B is for Figure 6 An example of a second image obtained by identifying image A, Figure 6 In B, the cargo grid line area is marked with a first pixel value, which is displayed as black in the image, and the non-cargo grid line area is marked with a second pixel value, which is displayed as white in the image.

[0122] Compare the pixel values ​​of each position on the cargo grid line (area) in the first image with the pixel values ​​of the corresponding position on the cargo grid line in the second image. If the pixel values ​​of the corresponding positions are all the same, it is determined that there is no cargo crossing the border. If the pixel values ​​of the corresponding positions are different, it is determined that there is cargo crossing the border. The pixel values ​​of the corresponding positions are different, including: the pixel values ​​of one or more positions are different.

[0123] For example, Figure 3B The pixel values ​​at each position of the cargo grid line area in the first image are Figure 6 By comparing the pixel values ​​at the corresponding positions of the cargo grid line area in the second image in B, it is found that the pixel values ​​at the position where the upper edge line of the grid 9 is located are not completely the same, and it is determined that the cargo has crossed the border.

[0124] In one implementation, the comparison process can be implemented by an XOR operation. If the pixel values ​​at the two positions are the same, the output is 255. If the pixel values ​​at the two positions are different, the output is 0.

[0125]

[0126] In specific implementation, the color of the cargo grid lines in the output image can be used to determine whether there is cargo crossing the border. If the pixel values ​​at the two positions are the same, the output is 255, and 255 is displayed as white in the image, that is, if the cargo grid lines in the output image are white, it means that there is no cargo crossing the border; if the pixel values ​​at the two positions are different, the output is 0, and 0 is displayed as black in the image, that is, if the cargo grid lines in the output image are black, it means that there is cargo crossing the border. This is just an example. In actual applications, other colors (such as red) or other pixel values ​​can also be defined to determine whether there is cargo crossing the border.

[0127] Optionally, the color and position of the grid lines of the goods in the output image can be used to determine which goods have crossed the border. For example, the cross-border position can be determined based on the color of the grid lines of the goods in the output image. Figure 7 As shown, Figure 3B The pixel values ​​at each position of the cargo grid line in Figure 6 The pixel values ​​at the corresponding positions of the cargo grid lines in B are XORed and the following is obtained: Figure 7 The output image shown in the figure shows the black block in the output image, indicating the location of the cross-border. The Euclidean distance between the center point of the black block and the center point of each grid is calculated. The grid with the smallest Euclidean distance to the center point of the grid determines the grid in which the goods have crossed the border. Figure 7 In the calculation, the Euclidean distance between the center point of the black block and the center point of grid 9 is the shortest, so it is determined that the goods in grid 9 have crossed the border. It should be noted that Figure 7 The dotted lines in the output image represent the grid lines of the storage area, which can be displayed in the output image or not. Even if it is not displayed in the output image, the server "knows" the location of each grid in the storage area, and can determine the center point of each grid, thereby calculating the Euclidean distance between the center point of the black block and the center point of each grid. If it is convenient for people to check the out-of-bounds location and the goods that have crossed the boundary, it can be displayed in the output image.

[0128] In one implementation, the comparison process may be implemented through a XOR operation, and the specific implementation scheme will not be introduced here.

[0129] S104. If the pixel values ​​at corresponding positions are all the same, it is determined that there is no cross-border situation of goods.

[0130] S105. If the pixel values ​​at the corresponding positions are different, it is determined that the goods have crossed the border.

[0131] Optionally, when it is determined that goods have crossed the boundary, the position of the goods can be adjusted manually or by using an AGV so that the goods are stored within the grid lines.

[0132] It can be seen that the neural network model in this application is improved on the basis of U-Net. In the encoder part, the deep separable convolution module is used instead of the convolution operation, which can reduce the number of parameters in the neural network model and make the neural network model more lightweight. In the encoder and decoder parts, a CSA module is added. The CSA module extracts the interactive information of spatial features and channel features through multiple convolution operations with different convolution kernel sizes to obtain more detailed and global features, making the recognition results of the neural network more accurate. The loss function of the neural network model adopts the weighted sum of the first function Dice Loss and the second function Focal Loss, which makes the neural network model converge faster and the recognition results more accurate.

[0133] The above is a description of a method embodiment. The following is an introduction to a device embodiment corresponding to the method embodiment.

[0134] See also Figure 8 , Figure 8 The present invention provides a schematic diagram of a storage location storage goods crossing boundary detection device 800, the device 800 comprising:

[0135] An acquisition module 810 is used to acquire a cargo image of a storage area, wherein the cargo image includes cargo grid lines;

[0136] The recognition module 820 is used to recognize the cargo grid lines in the cargo image, and mark the cargo grid lines in the cargo image with a first pixel value, and mark the non-cargo grid line area in the cargo image with a second pixel value, so as to obtain a first image;

[0137] The determination module 830 is used to compare the pixel values ​​of each position on the cargo grid line in the first image with the pixel values ​​of the corresponding position on the cargo grid line in the second image. If the pixel values ​​of the corresponding positions are all the same, it is determined that there is no cargo crossing the boundary. If the pixel values ​​of the corresponding positions are different, it is determined that there is cargo crossing the boundary. The second image is an image of a storage area without cargo stored therein, and the cargo grid line in the second image is marked by the first pixel value.

[0138] In a possible implementation, the identification module 820 is used to:

[0139] The cargo grid lines in the cargo image are identified by a neural network model to obtain a first image, wherein the cargo grid lines in the first image are marked by a first pixel value, and the non-cargo grid line area in the first image is marked by a second pixel value; wherein the neural network model includes an encoder and a decoder, the encoder includes a deep separable convolution module and a compact split attention CSA module, the deep separable convolution module is used to extract the spatial dimension features and channel dimension features in the cargo image by splitting, and the decoder includes a compact split attention CSA module, and the compact split attention CSA module is used to extract the interactive information between the spatial dimension features and the channel dimension features of the feature map input into the compact split attention CSA module.

[0140] In a possible implementation, the encoder includes multiple depth-wise separable convolution modules, and the multiple depth-wise separable convolution modules use jump connections.

[0141] In a possible implementation, the compact split attention CSA module extracts features of the feature map input to the compact split attention CSA module through multiple paths based on the attention mechanism, and the multiple paths use different convolution operations on multiple different channel dimensions.

[0142] In a possible implementation, the identification module 820 is used to:

[0143] Performing a convolution operation on the image input into the compact split attention CSA module through the compact split attention CSA module to obtain a first feature map;

[0144] The first feature map is split into two feature maps according to the channel dimension by a compact split attention CSA module, and a second feature map on the first channel dimension and a third feature map on the second channel dimension are obtained respectively;

[0145] The first path: convolve the second feature map through the compact split attention CSA module to obtain the fourth feature map in the first channel dimension;

[0146] The second path: convolve the third feature map through the compact split attention CSA module, then sum the feature map obtained after the convolution with the fourth feature map on the first channel dimension and convolve it again to obtain the fifth feature map on the second channel dimension;

[0147] The attention weight parameters corresponding to the fourth feature map on the first channel dimension and the fifth feature map on the second channel dimension are calculated respectively through the compact split attention CSA module;

[0148] The fourth feature map and the fifth feature map are weighted summed according to the corresponding attention weight parameter by the compact split attention CSA module to obtain the sixth feature map;

[0149] The sixth feature map is convolved by the compact split attention CSA module and then summed with the image input to the compact split attention CSA module to obtain the output feature map.

[0150] In a possible implementation, the encoder includes a plurality of feature extraction layers, a first feature extraction layer in the plurality of feature extraction layers includes a plurality of depth-separable convolution modules and a pooling layer, a last feature extraction layer in the plurality of feature extraction layers includes a compact split attention CSA module, and other feature extraction layers in the plurality of feature extraction layers include a compact split attention CSA module and a pooling layer;

[0151] The decoder includes at least multiple feature restoration layers, each of which includes an upsampling module, a connection module and a compact split attention CSA module, and the connection module is used to connect the multiple feature extraction layers in the encoder with the multiple feature restoration layers in the decoder respectively.

[0152] In a possible implementation, the loss function of the neural network model is a weighted sum of a first function, Dice Loss, and a second function, Focal Loss.

[0153] Figure 8 The various functional modules in are used to implement Figures 2 to 7 For details of the steps of the method embodiment, please refer to Figures 2 to 7 The description of the relevant contents in the method embodiment will not be repeated here for the sake of brevity of the specification.

[0154] This application also provides a server, see Fig. 9 , Fig. 9 A schematic diagram of the structure of a server 900 provided in this application, the server is used to implement Figures 2 to 7 In the method embodiment, the server 900 includes: a processor 910, a communication interface 920, and a memory 930. The processor 910, the communication interface 920, and the memory 930 may be interconnected via an internal bus 940, or may communicate via other means such as wireless transmission.

[0155] Taking the connection through bus 940 as an example, bus 940 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. Bus 940 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0156] The processor 910 may be composed of at least one general-purpose processor, such as a CPU, or a combination of a CPU and a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The processor 910 executes various types of digital storage instructions, such as software or firmware programs stored in the memory 930, which enables the server 900 to provide a wide variety of services.

[0157] The memory 930 is used to store program codes and is controlled by the processor 910 to execute the above Figures 2 to 7 For the steps described in the embodiment, please refer to the relevant description of the embodiment shown above, which will not be elaborated here.

[0158] The memory 930 may include a volatile memory, such as a RAM; the memory 930 may also include a non-volatile memory, such as a ROM, a flash memory; the memory 930 may also include a combination of the above types.

[0159] The communication interface 920 can be a wired interface (such as an Ethernet interface), an internal interface (such as a high-speed serial computer expansion bus (peripheral component interconnect express, PCIE) bus interface), a wired interface (such as an Ethernet interface) or a wireless interface (such as a cellular network interface or a wireless local area network interface) for communicating with other devices or modules.

[0160] The processor 910, communication interface 920, etc. in the server 900 can implement the functions and / or various steps and methods implemented in the above-mentioned various method embodiments, and for the sake of brevity, they are not repeated here. The identification module 820 and the determination module 830 in the device 800 can be located in the processor 910 in the server 900; the acquisition module 810 can be located in the communication interface 920 in the server 900.

[0161] Need to explain, Fig. 9This is only a possible implementation of the embodiment of the present application. In practical applications, the server may also include more or fewer components, which is not limited here. For content not shown or described in the embodiment of the present application, please refer to the relevant description in the embodiment described in the aforementioned method, which will not be repeated here.

[0162] The present application also provides a computer storage medium, including program instructions, which, when executed by a computer, enable the computer to execute part or all of the steps described in the above method embodiment.

[0163] The present application also provides a computer program product, including program instructions, which, when executed by a computer, enable the computer to execute part or all of the steps described in the above method embodiments.

[0164] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0165] In the above embodiments, it can be implemented in whole or in part by software, hardware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product may include code. When the computer program product is read and executed by a computer, some or all steps of the method recorded in the above method embodiment may be implemented. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium, or a semiconductor medium, etc.

[0166] The steps in the method of the embodiment of the present application can be adjusted in order, combined or deleted according to actual needs; the units in the device of the embodiment of the present application can be divided, combined or deleted according to actual needs.

[0167] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for detecting cross-boundary storage of goods, characterized in that: include: Acquire a cargo image of a storage area, wherein the cargo image includes cargo grid lines; Identify the cargo grid lines in the cargo image, and mark the cargo grid lines in the cargo image by a first pixel value, and mark the non-cargo grid line area in the cargo image by a second pixel value, to obtain a first image; The pixel values ​​at each position on the cargo grid line in the first image are compared with the pixel values ​​at the corresponding positions on the cargo grid line in the second image. If the pixel values ​​at the corresponding positions are all the same, it is determined that there is no cargo crossing the boundary. If the pixel values ​​at the corresponding positions are different, it is determined that there is cargo crossing the boundary. The second image is an image of the storage area without cargo stored therein, and the cargo grid line in the second image is marked by the first pixel value.

2. The method according to claim 1, characterized in that Recognizing the cargo grid lines in the cargo image by using a neural network model to obtain the first image; Among them, the neural network model includes an encoder and a decoder, the encoder includes a deep separable convolution module and a compact split attention CSA module, the deep separable convolution module is used to extract the spatial dimension features and channel dimension features in the cargo image, and the decoder includes the compact split attention CSA module, and the compact split attention CSA module is used to extract the interaction information between the spatial dimension features and channel dimension features of the feature map input into the compact split attention CSA module.

3. The method according to claim 2, characterized in that The encoder includes a plurality of depth-wise separable convolution modules, and the plurality of depth-wise separable convolution modules adopt skip connections.

4. The method according to claim 2 or 3, characterized in that: The compact split attention CSA module extracts features of the feature map input into the compact split attention CSA module through multiple paths based on the attention mechanism, and the multiple paths use different convolution operations on multiple different channel dimensions.

5. The method according to claim 4, characterized in that The compact split attention CSA module extracts features of the feature graph input to the compact split attention CSA module through multiple paths based on the attention mechanism, including: The compact split attention CSA module performs a convolution operation on the image input into the compact split attention CSA module to obtain a first feature map; Splitting the first feature map into two feature maps according to the channel dimension, and obtaining a second feature map on the first channel dimension and a third feature map on the second channel dimension respectively; First path: performing a convolution operation on the second feature map to obtain a fourth feature map on the first channel dimension; The second path: performing a convolution operation on the third feature map, then summing the feature map obtained after the convolution with the fourth feature map on the first channel dimension and performing a convolution operation again to obtain a fifth feature map on the second channel dimension; Calculate the attention weight parameters corresponding to the fourth feature map on the first channel dimension and the fifth feature map on the second channel dimension respectively; According to the corresponding attention weight parameter, performing weighted summation on the fourth feature map and the fifth feature map to obtain a sixth feature map; After performing a convolution operation on the sixth feature map, the convolution operation is performed and the image input into the compact split attention CSA module is summed to obtain an output feature map.

6. The method according to any one of claims 3 to 5, characterized in that: The encoder comprises a plurality of feature extraction layers, a first feature extraction layer of the plurality of feature extraction layers comprises the plurality of depth-separable convolution modules and a pooling layer, a last feature extraction layer of the plurality of feature extraction layers comprises the compact split attention CSA module, and other feature extraction layers of the plurality of feature extraction layers comprise the compact split attention CSA module and the pooling layer; The decoder includes at least multiple feature restoration layers, each of the multiple feature restoration layers includes an upsampling module, a connection module and the compact split attention CSA module, and the connection module is used to respectively connect the multiple feature extraction layers in the encoder with the multiple feature restoration layers in the decoder.

7. The method according to any one of claims 3 to 6, characterized in that: The loss function of the neural network model is the weighted sum of the first function Dice Loss and the second function Focal Loss.

8. A device for detecting goods crossing the storage boundary, characterized in that: include: An acquisition module, used to acquire a cargo image of a storage area, wherein the cargo image includes cargo grid lines; a recognition module, configured to recognize the cargo grid lines in the cargo image, mark the cargo grid lines in the cargo image by a first pixel value, and mark a non-cargo grid line area in the cargo image by a second pixel value, so as to obtain a first image; A determination module is used to compare the pixel values ​​of each position on the cargo grid line in the first image with the pixel values ​​of the corresponding positions on the cargo grid line in the second image. If the pixel values ​​of the corresponding positions are all the same, it is determined that there is no cargo crossing the boundary; if the pixel values ​​of the corresponding positions are different, it is determined that there is cargo crossing the boundary; wherein, the second image is an image of the storage area without cargo stored, and the cargo grid line in the second image is marked by the first pixel value.

9. A server, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The method comprises program instructions, and when the program instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images