Cargo area segmentation method, device and equipment and storage medium

Abstract

The invention discloses a cargo area segmentation method, apparatus and device, and a storage medium. The cargo area segmentation method comprises the steps of obtaining a to-be-segmented cargo picture; inputting the to-be-segmented cargo picture into a trained segmentation model, and sequentially processing the to-be-segmented cargo picture through a constructed initial convolution module, a plurality of enhanced local Transform modules and a segmentation head to obtain a cargo region segmentation result; each enhanced local Transform module comprises an image embedding layer, an edge enhancement convolution module and a local Transform module, and the edge enhancement convolution module calculates the difference between a first feature map obtained through convolution processing of a multi-head convolution attention mechanism constructed by the edge enhancement convolution module and an original feature map obtained through processing of the image embedding layer; and obtaining a second feature map containing the local texture and edge information of the cargo picture. By using the cargo area segmentation method disclosed by the invention, the accuracy and effectiveness of cargo area segmentation can be improved.

Description

Technical Field

[0001] The present invention relates to the field of logistics technology, and in particular to a cargo area segmentation method, device, equipment and storage medium. Background Art

[0002] With the rapid growth of global e-commerce, the logistics industry faces huge challenges, especially in terms of efficient handling of goods, accurate billing and cost control. Logistics companies must adopt innovative technologies to improve service quality, reduce operating costs, and increase customer satisfaction. In this context, cargo area segmentation can be used to optimize logistics companies' operations in cargo handling and billing.

[0003] At present, the cargo area segmentation method of SAM (SegmentAnything Model, an image segmentation algorithm) and YOLOV8n (YouOnly Live Once, an object detection and image segmentation algorithm; YOLOV8n is a Yolo series algorithm) can be used. This method uses the target detection model to first locate the cargo area, and then obtains the cargo area by the semantic segmentation model. However, this multi-model serial method needs to process multiple models separately during model optimization and cargo area segmentation, which has a high training cost and low efficiency in cargo area segmentation. The cargo area segmentation method based on either a convolutional neural network or a visual Transformer model can also be used alone. However, the cargo area segmentation method based on a convolutional neural network is insufficient in capturing the global context, resulting in low accuracy and effectiveness in cargo area segmentation, while the cargo area segmentation method based on a visual Transformer model is inefficient in processing local features, resulting in low efficiency in cargo area segmentation. Summary of the invention

[0004] The present invention provides a cargo area segmentation method, device, equipment and storage medium, which are used to solve the technical problems of low accuracy, effectiveness and efficiency of cargo area segmentation in the prior art.

[0005] In order to solve the above technical problems, in a first aspect, the present invention provides a cargo area segmentation method, the cargo area segmentation method comprising:

[0006] Get the pictures of the goods to be divided;

[0007] The cargo image to be segmented is input into the trained segmentation model, and is processed in sequence by the constructed initial convolution module, multiple enhanced local Transformer modules and the segmentation head to obtain the cargo area segmentation result; each of the enhanced local Transformer modules includes an image embedding layer, an edge enhancement convolution module and a local Transformer module, and the edge enhancement convolution module calculates the difference between the first feature map obtained by the convolution processing of the multi-head convolution attention mechanism constructed by it and the original feature map obtained by the image embedding layer processing to obtain a second feature map containing the local texture and edge information of the cargo image.

[0008] Optionally, the edge enhancement convolution module includes a grouped convolution layer, a first batch of normalization layers, an activation layer, a first 1×1 convolution layer, a pixel difference convolution layer and a first multi-layer perceptron; each of the enhanced local Transformer modules calculates the difference between the first feature map obtained by the convolution processing of the multi-head convolution attention mechanism and the original feature map obtained by the image embedding layer processing to obtain a second feature map containing the local texture and edge information of the cargo picture, including:

[0009] The pixel difference convolution layer calculates the difference between the first feature map obtained by processing the original feature map through the grouped convolution layer, the first normalization layer, the activation layer, and the first 1×1 convolution layer and the original feature map spliced ​​to the pixel difference convolution layer through the residual connection, and obtains a second feature map containing local texture and edge information of the cargo image to be input into the first multi-layer perceptron.

[0010] Optionally, the local Transformer module includes a second 1×1 convolutional layer, a multi-head self-attention layer and a second multi-layer perceptron, and the second 1×1 convolutional layer is composed of Point-Wise convolution; the second feature map is processed by the Point-Wise convolution of the second 1×1 convolutional layer, and then processed by the multi-head self-attention layer, and finally input into the second multi-layer perceptron.

[0011] Optionally, residual connections are added before and after the multi-head attention layer, and the multi-head attention layer processes the output of the second 1×1 convolutional layer and performs residual connection processing, and finally inputs it into the second multi-layer perceptron.

[0012] Optionally, the image embedding layer includes a mean pooling layer and a third 1×1 convolutional layer.

[0013] Optionally, the initial convolution module includes multiple 3×3 convolution layers, each of which is followed by a second batch of normalization layers and activation layers.

[0014] Optionally, the segmentation head includes a third 1×1 convolutional layer and an upsampling layer.

[0015] In a second aspect, the present invention provides a cargo area segmentation device, comprising a data acquisition unit and a segmentation unit;

[0016] The data acquisition unit is used to acquire pictures of goods to be segmented;

[0017] The segmentation unit is used to input the cargo image to be segmented into a trained segmentation model, and sequentially process it through the constructed initial convolution module, multiple enhanced local Transformer modules and segmentation head processing to obtain a cargo area segmentation result; each of the enhanced local Transformer modules includes an image embedding layer, an edge enhancement convolution module and a local Transformer module, and the edge enhancement convolution module calculates the difference between a first feature map obtained by convolution processing of the multi-head convolution attention mechanism constructed by it and an original feature map obtained by processing the image embedding layer, to obtain a second feature map containing local texture and edge information of the cargo image.

[0018] In a third aspect, the present invention provides a cargo area segmentation device, including a memory and a processor, wherein:

[0019] The memory is used to store computer programs;

[0020] The processor is used to read the program in the memory and execute the steps of the cargo area segmentation method provided in the first aspect above.

[0021] In a fourth aspect, the present invention provides a computer-readable storage medium having a readable computer program stored thereon, which, when executed by a processor, implements the steps of the cargo area segmentation method provided in the first aspect above.

[0022] Compared with the prior art, the cargo area segmentation method, device, equipment and storage medium provided by the present invention have the following beneficial effects:

[0023] This method combines a multi-head convolutional attention mechanism, calculates the difference between the first feature map obtained by convolution processing of the multi-head convolutional attention mechanism and the original feature map obtained by processing the image embedding layer, and introduces a residual connection. It can more effectively capture edge features and enhance the sensitivity of the segmentation model to image details, thereby improving the accuracy and effectiveness of cargo area segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only part of the embodiments of the present invention, rather than all of the embodiments. For ordinary technicians in this field, without paying creative work, other drawings obtained based on these drawings all fall within the scope of protection of this application.

[0025] Figure 1 is a flow chart of a cargo area segmentation method provided by an embodiment of the present invention;

[0026] Figure 2 is a structural schematic diagram of a segmentation model provided by an embodiment of the present invention;

[0027] Figure 3 is a structural diagram of an enhanced local Transformer module provided by an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of the structure of an edge enhancement convolution module provided by an embodiment of the present invention;

[0029] Figure 5 is a structural diagram of a local Transformer module provided by an embodiment of the present invention;

[0030] Figure 6 is a structural schematic diagram of a cargo area segmentation device provided by an embodiment of the present invention;

[0031] Figure 7 is a structural schematic diagram of a cargo area segmentation device provided by an embodiment of the present invention;

[0032] Figure 8 It is a structural schematic diagram of a computer-readable storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0034] In order to make the description of the disclosed content more detailed and complete, the following is an illustrative description of the implementation mode and specific examples of the present invention; however, this is not the only form of implementing or applying the specific embodiments of the present invention. The implementation mode covers the features of multiple specific embodiments and the method steps and their sequence for constructing and operating these specific embodiments. However, other specific embodiments can also be used to achieve the same or equal functions and step sequences. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0036] In the description of the embodiments of the present application, "multiple" refers to two or more than two, and other quantifiers are similar and should be understood. The preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention. In addition, the embodiments of the present application and the features therein may be combined with each other if there is no conflict.

[0037] Example 1

[0038] like Figure 1 As shown, it is a flow chart of a cargo area segmentation method provided by an embodiment of the present invention. The cargo area segmentation method includes the following steps.

[0039] Step S101, obtaining pictures of goods to be segmented;

[0040] In the logistics links such as sorting and transportation, take pictures of the area containing the goods to obtain pictures of the goods to be segmented. In order to adapt to the diversity of the pictures of the goods to be segmented for segmentation detection, when training the segmentation model, the goods in the actual logistics links can be photographed from different angles and under different environmental conditions to collect sample pictures of goods of different types and angles. In some embodiments, data enhancement processing can also be performed on the sample pictures of goods, such as random cropping, scaling, translation and rotation operations, to improve the generalization ability of the segmentation model.

[0041] Step S102, inputting the cargo image to be segmented into the trained segmentation model, and sequentially processing through the constructed initial convolution module, multiple enhanced local Transformer modules and segmentation head to obtain the cargo area segmentation result; each of the enhanced local Transformer modules includes an image embedding layer, an edge enhancement convolution module and a local Transformer module, and the edge enhancement convolution module calculates the difference between the first feature map obtained by the convolution processing of the multi-head convolution attention mechanism constructed by it and the original feature map obtained by the image embedding layer processing, and obtains the second feature map containing the local texture and edge information of the cargo image.

[0042] The cargo images to be segmented are input into the trained segmentation model for segmentation processing. The training process of the segmentation model is to first label the segmentation labels of the cargo area on the collected sample cargo images; then the labeled sample cargo image data is randomly divided into a training set, a validation set and a test set according to a certain ratio; the segmentation model is trained using the training set, the validation set and the test set, thereby obtaining a trained segmentation model.

[0043] In some embodiments, a schematic diagram of a segmentation model is shown in FIG. Figure 2 As shown in the figure, the segmentation model includes an initial convolution module (Conv 3×3), three enhanced local Transformer modules (ELT Block1, ELT Block2, ELTBlock3) and a segmentation head. If the image size of the cargo picture to be segmented is 512×512, the input (Input) segmentation model enters the initial convolution module. The initial convolution module performs initial convolution processing on the cargo picture to be segmented to obtain a feature map with a size of 128×128 and 64 channels. The feature map is sequentially input into the ELT Block1 module, the ELT Block2 module and the ELTBlock3 module. After processing by the ELT Block1 module, a feature map with an image size of 64×64 and a number of channels of 128 is obtained. After processing by the ELT Block2 module, a feature map with an image size of 32×32 and a number of channels of 256 is obtained. After processing by the ELT Block3 module, a feature map with an image size of 16×16 and a number of channels of 512 is obtained. Finally, the ELT After the feature map processed by the Block3 module is processed by the segmentation head, the cargo area segmentation result is output from the segmentation model (Output). It should be noted that the present invention does not limit the number of enhanced local Transformer modules, and the size and number of channels of the feature map obtained by each Transformer module.

[0044] In some embodiments, a structural diagram of an enhanced local Transformer module is shown as follows: Figure 3 As shown, the enhanced local transformer module (ELT Block) includes an image embedding layer (Patch Embedding), an edge enhanced convolution module (EECB) and a local transformer module (LTB). The edge enhanced convolution module (EECB) in each enhanced local transformer module (ELTBlock) calculates the difference between the first feature map obtained by the convolution processing of the multi-head convolution attention mechanism constructed by it and the original feature map obtained by the image embedding layer processing to obtain a second feature map containing the local texture and edge information of the cargo image.

[0045] Taking ELT Block1 as an example, the original feature map obtained by the image embedding layer (Patch Embedding) is input into the edge enhancement convolution module (EECB). The edge enhancement convolution module (EECB) calculates the difference between the first feature map obtained by the convolution processing of the multi-head convolution attention mechanism it constructs and the original feature map, and obtains the second feature map containing the local texture and edge information of the cargo picture. The whole process can keep the size and number of channels of the feature map unchanged, but enhance the expressiveness of the feature map and the capture of details. The processing of ELT Block2, ELT Block3 is the same as that of ELTBlock1, which will not be repeated here.

[0046] The embodiment of the present invention, by combining a multi-head convolutional attention mechanism and calculating the difference between the first feature map obtained by convolution processing of the multi-head convolutional attention mechanism constructed therewith and the original feature map obtained by processing the image embedding layer, introduces a residual connection, which can more effectively capture edge features and enhance the sensitivity of the segmentation model to image details, thereby improving the accuracy and effectiveness of cargo area segmentation.

[0047] As an optional implementation, the edge enhancement convolution module includes a grouped convolution layer, a first normalization layer, an activation layer, a first 1×1 convolution layer, a pixel difference convolution layer and a first multi-layer perceptron; each of the enhanced local Transformer modules calculates the difference between the first feature map obtained by convolution processing with a multi-head convolution attention mechanism and the original feature map obtained by processing the image embedding layer to obtain a second feature map containing local texture and edge information of the cargo image, including:

[0048] The pixel difference convolution layer calculates the difference between the first feature map obtained by processing the original feature map through the grouped convolution layer, the first normalization layer, the activation layer, and the first 1×1 convolution layer and the original feature map spliced ​​to the pixel difference convolution layer through residual connection, and obtains a second feature map containing the local texture and edge information of the cargo image to be input into the first multi-layer perceptron. Among them, the grouped convolution layer, the first normalization layer, the activation layer, and the first 1×1 convolution layer together constitute the multi-head convolution attention mechanism convolution processing.

[0049] In some embodiments, a schematic diagram of the structure of an edge enhancement convolution module is shown as follows: Figure 4 As shown in the figure, the edge enhancement convolution module (EECB) includes a group convolution layer (Group Conv3×3), a first batch normalization layer (BN), an activation layer (ReLU), a first 1×1 convolution layer (Conv 1×1), a pixel difference convolution layer (PDC) and a first multi-layer perceptron (MLP). Through the residual connection operation, the original feature map of the input edge enhancement convolution module (EECB) is spliced ​​to the input of the pixel difference convolution layer (PDC); the original feature map input to the edge enhancement convolution module (EECB) is convolved by a multi-head convolution attention mechanism formed by a group convolution layer (Group Conv 3×3), a first batch normalization layer (BN), an activation layer (ReLU), and a first 1×1 convolution layer (Conv 1×1) to obtain a first feature map and input it to the pixel difference convolution layer (PDC); the pixel difference convolution layer (PDC) calculates the difference between the first feature map and the original feature map to obtain a second feature map that extracts local texture and edge information, which is input to the first multi-layer perceptron (MLP). The edge enhancement convolution module combines the multi-head convolution attention mechanism convolution processing and the pixel difference convolution layer processing to enhance the expressiveness and detail capture of the feature map. The entire processing process can keep the size and number of channels of the first feature map unchanged, which can be applied to efficient feature processing and information integration in complex visual tasks.

[0050] The embodiments of the present invention, by combining a multi-head convolutional attention mechanism and a pixel differential convolutional layer, can more effectively capture edge features and enhance the model's sensitivity to image details, thereby improving the accuracy and effectiveness of cargo area segmentation.

[0051] As an optional implementation, the local Transformer module includes a second 1×1 convolutional layer, a multi-head self-attention layer and a second multi-layer perceptron, and the second 1×1 convolutional layer is composed of Point-Wise convolution; the second feature map is processed by the Point-Wise convolution of the second 1×1 convolutional layer, and then processed by the multi-head self-attention layer, and finally input into the second multi-layer perceptron.

[0052] In some embodiments, a schematic diagram of the structure of a local Transformer module (LTB) is shown as follows: Figure 5 As shown, the local Transformer module includes a second 1×1 convolutional layer (Conv 1×1), a multi-head self-attention layer (MHSA) and a second multi-layer perceptron (MLP). Among them, the second 1×1 convolutional layer is composed of Point-Wise convolution, which can reduce the dimension of the input channel and improve the computational efficiency. Exemplarily, the size of the feature map processed by the aforementioned edge enhancement convolution module is 128×128, and the number of channels is 128. After being processed by the Point-Wise convolutional layer of the local Transformer module (LTB), the number of channels is reduced from 128 to 64, that is, the size of the feature map output by the second 1×1 convolutional layer (Conv 1×1) is 128×128, and the number of channels is 64. After being processed by the multi-head self-attention layer (MHSA), the size of the output feature map is 64X64, and the number of channels is 128, and finally input to the second multi-layer perceptron (MLP).

[0053] The embodiments of the present invention, by combining Point-Wise convolution and multi-head self-attention mechanism, can capture the dependencies between input features, improve the computational efficiency of the model and the model's ability to pay attention to different areas, and thus improve the efficiency, accuracy and effectiveness of cargo area segmentation.

[0054] As an optional implementation, residual connections are added before and after the multi-head attention layer, and the multi-head attention layer processes the output of the second 1×1 convolutional layer and performs residual connection processing, and finally inputs it into the second multi-layer perceptron.

[0055] In some embodiments, adding residual connections before and after the multi-head self-attention layer (MHSA) can better retain and transmit information. The second multi-layer perceptron (MLP) further processes the features after the multi-head self-attention layer, which can achieve efficient processing of feature maps while maintaining the expressiveness and learning ability of the segmentation model. Taking the local Transformer module (LTB) in ELT Block1 as an example, the feature map processed by the second 1×1 convolutional layer (Conv 1×1) is input into the multi-head self-attention layer (MHSA), and after being processed by the multi-head self-attention layer (MHSA) and subjected to residual connection processing, it is input into the second multi-layer perceptron (MLP).

[0056] The embodiment of the present invention improves the accuracy and effectiveness of the segmentation of the model by adding residual connections before and after the multi-head attention layer and retaining and transmitting information through the residual connections. At the same time, the second multi-layer perceptron further processes the features after the multi-head self-attention layer, realizes efficient processing of the feature map, maintains the expressive power and learning ability of the segmentation model, and improves the efficiency of the model calculation.

[0057] As an optional implementation, the initial convolution module includes multiple 3×3 convolution layers, each of which is followed by a second batch of normalization layers and activation layers.

[0058] In some embodiments, the above-mentioned initial convolution module may include 4 3×3 convolution layers, each of which is connected to a second batch of normalization layers and activation function layers. The output channel of the first 3×3 convolution layer can be 64, and the stride can be 2, the output channel of the second 3×3 convolution layer can be 32, and the stride can be 1, the output channel of the third 3×3 convolution layer can be 64, and the stride can be 1, the output channel of the fourth 3×3 convolution layer can be 64, and the stride can be 2, and the padding of the above 4 3×3 convolution layers can all be 1. Exemplarily, a 512×512 image is input to the initial convolution module. After being processed by the above-mentioned initial convolution module, the size of the output feature map is 128×128, and the number of channels is 64.

[0059] As an optional implementation, the image embedding layer includes a mean pooling layer and a third 1×1 convolutional layer.

[0060] In some embodiments, the image embedding layer in the enhanced local Transformer module includes a mean pooling layer and a third 1×1 convolution layer. The stride of the mean pooling layer of the image embedding layer can be 2. The image embedding layer divides the input feature map into multiple Patches (small areas extracted from the original image), and each Patch generates a feature vector through a convolution operation. These feature vectors represent the local information of the image.

[0061] As an optional implementation, the segmentation head includes a third 1×1 convolutional layer and an upsampling layer.

[0062] The segmentation head uses a common model architecture, including the third 1×1 convolution layer and the upsampling layer. After the feature map processed by the last enhanced local Transformer module is processed by the segmentation head, the probability distribution of each pixel in the feature map can be obtained, thereby obtaining the cargo area segmentation result.

[0063] In some embodiments, the segmentation head is located at the end of the segmentation model, and is used to convert the feature map extracted by the previous network into pixel-level classification predictions. Specifically, the segmentation head can obtain the probability distribution of each pixel in the feature map, and the category of each pixel in the cargo image can be determined according to the probability distribution of each pixel, thereby achieving cargo image segmentation and obtaining cargo area segmentation results.

[0064] The cargo area segmentation method provided by the embodiment of the present invention obtains a cargo image to be segmented; inputs the cargo image to be segmented into a trained segmentation model, and sequentially processes the initial convolution module, multiple enhanced local Transformer modules and segmentation head to obtain the cargo area segmentation result; each enhanced local Transformer module includes an image embedding layer, an edge enhancement convolution module and a local Transformer module, and the edge enhancement convolution module calculates the difference between the first feature map obtained by the convolution processing of the multi-head convolution attention mechanism constructed by it and the original feature map obtained by the image embedding layer processing to obtain the second feature map containing the local texture and edge information of the cargo image. The method combines the multi-head convolution attention mechanism, calculates the difference between the first feature map obtained by the convolution processing of the multi-head convolution attention mechanism constructed by it and the original feature map obtained by the image embedding layer processing, and introduces residual connection, which can more effectively capture edge features and enhance the sensitivity of the segmentation model to image details, thereby improving the accuracy and effectiveness of cargo area segmentation.

[0065] This method combines the initial convolution module and the enhanced local Transformer module, which can not only improve the ability to capture the global context, but also avoid the separate processing of multiple models when segmenting the cargo area, further improving the accuracy and effectiveness of cargo area segmentation, and can also improve the efficiency of processing local features and improve the efficiency of cargo area segmentation.

[0066] The cargo area segmentation method provided in the embodiment of the present invention improves the model's ability to capture edge features and global dependencies by combining pixel differential convolution and optimized local Transformer modules, thereby improving the accuracy and effectiveness of cargo area segmentation. At the same time, a residual connection method is added to the segmentation model to alleviate the gradient vanishing problem in deep neural networks, thereby promoting information transfer and improving model training effects.

[0067] Example 2

[0068] Based on the above cargo area segmentation method, an embodiment of the present invention provides a cargo area segmentation device, and its structural schematic diagram is as follows: Figure 6 As shown, the cargo area segmentation device 60 includes a data acquisition unit 61 and a segmentation unit 62;

[0069] The data acquisition unit 61 is used to acquire the image of the goods to be segmented;

[0070] The segmentation unit 62 is used to input the cargo image to be segmented into the trained segmentation model, and sequentially process it through the constructed initial convolution module, multiple enhanced local Transformer modules and segmentation head processing to obtain the cargo area segmentation result; each of the enhanced local Transformer modules includes an image embedding layer, an edge enhancement convolution module and a local Transformer module, and the edge enhancement convolution module calculates the difference between the first feature map obtained by the convolution processing of the multi-head convolution attention mechanism constructed by it and the original feature map obtained by the image embedding layer processing to obtain a second feature map containing the local texture and edge information of the cargo image.

[0071] For other details about how the modules in the cargo area segmentation device implement the above technical solution, please refer to the description of the cargo area segmentation method provided in the above invention embodiment, which will not be repeated here.

[0072] Example 3

[0073] Based on the above cargo area segmentation method, the embodiment of the present invention further provides a cargo area segmentation device, the structural diagram of which is as follows: Figure 7 As shown, the device 70 includes a processor 71 and a memory 72 coupled to the processor 71. The memory 72 stores a computer program, and when the computer program is executed by the processor 71, the processor 71 executes the steps of the cargo area segmentation method in the above embodiment.

[0074] For other details about how the processor 71 in the cargo area segmentation device implements the above technical solution, please refer to the description of the cargo area segmentation method provided in the above invention embodiment, which will not be repeated here.

[0075] Among them, the processor 71 can also be called a CPU (Central Processing Unit), and the processor 71 may be an integrated circuit chip with signal processing capabilities; the processor 71 can also be a general-purpose processor, DSP (Digital Signal Process), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, among which the general-purpose processor can be a microprocessor or the processor 71 can also be any conventional processor, etc.

[0076] Example 4

[0077] The embodiment of the present invention further provides a computer-readable storage medium, a schematic diagram of which is shown in FIG. Figure 8 As shown, a readable computer program 81 is stored on the storage medium 80; wherein, the computer program 81 can be stored in the above storage medium 80 in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium 80 includes: U disk, mobile hard disk, magnetic disk or optical disk, ROM (Read-Only Memory), RAM (Random Access Memory) and other media that can store program codes, or terminal devices such as computers, servers, mobile phones, and tablets.

[0078] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0079] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0080] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0081] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0082] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0083] The technical solution provided by the present application is introduced in detail above. The principles and implementation methods of the present application are explained by using specific examples in the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0084] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0085] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0086] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0088] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A cargo area segmentation method, characterized in that: include: Get the pictures of the goods to be divided; The cargo image to be segmented is input into the trained segmentation model, and is processed in sequence by the constructed initial convolution module, multiple enhanced local Transformer modules and the segmentation head to obtain the cargo area segmentation result; each of the enhanced local Transformer modules includes an image embedding layer, an edge enhancement convolution module and a local Transformer module, and the edge enhancement convolution module calculates the difference between the first feature map obtained by the convolution processing of the multi-head convolution attention mechanism constructed by it and the original feature map obtained by the image embedding layer processing to obtain a second feature map containing the local texture and edge information of the cargo image.

2. The cargo area segmentation method according to claim 1, characterized in that: The edge enhancement convolution module includes a grouped convolution layer, a first batch of normalization layers, an activation layer, a first 1×1 convolution layer, a pixel difference convolution layer and a first multi-layer perceptron; each of the enhanced local Transformer modules calculates the difference between the first feature map obtained by the convolution processing of the multi-head convolution attention mechanism and the original feature map obtained by the image embedding layer, and obtains a second feature map containing the local texture and edge information of the cargo picture, including: The pixel difference convolution layer calculates the difference between the first feature map obtained by processing the original feature map through the grouped convolution layer, the first normalization layer, the activation layer, and the first 1×1 convolution layer and the original feature map spliced ​​to the pixel difference convolution layer through the residual connection, and obtains a second feature map containing local texture and edge information of the cargo image to be input into the first multi-layer perceptron.

3. The cargo area segmentation method according to claim 1 or 2, characterized in that: The local Transformer module includes a second 1×1 convolutional layer, a multi-head self-attention layer and a second multi-layer perceptron, wherein the second 1×1 convolutional layer is composed of Point-Wise convolutions; the second feature map is processed by the Point-Wise convolution of the second 1×1 convolutional layer, then processed by the multi-head self-attention layer, and finally input into the second multi-layer perceptron.

4. The cargo area segmentation method according to claim 3, characterized in that: Residual connections are added before and after the multi-head attention layer. The multi-head attention layer processes the output of the second 1×1 convolutional layer and performs residual connection processing, and finally inputs it into the second multi-layer perceptron.

5. The cargo area segmentation method according to claim 1, characterized in that: The image embedding layer includes a mean pooling layer and a third 1×1 convolutional layer.

6. The cargo area segmentation method according to claim 1, characterized in that: The initial convolution module includes a plurality of 3×3 convolution layers, each of which is followed by a second batch of normalization layers and activation layers.

7. The cargo area segmentation method according to claim 1, characterized in that: The segmentation head includes a third 1×1 convolutional layer and an upsampling layer.

8. A cargo area partitioning device, characterized in that: It includes a data acquisition unit and a segmentation unit; The data acquisition unit is used to acquire pictures of goods to be segmented; The segmentation unit is used to input the cargo image to be segmented into a trained segmentation model, and sequentially process it through the constructed initial convolution module, multiple enhanced local Transformer modules and segmentation head processing to obtain a cargo area segmentation result; each of the enhanced local Transformer modules includes an image embedding layer, an edge enhancement convolution module and a local Transformer module, and the edge enhancement convolution module calculates the difference between a first feature map obtained by convolution processing of the multi-head convolution attention mechanism constructed by it and an original feature map obtained by processing the image embedding layer, to obtain a second feature map containing local texture and edge information of the cargo image.

9. A cargo area segmentation device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store computer programs; The processor is used to read the computer program in the memory and execute the steps of the cargo area segmentation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A readable computer program is stored thereon, and when the program is executed by a processor, the steps of the cargo area segmentation method as claimed in any one of claims 1 to 7 are implemented.

Images