Shelf map region segmentation method and device, terminal equipment and computer medium

Abstract

Embodiments of the present disclosure disclose a shelf map region segmentation method, device, terminal equipment and computer medium. A specific implementation of the method comprises: obtaining an original image; inputting the original image into a pre-determined segmentation model to obtain a target image; performing fusion post-processing on the target image to obtain a result image; pushing the result image to a target device having a display function, and controlling the target device to display the result image. The implementation further performs fusion post-processing on the target image processed by the segmentation model, improves the positioning accuracy of the effective region to be recognized in the original image and the effectiveness of hierarchical positioning, and improves the display effect of region segmentation.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, in particular to a shelf image region segmentation method and device, a terminal device and a computer medium. BACKGROUND

[0002] In the intelligent retail business scenario, artificial intelligence technology is increasingly applied. The most common one is the identification of product images. Product identification is the last link in the entire intelligent algorithm chain. Shelf image recognition is mainly applied in the identification of each layer and each product in a single shelf. Shelf images are mainly images taken by people through mobile phones or cameras installed around the shelves. The recognition of shelf images mainly involves shelf positioning and the identification of different regions in the shelf.

[0003] However, in the process of identifying and segmenting the shelf region in the actual application scenario, the following technical problems often exist:

[0004] First, the angle of the shelf image is relatively large, which causes the shelf to not appear in a standard "front view" in the image. The detection boxes of the adjacent shelf layers will have a large overlap area, resulting in inaccurate shelf image region segmentation and poor segmentation display effect.

[0005] Second, the existing image segmentation technology cannot accurately "separate" adjacent shelves, so it cannot effectively filter out the invalid shelf region in the image, and thus cannot accurately locate the layered region of the shelf, resulting in poor shelf image region segmentation effect. SUMMARY

[0006] The summary part of the present disclosure is used to introduce the concepts in a brief form, which will be described in detail in the specific embodiments part. The summary part of the present disclosure is not intended to identify the key features or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0007] Some embodiments of the present disclosure propose a shelf image region segmentation method, device, terminal device and computer medium to solve one or more of the technical problems mentioned in the background part.

[0008] In a first aspect, some embodiments of the present disclosure provide a shelf image region segmentation method, which comprises: acquiring an original image; inputting the original image into a pre-determined segmentation model to obtain a target image; performing fusion post-processing on the target image to obtain a result image; pushing the result image to a target device with display function, and controlling the target device to display the result image.

[0009] In a second aspect, some embodiments of the present disclosure provide a shelf map region segmentation apparatus, which comprises: a receiving unit configured to obtain an original image; a processing unit configured to input the original image into a predetermined segmentation model to obtain a target image; a generating unit configured to perform fusion post-processing on the target image to obtain a result image; a control unit configured to push the result image to a target device having a display function, and control the target device to display the result image.

[0010] In a third aspect, some embodiments of the present disclosure provide a terminal device, comprising: one or more processors; a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of the first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method of any one of the first aspect.

[0012] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the shelf map region segmentation method of some embodiments of the present disclosure improves the positioning accuracy of the effective region to be recognized in the original image and the effectiveness of hierarchical positioning by further performing fusion post-processing on the target image processed by the segmentation model, and improves the display effect of region segmentation. Specifically, the inventors found that the reason for the poor current shelf map region segmentation effect is that the angle of shooting the shelf image is relatively large, which causes the shelf to not present a standard "front view" in the image, and the detection boxes of the adjacent shelf layers above and below will have a large overlap area, resulting in inaccurate shelf map region segmentation and poor display effect. Based on this, first, some embodiments of the present disclosure obtain an original image. Second, the original image is input into a predetermined segmentation model to obtain a target image. Specifically, the segmentation model considers the characteristics of the shelf hierarchical structure and is trained using a sample set after distortion processing. Then, the target image is subjected to fusion post-processing to obtain a result image. Through fusion post-processing, the region between the layer region and the shelf in the target image output by the model can be filtered out which does not belong to the shelf region to be recognized. Finally, the result image is pushed to a target device having a display function, and the target device is controlled to display the result image. This method applies a segmentation model specially suitable for shelf maps to region segment the original image, and applies a fusion post-processing method to merge the segmented shelf region, which can improve the segmentation accuracy of the shelf region and improve the display effect of the result image after segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0013] The above and other features, aspects and advantages of various embodiments of the present disclosure will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings, in which like reference numerals refer to like elements throughout. The drawings are not necessarily to scale, the emphasis instead being placed upon illustrating the principles of the embodiments of the present disclosure.

[0014] Figure 1 is an architectural diagram of an exemplary system to which some embodiments of the present disclosure can be applied;

[0015] Figure 2 is a flowchart of some embodiments of a shelf map region segmentation method according to the present disclosure;

[0016] Figure 3 is a flowchart of a spatial self-attention module according to some embodiments of the present disclosure;

[0017] Figure 4 is a flowchart of a channel self-attention module according to some embodiments of the present disclosure;

[0018] Figure 5 is a flowchart of one embodiment of a training step for training a segmentation model according to the present disclosure;

[0019] Figure 6 is a flowchart of some embodiments of a shelf map region segmentation apparatus according to the present disclosure;

[0020] Figure 7 is a structural schematic diagram of a terminal device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly and completely understood. It should be understood that the drawings of the present disclosure are only for illustrative purposes and should not be construed as limiting the scope of protection of the present disclosure.

[0022] It should also be noted that, for the sake of brevity, only the parts of the drawings that are relevant to the present disclosure are shown. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0023] It should be noted that the terms “first”, “second”, and the like in the present disclosure are only used to distinguish different devices, modules or units, and do not imply the order or interdependence of the functions performed by these devices, modules or units.

[0024] It should be noted that the modification of "one", "multiple" mentioned in the present disclosure is illustrative but not restrictive, and those skilled in the art should understand that unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0025] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0026] Figure 1 An exemplary system architecture 100 of an embodiment of the shelf map region segmentation method of the present disclosure is shown.

[0027] As Figure 1 shown, the system architecture 100 can include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0028] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as information processing applications, image segmentation applications, data analysis applications, etc.

[0029] The terminal devices 101, 102, 103 can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various terminal devices with display screens, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the terminal devices listed above. They can be implemented as multiple software or software modules (such as for providing raw image input, etc.), or as a single software or software module. No specific limitation is made here.

[0030] The server 105 can be a server that provides various services, such as a server that stores the raw image input by the terminal devices 101, 102, 103, etc. The server can process the received raw image and feed back the processing result (such as the result image) to the terminal device.

[0031] It should be noted that the shelf map region segmentation method provided by the embodiments of the present disclosure can be executed by the server 105 or by the terminal device.

[0032] It should be noted that the local of the server 105 can also directly store the original image, and the server 105 can directly extract the local original image to obtain a result image after processing. In this case, the exemplary system architecture 100 can not include the terminal devices 101, 102, and 103 and the network 104.

[0033] It should also be noted that the terminal devices 101, 102, and 103 can also be installed with a shelf map region segmentation application. In this case, the processing method can also be executed by the terminal devices 101, 102, and 103. In this case, the exemplary system architecture 100 can not include the server 105 and the network 104.

[0034] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules (for example, to provide a shelf map region segmentation service), or as a single software or software module. No specific limitation is made herein.

[0035] It should be understood that Figure 1 The number of terminal devices, networks, and servers in

[0036] With reference to Figure 2 , a flow 200 of some embodiments of a shelf map region segmentation method according to the present disclosure is shown. The shelf map region segmentation method includes the following steps:

[0037] Step 201, obtaining an original image.

[0038] In some embodiments, the execution subject (for example, the server shown in Figure 1 ) of the shelf map region segmentation method obtains the original image. Specifically, the original image can be a shelf image in a supermarket. The shelf image is mainly an image captured by a mobile phone or a camera installed around the shelf by a person. Therefore, it is difficult to guarantee that the captured image will not have a region of an adjacent shelf.

[0039] Step 202, inputting the original image into a pre-determined segmentation model to obtain a target image.

[0040] In some embodiments, the execution subject inputs the original image into a pre-determined segmentation model to obtain a target image.

[0041] Optionally, the predetermined segmentation model comprises an encoder, a dual-stream self-attention module, and a decoder. The dual-stream self-attention module comprises a spatial self-attention module, a channel self-attention module, and a fusion module. Specifically, the predetermined segmentation model can be regarded as a fully convolutional neural network comprising the dual-stream self-attention module. Specifically, the encoder can be an encoder of the image segmentation model, and the task of the encoder is to learn the image features of the input image through the neural network after receiving the input image. The spatial self-attention module comprises a convolution module, a matrix multiplication module, a Softmax operator, and an output module. Specifically, Figure 3 A flowchart of the spatial self-attention module is shown. As shown in Figure 3 , the image features obtained by the encoder are determined as the input of the spatial self-attention module. The input is subjected to convolution processing in steps 301 and 302 to obtain features A and B. Then, the transpose of A is input into the matrix multiplication module to obtain the spatial attention map M (step 303), and the Softmax operation is performed on M in the last dimension (step 304). Then, B and M are subjected to matrix multiplication processing to obtain the feature W (step 305), and finally the feature W is added to the corresponding position of the input to obtain the output. Specifically, Figure 4 A flowchart of the channel self-attention module is shown. As shown in Figure 4 , the image features obtained by the encoder are determined as the input of the channel self-attention module. The input of the module is subjected to average pooling processing in steps 401 and 402 to perform downsampling operation, which can greatly reduce the computational complexity, to obtain features A and B. Then, the feature A and the transpose of A are input into the matrix multiplication module to obtain the channel attention map M, and the Softmax operation is performed on M in the last dimension. Then, B and M are subjected to matrix multiplication to obtain the feature W, and the bilinear upsampling is performed on W to the same scale as I, and finally the output is obtained by adding the corresponding position of the input (step 403). Specifically, the spatial self-attention module and the channel self-attention module are combined in parallel, which can improve the resolution of the spatial features and the channel features at the same time to obtain the dual-stream self-attention module. The fusion module can be a module that adds the outputs of the spatial self-attention module and the channel self-attention module. Specifically, the decoder can be a decoder of the image segmentation model, and the task of the decoder is to gradually realize the class labeling of each pixel according to the output of the dual-stream self-attention module, that is, to obtain the target image after segmentation. Specifically, the value of each pixel in the target image is the probability value of the pixel belonging to each class. The classes can include the layer region, the gap region, and the background region. The pixel value can represent that the probability of the current pixel belonging to the layer region is 0.7, the probability of the current pixel belonging to the gap region is 0.2, and the probability of the current pixel belonging to the background region is 0.1. Therefore, the pixel has a high probability of belonging to the layer region.

[0042] Optionally, the original image is input into a predetermined segmentation model to obtain the target image, including: inputting the original image into an encoder to obtain pre-processed image features. The pre-processed image features are input into a double-flow self-attention module to obtain post-processed image features. Optionally, the pre-processed image features are input into a spatial self-attention module to obtain pre-processed spatial features. The pre-processed image features are input into a channel self-attention module to obtain pre-processed channel features. The pre-processed spatial features and the pre-processed channel features are input into a fusion module to obtain the post-processed image features. The post-processed image features are input into a decoder to obtain the target image.

[0043] In step 203, the target image is subjected to fusion post-processing to obtain a result image.

[0044] In some embodiments, the execution subject described above subjects the target image to fusion post-processing to obtain a result image.

[0045] Optionally, a set of gap regions is extracted from the target image, wherein the set of gap regions includes a first number of gap regions. Specifically, the gap region can be a gap region between adjacent shelves of a shelf image. Specifically, a first threshold value and a second threshold value can be determined according to historical experience, and the first threshold value and the second threshold value are used to determine the layer region of the shelf and the gap region between the shelves. Let the first threshold value be Tc and the second threshold value be Tf. For each pixel point in the target image, the probability value of the pixel point belonging to the layer region is determined as Pc, and the probability value of the pixel point belonging to the gap region is determined as Pf. Each pixel point in the target image is marked as a background region pixel point. For each pixel point in the target image, in response to the pixel point satisfying Pc>Tc and Pc>Pf, the pixel point is marked as a layer region pixel point, and in response to the pixel point satisfying Pf>Tf and Pf>Pc, the pixel point is marked as a gap region pixel point. The set of background region pixel points, layer region pixel points and gap region pixel points is determined as the set of gap regions.

[0046] The gap region set is fitted to obtain a split line region set, wherein the split line region set includes a second number of split line regions. Specifically, the least square method can be used to process each gap region in the gap region set to obtain the split line region set. The least square method can connect the gap regions belonging to the same straight line to form a split line region to obtain the split line region set. The straight line obtained by fitting the gap can separate the layer regions. The straight line can form a plurality of region bands, and each layer region belongs to a region band, which facilitates subsequent determination of which layer regions belong to the target shelf to be identified. Based on the result image and the split line region set, a process result image is generated. The process result image is filtered to obtain a result image. Specifically, small layer regions are filtered out. According to the width of each layer region, layer regions with a short width are filtered out. A filtering threshold Tw is determined, which can be set to 0.25*W, where W is the width of the target image and Tw is the filtering threshold. Layer regions with a width less than Tw are filtered out. The average height of all layer regions is calculated and denoted as CH. A height threshold Th is determined, which is 0.15*CH, where Th represents the height threshold. Layer regions with a height less than Th are filtered out. Finally, layer regions in the middle region band can be retained as the effective region of the target shelf to be identified. According to the plurality of region bands separated by the straight line, it can be generally considered that the target shelf to be identified is usually in the central region of the image, so the shelf in the middle region band is taken as the target shelf to be identified to obtain the result image.

[0047] The optional content in steps 202-203 above, i.e., "introducing a double-flow self-attention module and fusion post-processing to improve the shelf map segmentation effect", is an invention point of an embodiment of the present disclosure, which solves the second technical problem mentioned in the background art that "the existing image segmentation technology cannot accurately'separate' adjacent shelves, so it cannot effectively filter out the invalid shelf region in the image, thus cannot accurately locate the layered region of the shelf, resulting in poor shelf map region segmentation effect". The reason why the existing image segmentation technology cannot accurately segment the shelf map region is that the shelf map is different from general images, with complex background and numerous lines, making it difficult to distinguish the position of the target shelf. If the above factors are solved, the effect of improving the shelf map segmentation level can be achieved. To achieve this effect, the embodiment of the present disclosure first proposes a segmentation model that fuses a double-flow self-attention module, which combines a spatial self-attention module and a channel self-attention module in parallel, which can improve the resolution of spatial features and channel features at the same time. Secondly, further post-processing is taken for the target image output by the segmentation model, which can fuse the lines in the target image and distinguish adjacent shelf regions to accurately locate and segment the shelf map, thereby outputting the effective region of the target shelf to be identified, accurately locating the layered region of the shelf, and thus improving the shelf map region segmentation effect.

[0048] In step 204, the result image is pushed to a target device with a display function, and the target device is controlled to display the result image.

[0049] In some embodiments, the execution subject described above pushes the result image to a target device with a display function, and controls the target device to display the result image. The target terminal device can be a device in communication connection with the execution subject described above. The target terminal device can be a "mobile phone" or a "computer". Specifically, the target terminal device can directly display the result image, accurately positioning the region of the to-be-identified shelf, and laying a foundation for subsequent article detection and article identification.

[0050] Figure 2 An embodiment given has the following beneficial effects: obtaining an original image; inputting the original image into a predetermined segmentation model to obtain a target image; performing fusion post-processing on the target image to obtain a result image; pushing the result image to a target device with a display function, and controlling the target device to display the result image. This implementation further performs fusion post-processing on the target image processed by the segmentation model, improves the positioning accuracy of the effective region to be identified in the original image and the effectiveness of hierarchical positioning, and improves the display effect of region segmentation.

[0051] With reference to Figure 5 , a flow 500 of one embodiment of a training step of the predetermined segmentation model according to the present disclosure is shown. The training step can include the following steps:

[0052] In step 501, a sample set is obtained.

[0053] In some embodiments, the execution subject of the training step can be the same as or different from the execution subject of the shelf map region segmentation method (e.g., the terminal device shown in Figure 1 If the same, the execution subject of the training step can store the model structure information and the parameter values of the model parameters of the trained segmentation model locally after training the segmentation model. If different, the execution subject of the training step can send the model structure information and the parameter values of the model parameters of the trained segmentation model to the execution subject of the shelf map region segmentation method after training the segmentation model.

[0054] In some embodiments, the execution subject of the training step can obtain the sample set from other terminal devices connected to the execution subject network locally or remotely. The samples in the sample set include sample original images and sample target images corresponding to the sample original images. Optionally, before obtaining the sample set, the method further includes obtaining an initial sample set. The initial sample set includes a third number of initial sample original images and initial sample target images. Specifically, the initial sample original images are historical collected shelf images, and the initial sample target images are segmentation result images corresponding to the historical collected shelf images. For each initial sample in the initial sample set, the initial sample original image of the initial sample is distorted to generate a set of distorted sample original image sets and a set of distorted sample target image sets corresponding to the initial sample original image, so as to obtain the set of distorted sample original image sets and the set of distorted sample target image sets. Specifically, for each initial sample in the initial sample set, the initial sample original image of the initial sample is distorted by using a perspective transformation method to generate a set of distorted sample original images corresponding to the initial sample original image. Perspective transformation is a processing method of projecting imaging to a new view plane to obtain an image under the new view plane. Through perspective transformation, a fourth number of distorted sample original images under the new view plane can be obtained to obtain the set of distorted sample original images. Each distorted sample original image in the set of distorted sample original images is subjected to region segmentation processing to obtain a distorted sample target image of the distorted sample original image, and a set of distorted sample target images is generated. The set of distorted sample original image sets and the set of distorted sample target image sets are put into the initial sample set. The initial sample set is determined as the sample set.

[0055] In step 502, the model structure of the initial segmentation model is determined, and the model parameters of the initial segmentation model are initialized.

[0056] In some embodiments, the execution subject of the training step can first determine the model structure of the initial segmentation model. Optionally, the initial segmentation model includes an encoder, a dual-stream self-attention module, and a decoder.

[0057] Then, the execution subject of the training step can initialize the model parameters of the initial segmentation model. In practice, each model parameter (for example, a weight parameter and a bias parameter) of the initial segmentation model can be initialized with some different small random numbers. The "small random numbers" are used to ensure that the model will not enter a saturated state due to excessively large weights, thereby causing training failure. The "different" is used to ensure that the model can normally learn.

[0058] At step 503, the sample original image included in the sample in the sample set is input as the input of the initial segmentation model by using the machine learning method, and the target image of the sample corresponding to the input sample original image is taken as the expected output of the initial segmentation model, so as to train the prediction model.

[0059] In some embodiments, the execution subject of the training step can train the prediction model by using the machine learning method, inputting the sample original image included in the sample in the sample set as the input of the initial segmentation model, and taking the target image of the sample corresponding to the input sample original image as the expected output of the initial segmentation model.

[0060] Specifically, the sample original image of the selected sample is input into the initial segmentation model to obtain the target image of the selected sample. The target image of the selected sample is compared with the corresponding sample target image. According to the comparison result, it is determined whether the initial segmentation model reaches the preset optimization target. Specifically, the optimization target can be less than a predetermined threshold, and the optimization target can also be to reach a predetermined number of iterations. In response to determining that the initial segmentation model reaches the optimization target, the initial segmentation model is taken as the pre-trained segmentation model. Specifically, the comparison result can be determined based on the target image of the selected sample and the corresponding sample target image by using a loss function. Optionally, the loss function result is generated by using the following formula:

[0061] L = aL w + bL A

[0062]

[0063]

[0064] wherein x is a pixel in the target image of the selected sample, M is the total number of pixels in the target image of the selected sample, a and b are predetermined control parameters, specifically, a and b can take the value of 1, L w is a weighted cross-entropy loss function, w represents the weighted cross-entropy loss function, L A is an edge attention loss function, A represents the edge attention loss function, c is the class label of the pixel in the corresponding sample target image, W c is the weight of the class label corresponding to x, y x is the true class corresponding to x, p x is the prediction probability of the segmentation model for x. Specifically, the introduction of the cross-entropy loss function can assign a larger weight to a smaller number of classes, and the introduction of the edge attention loss function can pay more attention to the edge region of the connected region. In the training process, by using the combination of the edge attention loss and the cross-entropy loss, only the loss at the edge of all connected regions can be calculated.

[0065] In response to determining that the initial segmentation model is not trained completely, adjusting the related parameters in the initial segmentation model, and reselecting samples from the sample set, using the adjusted initial segmentation model as the initial segmentation model, and executing the training step again.

[0066] Figure 5 An embodiment has the beneficial effect that by introducing the edge attention loss and the cross-entropy loss function, the model obtained by training can pay more attention to the loss at the edge of the connected region in the shelf image, so as to better segment the layer region and the gap region of the shelf image, and improve the effect of shelf image segmentation.

[0067] Further reference Figure 6 , as the implementation of the above method, the present disclosure provides some embodiments of a shelf image region segmentation device, which device embodiments correspond to Figure 2 the above method embodiments, and the device can be specifically applied to various terminal devices.

[0068] As Figure 6 shown, the shelf image region segmentation device 600 of some embodiments includes a receiving unit 601, a processing unit 602, a generating unit 603, and a control unit 604. The receiving unit 601 is configured to obtain an original image. The processing unit 602 is configured to input the original image into a pre-determined segmentation model to obtain a target image. The generating unit 603 is configured to perform fusion post-processing on the target image to obtain a result image. The control unit 604 is configured to push the result image to a target device having a display function, and control the target device to display the result image.

[0069] It can be understood that the units described in the device 600 correspond to the respective steps in the method described with reference to Figure 2 the foregoing. Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 600 and the units contained therein, and will not be described here.

[0070] Reference is made to Figure 7 , which shows a structural schematic diagram of a computer system 700 of a terminal device suitable for implementing embodiments of the present disclosure. Figure 7 The terminal device shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0071] As Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701 which can perform various appropriate actions and processes in accordance with a program stored in a read only memory (ROM) 702 or a program loaded from the storage section 706 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 400 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0072] Connected to the I / O interface 705 are a storage section 706 including a hard disk and the like, and a communication section 707 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication section 707 performs communication processing via a network such as the Internet. A drive 708 is also connected to the I / O interface 705 as necessary. A removable recording medium 709 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 708 as necessary, so that a computer program read out therefrom is installed in the storage section 706 as necessary.

[0073] In particular, the processes described above with reference to the flow charts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flow charts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 707, and / or installed from the removable media 709. When the computer program is executed by the central processing unit (CPU) 701, the above-described functions defined in the methods of the present disclosure are performed. It should be noted that the computer readable medium of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. Program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the above.

[0074] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0075] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0076] The above description is only preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the application involved in the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with the technical features disclosed in the present disclosure (but not limited to) having similar functions to form technical solutions.

Claims

1. A method for segmenting a shelf diagram area, comprising: Obtain the original image; The original image is input into a predetermined segmentation model to obtain the target image; The target image is fused and processed to obtain a result image, including: A set of gap regions is extracted from the target image, wherein the set of gap regions includes a first number of gap regions. A first threshold and a second threshold are determined based on historical experience. The first threshold and the second threshold are used to determine the layer region of the shelf and the gap region between the shelf. The first threshold is denoted as Tc and the second threshold is denoted as Tf. For each pixel in the target image, the probability value of the pixel belonging to the layer region is determined as Pc, and the probability value of the pixel belonging to the gap region is determined as Pf. Each pixel in the target image is marked as a background region pixel. For each pixel in the target image, in response to the pixel satisfying Pc > Tc and Pc > Pf, the pixel is marked as a layer region pixel. In response to the pixel satisfying Pf > Tf and Pf > Pc, the pixel is marked as a gap region pixel. The set of background region pixels, layer region pixels and gap region pixels is determined as the gap region set. The set of gap regions is fitted to obtain a set of dividing line regions, wherein the set of dividing line regions includes a second number of dividing line regions. Based on the target image and the set of segmentation regions, a process result image is generated; The resulting image is filtered to remove layer regions with a width less than Tw, where Tw is a filtering threshold, and to remove layer regions with a height less than Th, where Th is a height threshold, so as to obtain the resulting image. The result image is pushed to a target device with display capabilities, and the target device is controlled to display the result image.

2. The method according to claim 1, wherein, The predetermined segmentation model is obtained through the following steps: Obtain a sample set, wherein the samples in the sample set include original sample images and sample target images corresponding to the original sample images; Determine the structure of the initial segmentation model and initialize the parameters of the initial segmentation model; Samples are selected from the sample set, and the following training steps are performed: The original image of the selected sample is input into the initial segmentation model to obtain the target image of the selected sample; The target image of the selected sample is compared with the corresponding target image of the sample; Based on the comparison results, determine whether the initial segmentation model has achieved the preset optimization objective; In response to determining that the initial segmentation model achieves the optimization objective, the initial segmentation model is determined as a pre-determined segmentation model.

3. The method according to claim 2, wherein, The predetermined segmentation model includes an encoder, a two-stream self-attention module, and a decoder. The step of inputting the original image into a pre-determined segmentation model to obtain the target image includes: The original image is input into the encoder to obtain preprocessed image features; The preprocessed image features are input into the dual-stream self-attention module to obtain post-processed image features; The post-processed image features are input into the decoder to obtain the target image.

4. The method according to claim 3, wherein, The dual-stream self-attention module includes a spatial self-attention module, a channel self-attention module, and a fusion module; as well as The step of inputting the preprocessed image features into the dual-stream self-attention module to obtain post-processed image features includes: The preprocessed image features are input into the spatial self-attention module to obtain preprocessed spatial features; The preprocessed image features are input into the channel self-attention module to obtain preprocessed channel features; The preprocessed spatial features and the preprocessed channel features are input into the fusion module to obtain the post-processed image features.

5. The method according to claim 2, wherein, The step of comparing the selected sample target image with the corresponding sample target image includes: Based on the selected target image of the sample and the corresponding sample target image, the loss function result is generated using the following formula: L=αL w +βL A Where x is the number of pixels in the target image of the selected sample, M is the total number of pixels in the target image of the selected sample, α and β are predetermined control parameters, and L w Let w represent the weighted cross-entropy loss function, and L be the weighted cross-entropy loss function. A Let A be the edge attention loss function, c be the category label of the corresponding pixel in the sample target image, and W be the edge attention loss function. c The weight of the category label corresponding to x, y x p represents the true category corresponding to x. x This represents the predicted probability of x by the segmentation model.

6. The method according to claim 2, wherein, Before obtaining the sample set, the process also includes: Obtain an initial sample set, wherein the initial sample set includes a third number of initial sample original images and initial sample target images; For each initial sample in the initial sample set, the original image of the initial sample is distorted to generate a set of distorted sample original images and a set of distorted sample target images corresponding to the original image of the initial sample, so as to obtain a set of distorted sample original images and a set of distorted sample target images, wherein the set of distorted sample original images includes a fourth number of distorted sample original images. The set of original distorted sample images and the set of target distorted sample images are placed into the initial sample set to obtain the sample set.

7. A shelving diagram area segmentation device, comprising: The receiving unit is configured to acquire the raw image; The processing unit is configured to input the original image into a predetermined segmentation model to obtain a target image; The generation unit is configured to perform post-fusion processing on the target image to obtain a result image, including: A set of gap regions is extracted from the target image, wherein the set of gap regions includes a first number of gap regions. A first threshold and a second threshold are determined based on historical experience. The first threshold and the second threshold are used to determine the layer region of the shelf and the gap region between the shelf. The first threshold is denoted as Tc and the second threshold is denoted as Tf. For each pixel in the target image, the probability value of the pixel belonging to the layer region is determined as Pc, and the probability value of the pixel belonging to the gap region is determined as Pf. Each pixel in the target image is marked as a background region pixel. For each pixel in the target image, in response to the pixel satisfying Pc > Tc and Pc > Pf, the pixel is marked as a layer region pixel. In response to the pixel satisfying Pf > Tf and Pf > Pc, the pixel is marked as a gap region pixel. The set of background region pixels, layer region pixels and gap region pixels is determined as the gap region set. The set of gap regions is fitted to obtain a set of dividing line regions, wherein the set of dividing line regions includes a second number of dividing line regions. Based on the target image and the set of segmentation regions, a process result image is generated; The process result image is filtered to remove layer regions with a width less than Tw, where Tw is a filtering threshold, and to remove layer regions with a height less than Th, where Th is a height threshold, to obtain the result image; the control unit is configured to push the result image to a target device with display function, and to control the target device to display the result image.

8. A terminal device, comprising: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Images