An image segmentation method, device, equipment and computer readable storage medium

Abstract

The application provides an image segmentation method, device and equipment and a computer readable storage medium. The image segmentation method comprises: acquiring imaging information of a to-be-detected box to obtain a to-be-segmented image; performing initial plate surface segmentation on the to-be-segmented image to obtain an initial plate surface segmentation result, wherein the initial plate surface segmentation result comprises segmentation information corresponding to each initial box plate surface; acquiring a splicing result of the to-be-segmented image and the initial plate surface segmentation result; based on the splicing result, performing component segmentation on each initial box plate surface to obtain a component segmentation result, wherein the component segmentation result comprises segmentation information of components on each initial box plate surface; and determining the component segmentation result as a box segmentation result corresponding to the to-be-detected box of the to-be-segmented image. Through the application, the accuracy of image segmentation can be improved.

Description

Technical Field

[0001] This application relates to image processing technology in the field of artificial intelligence, and more particularly to an image segmentation method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] In practical applications, there is often a need to inspect container bodies, such as shipping containers and cargo boxes. Container bodies are composed of different components (such as container doors, beams, and internal corner posts), and different components correspond to different quality inspection standards. Therefore, before conducting quality inspection on the container body, it is necessary to segment the container body image and then conduct quality inspection on the components on the container body according to the corresponding quality inspection standards based on the segmentation results.

[0003] Generally, to segment an image of a box, the image features are first extracted, and then segmentation is performed based on those features to obtain the segmentation result. However, in the above process of segmenting an image of a box, since the segmentation result is obtained directly based on the image features, the segmented parts are often inaccurate, resulting in low image segmentation accuracy. Summary of the Invention

[0004] This application provides an image segmentation method, apparatus, device, and computer-readable storage medium that can improve the accuracy of image segmentation.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides an image segmentation method, including:

[0007] Obtain the imaging information of the box to be detected to obtain the image to be segmented;

[0008] The image to be segmented is subjected to initial panel segmentation to obtain initial panel segmentation results, wherein the initial panel segmentation results include segmentation information corresponding to each initial box panel;

[0009] Obtain the stitching result of the image to be segmented and the initial board surface segmentation result;

[0010] Based on the splicing result, each of the initial box panels is segmented into components to obtain component segmentation results, wherein the component segmentation results include segmentation information of the components on each of the initial box panels;

[0011] The component segmentation result is determined as the box segmentation result of the image to be segmented corresponding to the box to be detected.

[0012] This application provides an image segmentation apparatus, including:

[0013] The image acquisition module is used to acquire the imaging information of the box to be detected and obtain the image to be segmented;

[0014] The initial segmentation module is used to perform initial panel segmentation on the image to be segmented to obtain initial panel segmentation results, wherein the initial panel segmentation results include segmentation information corresponding to each initial box panel.

[0015] The result stitching module is used to obtain the stitching result of the image to be segmented and the initial panel segmentation result;

[0016] The component segmentation module is used to segment each of the initial box panels based on the splicing result to obtain the component segmentation result, wherein the component segmentation result includes the segmentation information of the components on each of the initial box panels;

[0017] The image segmentation module is used to determine the segmentation result of the component as the box segmentation result of the image to be segmented corresponding to the box to be detected.

[0018] In this embodiment of the application, the initial segmentation module is further configured to extract features of the image to be segmented to obtain panel features to be segmented; based on the panel features to be segmented, perform box edge segmentation on the image to be segmented to obtain box edges; based on the panel features to be segmented, perform panel region segmentation on the image to be segmented to obtain initial panel regions; and determine the box edges and the initial panel regions as the initial panel segmentation result.

[0019] In this embodiment of the application, the initial segmentation module is further configured to perform box contour segmentation on the image to be segmented based on the features of the board surface to be segmented, to obtain the box contour; and to obtain the background region in the image to be segmented based on the box contour.

[0020] In this embodiment of the application, the image segmentation module is further configured to determine the segmentation result of the background region and the component as the box segmentation result of the image to be segmented corresponding to the box to be detected.

[0021] In this embodiment, the component segmentation module is further configured to extract features from the splicing result to obtain features of the component to be segmented; based on the features of the component to be segmented, perform component region segmentation on each initial box panel to obtain component regions; based on the features of the component to be segmented, perform component edge segmentation on each initial box panel to obtain component edges; determine the component regions and component edges as segmentation information of the component on each initial box panel; and combine the segmentation information of the component on each initial box panel to form the component segmentation result.

[0022] In this embodiment of the application, the component segmentation module is further configured to extract features of the splicing result to obtain semantic features of the component to be segmented and edge features of the component to be segmented; and to determine the semantic features of the component to be segmented and the edge features of the component to be segmented as the features of the component to be segmented.

[0023] In this embodiment, the initial segmentation module is further configured to train the region segmentation model: acquire region segmentation samples, wherein the region segmentation samples include a first box image sample and a first initial board surface annotation region; use the region segmentation model to be trained to segment the board surface region of the first box image sample to obtain a first initial board surface prediction region and a prediction feature vector corresponding to the first initial board surface prediction region; based on the difference between the first initial board surface prediction region and the first initial board surface annotation region, and the difference between the annotation feature vector corresponding to the first initial board surface annotation region and the prediction feature vector, perform backpropagation in the region segmentation model to be trained to obtain the region segmentation model.

[0024] In this embodiment, the initial segmentation module is further configured to train the initial segmentation model: obtain initial segmentation samples, wherein the initial segmentation samples include a second box image sample, box labeled edges, a second initial board surface labeled region, and a box labeled contour; use the initial segmentation model to be trained to perform initial segmentation on the second box image sample to obtain box estimated edges, a second initial board surface estimated region, and a box estimated contour; obtain the edge difference between the box estimated edges and the box labeled edges, obtain the board surface difference between the second initial board surface estimated region and the second initial board surface labeled region, and obtain the contour difference between the box estimated contour and the box labeled contour; based on the edge difference, the board surface difference, and the contour difference, perform backpropagation in the initial segmentation model to be trained to obtain the initial segmentation model.

[0025] In this embodiment, the initial segmentation module is further configured to: determine contour weights negatively correlated with the estimated ratio of contours to non-contours in the second box image sample; obtain non-box estimated contours based on the estimated box contours, and obtain non-box labeled contours based on the box labeled contours; obtain an initial contour difference between the estimated box contours and the box labeled contours, and obtain an initial non-contour difference between the non-box estimated contours and the non-box labeled contours; and combine the fusion result between the contour weights and the initial contour difference, and the fusion result between the non-contour weights and the initial non-contour difference, into the contour difference.

[0026] In this embodiment of the application, the second initial board surface estimation region includes at least one sub-initial board surface estimation region, and the second initial board surface annotation region includes at least one sub-initial board surface annotation region; the initial segmentation module is further configured to obtain at least one sub-board surface difference between the at least one sub-initial board surface estimation region and the at least one sub-initial board surface annotation region; and fuse at least one of the sub-board surface differences to obtain the board surface difference.

[0027] In this embodiment of the application, the image segmentation device further includes a quality inspection module, which is used to perform quality inspection on the box to be inspected based on the box segmentation result; and to determine the defect location information in the quality inspection result based on the box segmentation result.

[0028] In this embodiment of the application, the image segmentation device further includes a post-processing module for denoising the box segmentation result to obtain the target box segmentation result.

[0029] In this embodiment of the application, the quality inspection module is further configured to perform quality inspection on the box to be inspected based on the target box segmentation result; and to determine the defect location information in the quality inspection result based on the target box segmentation result.

[0030] In this embodiment of the application, the image acquisition module is further configured to acquire first imaging information of the box to be detected to obtain a color image to be segmented; acquire second imaging information of the box to be detected to obtain a depth image to be segmented; and determine the color image to be segmented and the depth image to be segmented as the image to be segmented.

[0031] This application provides an image segmentation device, including:

[0032] Memory, used to store executable instructions;

[0033] The processor, when executing executable instructions stored in the memory, implements the image segmentation method provided in the embodiments of this application.

[0034] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the image segmentation method provided in this application.

[0035] The embodiments of this application have at least the following beneficial effects: by utilizing the positional relationship between the box panel and the components on the panel, the imaging information of the box to be detected is first initially segmented, and then the components are segmented based on the initial segmentation result. In this way, the imaging information of the box to be detected is segmented step by step, so that the segmentation information of the components in the obtained box segmentation result has high robustness; therefore, the accuracy of image segmentation can be improved. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of an optional architecture of the image segmentation system provided in an embodiment of this application;

[0037] Figure 2 This is one of the embodiments provided in this application. Figure 1 A schematic diagram of the server's structure;

[0038] Figure 3 This is an optional flowchart illustrating the image segmentation method provided in an embodiment of this application;

[0039] Figure 4 This is a schematic diagram of an exemplary image to be segmented provided in an embodiment of this application;

[0040] Figure 5 This is a schematic diagram of an exemplary box panel provided in an embodiment of this application;

[0041] Figure 6 This is a schematic diagram of an exemplary initial panel segmentation result provided in an embodiment of this application;

[0042] Figure 7 This is a schematic diagram of an exemplary box segmentation result provided in an embodiment of this application;

[0043] Figure 8 This is another optional flowchart illustrating the image segmentation method provided in the embodiments of this application;

[0044] Figure 9 This is a schematic diagram of an exemplary box edge provided in an embodiment of this application;

[0045] Figure 10 This is a schematic diagram of an exemplary box outline provided in an embodiment of this application;

[0046] Figure 11 This is another optional flowchart illustrating the image segmentation method provided in the embodiments of this application;

[0047] Figure 12 This is a schematic diagram of an exemplary container component provided in an embodiment of this application;

[0048] Figure 13 This is a schematic diagram of an exemplary container component taken from different shooting angles, provided in an embodiment of this application.

[0049] Figure 14 This is a flowchart of an exemplary image segmentation method provided in an embodiment of this application;

[0050] Figure 15This is a schematic diagram of another exemplary image to be segmented provided in an embodiment of this application;

[0051] Figure 16 This is a schematic diagram of an exemplary initial panel segmentation result provided in an embodiment of this application;

[0052] Figure 17 This is a schematic diagram of an exemplary first convolutional neural network provided in an embodiment of this application;

[0053] Figure 18 This is a schematic diagram of an exemplary second-level convolutional neural network provided in an embodiment of this application;

[0054] Figure 19 This is a schematic diagram illustrating an exemplary component segmentation result provided in an embodiment of this application;

[0055] Figure 20 This is a schematic diagram of an exemplary cascaded neural network provided in an embodiment of this application;

[0056] Figure 21 This is a schematic diagram illustrating an exemplary post-processing method provided in an embodiment of this application;

[0057] Figure 22 This is a schematic diagram of an exemplary image segmentation provided in an embodiment of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0060] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0061] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0062] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0063] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0064] 1) Artificial Intelligence (AI) is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. Therefore, AI is a comprehensive technology within computer science that aims to capture the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. In other words, AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0065] 2) Image segmentation refers to the process of dividing an image into several specific regions with unique properties. It is a key step between image processing and image analysis. In this embodiment, image segmentation technology is applied to the image to be segmented corresponding to the box to be detected.

[0066] 3) Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It primarily determines how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Furthermore, machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence. Its applications are widespread across all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning.

[0067] 4) Deep learning (DL) is a method of learning representations of data in machine learning, including supervised learning and unsupervised learning; among them, deep neural networks (DNNs) are the foundation of deep learning.

[0068] 5) Convolutional Neural Networks (CNNs) are a class of feedforward neural networks that include convolutional computations and have a deep structure. They are one of the representative algorithms of deep learning.

[0069] 6) A cascaded neural network structure, which is a neural network structure formed by connecting at least two independent neural network structures; wherein the input of the downstream neural network structure is the output of the upstream neural network; in this embodiment of the application, the box segmentation result can be obtained through the cascaded neural network structure.

[0070] 7) High-Resolution Network (HR-net) is a convolutional neural network structure in the field of deep learning. When extracting high-level semantic information, it also preserves high-resolution edge detail feature maps. Thus, at the output of the high-resolution network, it can better preserve the edge detail information in the image and improve the feature extraction effect on multi-scale information. In the embodiments of this application, the features of the parts to be segmented can be obtained through the high-resolution network.

[0071] 8) A shipping container is a steel box made according to standardized specifications and used to transport various goods. Because of its standardized format, it is easy to stack it layer by layer, so it can be placed in large quantities on specially designed ocean-going ships for cargo transportation.

[0072] 9) Container decking refers to the decking that enables assembly into a container. It consists of six decking sections: the bottom deck, door deck, front deck, left deck, right deck, and top deck. These decking sections are connected and reinforced by beams or corner posts.

[0073] 10) Container components, which are used to assemble containers, such as door panels, upper door beams, lower door beams, left and right panels (left and right panels), front panel, top panel, inner floor panel, inner panel, bottom beam, beam, corner fittings, corner posts, lower left and right side beams, lower front beam, forklift ramp, bottom panel, inner corner posts, and inner door panel; it is easy to see that container components include the panels of the container.

[0074] Generally, to segment an image of a box, the image features are first extracted, and then segmentation is performed based on those features to obtain the segmentation result. However, in the above process of segmenting an image of a box, since the segmentation result is obtained directly based on the image features, the segmented parts are often inaccurate, resulting in low image segmentation accuracy.

[0075] Based on this, embodiments of this application provide an image segmentation method, apparatus, device, and computer-readable storage medium, which can improve the accuracy of box segmentation results. The following describes exemplary applications of the image segmentation device provided in this application. The image segmentation device provided in this application can be implemented as various types of terminals such as smartphones, smartwatches, laptops, tablets, desktop computers, smart TVs, set-top boxes, smart in-vehicle devices, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices, and smart speakers, or it can be implemented as a server. The following will describe exemplary applications when the device is implemented as a server.

[0076] See Figure 1 , Figure 1 This is a schematic diagram of an optional architecture of the image segmentation system provided in this application embodiment; as shown... Figure 1 As shown, to support an image segmentation application, in the image segmentation system 100, terminals (terminals 200-1 and 200-2 are shown exemplarily) connect to a server 400 (image segmentation device) via a network 300. The network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both. Additionally, the image segmentation system 100 includes a database 500 for providing data support to the server 400; and... Figure 1 The example shown illustrates a scenario where the database 500 is independent of the server 400. However, the database 500 can also be integrated into the server 400, and this embodiment does not limit this to any particular case.

[0077] Terminal 200-1 is used to capture images of the box 600 to be inspected, and obtain images to be segmented; the images to be segmented are sent to server 400 via network 300.

[0078] Server 400 is used to receive the image to be segmented sent by terminal 200-1 via network 300; perform initial panel segmentation on the image to be segmented to obtain initial panel segmentation results, wherein the initial panel segmentation results include segmentation information corresponding to each initial box panel; obtain the stitching result of the image to be segmented and the initial panel segmentation results; based on the stitching result, perform component segmentation on each initial box panel to obtain component segmentation results, wherein the component segmentation results include segmentation information of components on each initial box panel; and determine the component segmentation results as the box segmentation results corresponding to the box to be detected in the image to be segmented. It is also used to send the box segmentation results to terminal 200-2 via network 300.

[0079] Terminal 200-2 is used to receive the cabinet segmentation results sent by server 400 via network 300 and display the cabinet segmentation results.

[0080] In some embodiments, server 400 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal 200 may be a smartphone, tablet, laptop, desktop computer, smart speaker, or smartwatch, but is not limited thereto. Terminals and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0081] See Figure 2 , Figure 2 This is one of the embodiments provided in this application. Figure 1 A schematic diagram of the server's structure. Figure 2 The server 400 shown includes at least one processor 410, memory 450, at least one network interface 420, and a user interface 430. The various components in the server 400 are coupled together via a bus system 440. It can be understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0082] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0083] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0084] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0085] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0086] In some embodiments of this application, memory 450 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, as illustrated below.

[0087] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0088] The network communication module 452 is used to reach other computer devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, Wi-Fi, and Universal Serial Bus (USB), etc.

[0089] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with user interface 430 (e.g., a display screen, a speaker, etc.).

[0090] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0091] In some embodiments of this application, the image segmentation apparatus provided in this application can be implemented in software. Figure 2 An image segmentation apparatus 455 stored in memory 450 is shown. This apparatus can be software in the form of programs and plug-ins, and includes the following software modules: an image acquisition module 4551, an initial segmentation module 4552, a result stitching module 4553, a component segmentation module 4554, an image segmentation module 4555, a quality detection module 4556, and a post-processing module 4557. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0092] In some other embodiments of this application, the image segmentation apparatus provided in this application can be implemented in hardware. As an example, the image segmentation apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the image segmentation method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0093] The image segmentation method provided in this application will be described below with reference to exemplary applications and implementations of the image segmentation device provided in the embodiments of this application.

[0094] See Figure 3 , Figure 3 This is an optional flowchart illustrating an image segmentation method provided in an embodiment of this application; wherein, the image segmentation method is applied in an image segmentation device, and combines... Figure 3 The steps shown are explained.

[0095] S301. Obtain the imaging information of the box to be detected to obtain the image to be segmented.

[0096] In this embodiment, the image segmentation device acquires the imaging information of the box to be inspected, thus obtaining the image to be segmented. This can be achieved by the image segmentation device acquiring an image of the box to be inspected; or by a device independent of the image segmentation device acquiring an image of the box to be inspected, obtaining the image to be segmented, and then sending the image to the image segmentation device to obtain the image to be segmented; etc. This embodiment does not limit the scope of the application.

[0097] It should be noted that the image acquisition device can be a handheld device such as a smartphone or camera, or a fixed device such as a fixed camera; this application does not limit this. However, it is understood that handheld devices are convenient to carry, compact, and easy to use, and are widely used in people's daily lives. By taking pictures of the box to be inspected using a handheld device, and then realizing the identification of the box's components, the imaging cost can be greatly reduced, and the applicability is wide.

[0098] It should also be noted that the container to be inspected is the container to be subjected to quality inspection, and the container to be inspected includes various components, the quality inspection standards of each component can be different or the same; the container to be inspected can be various containers such as shipping containers, loading boxes, trolley cases, packaging boxes, container houses and wooden boxes, and the container to be inspected can be containers of various shapes such as cubes, cuboids and tetrahedrons, which are not limited in this application embodiment.

[0099] For example, see Figure 4 , Figure 4 This is a schematic diagram of an exemplary image to be segmented provided in an embodiment of this application; as shown... Figure 4 As shown, when the container to be detected is a shipping container, the image to be segmented, 4-1, is the imaging information of the container to be detected.

[0100] S302. Perform initial panel segmentation on the image to be segmented to obtain initial panel segmentation results, wherein the initial panel segmentation results include the segmentation information corresponding to each initial box panel.

[0101] In this embodiment of the application, since the box to be detected includes multiple box panels, and each box panel includes specific components, and the position information of the specific components on the box panel is usually fixed; therefore, the image segmentation device performs step-by-step segmentation on the obtained image to be segmented according to the appearance characteristics of the box to be detected; the image segmentation device first performs the first level of segmentation: segmenting the image to be segmented for the box panels, that is, performing initial panel segmentation on the image to be segmented, and obtaining the initial panel segmentation result including the segmentation information corresponding to each initial box panel.

[0102] It should be noted that the initial box panel is a predicted box panel obtained by segmenting the initial panel; the segmentation information corresponding to each initial box panel can be at least one of the region information and edge information corresponding to the box panel.

[0103] For example, see Figure 5 , Figure 5 This is a schematic diagram of an exemplary box panel provided in an embodiment of this application; as shown... Figure 5 As shown, the six sides of the container body panel 5-1 are the door panel 5-11, floor panel 5-12, left panel 5-13, right panel 5-14, front panel 5-15, and top panel 5-16. Among them, the left panel 5-13 and the right panel 5-14 are quite similar in shape.

[0104] For example, see Figure 6 , Figure 6 This is a schematic diagram of an exemplary initial board segmentation result provided in an embodiment of this application; as shown... Figure 6 As shown, the initial panel segmentation result 6-11 in image 6-1 is... Figure 4 Obtained by initial panel segmentation.

[0105] S303. Obtain the stitching result of the image to be segmented and the initial plate segmentation result.

[0106] It should be noted that after obtaining the initial plate segmentation result, the image segmentation device uses both the initial plate segmentation result and the image to be segmented as input data for the second-level segmentation. Therefore, the image segmentation device stitches the image to be segmented and the initial plate segmentation result together to perform the second-level segmentation based on the obtained stitching result.

[0107] S304. Based on the splicing results, each initial box panel is divided into components to obtain the component segmentation results.

[0108] In this embodiment, the image segmentation device uses the stitching result as input data for the second-level segmentation; here, the second-level segmentation refers to segmenting each initial box panel into components. The component segmentation result includes segmentation information of the components on each initial box panel, and the component segmentation information is at least one of the region information and edge information corresponding to the component.

[0109] It should be noted that the enclosure panel is also a component.

[0110] S305. Determine the component segmentation result as the box segmentation result corresponding to the box to be detected in the image to be segmented.

[0111] In this embodiment of the application, the component segmentation result is the box segmentation result of the image to be segmented corresponding to the box to be detected.

[0112] For example, see Figure 7 , Figure 7 This is a schematic diagram illustrating an exemplary box segmentation result provided in an embodiment of this application; as shown... Figure 7 As shown, when the container to be inspected is a shipping container, in the container segmentation result 7-1, area 7-11 is the background area, area 7-12 is the left panel area, area 7-13 is the front panel area, area 7-14 is the beam area, area 7-15 is the front bottom beam area, area 7-16 is the left bottom beam area, area 7-17 is the corner post area, and area 7-18 is the corner fitting area.

[0113] It is understandable that image segmentation devices utilize the positional relationship information between components and obtain segmentation information of different segmentation granularities of components through cascaded segmentation (first-level segmentation and second-level segmentation). In other words, each initial plate surface is segmented first, and then the fine components on each plate surface are segmented from coarse to fine, which can improve the segmentation accuracy and thus improve the image segmentation accuracy.

[0114] See Figure 8 , Figure 8 This is another optional flowchart illustrating the image segmentation method provided in the embodiments of this application; as shown below. Figure 8 As shown in the embodiment of this application, S302 can be implemented by S3021 to S3024; that is, the image segmentation device performs initial panel segmentation on the image to be segmented to obtain the initial panel segmentation result, including S3021 to S3024. Each step is described below.

[0115] S3021. Extract the features of the image to be segmented to obtain the features of the plate surface to be segmented.

[0116] It should be noted that when the image segmentation device extracts features from the image to be segmented, it also extracts the features of the board surface to be segmented corresponding to the image to be segmented; here, the features of the board surface to be segmented are used for initial board surface segmentation.

[0117] S3022. Based on the features of the plate to be segmented, perform box edge segmentation on the image to be segmented to obtain the box edges.

[0118] It should be noted that the initial panel segmentation includes box edge segmentation; the image segmentation device performs box edge segmentation on the image to be segmented based on the features of the panel to be segmented, and the obtained box edge segmentation result is the box edge; where the box edge is the edge information of the edge of the box to be detected in the image to be segmented.

[0119] For example, see Figure 9 , Figure 9 This is a schematic diagram of an exemplary box edge provided in an embodiment of this application; as shown. Figure 9 As shown, when the container to be detected is a shipping container, the edge information 9-11 in the corresponding image 9-1 is the edge of the container.

[0120] S3023. Based on the features of the board surface to be segmented, the board surface region of the image to be segmented is segmented to obtain the initial board surface region.

[0121] It should be noted that the initial board surface segmentation also includes board surface region segmentation; the image segmentation device performs board surface region segmentation on the image to be segmented based on the features of the board surface to be segmented, and the obtained board surface region segmentation result is the initial board surface region; where the initial board surface region is the estimated region of the box surface of the box to be detected in the image to be segmented.

[0122] For example, see [link to previous article] Figure 6 The initial plate segmentation result 6-11 in image 6-1 is the initial plate region.

[0123] S3024. Determine the box edge and the initial plate area as the initial plate segmentation result.

[0124] In this embodiment of the application, the image segmentation device uses the box edge and the initial plate area obtained from the initial plate segmentation as the initial plate segmentation result.

[0125] Understandably, image segmentation equipment can segment the initial board surface area while also segmenting the edge lines of the box, thus obtaining a more refined initial board surface area segmentation result; in this way, the precision of the initial board surface area segmentation can be improved.

[0126] See also Figure 8 In this embodiment of the application, S3021 is followed by S3025 and S3026; that is, after the image segmentation device extracts the features of the image to be segmented and obtains the features of the plate surface to be segmented, the image segmentation method further includes S3025 and S3026. Each step is described below.

[0127] S3025. Based on the features of the plate to be segmented, perform box contour segmentation on the image to be segmented to obtain the box contour.

[0128] It should be noted that the initial panel segmentation also includes box contour segmentation; the image segmentation device performs box contour segmentation on the image to be segmented based on the features of the panel to be segmented, and the obtained box contour segmentation result is the box contour; where the box contour is the edge information of the outer contour of the box to be detected in the image to be segmented.

[0129] For example, see Figure 10 , Figure 10 This is a schematic diagram of an exemplary box outline provided in an embodiment of this application; as shown... Figure 10 As shown in image 10-1, when the container to be detected is a shipping container, the edge information 10-11 is the outline of the container.

[0130] S3026. Based on the box contour, obtain the background region in the image to be segmented.

[0131] It should be noted that the box contour obtained by the image segmentation device is helpful in distinguishing the box from the background in the image to be segmented, and is a detail dividing line between the main body area and the background area in the image to be segmented; thus, the image segmentation device can determine the background area in the image to be segmented based on the box contour.

[0132] It is understandable that when segmenting the initial plate area, the image segmentation device also outputs the box edges and box contours, thus achieving the segmentation of the region and the edge; therefore, it can improve the segmentation precision and thus improve the segmentation accuracy of the image to be segmented.

[0133] See also Figure 8 In this embodiment of the application, S305 can be implemented through S3051; that is, the image segmentation device determines the component segmentation result as the box segmentation result of the image to be segmented corresponding to the box to be detected, including S3051, which will be described below.

[0134] S3051. The background area and component segmentation results are determined as the box segmentation results of the image to be segmented corresponding to the box to be detected.

[0135] It should be noted that the background region refers to the background area in the image to be segmented, and the component segmentation result refers to the segmentation result in the main body region of the image to be segmented; thus, the background region and the box segmentation result together constitute the segmentation result of the image to be segmented.

[0136] See also Figure 8 In this embodiment of the application, S304 can be implemented by S3041 to S3045; that is, the image segmentation device performs component segmentation on each initial box panel based on the stitching result to obtain the component segmentation result, including S3041 to S3045. The steps are described below.

[0137] S3041. Extract the features of the splicing result to obtain the features of the component to be segmented.

[0138] It should be noted that the image segmentation device extracts features from the stitching result, and the extracted features are the features of the component to be segmented corresponding to the stitching result; here, the features of the component to be segmented are used for component segmentation.

[0139] S3042. Based on the features of the components to be segmented, segment the component regions of each initial box panel to obtain the component regions.

[0140] It should be noted that component segmentation includes component region segmentation; the image segmentation device performs component region segmentation on each initial box panel based on the features of the component to be segmented, and the obtained component region segmentation result is the component region; the component region is the area occupied by each component that makes up the box to be detected in the image to be segmented. For example, Figure 7 Regions 7-11 to 7-18 are all component regions.

[0141] S3043. Based on the features of the component to be segmented, perform component edge segmentation on each initial box panel to obtain the component edge.

[0142] It should be noted that component segmentation also includes component edge segmentation; the image segmentation device segments the component edges of each initial box panel based on the features of the component to be segmented, and the obtained component edge segmentation result is the component edge.

[0143] S3044. Determine the component area and component edge as the segmentation information of the component on each initial box panel.

[0144] It should be noted that the segmentation information of the components on each initial box panel includes the component area and component edge.

[0145] S3045. Combine the segmentation information of the components on each initial box panel into a component segmentation result.

[0146] It should be noted that after the image segmentation device obtains the segmentation information of the components on each initial box plate, it combines the segmentation information of at least one component on the initial box plate corresponding to the image to be segmented, thus obtaining the component segmentation result.

[0147] It is understandable that image segmentation devices can segment not only the component region but also the component edge during the final component segmentation. Therefore, when the component region and component edge are used together as the segmentation result, the fineness of the image style can be improved and the accuracy of image segmentation can be improved.

[0148] In this embodiment, S3041 can be implemented through S30411 and S30412; that is, the image segmentation device extracts the features of the stitching result to obtain the features of the component to be segmented, including S30411 and S30412. Each step is described below.

[0149] S30411. Extract the features of the splicing result to obtain the semantic features and edge features of the component to be segmented.

[0150] In this embodiment of the application, the features extracted by the image segmentation device from the stitching result include semantic features of the component to be segmented and edge features of the component to be segmented; wherein, the semantic features of the component to be segmented refer to the high-level semantic features of the stitching result; the edge features of the component to be segmented refer to the low-level edge features of the stitching result, which are a kind of detail features.

[0151] It should be noted that image segmentation devices can obtain features of the stitching results through high-resolution network models. These high-resolution network models extract high-level semantic features layer by layer while retaining high-resolution low-level features, providing data support for the recognition of detailed information. This improves the segmentation effect of components and obtains accurate edge information.

[0152] S30412. The semantic features and edge features of the component to be segmented are determined as the features of the component to be segmented.

[0153] In this embodiment of the application, the features of the component to be segmented include semantic features of the component to be segmented and edge features of the component to be segmented.

[0154] It is understandable that when image segmentation devices extract features from the stitched results, they not only acquire the high-level semantic features of the stitched results but also the low-level edge features of the stitched results, resulting in stronger expressive power of the final segmented component features. Consequently, when segmenting components based on the features of the segmented components, the precision and accuracy of image segmentation can be improved.

[0155] In the embodiments of this application, S3021 and S3023 can be implemented through a network model; that is, the image segmentation device extracts the features of the image to be segmented to obtain the features of the board surface to be segmented, and the image segmentation device performs board surface region segmentation on the image to be segmented based on the features of the board surface to be segmented to obtain the initial board surface region, which is implemented through a region segmentation model; wherein, the region segmentation model is trained through S30271 to S30273, and each step is described below.

[0156] S30271. Obtain region segmentation samples, wherein the region segmentation samples include a first box image sample and a first initial plate surface annotation region.

[0157] It should be noted that the image segmentation device obtains training samples for training the region segmentation model, which in turn obtains region segmentation samples; among them, the first box image sample refers to the imaging information of the first box sample to be detected, and the first initial plate surface annotation area refers to the estimated plate surface area annotated in the first box image sample.

[0158] S30272. The first box image sample is segmented into a plate area using the region segmentation model to be trained, so as to obtain the first initial plate area prediction region and the prediction feature vector corresponding to the first initial plate area prediction region.

[0159] It should be noted that the image segmentation device obtains the training region segmentation model by initializing the network model; here, the image segmentation device uses the training region segmentation model to segment the surface region of the first box image sample, and the obtained surface region segmentation result is the first initial estimated surface region; at the same time, the image segmentation device can also obtain the estimated feature vector corresponding to the first initial estimated surface region; where the estimated feature vector is the feature corresponding to the first initial estimated surface region.

[0160] S30273. Based on the difference between the first initial board surface estimated region and the first initial board surface labeled region, and the difference between the labeled feature vector corresponding to the first initial board surface labeled region and the estimated feature vector, backpropagation is performed in the region segmentation model to be trained to obtain the region segmentation model.

[0161] In this embodiment, the image segmentation device adjusts the parameters of the region segmentation model to be trained from two aspects to train the model and obtain a region segmentation model. One aspect is the difference between the estimated region and the labeled region on the first initial board surface; the second aspect is the difference between the labeled feature vector and the estimated feature vector corresponding to the labeled region on the first initial board surface. The adjustment of the parameters of the region segmentation model is achieved through backpropagation. Here, the region segmentation model refers to the trained region segmentation model.

[0162] Understandably, during the training process, the image segmentation device obtains the feature vectors corresponding to each type of predicted board area while acquiring the predicted board area, thus obtaining the predicted feature vectors. Then, based on the predicted feature vectors, it helps to distinguish the feature differences between each predicted board area, improves the distinguishability between each predicted board area, and makes the segmentation accuracy of the region segmentation model higher.

[0163] In this embodiment of the application, the box edge, the initial plate area and the box outline are all obtained through the initial segmentation model; and the initial segmentation model is trained through S30281 to S30284. The steps are described below.

[0164] S30281. Obtain initial segmentation samples, wherein the initial segmentation samples include second box image samples, box labeled edges, second initial plate labeled area and box labeled contour.

[0165] It should be noted that the image segmentation device acquires training samples used to train the initial segmentation model, thus obtaining the initial segmentation samples; among them, the second box image sample refers to the imaging information of the second box sample to be detected, the box labeled edge refers to the edge information of the edge of the second box sample to be detected in the second box image sample, the second initial plate surface labeled area refers to the estimated plate surface area labeled in the second box image sample, and the box labeled contour refers to the edge information of the contour of the second box sample to be detected in the second box image sample.

[0166] S30282. Using the initial segmentation model to be trained, the second box image sample is initially segmented to obtain the box estimated edge, the second initial plate surface estimated region and the box estimated contour.

[0167] In this embodiment, the image segmentation device obtains an initial segmentation model to be trained through model parameter initialization. This initial segmentation model is a network model to be trained for initial segmentation. Here, the image segmentation device uses the initial segmentation model to perform initial segmentation on the second box image sample, obtaining a three-branch output: box predicted edges, second initial board surface predicted region, and box predicted contour. Specifically, the box predicted edges refer to the edge information of the predicted edges of the second box sample to be detected in the second box image sample; the second initial board surface predicted region refers to the predicted board surface region in the second box image sample; and the box predicted contour refers to the edge information of the predicted contour of the second box sample to be detected in the second box image sample.

[0168] S30283. Obtain the edge difference between the estimated edge of the box and the marked edge of the box, obtain the plate surface difference between the estimated area of ​​the second initial plate surface and the marked area of ​​the second initial plate surface, and obtain the contour difference between the estimated contour of the box and the marked contour of the box.

[0169] S30284. Based on edge differences, surface differences, and contour differences, backpropagation is performed in the initial segmentation model to be trained to obtain the initial segmentation model.

[0170] In this embodiment, the image segmentation model adjusts the parameters of the initial segmentation model to be trained from three aspects to obtain the initial segmentation model. These aspects include: firstly, the edge difference between the estimated edges of the box and the labeled edges of the box; secondly, the surface difference between the estimated region of the second initial board and the labeled region of the second initial board; and thirdly, the contour difference between the estimated contour of the box and the labeled contour of the box. The adjustment of the parameters of the initial segmentation model to be trained is achieved through backpropagation. Here, the initial segmentation model is the initial segmentation model to be trained after training.

[0171] Understandably, since the initial segmentation model is a multi-branch network structure, it can identify the corresponding edge lines of the region while outputting the segmented region, so that more refined edge results can be obtained when learning high-level semantic features, which can improve the initial segmentation effect; and, as a backbone structure of the network, the initial segmentation can improve the efficiency of the initial segmentation.

[0172] It should be noted that S30271 to S30273 describe the process of improving the accuracy of the estimated board area obtained by feature vector enhancement; S30281 to S30284 describe the process of improving the accuracy of the estimated board area obtained by multi-branch output. It is easy to see that in practical applications, S30271 to S30273 can be combined with S30281 to S30284 to further improve the accuracy of the estimated board area obtained by four branches (feature vector, estimated board area, box edge and box contour).

[0173] In this embodiment of the application, the image segmentation device in S30283 obtains the contour difference between the estimated contour of the box and the marked contour of the box, which can be achieved through S302831 to S302834. Each step is described below.

[0174] S302831. Based on the estimated ratio of contours to non-contours in the second box image sample, determine the contour weights that are negatively correlated with the estimated ratios and the non-contour weights that are positively correlated with the estimated ratios.

[0175] It should be noted that, for the training of the contour branch, since the proportion of contours in the box image samples is relatively small, while the proportion of non-contours in the box image samples is relatively large, the image segmentation device sets the contour weights corresponding to contours to be negatively correlated with the estimated proportion, and sets the non-contour weights corresponding to non-contours to be positively correlated with the estimated proportion, in order to increase the contour weights and decrease the non-contour weights.

[0176] For example, when the estimated ratio is 0.1, the contour weight is determined to be 1 and the non-contour weight is 0.1.

[0177] S302832. Obtain the non-box estimated contour based on the box estimated contour, and obtain the non-box labeled contour based on the box labeled contour.

[0178] In this embodiment of the application, the area in the second box image sample other than the estimated box contour is the non-estimated box contour; the area in the second box image sample other than the box labeled contour is the non-labeled box contour.

[0179] S302833. Obtain the initial contour difference between the estimated contour of the box and the marked contour of the box, and obtain the initial non-contour difference between the estimated contour of the non-box and the marked contour of the non-box.

[0180] It should be noted that the initial contour difference refers to the difference between the estimated contour of the box and the marked contour of the box, while the initial non-contour difference refers to the difference between the estimated contour of the non-box and the marked contour of the non-box.

[0181] S302834. Combine the fusion result of contour weight and initial contour difference, and the fusion result of non-contour weight and initial non-contour difference, into contour difference.

[0182] In the embodiments of this application, the image segmentation device fuses the obtained contour weights with the obtained initial contour differences, and fuses the obtained non-contour weights with the obtained non-initial contour differences. Then, the two fused weights are combined and fused again to obtain the final contour differences.

[0183] In this embodiment of the application, the second initial board surface estimation region includes at least one sub-initial board surface estimation region, and the second initial board surface annotation region includes at least one sub-initial board surface annotation region; the image segmentation device in S30283 obtains the board surface difference between the second initial board surface estimation region and the second initial board surface annotation region, which can be achieved through S302835 and S302836. The steps are described below.

[0184] S302835. Obtain at least one sub-plane difference between at least one sub-initial plane estimated region and at least one sub-initial plane marked region.

[0185] It should be noted that at least one sub-initial board surface prediction region corresponds one-to-one with at least one sub-initial board surface annotation region. Here, the image segmentation device obtains the difference between a sub-initial board surface prediction region and the sub-initial board surface annotation region corresponding to the sub-initial board surface prediction region, thus obtaining a sub-board surface difference. Therefore, for at least one sub-initial board surface prediction region and at least one sub-initial board surface annotation region, at least one sub-board surface difference can be obtained.

[0186] S302836. Merge at least one sub-panel surface difference to obtain the panel surface difference.

[0187] It should be noted that since the size differences between the various panels of the box are not significant, when the image segmentation device fuses the differences of at least one sub-panel to obtain the differences, the weights of the differences between the at least one sub-panel can be the same or different.

[0188] It is understandable that image segmentation devices can reduce the impact of significant differences in pixel count on segmentation accuracy by setting contour weights, non-contour weights, and weights corresponding to at least one sub-plane based on the region size and contour size of various planes in the sample image. In this way, they can improve the precision of image segmentation.

[0189] See Figure 11 , Figure 11 This is another optional flowchart illustrating the image segmentation method provided in the embodiments of this application; as shown Figure 11 As shown in the embodiment of this application, S305 is followed by S306 and S307; that is, after the image segmentation device determines the component segmentation result as the box segmentation result corresponding to the box to be detected in the image to be segmented, the image segmentation method further includes S306 and S307. Each step will be described below.

[0190] S306. Based on the box segmentation results, perform quality inspection on the box to be inspected.

[0191] S307. Based on the box segmentation results, determine the defect location information in the quality inspection results.

[0192] It should be noted that the results of the enclosure segmentation provide a prerequisite for subsequent quality inspection of different components and reporting of defect locations.

[0193] In this embodiment of the application, S305 is followed by S308; that is, after the image segmentation device determines the component segmentation result as the box segmentation result corresponding to the box to be detected in the image to be segmented, the image segmentation method further includes S308. Each step is described below.

[0194] S308. Denoise the box segmentation results to obtain the target box segmentation results.

[0195] It should be noted that the box segmentation results obtained by the image segmentation device may contain misidentified components; therefore, the image segmentation device performs noise reduction processing on the box segmentation results based on the positional relationship between the components that make up the box to be detected, and the noise-reduced box segmentation results are the target box segmentation results.

[0196] Accordingly, in this embodiment of the application, the image segmentation device in S306 performs quality inspection on the box to be inspected based on the box segmentation result, including: the image segmentation device performs quality inspection on the box to be inspected based on the target box segmentation result.

[0197] Accordingly, in this embodiment of the application, the image segmentation device in S307 determines the defect location information in the quality inspection result based on the box segmentation result, including: the image segmentation device determines the defect location information in the quality inspection result based on the target box segmentation result.

[0198] It should be noted that after the image segmentation device completes the denoising of the box segmentation result and obtains the target box segmentation result, quality inspection and defect location information are determined based on the target box segmentation result.

[0199] In this embodiment, S301 can be implemented through S3011 to S3013; that is, the image segmentation device acquires the imaging information of the box to be detected and obtains the image to be segmented, including S3011 to S3013. Each step is described below.

[0200] S3011. Obtain the first imaging information of the box to be detected, and obtain the color image to be segmented.

[0201] S3012. Obtain the second imaging information of the box to be detected to obtain the depth image to be segmented.

[0202] S3013. The color image to be segmented and the depth image to be segmented are determined as the images to be segmented.

[0203] It should be noted that the image to be segmented includes at least one of a color image to be segmented and a depth image to be segmented; wherein, the color image to be segmented refers to a three-channel color image, and the depth image to be segmented is used to characterize the depth information of the box to be detected.

[0204] Understandably, combining the color image and depth image of the box to be inspected for image segmentation can improve the accuracy of image segmentation.

[0205] In this embodiment, before S302, there is a process in which the image segmentation device preprocesses the image to be segmented based on image processing technology, so that the initial segmentation object in S302 is the preprocessed image to be segmented.

[0206] Understandably, combining deep learning image segmentation and image processing techniques to identify components from the imaging information of the box to be inspected is highly robust; moreover, it can identify complex backgrounds and a wide variety of components, thus improving the accuracy of image segmentation.

[0207] The following describes an exemplary application of the embodiments of this application in a real-world application scenario. This exemplary application describes a real-world application scenario for quality inspection of containers.

[0208] It should be noted that a shipping container is a standardized steel box used to load various goods. Due to its uniform format and ease of mechanized loading and unloading, it is widely used in various transportation scenarios such as ports and railways. Quality control of shipping containers is a crucial aspect of their continuous and long-term use. If any component of the container is damaged or bulges outward, it will affect the container's usability and the successful transport of goods. Furthermore, different container components correspond to different quality control standards. Therefore, container quality control includes quality inspection of each component. The large number of quality inspections and the complex process result in low efficiency. Here, the image segmentation method provided in this application is applied to container quality control to improve the efficiency of container quality control and the accuracy of container component segmentation, thereby improving the overall inspection quality of the container.

[0209] It should also be noted that although a shipping container comprises hundreds of components, the perspective for capturing images varies when these components are segmented, and the components come in a variety of sizes and colors. Furthermore, the shapes of the components are diverse; for example, corner fittings and cam seats on doors are intricate and small, while the left and right panels, floor, and doors have larger surfaces, and the corner posts and beams are slender. (See also...) Figure 12 , Figure 12 This is a schematic diagram of an exemplary container component provided in an embodiment of this application; as shown... Figure 12 As shown, panel 12-11 in image 12-1 and panel 12-21 in image 12-2 are large-area components; door lock bar 12-31 in image 12-3 and corner piece 12-41 in image 12-4 are small and delicate components; corner post 12-51 in image 12-5 and beam 12-61 in image 12-6 are slender components. By using the image segmentation method provided in the embodiments of this application, the container panel is segmented first, and then the components on the panel are segmented, and the segmentation of container components can still be accurately achieved.

[0210] Furthermore, the variable shooting position and angle can lead to variations in the shape of the container in the resulting image; see also Figure 13 , Figure 13 This is a schematic diagram of an exemplary container component taken from different shooting angles, as provided in an embodiment of this application; Figure 13As shown, image 13-1 is taken from the front of the corner piece, image 13-2 is taken from the upper side of the top panel, image 13-3 is taken from the side of the door panel, image 13-4 is taken from one side of the panel, image 13-5 is taken from the upper side of the panel, and image 13-6 is taken from the other side of the panel. By using the image segmentation method provided in this application embodiment, the container panel is segmented first, and then the components on the panel are segmented, and the segmentation of the container components can still be accurately achieved.

[0211] In addition, the background area in the imaging image corresponding to the container is also quite diverse, in addition to the container area, such as the sky, the ground and other stacked container bodies; by using the image segmentation method provided in the embodiments of this application, the container panel is first segmented to obtain the outer contour of the container, and then the components on the panel are segmented, which can still accurately achieve the segmentation of container components.

[0212] See Figure 14 , Figure 14 This is a flowchart of an exemplary image segmentation method provided in an embodiment of this application; as shown... Figure 14 As shown, this exemplary image segmentation method includes:

[0213] S1401, Begin.

[0214] S1402. Acquire image (image to be detected).

[0215] It should be noted that the handheld device is used to capture images of the container to be inspected. The captured images can be color (RGB, Red, Green, Blue) images (the color image to be inspected) or depth images (the depth image to be inspected); here, the acquired image is the image captured, such as... Figure 15 As shown, Figure 15 This is a schematic diagram of another exemplary image to be segmented provided in the embodiments of this application; as shown... Figure 15 As shown, image 15-1 is the acquired image.

[0216] S1403. The image is segmented using a first-level convolutional neural network (initial segmentation model) to obtain the panel segmentation result (initial panel segmentation result).

[0217] It should be noted that since the left and right panels are difficult to distinguish at close range, they are classified as a single panel type: left and right panels. Additionally, the internal panels of the container are classified as another panel type, resulting in six panel types for the container: door panels, left and right panels, front panel, top panel, internal panel, and bottom panel. A first-level convolutional neural network is used to obtain the relevant segmentation results for these six panel types in the image.

[0218] For example, see Figure 16 , Figure 16 This is a schematic diagram of an exemplary initial board segmentation result provided in an embodiment of this application; as shown... Figure 16 As shown, regions 16-11 in image 16-1 have different fill patterns. Figure 15 The results of panel segmentation in image 15-1.

[0219] It should also be noted that the first-level convolutional neural network includes three branches: one branch outputs the outer contour (box contour), one branch outputs the board edges (box edges), and one branch outputs the board area (initial board area). Additionally, the branch outputting the board area is also used during training to output the center feature vector (estimated feature vector) for each board class. The center feature vector refers to the last set of feature maps before the network activation layer (e.g., the "softmax" layer). By using the squared difference between the center feature vector and the corresponding pixel features (labeled feature vector) for each board class as the loss value calculation data, the segmentation accuracy of the first-level convolutional neural network is improved.

[0220] For example, see Figure 17 , Figure 17 This is a schematic diagram of an exemplary first-level convolutional neural network provided in an embodiment of this application; as shown... Figure 17 As shown, the first-level convolutional neural network 17-1 uses a residual network 17-11 (e.g., "ResNet 18") to extract features from the captured image 17-2. Then, the extracted features are input into three network branches, each of which includes two convolutional layers. Each convolutional layer has a kernel size of "1*1", 270 channels, and a stride of 1. Finally, network branch 17-121 outputs the outer contour 17-31, network branch 17-122 outputs the edge of the plate 17-32, and network branch 17-123 outputs the plate area 17-33.

[0221] S1404. The second-level convolutional neural network is used to segment the board surface to obtain the component segmentation result.

[0222] It should be noted that, from the various components of the container, 18 types of components to be separated are identified: door panel, upper door beam, lower door beam, left and right panels, front panel, top panel, inner floor panel, inner panel, inner door panel, bottom beam, beam, corner fittings, corner posts, lower left and right side beams, lower front beam, forklift pallet, bottom floor panel, and inner corner posts.

[0223] For example, see Figure 18 , Figure 18This is a schematic diagram of an exemplary second-level convolutional neural network provided in an embodiment of this application; as shown... Figure 18 As shown, first, obtain Figure 17 The image 17-2, the edge of the board 17-32, and the board region 17-33 are concatenated across channels to form result 18-1. Then, the high-resolution network 18-21 in the second-level convolutional neural network 18-2 extracts features from result 18-1. Next, the extracted features are input into two network branches, each consisting of two convolutional layers with a 1x1 kernel size, 270 channels, and a stride of 1. Finally, network branch 18-221 outputs the part region 18-31, and network branch 18-222 outputs the part edge 18-32. It is evident that the features extracted using the high-resolution network 18-21 include not only high-level semantic features but also high-resolution low-level features, resulting in better recognition performance for smaller parts.

[0224] For example, see Figure 19 , Figure 19 This is a schematic diagram illustrating an exemplary component segmentation result provided in an embodiment of this application; as shown... Figure 19 As shown, edge line 19-11 in image 19-1 is the segmented component edge; region 19-21 in image 19-2 is the segmented component region.

[0225] It should also be noted that the first-level convolutional neural network and the second-level convolutional neural network are cascaded neural networks; based on Figure 17 and Figure 18 See Figure 20 , Figure 20 This is a schematic diagram of an exemplary cascaded neural network provided in an embodiment of this application; as shown... Figure 20 As shown, image 17-2 is processed by the first-level convolutional neural network 17-1 to obtain the outer contour 17-31, the board edge 17-32, and the board region 17-33; then, the stitched result 18-1 of image 17-2, board edge 17-32, and board region 17-33 is processed by the second-level convolutional neural network 18-2 to obtain the component region 18-31 and component edge 18-32; wherein, the first-level convolutional neural network 17-1 and the second-level convolutional neural network 18-2 are cascaded neural networks 20-1.

[0226] Understandably, a shipping container is a relatively regular rectangular structure with different components and shapes on each face. When identifying container components using a cascaded neural network, each level of the convolutional neural network has different network branches designed to identify the regions and edges of the container components separately. Furthermore, in the first-level convolutional neural network, the network branch is trained by acquiring the central feature vector for each type of panel, making the output panel regions of the first-level convolutional neural network more discriminative. In addition, the cascaded neural network achieves the recognition logic of first identifying the various faces of the container, and then identifying the fine components on each face, from coarse to fine, from components to edge lines, resulting in better edge segmentation of the obtained components, thereby improving the accuracy of container component recognition.

[0227] S1405. Post-process the component segmentation results to obtain the final component segmentation result (target box segmentation result).

[0228] It should be noted that in the component segmentation results, sometimes small areas of other component categories may appear in large segmented areas such as the board area and background area, which will form noise. Therefore, identifying and removing noise in the component segmentation results can improve the accuracy of the final component segmentation results.

[0229] For example, see Figure 21 , Figure 21 This is a schematic diagram illustrating an exemplary post-processing method provided in an embodiment of this application; as shown... Figure 21 As shown, image 21-1 is a color image of the container to be inspected, taken by a handheld device; in the segmentation result 21-2, region 21-211 in the background region 21-21 is missegmented as a forklift ramp; here, region 21-211 is the noise to be removed.

[0230] S1406, End.

[0231] For example, see Figure 22 , Figure 22 This is a schematic diagram illustrating an exemplary image segmentation method provided in an embodiment of this application; as shown... Figure 22As shown, images 22-11, 22-12, and 22-13 are all images obtained by handheld devices capturing images of the containers to be inspected. Specifically, image 22-21 is segmented, resulting in the component region shown in image 22-21, the component edges shown in image 22-31, the outer contour shown in image 22-41, and the container edge shown in image 22-51. Similarly, image 22-12 is segmented, resulting in the component region shown in image 22-22, the component edges shown in image 22-32, the outer contour shown in image 22-42, and the container edge shown in image 22-52. Finally, image 22-13 is segmented, resulting in the component region shown in image 22-23, the component edges shown in image 22-33, the outer contour shown in image 22-43, and the container edge shown in image 22-53.

[0232] It is understandable that shipping containers are a widely used standardized means of transporting goods, whether packaged or unpackaged, making quality monitoring a crucial aspect. The image segmentation method provided in this application intelligently and accurately segments the imaging information of shipping containers, improving the efficiency of quality monitoring. When a handheld device is used to photograph a shipping container, and various components are identified within that image, the average intersection-union ratio (IUU) of the obtained region localization reaches 87%. This provides a prerequisite for subsequent defect detection and reporting of defect locations for different components. Furthermore, the flexibility and low cost of mobile phone photography significantly enhance the feasibility and simplicity of the image segmentation method provided in this application. Additionally, the image segmentation method provided in this application, based on the appearance characteristics of the shipping container, designs a cascaded neural network structure, progressing from coarse to fine, to better utilize the positional features of different components on each panel, thereby obtaining more robust component localization information. Moreover, each level of the convolutional neural network structure incorporates multi-branch network structures at different levels, achieving detailed edge segmentation and further improving the edge segmentation effect of the components.

[0233] The following description continues to illustrate the exemplary structure of the image segmentation device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software module stored in the image segmentation device 455 in the memory 450 may include:

[0234] Image acquisition module 4551 is used to acquire imaging information of the box to be detected and obtain an image to be segmented;

[0235] The initial segmentation module 4552 is used to perform initial panel segmentation on the image to be segmented to obtain initial panel segmentation results, wherein the initial panel segmentation results include segmentation information corresponding to each initial box panel.

[0236] The result stitching module 4553 is used to obtain the stitching result of the image to be segmented and the initial board surface segmentation result;

[0237] The component segmentation module 4554 is used to segment each of the initial box panels based on the splicing result to obtain a component segmentation result, wherein the component segmentation result includes segmentation information of the components on each of the initial box panels;

[0238] The image segmentation module 4555 is used to determine the segmentation result of the component as the box segmentation result of the image to be segmented corresponding to the box to be detected.

[0239] In this embodiment of the application, the initial segmentation module 4552 is further configured to extract features of the image to be segmented to obtain features of the board surface to be segmented; based on the features of the board surface to be segmented, perform box edge segmentation on the image to be segmented to obtain box edges; based on the features of the board surface to be segmented, perform board surface region segmentation on the image to be segmented to obtain an initial board surface region; and determine the box edges and the initial board surface region as the initial board surface segmentation result.

[0240] In this embodiment of the application, the initial segmentation module 4552 is further configured to perform box contour segmentation on the image to be segmented based on the features of the board surface to be segmented, to obtain the box contour; and to obtain the background region in the image to be segmented based on the box contour.

[0241] In this embodiment of the application, the image segmentation module 4555 is further configured to determine the segmentation result of the background region and the component as the box segmentation result of the image to be segmented corresponding to the box to be detected.

[0242] In this embodiment of the application, the component segmentation module 4554 is further configured to extract features of the splicing result to obtain features of the component to be segmented; based on the features of the component to be segmented, perform component region segmentation on each initial box panel to obtain component regions; based on the features of the component to be segmented, perform component edge segmentation on each initial box panel to obtain component edges; determine the component regions and the component edges as segmentation information of the component on each initial box panel; and combine the segmentation information of the component on each initial box panel to form the component segmentation result.

[0243] In this embodiment of the application, the component segmentation module 4554 is further configured to extract features of the splicing result to obtain semantic features of the component to be segmented and edge features of the component to be segmented; and to determine the semantic features of the component to be segmented and the edge features of the component to be segmented as the features of the component to be segmented.

[0244] In this embodiment, the initial segmentation module 4552 is further configured to train the region segmentation model: acquire region segmentation samples, wherein the region segmentation samples include a first box image sample and a first initial board surface annotation region; use the region segmentation model to be trained to segment the board surface region of the first box image sample to obtain a first initial board surface estimated region and an estimated feature vector corresponding to the first initial board surface estimated region; based on the difference between the first initial board surface estimated region and the first initial board surface annotation region, and the difference between the annotation feature vector corresponding to the first initial board surface annotation region and the estimated feature vector, perform backpropagation in the region segmentation model to be trained to obtain the region segmentation model.

[0245] In this embodiment, the initial segmentation module 4552 is further configured to train the initial segmentation model: obtain initial segmentation samples, wherein the initial segmentation samples include a second box image sample, box labeled edges, a second initial board surface labeled region, and a box labeled contour; use the initial segmentation model to be trained to perform initial segmentation on the second box image sample to obtain box estimated edges, a second initial board surface estimated region, and a box estimated contour; obtain the edge difference between the box estimated edges and the box labeled edges, obtain the board surface difference between the second initial board surface estimated region and the second initial board surface labeled region, and obtain the contour difference between the box estimated contour and the box labeled contour; based on the edge difference, the board surface difference, and the contour difference, perform backpropagation in the initial segmentation model to be trained to obtain the initial segmentation model.

[0246] In this embodiment, the initial segmentation module 4552 is further configured to: determine contour weights negatively correlated with the estimated ratio of contours to non-contours in the second box image sample; obtain non-box estimated contours based on the estimated box contours, and obtain non-box labeled contours based on the box labeled contours; obtain an initial contour difference between the estimated box contours and the box labeled contours, and obtain an initial non-contour difference between the non-box estimated contours and the non-box labeled contours; and combine the fusion result between the contour weights and the initial contour difference, and the fusion result between the non-contour weights and the initial non-contour difference, into the contour difference.

[0247] In this embodiment of the application, the second initial board surface estimation region includes at least one sub-initial board surface estimation region, and the second initial board surface annotation region includes at least one sub-initial board surface annotation region; the initial segmentation module 4552 is further configured to obtain at least one sub-board surface difference between the at least one sub-initial board surface estimation region and the at least one sub-initial board surface annotation region; and fuse at least one of the sub-board surface differences to obtain the board surface difference.

[0248] In this embodiment of the application, the image segmentation device 455 further includes a quality inspection module 4556, which is used to perform quality inspection on the box to be inspected based on the box segmentation result; and to determine the defect location information in the quality inspection result based on the box segmentation result.

[0249] In this embodiment of the application, the image segmentation device 455 further includes a post-processing module 4557, which is used to perform noise reduction processing on the box segmentation result to obtain the target box segmentation result.

[0250] In this embodiment of the application, the quality inspection module 4556 is further configured to perform quality inspection on the box to be inspected based on the target box segmentation result; and to determine the defect location information in the quality inspection result based on the target box segmentation result.

[0251] In this embodiment of the application, the image acquisition module 4551 is further configured to acquire the first imaging information of the box to be detected to obtain a color image to be segmented; acquire the second imaging information of the box to be detected to obtain a depth image to be segmented; and determine the color image to be segmented and the depth image to be segmented as the image to be segmented.

[0252] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. The processor of a computer device (image segmentation device) reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image segmentation method described in this application.

[0253] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to perform the image segmentation method provided in this application. For example, ... Figure 3 The image segmentation method shown is illustrated.

[0254] In some embodiments of this application, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a device that includes one or any combination of the above-mentioned memories.

[0255] In some embodiments of this application, executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0256] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0257] As an example, executable instructions can be deployed to execute on a single computer device (in this case, the single computer device is an image segmentation device), or to execute on multiple computer devices located in one location (in this case, the multiple computer devices located in one location are image segmentation devices), or to execute on multiple computer devices distributed across multiple locations and interconnected via a communication network (in this case, the multiple computer devices distributed across multiple locations and interconnected via a communication network are image segmentation devices).

[0258] In summary, through the embodiments of this application, by utilizing the positional relationship between the box panel and the components on the panel, the imaging information of the box to be detected is first initially segmented into panel surfaces, and then component segmentation is performed based on the initial panel surface segmentation results. This achieves hierarchical segmentation of the imaging information of the box to be detected, resulting in high robustness of the component segmentation information in the obtained box segmentation results; therefore, it can improve the accuracy of image segmentation. Furthermore, using a handheld device to acquire the image to be segmented improves the executability and flexibility of image segmentation. Also, by acquiring high-level semantic features and edge detail features, the accuracy of image segmentation can be improved. In addition, each level of segmentation acquires the segmentation edge lines while acquiring the segmentation region, which can improve the fineness of image segmentation and enhance the image segmentation effect.

[0259] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An image segmentation method characterized by, The method comprises: obtaining imaging information of a to-be-detected box to obtain a to-be-segmented image; performing initial plate surface segmentation on the to-be-segmented image through a region segmentation model to obtain an initial plate surface segmentation result, wherein the initial plate surface segmentation result comprises segmentation information corresponding to each initial box plate surface, and the region segmentation model is obtained through the following steps: obtaining a region segmentation sample, wherein the region segmentation sample comprises a first box image sample and a first initial plate surface labeled region; performing plate surface region segmentation on the first box image sample through a to-be-trained region segmentation model to obtain a first initial plate surface estimated region and an estimated feature vector corresponding to the first initial plate surface estimated region; performing back propagation in the to-be-trained region segmentation model based on differences between the first initial plate surface estimated region and the first initial plate surface labeled region and differences between a labeled feature vector corresponding to the first initial plate surface labeled region and the estimated feature vector to obtain the region segmentation model; obtaining a splicing result of the to-be-segmented image and the initial plate surface segmentation result; performing component segmentation on each initial box plate surface based on the splicing result to obtain a component segmentation result, wherein the component segmentation result comprises segmentation information of components on each initial box plate surface; determining the component segmentation result as a box segmentation result corresponding to the to-be-detected box of the to-be-segmented image.

2. The method of claim 1, wherein, The method comprises: extracting features of the to-be-segmented image through a region segmentation model to obtain to-be-segmented plate surface features; performing box edge segmentation on the to-be-segmented image based on the to-be-segmented plate surface features to obtain a box edge; performing plate surface region segmentation on the to-be-segmented image based on the to-be-segmented plate surface features to obtain an initial plate surface region; determining the box edge and the initial plate surface region as the initial plate surface segmentation result.

3. The method of claim 2, wherein, After the features of the to-be-segmented image are extracted to obtain to-be-segmented plate surface features, the method further comprises: performing box contour segmentation on the to-be-segmented image based on the to-be-segmented plate surface features to obtain a box contour; obtaining a background region in the to-be-segmented image based on the box contour; The method comprises: determining the background region and the component segmentation result as the box segmentation result corresponding to the to-be-detected box of the to-be-segmented image.

4. The method according to any one of claims 1 to 3, characterized in that, The method comprises: extracting features of the splicing result to obtain to-be-segmented component features; performing component region segmentation on each initial box plate surface based on the to-be-segmented component features to obtain a component region; performing component edge segmentation on each initial box plate surface based on the to-be-segmented component features to obtain a component edge; determining the component region and the component edge as the segmentation information of components on each initial box plate surface; Combine the segmentation information of the components on each of the initial box panel surfaces into the component segmentation result.

5. The method of claim 4, wherein, The feature extraction of the spliced result obtains a component-to-be-segmented feature, comprising: extracting features of the spliced result to obtain a component-to-be-segmented semantic feature and a component-to-be-segmented edge feature; determining the component-to-be-segmented semantic feature and the component-to-be-segmented edge feature as the component-to-be-segmented feature.

6. The method of claim 2, wherein, The feature extraction of the image-to-be-segmented obtains a panel surface feature, and the panel surface region segmentation based on the panel surface feature is performed on the image-to-be-segmented to obtain an initial panel surface region, which is realized by the region segmentation model.

7. The method of claim 3, wherein, The box edge, the initial panel surface region, and the box contour are all obtained by an initial segmentation model; The initial segmentation model is obtained by the following steps: obtain an initial segmentation sample, wherein the initial segmentation sample includes a second box image sample, a box labeled edge, a second initial panel labeled region, and a box labeled contour; use a to-be-trained initial segmentation model to perform initial segmentation on the second box image sample to obtain a box estimated edge, a second initial panel estimated region, and a box estimated contour; obtain an edge difference between the box estimated edge and the box labeled edge, a panel difference between the second initial panel estimated region and the second initial panel labeled region, and a contour difference between the box estimated contour and the box labeled contour; based on the edge difference, the panel difference, and the contour difference, perform back propagation in the to-be-trained initial segmentation model to obtain the initial segmentation model.

8. The method of claim 7, wherein, The contour difference between the box estimated contour and the box labeled contour comprises: based on an estimated proportion of contours and non-contours in the second box image sample, determine a contour weight negatively correlated with the estimated proportion and a non-contour weight positively correlated with the estimated proportion; based on the box estimated contour, obtain a non-box estimated contour, and based on the box labeled contour, obtain a non-box labeled contour; obtain an initial contour difference between the box estimated contour and the box labeled contour, and an initial non-contour difference between the non-box estimated contour and the non-box labeled contour; combine a fusion result between the contour weight and the initial contour difference and a fusion result between the non-contour weight and the initial non-contour difference into the contour difference.

9. The method of claim 7, wherein, The second initial panel estimated region includes at least one sub-initial panel estimated region, and the second initial panel labeled region includes at least one sub-initial panel labeled region. The panel difference between the second initial panel estimated region and the second initial panel labeled region comprises: obtain at least one sub-panel difference between the at least one sub-initial panel estimated region and the at least one sub-initial panel labeled region; fuse at least one of the sub-panel differences to obtain the panel difference.

10. The method according to any one of claims 1 to 3, characterized in that, After the component segmentation result is determined as the box segmentation result corresponding to the to-be-detected box based on the component segmentation result, the method further includes: performing quality detection on the to-be-detected box based on the box segmentation result; determining defect position information in a quality detection result based on the box segmentation result.

11. The method of claim 10, wherein, After the component segmentation result is determined as the box segmentation result corresponding to the to-be-detected box based on the component segmentation result, the method further includes: performing denoising processing on the box segmentation result to obtain a target box segmentation result; the quality detection on the to-be-detected box based on the box segmentation result includes: performing quality detection on the to-be-detected box based on the target box segmentation result; the determination of the defect position information in the quality detection result based on the box segmentation result includes: determining the defect position information in the quality detection result based on the target box segmentation result.

12. The method according to any one of claims 1 to 3, characterized in that, The obtaining of the imaging information of the to-be-detected box to obtain a to-be-segmented image includes: obtaining first imaging information of the to-be-detected box to obtain a to-be-segmented color image; obtaining second imaging information of the to-be-detected box to obtain a to-be-segmented depth image; determining the to-be-segmented color image and the to-be-segmented depth image as the to-be-segmented image.

13. An image segmentation apparatus characterized by comprising: The method includes: an image acquisition module, configured to obtain imaging information of a to-be-detected box to obtain a to-be-segmented image; an initial segmentation module, configured to perform initial panel segmentation on the to-be-segmented image by using a region segmentation model to obtain an initial panel segmentation result, wherein the initial panel segmentation result includes segmentation information corresponding to each initial box panel, and the region segmentation model is obtained by training through the following steps: obtaining a region segmentation sample, wherein the region segmentation sample includes a first box image sample and a first initial panel labeled region; performing panel region segmentation on the first box image sample by using a to-be-trained region segmentation model to obtain a first initial panel estimated region and an estimated feature vector corresponding to the first initial panel estimated region; performing back propagation in the to-be-trained region segmentation model based on differences between the first initial panel estimated region and the first initial panel labeled region and differences between a labeled feature vector corresponding to the first initial panel labeled region and the estimated feature vector to obtain the region segmentation model; a result splicing module, configured to obtain a splicing result of the to-be-segmented image and the initial panel segmentation result; a component segmentation module, configured to perform component segmentation on each initial box panel based on the splicing result to obtain a component segmentation result, wherein the component segmentation result includes segmentation information of components on each initial box panel; an image segmentation module, configured to determine the component segmentation result as a box segmentation result corresponding to the to-be-detected box based on the to-be-segmented image.

14. The apparatus of claim 13, wherein The initial segmentation module is further configured to extract features of the image to be segmented to obtain plate surface features to be segmented; perform box edge segmentation on the image to be segmented based on the plate surface features to be segmented to obtain a box edge; perform plate surface region segmentation on the image to be segmented based on the plate surface features to be segmented to obtain an initial plate surface region; and determine the box edge and the initial plate surface region as the initial plate surface segmentation result.

15. The apparatus of claim 14, wherein, The initial segmentation module is further configured to perform box contour segmentation on the image to be segmented based on the plate surface features to be segmented to obtain a box contour; and obtain a background region in the image to be segmented based on the box contour. The image segmentation module is further configured to determine the background region and the component segmentation result as the box segmentation result of the image to be segmented corresponding to the box to be detected.

16. The apparatus of any one of claims 13 to 15, wherein, The component segmentation module is further configured to extract features of the spliced result to obtain component features to be segmented; perform component region segmentation on each of the initial box plate surfaces based on the component features to be segmented to obtain a component region; perform component edge segmentation on each of the initial box plate surfaces based on the component features to be segmented to obtain a component edge; determine the component region and the component edge as segmentation information of a component on each of the initial box plate surfaces; and combine the segmentation information of the component on each of the initial box plate surfaces as the component segmentation result.

17. The apparatus of claim 16, wherein, The component segmentation module is further configured to extract features of the spliced result to obtain component semantic features to be segmented and component edge features to be segmented; and determine the component semantic features to be segmented and the component edge features to be segmented as the component features to be segmented.

18. The apparatus of claim 15, wherein, The initial segmentation module is further configured to train the initial segmentation model by: obtaining an initial segmentation sample, wherein the initial segmentation sample includes a second box image sample, box labeled edges, a second initial plate surface labeled region, and a box labeled contour; performing initial segmentation on the second box image sample by using a to-be-trained initial segmentation model to obtain box estimated edges, a second initial plate surface estimated region, and a box estimated contour; obtaining an edge difference between the box estimated edges and the box labeled edges, a plate surface difference between the second initial plate surface estimated region and the second initial plate surface labeled region, and a contour difference between the box estimated contour and the box labeled contour; and performing back propagation in the to-be-trained initial segmentation model based on the edge difference, the plate surface difference, and the contour difference to obtain the initial segmentation model.

19. The apparatus of claim 18, wherein, The initial segmentation module is further configured to determine a contour weight negatively correlated with a predicted proportion of contours and a non-contour weight positively correlated with the predicted proportion of non-contours based on the predicted proportion of contours and non-contours in the second box image sample; acquire a non-box predicted contour based on the box predicted contour and a non-box labeled contour based on the box labeled contour; acquire an initial contour difference between the box predicted contour and the box labeled contour and an initial non-contour difference between the non-box predicted contour and the non-box labeled contour; and combine a fusion result between the contour weight and the initial contour difference and a fusion result between the non-contour weight and the initial non-contour difference as the contour difference.

20. The apparatus of claim 18, wherein, The second initial plate surface predicted region includes at least one sub-initial plate surface predicted region, and the second initial plate surface labeled region includes at least one sub-initial plate surface labeled region. The initial segmentation module is further configured to acquire at least one sub-plate surface difference between the at least one sub-initial plate surface predicted region and the at least one sub-initial plate surface labeled region; and fuse at least one of the sub-plate surface differences to obtain the plate surface difference.

21. The apparatus of any one of claims 13 to 15, wherein, The device further includes: The quality detection module is configured to perform quality detection on the to-be-detected box based on the box segmentation result; and determine defect position information in a quality detection result based on the box segmentation result.

22. The apparatus of claim 21, wherein, The device further includes: The post-processing module is configured to perform denoising processing on the box segmentation result to obtain a target box segmentation result. The quality detection module is further configured to perform quality detection on the to-be-detected box based on the target box segmentation result; and determine the defect position information in the quality detection result based on the target box segmentation result.

23. The device of any one of claims 13-15, wherein: The image acquisition module is further configured to acquire first imaging information of the to-be-detected box to obtain a to-be-segmented color image; acquire second imaging information of the to-be-detected box to obtain a to-be-segmented depth image; and determine the to-be-segmented color image and the to-be-segmented depth image as the to-be-segmented image.

24. An image segmentation apparatus characterized by comprising: including: a memory configured to store executable instructions; a processor configured to execute the executable instructions stored in the memory to implement the image segmentation method of any one of claims 1-12.

25. A computer-readable storage medium, characterized in that, executable instructions stored in the memory, configured to be executed by the processor to implement the image segmentation method of any one of claims 1-12.

26. A computer program product comprising computer instructions, characterized in that, The computer instructions are executed by the processor to implement the image segmentation method of any one of claims 1-12.

Images