Image segmentation method and device, storage medium and program product

Abstract

The invention provides an image segmentation method and device, a storage medium and a program product. The image segmentation method comprises the steps of collecting a perspective image of a vehicle; extracting a feature map of the perspective image; processing the feature map by using a detection branch model in a machine learning model, and generating a plurality of mask confidence coefficients of each target object in the perspective image; processing the feature map by using a segmentation branch model in a machine learning model to generate a plurality of mask prototype maps in one-to-one correspondence with the plurality of mask confidence coefficients; generating a target mask by using the plurality of mask confidence coefficients and the plurality of mask prototype graphs; and obtaining an instance segmentation result of each target object according to the target mask.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image segmentation method and device, a storage medium and a program product. Background Art

[0002] With the deepening development of globalization and the increasing frequency of cross-border trade, customs inspection has become an important part of ensuring national economic security and maintaining international trade order. Traditional customs inspection methods often rely on manual inspection, which is not only inefficient but also easily affected by human factors, resulting in inaccurate and unpredictable inspection results. Therefore, in order to meet the growing trade needs and ensure the accuracy and efficiency of inspection, the customs intelligent inspection system came into being.

[0003] The vehicle perspective image scanned by the customs intelligent security inspection system contains relevant image information of the vehicle and the goods it transports. During the inspection process, customs officials need to conduct different inspections on different areas of the vehicle body. Therefore, a function to distinguish vehicle information is required in the intelligent security inspection system to facilitate subsequent identification and analysis of different vehicle body areas. Summary of the invention

[0004] The inventors noticed that the vehicle perspective image segmentation method in the prior art assigns a specific category label to each pixel in the image, thereby achieving pixel-level classification of the image, which is a single-label segmentation form. However, in vehicle perspective images, there are situations where the same pixel position actually corresponds to multiple categories. In this case, the traditional segmentation method will only classify the pixels at the overlapping position into one of the multiple categories, resulting in the loss of information of other categories at this position, affecting subsequent judgments.

[0005] Accordingly, the present disclosure provides an image segmentation method. Since each pixel in the image corresponds to all categories contained in the actual situation at that position, multiple mask confidences of each target object can be obtained through a machine learning model. On this basis, the instance segmentation results of each target object can be accurately obtained, thereby effectively solving the problem of missing aliasing position information.

[0006] In a first aspect of the present disclosure, an image segmentation method is provided, comprising: acquiring a perspective image of a vehicle; extracting a feature map of the perspective image; processing the feature map using a detection branch model in a machine learning model to generate a plurality of mask confidences for each target object in the perspective image; processing the feature map using a segmentation branch model in the machine learning model to generate a plurality of mask prototype maps corresponding one-to-one to the plurality of mask confidences; generating a target mask using the plurality of mask confidences and the plurality of mask prototype maps; and obtaining an instance segmentation result of each target object based on the target mask.

[0007] In some embodiments, the generating of the target mask by using the multiple mask confidences and the multiple mask prototype images includes: multiplying the i-th mask confidence and the i-th mask prototype image to generate the i-th product, 1≤i≤K, K is the total number of mask confidences; calculating the sum of K products as a target sum value; and generating the target mask according to the target sum value.

[0008] In some embodiments, the feature map is processed using the detection branch model to generate category confidence and bounding box information for each target object.

[0009] In some embodiments, multiple objects in the perspective image are filtered to obtain a first object set as a segmentation result of the perspective image, wherein the category confidence of each object in the first object set is greater than a predetermined confidence threshold.

[0010] In some embodiments, the first target set is filtered to obtain a second target set as a segmentation result of the perspective image; wherein filtering the first target set includes: taking the target with the maximum category confidence in the first target set as a reference object, and placing the reference object into the second target; determining whether the intersection-over-union (IOU) of the bounding box of the mth target object in the first target set excluding the reference object and the bounding box of the reference object is less than a predetermined IOU threshold, 1≤m≤M-1, M is the total number of targets in the first target set; if the intersection-over-union (IOU) of the bounding box of the mth target object and the bounding box of the reference object is less than the predetermined IOU threshold, then placing the mth target object into the second target.

[0011] In some embodiments, acquiring the perspective image of the vehicle includes: scanning the vehicle using X-rays to obtain the perspective image.

[0012] In some embodiments, air regions in the fluoroscopic image are deleted so as to update the fluoroscopic image.

[0013] In some embodiments, deleting the air area in the perspective image includes: binarizing the perspective image to obtain a first connected graph of the area where the vehicle is located, and a second connected graph of the air area located outside the first connected graph; and deleting the second connected graph.

[0014] In some embodiments, deleting the air area in the perspective image includes: detecting the air value of each area in the perspective image; and deleting the area in the perspective image whose air value is greater than a predetermined air value threshold.

[0015] In some embodiments, the machine learning model is trained, wherein the training of the machine learning model includes: generating a vehicle sample image; extracting a sample feature map of the vehicle sample image; processing the sample feature map using the detection branch model in the machine learning model to generate category information, border information and multiple sample mask confidences for each sample target object in the vehicle sample image; processing the sample feature map using the segmentation branch model in the machine learning model to generate multiple sample mask prototype maps corresponding one-to-one to the multiple sample mask confidences; generating a sample target mask using the multiple sample mask confidences and the multiple sample mask prototype maps; generating a loss function based on the category information, the border information, the sample target mask and the annotation information of the vehicle sample image; and training the machine learning model based on the loss function.

[0016] In some embodiments, generating the loss function includes: generating a first sub-loss function based on the binary cross entropy of the category information and the category annotation information of each sample target object; generating a second sub-loss function based on the intersection and union ratio of the bounding box information and the bounding box annotation information of each sample target object; generating a third sub-loss function based on the binary cross entropy of the sample target mask and the mask annotation information of each sample target object; generating the loss function based on the first sub-loss function, the second sub-loss function and the third sub-loss function.

[0017] In some embodiments, the loss function is a weighted sum of the first sub-loss function, the second sub-loss function, and the third sub-loss function.

[0018] In some embodiments, generating a vehicle sample image includes: scanning the vehicle using X-rays to obtain the vehicle sample image; identifying each sample target object in the vehicle sample image; and labeling category information and bounding box coordinate information for each sample target object.

[0019] In some embodiments, air areas in the vehicle sample image are deleted so as to update the vehicle sample image.

[0020] In some embodiments, image enhancement processing is performed on the vehicle sample image.

[0021] According to a second aspect of an embodiment of the present disclosure, there is provided an image segmentation apparatus, comprising: a memory; and a processor, coupled to the memory, the processor being configured to execute an image segmentation method as described in any of the above embodiments based on instructions stored in the memory.

[0022] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the image segmentation method as described in any of the above embodiments is implemented.

[0023] According to a fourth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, the image segmentation method as described in any of the above embodiments is implemented.

[0024] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 A schematic diagram of a top view of a vehicle according to an embodiment of the present disclosure;

[0027] Figure 2 A schematic diagram of image segmentation according to an embodiment of the present disclosure;

[0028] Figure 3 A schematic diagram of image segmentation according to another embodiment of the present disclosure;

[0029] Figure 4 A schematic diagram of a flow chart of an image segmentation method according to an embodiment of the present disclosure;

[0030] Figure 5 A flowchart of a model training method according to an embodiment of the present disclosure;

[0031] Figure 6 The figure is a schematic diagram of the structure of an image segmentation device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0033] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0034] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0035] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0036] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0037] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0038] At present, the vehicle perspective image segmentation method is to assign a specific category label to each pixel in the image, thereby realizing pixel-level classification of the image, which is a single-label segmentation form. However, in vehicle perspective images, there are situations where the same pixel position actually corresponds to multiple categories. For example, Figure 1 As shown, the pixel 11 in the image may represent the chassis, the cargo, the battery, the air inside the vehicle, or other objects. In the traditional segmentation method, only the pixel 11 will be classified into one of the multiple categories, resulting in the loss of information of other categories at this position, affecting subsequent judgment.

[0039] For example, when each pixel in the vehicle perspective image corresponds to only a single label, the vehicle perspective image is segmented using the deeplabv3+ method, and the segmentation results are as follows: Figure 2 shown.

[0040] For another example, when each pixel in the vehicle perspective image corresponds to multiple mask labels, the segmentation result obtained by segmenting the vehicle perspective image is as follows: Figure 3 shown.

[0041] By comparison Figure 2 and Figure 3 It is known that in Figure 2 In , each pixel exists in only one class’s mask. Figure 3In the aliasing method, pixels in the aliasing area will exist in multiple different masks, which can effectively avoid the problem of missing aliasing position information and help obtain accurate image segmentation results.

[0042] Figure 4 The following is a flow chart of an image segmentation method according to an embodiment of the present disclosure. In some embodiments, the following image segmentation method is performed by an image segmentation device, including steps 41-46.

[0043] In step 41 , a perspective image of the vehicle is acquired.

[0044] In some embodiments, the vehicle is scanned using X-rays to obtain a perspective image.

[0045] In some embodiments, after the vehicle is scanned by X-rays to obtain a perspective image, the perspective image is further preprocessed to delete the air area in the perspective image so as to update the perspective image.

[0046] It should be noted here that by cropping the air area in the perspective image, the processing efficiency of image segmentation can be effectively improved.

[0047] For example, the air area in the perspective image can be deleted by the connected domain judgment method and the air threshold judgment method.

[0048] For example, in the connected domain judgment method, by binarizing the perspective image, a first connected graph of the area where the vehicle is located and a second connected graph of the air area outside the first connected graph are obtained, and the second connected graph is deleted, so that the updated perspective image only includes the image of the area where the vehicle is located. This method can be applied to images scanned by top-view devices.

[0049] For example, in the air threshold determination method, the air value of each area in the perspective image is detected, and the area in the perspective image with an air value greater than a predetermined air value threshold is deleted. This method can be applied to images scanned by side-view devices.

[0050] In step 42, a feature map of the perspective image is extracted.

[0051] For example, firstly, features of different scales of the perspective image are extracted, and then the features of different scales of the perspective image are fused using FPN (Feature Pyramid Network) to generate a feature map of the perspective image.

[0052] In step 43, the feature map is processed using the detection branch model in the machine learning model to generate multiple mask confidences for each target object in the perspective image.

[0053] In addition, the feature map is processed using the detection branch model to generate category confidence and bounding box information for each target object.

[0054] In some embodiments, the machine learning model is trained so that the trained machine learning model can generate multiple mask confidences for each target object in the perspective image, and multiple mask prototype images corresponding one-to-one to the multiple mask confidences.

[0055] Figure 5 This is a flowchart of a machine learning model training method according to an embodiment of the present disclosure, including steps 51-57.

[0056] In step 51 , a vehicle sample image is generated.

[0057] In some embodiments, the vehicle is scanned using X-rays to obtain a vehicle sample image, each sample target object in the vehicle sample image is identified, and category information and bounding box coordinate information are annotated for each sample target object.

[0058] For example, each sample target object in the vehicle sample image is labeled by a polygonal labeling method.

[0059] In some embodiments, air regions in the vehicle sample image are deleted to update the vehicle sample image.

[0060] For example, the air area in the perspective image can be deleted by the connected domain judgment method and the air threshold judgment method.

[0061] In some embodiments, image enhancement processing is performed on the vehicle sample image.

[0062] For example, image enhancement processing is performed by using mosaic enhancement, random perspective, and color perturbation (HSV augment) to increase the diversity of samples.

[0063] In step 52, a sample feature map of the vehicle sample image is extracted.

[0064] In step 53, the sample feature map is processed using the detection branch model in the machine learning model to generate category information, border information and multiple sample mask confidences for each sample target object in the vehicle sample image.

[0065] In step 54, the sample feature map is processed using the segmentation branch model in the machine learning model to generate a plurality of sample mask prototype maps corresponding one-to-one to a plurality of sample mask confidences.

[0066] In step 55, a sample target mask is generated using a plurality of sample mask confidences and a plurality of sample mask prototype images.

[0067] In some embodiments, the i-th sample mask confidence and the i-th sample mask prototype image are multiplied to generate the i-th product, 1≤i≤K, K is the total number of sample mask confidences. Next, the sum of the K products is calculated as a target sum value, and a target mask is generated according to the target sum value.

[0068] For example, the target and the value are processed using an activation function to generate a target mask.

[0069] For example, the activation function includes a sigmoid function.

[0070] For example, if the detection branch model outputs K sample mask confidences of the first sample target object as a1, a2, ..., aK, the segmentation branch model outputs K sample mask prototype images b1, b2, ..., bK of the first sample target object corresponding to the K sample mask confidences. In this case, the target and value S are as shown in formula (1).

[0071] S=a1× b1+a2× b2+…+aK× bK (1)

[0072] In step 56, a loss function is generated based on the category information, the bounding box information, the sample object mask and the annotation information of the vehicle sample image.

[0073] In some embodiments, the step of generating a loss function includes the following.

[0074] 1) Generate the first sub-loss function based on the binary cross entropy of the category information and category label information of each sample target object.

[0075] For example, the first sub-loss function can also be called an object category loss function.

[0076] 2) Generate the second sub-loss function based on the IOU (Intersection Over Union) of the bounding box information and the bounding box annotation information of each sample target object.

[0077] For example, the second sub-loss function can also be called a bounding box regression loss function.

[0078] 3) Generate the third sub-loss function based on the binary cross entropy of the sample target mask and mask annotation information of each sample target object.

[0079] For example, the third sub-loss function can also be called a mask classification loss function.

[0080] 4) Generate a loss function based on the first sub-loss function, the second sub-loss function and the third sub-loss function.

[0081] For example, the loss function is a weighted sum of the first sub-loss function, the second sub-loss function, and the third sub-loss function.

[0082] In step 57, the machine learning model is trained according to the loss function.

[0083] return Figure 4 In step 44, the feature map is processed using the segmentation branch model in the machine learning model to generate a plurality of mask prototypes corresponding to the plurality of mask confidences.

[0084] In step 45, a target mask is generated using a plurality of mask confidences and a plurality of mask prototype images.

[0085] In some embodiments, the i-th mask confidence and the i-th mask prototype image are multiplied to generate the i-th product, 1≤i≤K, K is the total number of mask confidences. Next, the sum of the K products is calculated as a target sum value, and a target mask is generated according to the target sum value.

[0086] For example, the target and the value are processed using an activation function to generate a target mask.

[0087] For example, the activation function includes a sigmoid function.

[0088] For example, if the detection branch model outputs K mask confidences of the first target object as a1, a2, ..., aK, the segmentation branch model outputs K mask prototype images b1, b2, ..., bK of the first target object corresponding to the K mask confidences. In this case, the target and value S are as shown in formula (2).

[0089] S=a1× b1+a2× b2+…+aK× bK (2)

[0090] In step 46, an instance segmentation result of each target object is obtained according to the target mask.

[0091] In some embodiments, after obtaining the instance segmentation result of each target object, the segmentation result is further filtered to obtain a more accurate segmentation result.

[0092] In some embodiments, multiple objects in the fluoroscopic image are filtered to obtain a first object set as a segmentation result of the fluoroscopic image, wherein the category confidence of each object in the first object set is greater than a predetermined confidence threshold.

[0093] For example, in the perspective image, it is identified that object 1 belongs to category 1, and its category confidence is 0.95, and object 2 belongs to category 1, and its category confidence is 0.55. This indicates that the category confidence of object 1 is high, while the category confidence of object 2 is low. If the predetermined confidence threshold is 0.75, object 1 is placed in the first object set for subsequent processing, and object 2 is not placed in the first object set.

[0094] In some embodiments, the first set of target objects is filtered to obtain a second set of target objects as a segmentation result of the perspective image.

[0095] For example, the step of filtering the first target object set includes the following:

[0096] 1) The target with the largest category confidence in the first target set is used as a reference, and the reference is placed in the second target.

[0097] 2) Determine whether the IOU between the bounding box of the mth target object excluding the reference object in the first target object set and the bounding box of the reference object is less than a predetermined IOU threshold, 1≤m≤M-1, and M is the total number of targets in the first target object set.

[0098] 3) If the IOU of the bounding box of the mth target object and the bounding box of the reference object is less than a predetermined IOU threshold, the mth target object is placed in the second target object.

[0099] For example, the first target set includes 5 targets, namely, targets 1 to 5, among which target 1 has the largest category confidence, so target 1 is placed in the second target. Among targets 2 to 5, the IOU of the borders of targets 2, 3, and 4 with the border of target 1 is less than the predetermined IOU threshold, which indicates that the overlap between the borders of targets 2, 3, and 4 and the border of target 1 is small, so targets 2, 3, and 4 are placed in the second target. The IOU of the border of target 5 with the border of target 1 is greater than the predetermined IOU threshold, which indicates that the overlap between the border of target 5 and the border of target 1 is large, so target 5 is not placed in the second target.

[0100] It should be noted here that since the category confidence of each target object in the second target object is greater than the predetermined confidence threshold, and the IOU of the borders between the targets is less than the predetermined IOU threshold, it can help to subsequently process the perspective image accurately and efficiently.

[0101] In the image segmentation method provided by the above embodiment of the present disclosure, a detection branch model is used to generate multiple mask confidences for each target object in the perspective image, a segmentation branch model is used to generate multiple mask prototype images corresponding to the multiple mask confidences, a target mask is generated using the multiple mask confidences and the multiple mask prototype images, and an instance segmentation result of each target object is obtained according to the target mask. In this way, accurate image segmentation of the perspective image can be achieved, and the problem of inaccurate image segmentation caused by the lack of aliasing position information can be effectively solved.

[0102] Figure 6 The figure is a schematic diagram of the structure of an image segmentation device according to an embodiment of the present disclosure.

[0103] like Figure 6 As shown in FIG. 6 , the image segmentation apparatus 60 is in the form of a general-purpose computing device. The image segmentation apparatus 60 includes a memory 61, a processor 62, and a bus 63 for connecting different system components.

[0104] The memory 61 may include, for example, a system memory, a non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs. The system memory may include a volatile storage medium, such as a random access memory (RAM) and / or a cache memory. The non-volatile storage medium may store, for example, instructions of a corresponding embodiment of at least one image segmentation method being executed. The non-volatile storage medium includes, but is not limited to, a disk memory, an optical memory, a flash memory, etc.

[0105] The processor 62 can be implemented by a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistors, etc. Discrete hardware components. Accordingly, each module such as an acquisition module, a calculation module, and an adjustment module can be implemented by a central processing unit (CPU) running instructions in a memory that execute corresponding steps, or can be implemented by a dedicated circuit that executes corresponding steps.

[0106] For example, the processor 62 is configured to execute instructions stored in the memory to implement the following Figures 4 to 5 The method of any one of the embodiments.

[0107] The bus 63 may use any of a variety of bus structures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0108] The interfaces 64, 65, 66 of the image segmentation device 60 and the memory 61 and the processor 62 can be connected via a bus 63. The input / output interface 64 can provide a connection interface for input / output devices such as a display, a mouse, and a keyboard. The network interface 65 provides a connection interface for various networked devices. The storage interface 66 provides a connection interface for external storage devices such as a floppy disk, a USB flash drive, and an SD card.

[0109] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each frame of the flowchart and / or block diagram and the combination of frames can be implemented by computer-readable program instructions.

[0110] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0111] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to work in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0112] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0113] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the following is implemented: Figures 4 to 5 The method of any one of the embodiments.

[0114] The present disclosure also provides a computer program product, including computer instructions, wherein when the computer instructions are executed by a processor, the following is achieved: Figures 4 to 5 The method of any one of the embodiments.

[0115] In some embodiments, the functional unit described above can be implemented as a general-purpose processor, a programmable logic controller (PLC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component or any appropriate combination thereof for performing the functions described in the present disclosure.

[0116] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0117] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. An image segmentation method, comprising: Acquire a perspective image of the vehicle; extracting a feature map of the perspective image; Processing the feature map using a detection branch model in a machine learning model to generate multiple mask confidences for each target object in the perspective image; Processing the feature map using a segmentation branch model in the machine learning model to generate a plurality of mask prototype maps corresponding one-to-one to the plurality of mask confidences; Generate a target mask using the multiple mask confidences and the multiple mask prototype images; According to the target mask, an instance segmentation result of each target object is obtained.

2. The image segmentation method according to claim 1, wherein: The step of generating a target mask by using the multiple mask confidences and the multiple mask prototype images comprises: Multiply the i-th mask confidence and the i-th mask prototype image to generate the i-th product, 1≤i≤K, K is the total number of mask confidences; Calculate the sum of K products as the target sum value; The target mask is generated based on the target and value.

3. The image segmentation method according to claim 1, further comprising: The feature map is processed using the detection branch model to generate category confidence and bounding box information for each target object.

4. The image segmentation method according to claim 3, further comprising: A plurality of objects in the perspective image are filtered to obtain a first object set as a segmentation result of the perspective image, wherein a category confidence of each object in the first object set is greater than a predetermined confidence threshold.

5. The image segmentation method according to claim 4, further comprising: filtering the first set of target objects to obtain a second set of target objects as a segmentation result of the perspective image; Wherein, filtering the first target object set includes: Taking the target with the largest category confidence in the first target set as a reference object, and placing the reference object into the second target set; Determine whether the IOU of the bounding box of the mth target object excluding the reference object in the first target set and the bounding box of the reference object is less than a predetermined IOU threshold, 1≤m≤M-1, M is the total number of targets in the first target set; If the intersection over union (IOU) of the border of the m-th target object and the border of the reference object is less than the predetermined IOU threshold, the m-th target object is placed in the second target object.

6. The image segmentation method according to claim 1, wherein: The collecting of the perspective image of the vehicle comprises: The vehicle is scanned by X-ray to obtain the perspective image.

7. The image segmentation method according to claim 6, further comprising: The air region in the perspective image is deleted so as to update the perspective image.

8. The image segmentation method according to claim 7, wherein: Deleting the air area in the perspective image comprises: Binarizing the perspective image to obtain a first connected graph of the area where the vehicle is located and a second connected graph of the air area outside the first connected graph; Delete the second connected graph.

9. The image segmentation method according to claim 7, wherein: Deleting the air area in the perspective image comprises: detecting the air value of each area in the perspective image; The area in the perspective image having an air value greater than a predetermined air value threshold is deleted.

10. The image segmentation method according to any one of claims 1 to 9, further comprising: Training the machine learning model, wherein the training the machine learning model comprises: Generate vehicle sample images; Extracting a sample feature map of the vehicle sample image; Processing the sample feature map using the detection branch model in the machine learning model to generate category information, border information, and multiple sample mask confidences for each sample target object in the vehicle sample image; Processing the sample feature map using the segmentation branch model in the machine learning model to generate a plurality of sample mask prototype maps corresponding one-to-one to the plurality of sample mask confidences; Generate a sample target mask using the multiple sample mask confidences and the multiple sample mask prototype images; Generate a loss function according to the category information, the frame information, the sample object mask and the annotation information of the vehicle sample image; The machine learning model is trained according to the loss function.

11. The image segmentation method according to claim 10, wherein: The generated loss function includes: Generate a first sub-loss function according to the binary cross entropy of the category information and the category label information of each sample target object; Generate a second sub-loss function according to the intersection-over-union ratio of the bounding box information and the bounding box annotation information of each sample target object; Generating a third sub-loss function according to the binary cross entropy of the sample target mask and the mask annotation information of each sample target object; The loss function is generated according to the first sub-loss function, the second sub-loss function and the third sub-loss function.

12. The image segmentation method according to claim 11, wherein: The loss function is a weighted sum of the first sub-loss function, the second sub-loss function, and the third sub-loss function.

13. The image segmentation method according to claim 10, wherein: Generating a vehicle sample image comprises: Scanning the vehicle using X-rays to obtain a sample image of the vehicle; Identify each sample target object in the vehicle sample image; Label each sample object with category information and bounding box coordinate information.

14. The image segmentation method according to claim 13, further comprising: The air area in the vehicle sample image is deleted to update the vehicle sample image.

15. The image segmentation method according to claim 14, further comprising: Perform image enhancement processing on the vehicle sample image.

16. An image segmentation device, comprising: Memory; A processor is coupled to the memory, and the processor is configured to execute the image segmentation method according to any one of claims 1 to 15 based on instructions stored in the memory.

17. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the image segmentation method according to any one of claims 1 to 15 is implemented.

18. A computer program product, comprising computer instructions, wherein when the computer instructions are executed by a processor, the image segmentation method according to any one of claims 1 to 15 is implemented.

Images