Multi-exposure image segmentation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a multi-exposure-based image segmentation method and device, electronic equipment and a storage medium, and can be widely applied to the technical field of image processing. According to the method, after the to-be-trained multi-exposure image data is acquired based on the multi-exposure time sequence and the multi-channel model is constructed, the multi-channel model is trained through the to-be-trained multi-exposure image data and the cross entropy loss function with the area weight; and image segmentation is carried out on the exposure image data to be processed through the trained multi-channel model, so that the imaging quality of an image after fusion based on an image segmentation result can be effectively improved, small target impurities are reduced, and the detection accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for image segmentation based on multiple exposures, an electronic device, and a storage medium. Background Art

[0002] In related technologies, image exposure time has an important impact on image quality. Generally, before performing image segmentation, target detection, image classification and other tasks, the exposure image fusion method is used to generate a high-quality image. However, since the image is currently exposed only once, the image quality after fusion is poor, and there are problems of information loss such as local detail loss, partial color distortion and uneven spatial details, resulting in a large variety of small target impurities and difficulty in detection.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose an image segmentation method and device based on multi-exposure, an electronic device and a storage medium, which can effectively improve the imaging quality of the fused image, reduce small target impurities, and improve detection accuracy.

[0005] To achieve the above object, an embodiment of the present application provides a multi-exposure based image segmentation method, the method comprising the following steps:

[0006] Collecting multi-exposure image data to be trained based on a multi-exposure time series;

[0007] Build multi-channel models;

[0008] Training the multi-channel model using the multi-exposure image data to be trained and a cross entropy loss function, wherein an area weight is provided in the cross entropy loss function;

[0009] Acquire exposure image data to be processed;

[0010] The image data to be processed is segmented using a trained multi-channel model.

[0011] In some embodiments, the step of acquiring the multi-exposure image data to be trained based on the multi-exposure time series includes:

[0012] determining a target exposure time in the multiple exposure time sequence and determining a target impurity type in the impurity type set;

[0013] Acquire a first pixel value array in a preset channel under a target exposure time, wherein the first pixel value array includes pixels corresponding to a target impurity type;

[0014] Obtaining a second pixel value array in the preset channel, wherein the second pixel value array includes target object pixels;

[0015] Calculating a distribution difference metric value between the first pixel value array and the second pixel value array;

[0016] Reacquiring a first pixel value array according to the distribution difference measure value until all the impurity types in the impurity type set correspond to the first pixel value array;

[0017] All the first pixel value arrays are used as the multi-exposure image data to be trained.

[0018] In some embodiments, calculating the distribution difference measure of the first pixel value array and the second pixel value array includes:

[0019] The KL divergence of the first pixel value array and the second pixel value array is calculated, and the calculation formula of the KL divergence is as follows:

[0020]

[0021] In the formula, KL(P||C) represents the KL divergence, p i,j (x) represents the first pixel value array on the jth channel corresponding to the i-th target impurity type, and represents the second pixel value array on the j-th channel.

[0022] In some embodiments, taking all the first pixel value arrays as the multi-exposure image data to be trained includes:

[0023] Preprocessing the first pixel value array, wherein the preprocessing includes denoising and smoothing;

[0024] Performing type labeling on the image corresponding to the first pixel value array after preprocessing;

[0025] The labeled first pixel value array is used as the multi-exposure image data to be trained.

[0026] In some embodiments, the multi-channel model includes an input module, a first normalization processing branch, a second normalization processing branch, a merging module, a segmentation network module, and an output module;

[0027] The input end of the input module is used to receive exposure image data, and the output end of the input module is connected to the input end of the first normalization processing branch and the input end of the second normalization processing branch respectively;

[0028] The output end of the first normalization processing branch and the output end of the second normalization processing branch are both connected to the input end of the merging module;

[0029] The output end of the merging module is connected to the input end of the output module, and the output end of the output module is used to output the segmentation result.

[0030] In some embodiments, the first normalization processing branch is used to perform normalization processing on all exposure image data of the corresponding channel of the input exposure image data;

[0031] The second normalization processing branch is used to perform normalization processing on all the input exposure image data.

[0032] In some embodiments, the calculation formula of the cross entropy loss function is as follows:

[0033]

[0034] In the formula, WeightedLoss represents the training loss value, N represents the total number of multi-exposure image data to be trained during the training process, H represents the total number of types of multi-exposure image data to be trained, and y i,h represents the true type label of the multi-exposure image data i to be trained, p i,h represents the predicted probability that the multi-exposure image data i to be trained belongs to type h, ω i,h Represents the area weight.

[0035] To achieve the above object, another aspect of an embodiment of the present application provides an image segmentation device based on multiple exposures, the device comprising:

[0036] The first module is used to collect multi-exposure image data to be trained based on a multi-exposure time series;

[0037] The second module is used to build a multi-channel model;

[0038] A third module is used to train the multi-channel model using the multi-exposure image data to be trained and a cross entropy loss function, wherein an area weight is provided in the cross entropy loss function;

[0039] The fourth module is used to obtain the exposure image data to be processed;

[0040] The fifth module is used to perform image segmentation on the exposure image data to be processed by using a trained multi-channel model.

[0041] To achieve the above object, another aspect of an embodiment of the present application provides an electronic device, including:

[0042] at least one processor;

[0043] at least one memory for storing at least one program;

[0044] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0045] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0046] The embodiments of the present application include at least the following beneficial effects: The present application provides a multi-exposure based image segmentation method and device, electronic device and storage medium, which collects multi-exposure image data to be trained based on a multi-exposure time series and constructs a multi-channel model, then trains the multi-channel model through the multi-exposure image data to be trained and a cross-entropy loss function with an area weight, and then performs image segmentation on the exposure image data to be processed through the trained multi-channel model, thereby effectively improving the imaging quality of the image after fusion based on the image segmentation results, reducing small target impurities and improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of a multi-exposure based image segmentation method provided in an embodiment of the present application;

[0048] Figure 2 is a schematic diagram of a multi-channel model provided in an embodiment of the present application;

[0049] Figure 3 It is a schematic diagram of the experimental results provided in the embodiments of the present application;

[0050] Figure 4 is a structural schematic diagram of an image segmentation device based on multi-exposure provided in an embodiment of the present application;

[0051] Figure 5 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application, they are only examples of devices and methods consistent with some aspects of the embodiments of the present application.

[0053] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0054] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0056] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0057] Image segmentation refers to dividing an image into several non-intersecting regions based on grayscale, color, spatial texture, geometric shape and other features, so that these features show consistency or similarity in the same region, but show obvious differences between different regions. Simply put, it is to separate the target from the background in an image.

[0058] Multi-exposure refers to exposing the same frame image multiple times and saving images taken with different exposures.

[0059] Batch Normalization (BN for short) normalizes each feature on a small batch of data, that is, the mean and variance of each feature are calculated on each sample in a small batch, and then the feature of each sample is normalized.

[0060] Layer Normalization (LN for short) normalizes all features of a single sample, that is, a separate mean and variance are calculated for all features of each sample, and all features of the sample are normalized.

[0061] In related technologies, image exposure time has an important impact on image quality. Generally, before performing image segmentation, target detection, image classification and other tasks, the exposure image fusion method is used to generate a high-quality image. However, since the image is currently exposed only once, the image quality after fusion is poor, and there are problems of information loss such as local detail loss, partial color distortion and uneven spatial details, resulting in a large variety of small target impurities and difficulty in detection.

[0062] In view of this, an embodiment of the present application provides an image segmentation method and device based on multi-exposure, an electronic device and a storage medium. After collecting multi-exposure image data to be trained based on a multi-exposure time series and constructing a multi-channel model, the multi-channel model is trained using the multi-exposure image data to be trained and a cross-entropy loss function with an area weight. The trained multi-channel model is then used to perform image segmentation on the exposure image data to be processed, thereby effectively improving the imaging quality of the image after fusion based on the image segmentation results, reducing small target impurities and improving detection accuracy.

[0063] The image segmentation method based on multiple exposures provided in the embodiment of the present application relates to the field of image processing technology. The image segmentation method based on multiple exposures provided in the embodiment of the present application can be applied to a terminal, can be applied to a server, or can be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements an image segmentation method based on multiple exposures, etc., but is not limited to the above forms.

[0064] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0065] Figure 1 is an optional flowchart of the multi-exposure based image segmentation method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S110 to S150:

[0066] Step S110, collecting multi-exposure image data to be trained based on the multi-exposure time series;

[0067] Step S120, constructing a multi-channel model;

[0068] Step S130, training the multi-channel model using the multi-exposure image data to be trained and a cross entropy loss function, wherein an area weight is provided in the cross entropy loss function;

[0069] Step S140, obtaining exposure image data to be processed;

[0070] Step S150: performing image segmentation on the exposure image data to be processed using the trained multi-channel model.

[0071] In the embodiment of the present application, the process of acquiring the multi-exposure image data to be trained based on the multi-exposure time series may include but is not limited to the following steps:

[0072] Step S210, determining a target exposure time in a multi-exposure time sequence, and determining a target impurity type in an impurity type set;

[0073] Step S220, obtaining a first pixel value array in a preset channel under a target exposure time, wherein the first pixel value array includes pixels corresponding to a target impurity type;

[0074] Step S230, obtaining a second pixel value array in a preset channel, wherein the second pixel value array includes target object pixels;

[0075] Step S240, calculating a distribution difference measurement value between the first pixel value array and the second pixel value array;

[0076] Step S250, reacquiring the first pixel value array according to the distribution difference metric value, until all impurity types in the impurity type set correspond to the first pixel value array;

[0077] Step S260: taking all first pixel value arrays as multi-exposure image data to be trained.

[0078] For example, taking KL divergence as the distribution difference metric and taking a picture of tea leaves as an example, assuming that the multi-exposure time series is E, the data collection process for a scene is as follows:

[0079] Step 1: Count all possible impurity types in tea leaves, recorded as impurity type set Z.

[0080] Step 2: Initially set the exposure time to a us. The multi-exposure time sequence is defined as E, and the initial state is empty.

[0081] Step 3: Photograph all impurities in the impurity type set Z and extract the pixel value set P of the target area. i,j ∈P represents z i ∈Z is the pixel value array (first pixel value array) of the i-th target impurity type captured in the j-th channel at the current exposure time; the value range of j is 0, 1, 2, representing the three channels of R, G, and B. j (second pixel value array).

[0082] Step 4: Calculate p i,j ∈P and c j When the KL divergence is greater than the threshold t, the impurity set Z removes the i-th element, and the current exposure time is added to the exposure time sequence E. The formula for KL divergence is:

[0083]

[0084] In the formula, KL(P||C) represents the KL divergence, p i,j (x) represents the first pixel value array on the jth channel corresponding to the i-th target impurity type, represents the second pixel value array on the j-th channel;

[0085] Step 5: Increase the exposure time by s us, and repeat steps 3 and 4 until the impurity type set Z is an empty set or the exposure time reaches b us.

[0086] After completing the shooting of one scene, each of the remaining scenes is shot in sequence using the exposure time in the multi-exposure time sequence E. The image group shot for each scene is merged into one data. A total of 1000 data are shot and recorded as a set D consisting of multi-exposure image data to be trained. For the data in set D, this embodiment can perform a preprocessing including denoising and smoothing on the first pixel value array, and then perform type annotation on the image corresponding to the preprocessed first pixel value array. Specifically, each image in the set D contains tea leaves and impurities at the same time, each type of impurity appears at least 200 times, and the image is no longer taken after an impurity or tea leaf is shot. When the data in set D meets the above requirements, set D is divided into a training set, a validation set, and a test set in a ratio of 7:2:1 to train, validate, and test the multi-channel model.

[0087] In this embodiment, the distribution difference measurement value may also be measured by using a U test, a kernel density estimation, or the like, which may be specifically selected according to actual conditions.

[0088] It is understandable that if Figure 2 As shown, the multi-channel model includes an input module, a first normalization processing branch, a second normalization processing branch, a merging module, a segmentation network module and an output module. The input end of the input module is used to receive exposure image data, and the output end of the input module is connected to the input end of the first normalization processing branch and the input end of the second normalization processing branch respectively; the output end of the first normalization processing branch and the output end of the second normalization processing branch are both connected to the input end of the merging module; the output end of the merging module is connected to the input end of the output module, and the output end of the output module is used to output the segmentation result.

[0089] It can be understood that the first normalization processing branch is used to perform BatchNorm on the data to better preserve the size relationship between different layers; the second normalization processing branch is used to perform LayerNorm on the data to better preserve the size relationship between pixels at different positions in the same layer. Specifically, the first normalization processing branch is used to perform normalization processing on the input exposure image data in all exposure image data of the corresponding channel; the second normalization processing branch is used to perform normalization processing on the input exposure image data in all exposure image data.

[0090] In this embodiment, the pseudo code of BatchNorm may be as follows:

[0091] Input: x is an element in array B: B = {x 1...m}; The parameters to be learned are: β, γ

[0092] Output:{y i =BN γ,β (xi )}

[0093]

[0094] The calculation formula of LayerNorm is as follows:

[0095]

[0096] In the formula, γ and β are learnable parameters. ∈ is a small constant.

[0097] When both BatchNorm and LayerNorm are processed, the outputs of BatchNorm and LayerNorm are merged in the merging module, and then the merged result is input into the segmentation network module for image segmentation. Specifically, the segmentation network module of this embodiment can adopt segmentation models such as Unet, maskrcnn or segnet, which can be selected according to actual conditions.

[0098] In the present embodiment, the calculation formula of the cross entropy loss function when training the multi-channel model is as follows:

[0099]

[0100] In the formula, WeightedLoss represents the training loss value, N represents the total number of multi-exposure image data to be trained during the training process, and H represents the total number of types of multi-exposure image data to be trained (this embodiment includes three types: one background type, one tea type, and one impurity type); y i,h represents the true type label of the multi-exposure image data i to be trained. If i belongs to type h, then y i,h =1, otherwise y i,h =0; p i,h represents the predicted probability that the multi-exposure image data i to be trained belongs to type h; ω i,h Represents the area weight, which can improve the segmentation effect of small targets. When c = 3, Among them A i is the area of ​​the connected domain where the pixel is located, α and β are adjustment parameters, so that the smaller the area of ​​the connected domain, the greater the weight. When c = 1 or c = 2, then ω i,h =1.

[0101] In some embodiments, three experiments are conducted in the same data set. Experiment 1 uses single exposure (taking the middle value of the exposure time series) data to input into unet for training and prediction; Experiment 2 uses a multi-scale multi-fusion-based algorithm to perform multi-exposure fusion, and then inputs into unet for training and prediction; Experiment 3 uses the method of the present application to conduct experiments. Among them, the test set is 100 images, which contain 936 impurities. The results of the three experiments are as follows: Figure 3 As shown. Figure 3 It can be seen that the precision and recall rate of image segmentation performed by the method of this application are the highest.

[0102] From the above content, it can be seen that the method of this application can solve the problem of poor imaging quality of single exposure pictures by inputting the multi-exposure picture group into the model as a whole; and by optimizing the loss function, it can solve the problem of small target impurities of various types and difficult to detect. And through the tea impurity detection experiment, it is proved that the method of this application can effectively improve the model performance compared with the traditional method (using multi-exposure fused pictures or single exposure pictures).

[0103] Reference Figure 4 The embodiment of the present application provides an image segmentation device based on multi-exposure, the device comprising:

[0104] The first module 410 is used to collect multi-exposure image data to be trained based on a multi-exposure time series;

[0105] The second module 420 is used to construct a multi-channel model;

[0106] The third module 430 is used to train the multi-channel model using the multi-exposure image data to be trained and a cross entropy loss function, wherein an area weight is provided in the cross entropy loss function;

[0107] The fourth module 440 is used to obtain exposure image data to be processed;

[0108] The fifth module 450 is used to perform image segmentation on the exposure image data to be processed by using the trained multi-channel model.

[0109] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0110] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned image segmentation method based on multiple exposures when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0111] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0112] See also Figure 5 , Figure 5 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0113] The processor 510 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0114] The memory 520 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 520 may store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 520, and the processor 510 calls and executes the image segmentation method based on multiple exposures in the embodiment of this application;

[0115] Input / output interface 530, used to implement information input and output;

[0116] Communication interface 540, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0117] bus 550 , which transmits information between the various components of the device (e.g., processor 510 , memory 520 , input / output interface 530 , and communication interface 540 );

[0118] The processor 510 , the memory 520 , the input / output interface 530 and the communication interface 540 are connected to each other in communication within the device via the bus 550 .

[0119] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned image segmentation method based on multiple exposures is implemented.

[0120] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0121] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0122] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0123] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0124] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0126] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0127] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0128] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0129] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0131] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0132] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A multi-exposure based image segmentation method, characterized in that: The method comprises the following steps: Collecting multi-exposure image data to be trained based on a multi-exposure time series; Build multi-channel models; Training the multi-channel model using the multi-exposure image data to be trained and a cross entropy loss function, wherein an area weight is provided in the cross entropy loss function; Acquire exposure image data to be processed; The image data to be processed is segmented using a trained multi-channel model.

2. The method according to claim 1, characterized in that The method of collecting the multi-exposure image data to be trained based on the multi-exposure time series includes: determining a target exposure time in the multiple exposure time sequence and determining a target impurity type in the impurity type set; Acquire a first pixel value array in a preset channel under a target exposure time, wherein the first pixel value array includes pixels corresponding to a target impurity type; Obtaining a second pixel value array in the preset channel, wherein the second pixel value array includes target object pixels; Calculating a distribution difference metric value between the first pixel value array and the second pixel value array; Reacquiring a first pixel value array according to the distribution difference measure value until all the impurity types in the impurity type set correspond to the first pixel value array; All the first pixel value arrays are used as the multi-exposure image data to be trained.

3. The method according to claim 1, characterized in that The calculating of the distribution difference measure value of the first pixel value array and the second pixel value array includes: The KL divergence of the first pixel value array and the second pixel value array is calculated, and the calculation formula of the KL divergence is as follows: In the formula, KL(P||C) represents the KL divergence, p i,j (x) represents the first pixel value array on the jth channel corresponding to the i-th target impurity type, and represents the second pixel value array on the j-th channel.

4. The method according to claim 1, characterized in that The step of taking all the first pixel value arrays as the multi-exposure image data to be trained comprises: Preprocessing the first pixel value array, wherein the preprocessing includes denoising and smoothing; Performing type labeling on the image corresponding to the first pixel value array after preprocessing; The labeled first pixel value array is used as the multi-exposure image data to be trained.

5. The method according to claim 1, characterized in that The multi-channel model includes an input module, a first normalization processing branch, a second normalization processing branch, a merging module, a segmentation network module and an output module; The input end of the input module is used to receive exposure image data, and the output end of the input module is connected to the input end of the first normalization processing branch and the input end of the second normalization processing branch respectively; The output end of the first normalization processing branch and the output end of the second normalization processing branch are both connected to the input end of the merging module; The output end of the merging module is connected to the input end of the output module, and the output end of the output module is used to output the segmentation result.

6. The method according to claim 5, characterized in that The first normalization processing branch is used to perform normalization processing on all exposure image data of the corresponding channel of the input exposure image data; The second normalization processing branch is used to perform normalization processing on all the input exposure image data.

7. The method according to claim 1, characterized in that The calculation formula of the cross entropy loss function is as follows: In the formula, WeightedLoss represents the training loss value, N represents the total number of multi-exposure image data to be trained during the training process, H represents the total number of types of multi-exposure image data to be trained, and y i,h represents the true type label of the multi-exposure image data i to be trained, p i,h represents the predicted probability that the multi-exposure image data i to be trained belongs to type h, ω i,h Represents the area weight.

8. An image segmentation device based on multiple exposures, characterized in that: The device comprises: The first module is used to collect multi-exposure image data to be trained based on a multi-exposure time series; The second module is used to build a multi-channel model; A third module is used to train the multi-channel model using the multi-exposure image data to be trained and a cross entropy loss function, wherein an area weight is provided in the cross entropy loss function; The fourth module is used to obtain the exposure image data to be processed; The fifth module is used to perform image segmentation on the exposure image data to be processed by using a trained multi-channel model.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images