Method, apparatus and medium for determining a salient object detection model

Abstract

Embodiments of the present disclosure disclose a method, apparatus, device and medium for determining a salient object detection model. The method comprises: constructing an image detection network model having a U-shaped search structure, the image detection network model comprising: an encoding network, a decoding network, a pooling module for connecting the encoding network and the decoding network, a first intermediate network located between the encoding network and the decoding network, and a second intermediate network located between the pooling module and the decoding network; taking detection of a salient object in an image as a search target, performing search training on the image detection network model based on sample data, and determining a salient object detection model. Through the technical solution of the embodiments of the present disclosure, the salient object detection model can be automatically searched and determined without human intervention, and the accuracy and efficiency of salient object detection can be effectively guaranteed.

Description

Technical Field

[0001] This disclosure relates to computer technology, and more particularly to a method, apparatus, device, and medium for determining a salient target detection model. Background Technology

[0002] With the rapid development of computer technology, neural network models can be used to detect salient targets in images. Existing neural network models for salient target detection are typically designed manually based on expert experience. This manual design is time-consuming and labor-intensive, and the resulting models often contain significant computational redundancy, failing to effectively guarantee the accuracy and efficiency of salient target detection. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and medium for determining a salient target detection model, which automatically searches and determines the salient target detection model without human intervention, and can effectively ensure the accuracy and efficiency of salient target detection.

[0004] In a first aspect, embodiments of this disclosure provide a method for determining a salient target detection model, including:

[0005] An image detection network model with a U-shaped search structure is constructed. The image detection network model includes: an encoding network, a decoding network, a pooling module for connecting the encoding network and the decoding network, a first intermediate network located between the encoding network and the decoding network, and a second intermediate network located between the pooling module and the decoding network.

[0006] Using salient targets in the detected image as search targets, the image detection network model is trained based on sample data to determine the salient target detection model.

[0007] Secondly, embodiments of this disclosure also provide an apparatus for determining a salient target detection model, comprising:

[0008] An image detection network model construction module is used to construct an image detection network model with a U-shaped search structure. The image detection network model includes: an encoding network, a decoding network, a pooling module for connecting the encoding network and the decoding network, a first intermediate network located between the encoding network and the decoding network, and a second intermediate network located between the pooling module and the decoding network.

[0009] The salient target detection model determination module is used to search and train the image detection network model based on sample data, using salient targets in the detected image as search targets, to determine the salient target detection model.

[0010] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0011] One or more processors;

[0012] Memory, used to store one or more programs;

[0013] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining a salient target detection model as provided in any embodiment of this disclosure.

[0014] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements a method for determining a salient target detection model as provided in any embodiment of this disclosure.

[0015] This disclosure discloses an image detection network model with a U-shaped search structure. This model includes an encoding network, a decoding network, a pooling module connecting the encoding and decoding networks, a first intermediate network between the encoding and decoding networks, and a second intermediate network between the pooling module and the decoding network. By using salient targets in the detected image as the direct search target, the image detection network model is trained based on sample data. This automatically searches for and determines a salient target detection model without manual intervention. Furthermore, redundant computations are removed during the search and training process, effectively ensuring the detection accuracy and efficiency of the resulting salient target detection model. Attached Figure Description

[0016] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0017] Figure 1 This is a flowchart of a method for determining a salient target detection model provided in Embodiment 1 of this disclosure;

[0018] Figure 2 This is an example of the network structure of an image detection network model according to Embodiment 1 of this disclosure;

[0019] Figure 3(a) is an example of the network structure of another image detection network model according to Embodiment 1 of this disclosure;

[0020] Figure 3(b) is an example of the network structure of another image detection network model according to Embodiment 1 of this disclosure;

[0021] Figure 4 This is a flowchart of a method for determining a salient target detection model provided in Embodiment 2 of this disclosure;

[0022] Figure 5 This is a schematic diagram of the structure of a device for determining a salient target detection model provided in Embodiment 3 of this disclosure;

[0023] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this disclosure. Detailed Implementation

[0024] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0025] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0026] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0027] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0028] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0029] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0030] Example 1

[0031] Figure 1 This is a flowchart illustrating a method for determining a salient target detection model according to Embodiment 1 of this disclosure. This embodiment is applicable to determining a network model for detecting salient targets in an image. This method can be executed by a device for determining a salient target detection model, which can be implemented in software and / or hardware and integrated into an electronic device. Figure 1 As shown, the method specifically includes the following steps:

[0032] S110. Construct an image detection network model with a U-shaped search structure. The image detection network model includes: an encoding network, a decoding network, a pooling module for connecting the encoding network and the decoding network, a first intermediate network located between the encoding network and the decoding network, and a second intermediate network located between the pooling module and the decoding network.

[0033] In this model, the image detection network model can be a U-shaped search space constructed for the salient object detection task. The image detection network model focuses on the location information of salient objects in the image, rather than their category information. The encoding network can be a searchable network structure used for feature extraction in the image detection network model, which can downsample image features to obtain low-resolution feature maps. The decoding network can be a searchable network structure that, in contrast to the encoding network, can upsample the low-resolution feature maps to obtain high-resolution feature maps. The pooling module can be a searchable network structure that performs pooling operations on the feature maps output by the encoding network. For example, the pooling module can be a lightweight pyramid pooling module (LPPM) to further increase the network's overall receptive field and effectively collect global information, improving the efficiency of global information utilization. The first intermediate network can be a searchable network structure that aggregates feature information from the encoding network into the decoding network to restore feature details. The second intermediate network can be a searchable network structure that introduces feature information output from the pooling module into the decoding network to ensure the guidance of global information during feature upsampling and avoid the fading of global information.

[0034] Specifically, Figure 2 An example network structure for an image detection network model is given. Figure 2As shown, the constructed image detection network model with a U-shaped search structure can include: a searchable encoding network (i.e., a bottom-up path), a searchable decoding network (i.e., a top-down path), and a pooling module for connecting the two searchable paths of the encoding and decoding networks, thereby constructing a U-shaped search space. Furthermore, between these two searchable paths, there is also a first intermediate network for restoring details and a second intermediate network (i.e., the Global Guidance path) that guides the global information output by the pooling module into the top-down path.

[0035] For example, Figure 3(a) shows another example of an image detection network model. As shown in Figure 3(a), the encoding network may include: a stem convolutional module and a preset number of downsampling modules; the decoding network may include: an object detection module and a preset number of upsampling modules; the first intermediate network may include: a preset number of first short connection modules; the second intermediate network may include: a preset number of second short connection modules; wherein the downsampling module, the upsampling module, the first short connection module, and the second short connection module are all in one-to-one correspondence.

[0036] The stem convolutional module can refer to a standard 3×3 convolutional layer with 3 input channels. The preset number n can be set based on business needs and actual scenarios. For example, the preset number can be set to 4 to balance detection accuracy and efficiency. The encoding network can include four downsampling modules arranged in sequence, with downsampling rates of 2, 4, 8, and 16, respectively. To maintain the high spatial resolution of deep feature maps, this embodiment does not use downsampling modules with large downsampling rates (e.g., 32). The convolutional stride s of the first network layer of each downsampling module can be set to a value greater than or equal to 2 to achieve the effect of spatial downsampling. The upsampling modules correspond one-to-one with the downsampling modules, have the same sampling rate, and are in the reverse index order. The first upsampling module takes the output of the pooling module as input and passes it sequentially to the subsequent upsampling modules until the target detection module converts the feature map output by the last upsampling module into a single-channel feature map and processes it through the sigmoid activation function to obtain the final output result. During the sampling process of each upsampling module, the feature maps output by the corresponding downsampling modules are gradually aggregated through the corresponding first short connection modules to restore details. Meanwhile, the global information output by the pooling module is guided to the corresponding upsampling module through each second short connection module to ensure the guidance of global information during feature upsampling and avoid the fading of global information.

[0037] S120. Using salient targets in the detected image as the search target, the image detection network model is searched and trained based on sample data to determine the salient target detection model.

[0038] Specifically, this embodiment uses salient object detection (SOD) in the detected image as the direct search target. Based on a preset search method and sample data, it trains the image detection network model to automatically determine the network architecture and weights, thereby obtaining the optimal salient object detection model. The preset search method can be any pre-defined neural network architecture search method. For example, it can be, but is not limited to, Darts (Differentiable Architecture Search), or a search method that processes the search space continuously to utilize continuous optimization methods such as gradient descent to solve for the optimal network structure. In this embodiment, the architecture search process can refer to a path-level pruning process, i.e., directly training an overparameterized network containing all candidate paths, i.e., the image detection network model. In addition to network weight parameters, network architecture parameters are also introduced to determine redundant paths during training. Since the network architecture parameters do not directly participate in the computation graph, the real-valued weights can be updated by binarizing all candidate paths. The network weight parameters and network architecture parameters can be updated alternately on the training and validation sets, respectively. At the end of training, redundant paths are pruned to obtain a compact and optimized network architecture, thereby determining the best-performing salient object detection model and effectively ensuring the detection accuracy and efficiency of the searched salient object detection model.

[0039] For example, after step S120, the method may further include: acquiring the image to be detected; inputting the image to be detected into a salient target detection model; and obtaining the salient target in the image to be detected based on the output of the salient target detection model.

[0040] In this context, a salient target refers to the object with the most prominent visual information in the image to be detected. Specifically, after determining the salient target detection model, it can be directly used to detect salient targets in the image to be detected, highlighting salient target regions and thus obtaining more accurate salient targets in the image more quickly. This improves the accuracy and efficiency of salient target detection, thereby saving computational and analytical resources in large-scale video image processing. The salient target detection method in this embodiment can be applied to application scenarios such as subject segmentation, background blurring, and intelligent bullet screen generation.

[0041] The technical solution of this disclosure involves constructing an image detection network model with a U-shaped search structure. This model includes an encoding network, a decoding network, a pooling module connecting the encoding and decoding networks, a first intermediate network between the encoding and decoding networks, and a second intermediate network between the pooling module and the decoding network. By using salient targets in the detected image as the direct search target, the image detection network model is trained based on sample data. This automatically searches for and determines a salient target detection model without manual intervention. Furthermore, redundant calculations are removed during the search and training process, effectively ensuring the detection accuracy and efficiency of the searched salient target detection model.

[0042] Based on the above technical solution, each downsampling module includes: a first number of downsampling operations; the candidate search space corresponding to the downsampling operations is: each MBConv inverted bottleneck convolutional layer obtained by combining a first preset number of preset convolutional kernel sizes and a second preset number of preset expansion ratios; each upsampling module includes: a second number of upsampling operations; the candidate search space corresponding to the upsampling operations is: each MBConv inverted bottleneck convolutional layer obtained by combining a third preset number of preset convolutional kernel sizes and a fourth preset number of preset expansion ratios; each first short connection module includes: a third number of first short connection operations; the candidate search space corresponding to the first short connection operations is: each MBConv inverted bottleneck convolutional layer obtained by combining a fifth preset number of preset convolutional kernel sizes and a sixth preset number of preset expansion ratios; each second short connection module includes: a fourth number of second short connection operations; the candidate search space corresponding to the second short connection operations is: each MBConv inverted bottleneck convolutional layer obtained by combining a seventh preset number of preset convolutional kernel sizes and an eighth preset number of preset expansion ratios.

[0043] Specifically, Figure 3(b) shows an example of an image detection network model with a preset number n of 4. As shown in Figure 3(b), the first, third, and fourth downsampling modules in the encoding network can each include 4 repeated downsampling operations; the second downsampling module can include 6 repeated downsampling operations. Each downsampling operation can be a mixed operation composed of multiple alternative paths. The first, third, and fourth upsampling modules in the decoding network can each include 2 repeated upsampling operations; the second upsampling module can include 3 repeated upsampling operations. Each upsampling operation can be a mixed operation composed of multiple alternative paths. Each first short connection module in the first intermediate network can include 1 first short connection operation, i.e., 1 mixed operation composed of multiple alternative paths. Each second short connection module in the second intermediate network can include 1 second short connection operation, i.e., 1 mixed operation composed of multiple alternative paths.

[0044] It should be noted that each hybrid operation in the image detection network model is searchable and can be defined individually. To improve search efficiency, this embodiment can search only the downsampling modules, upsampling modules, first short connection modules, and second short connection modules, while not searching the stem convolution module, pooling module, and object detection module. For example, Figure 3(b) shows a network structure for an LPPM pooling module, which can consist of three pooling layers and an Identity activation function layer. The search space of the image detection network model in this embodiment can be as shown in Table 1. All learnable hybrid operations in the downsampling and upsampling modules of the image detection network model can share the same alternative search space: MBConv inverted bottleneck convolutional layers with kernel sizes k of any k in the set {3,5,7} and expansion ratios e of any e in the set {3,6}. The MBConv inverted bottleneck convolutional layer can consist of 1x1 upscaling convolution operations, Depthwise convolution operations, SENet operations for feature channel weight allocation, and 1x1 downscaling convolution operations. The expansion ratio *e* can refer to the ratio of the number of output channels to the number of input channels in the first 1x1 up-dimensional convolution operation in MBConv. The number of blocks *n* can refer to the number of mixing operations included in each module. In this embodiment, the alternative expansion ratios *e* for the learnable blocks in each of the first and second short connection modules can be reduced to 1 and 3, respectively, to reduce unnecessary complexity. For the second short connection module, when the expansion ratio *e* equals 0, it represents zero operations, i.e., allowing the use of residual connections to skip the corresponding learnable blocks. It should be noted that this embodiment does not allow skipping learnable blocks with sampling operations (i.e., *s*>1). To maintain the basic U-shaped search structure, skipping learnable blocks in each of the first short connection modules is not allowed. In this way, the image detection network model can adjust its depth and width accordingly by retaining or skipping more blocks and using larger or smaller MBConv layers.

[0045] Table 1. Alternative search space information for each operation in the image detection network model.

[0046]

[0047]

[0048] Based on the above technical solutions, such as Figure 3(a) and 3(b)As shown, each downsampling module can be used to: downsample the feature information output by the previous downsampling module to determine the feature information output by the current downsampling module. Specifically, the first downsampling module is used to: downsample the feature information output by the STEM convolution module to determine the feature information output by the first downsampling module. Correspondingly, the pooling module is used to: perform pooling operations on the feature information output by the last downsampling module to determine the feature information output by the pooling module.

[0049] Each first short connection module can be used to: perform short connection processing on the feature information output by the previous downsampling module to determine the feature information output by the current first short connection module. Specifically, the first first short connection module is used to: perform short connection processing on the feature information output by the stem convolution module to determine the feature information output by the first first short connection module. Each second short connection module can be used to: perform short connection processing on the feature information output by the pooling module to determine the feature information output by the current second short connection module.

[0050] Each upsampling module can be used to: upsample the feature information output by the previous upsampling module, and then add the upsampled feature information, the feature information output by the first short connection module corresponding to the current upsampling module, and the feature information output by the second short connection module corresponding to the current upsampling module to determine the feature information output by the current upsampling module. Specifically, the first upsampling module is used to: upsample the feature information output by the pooling module, and then add the upsampled feature information, the feature information output by the last first short connection module, and the feature information output by the first second short connection module to determine the feature information output by the first upsampling module. The object detection module is used to: convert the feature information output by the last upsampling module into a single-channel feature map, process it using the sigmoid activation function, obtain the final output result, and output it, thereby realizing the salient object detection process.

[0051] Example 2

[0052] Figure 4 This is a flowchart of a method for determining a salient target detection model according to Embodiment 2 of this disclosure. Based on the above embodiments, this embodiment further optimizes the step of "using salient targets in the detected image as search targets, training the image detection network model based on sample data, and determining the salient target detection model". Explanations of terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0053] See Figure 4 The method for determining the salient target detection model provided in this embodiment specifically includes the following steps:

[0054] S410. Construct an image detection network model with a U-shaped search structure.

[0055] S420. Based on the preset search loss function and the first sample data, the network architecture parameters in the image detection network model are searched and trained to obtain the target network model after the search training is completed. The preset search loss function is constructed based on the complexity importance of each learnable mixture operation in the image detection network model.

[0056] The first sample data may include the first sample image used to train the network architecture and the standard salient target image corresponding to the first sample image. Specifically, by using a pre-built search loss function based on complexity importance, the network architecture parameters in the image detection network model are searched and trained using the first sample data to obtain a well-searched target network model. This effectively reduces the computational cost of the search process while achieving high-quality target extraction.

[0057] For example, S420 may include: inputting the first sample data into the image detection network model, searching and training the network architecture parameters of the image detection network model based on gradient descent; stopping training when the preset search loss function reaches its minimum value, and obtaining the target network model after training.

[0058] It should be noted that, compared to using overall complexity as a unified optimization target, this embodiment introduces the importance of complexity into the overall complexity calculation process during the architecture search phase, recalibrating the importance of the complexity of each hybrid operation, thereby enabling the search for a network architecture with better performance.

[0059] S430. Based on the preset training loss function and the second sample data, the network weight parameters in the target network model are trained to obtain the salient target detection model after training.

[0060] The second sample data may include the second sample image used to train the network weights and the standard salient target image corresponding to the second sample image. For example, the preset training loss function may be, but is not limited to, the BCE (Binary Cross Entropy) function, in order to improve the robustness of salient target detection by accumulating the BCE loss for each pixel in the image.

[0061] For example, S430 may include: inputting the second sample data into the target network model, training the network weight parameters of the target network model based on gradient descent, and stopping training when the preset training loss function reaches its minimum value, thereby obtaining a salient object detection model after training.

[0062] The technical solution of this embodiment constructs a preset search loss function based on the complexity importance of each learnable hybrid operation in the image detection network model in advance, and performs search training on the network architecture parameters in the image detection network model based on the preset search loss function and the first sample data to obtain the target network model after the search training is completed. In this way, the complexity importance is introduced into the overall complexity calculation process of the architecture search stage, and the importance of the complexity of each hybrid operation is recalibrated, so as to search for a network architecture with better performance.

[0063] Based on the above technical solution, the process of constructing the preset search loss function in step S420 may include the following steps S421-S424:

[0064] S421. During the search training process, obtain the path weight and path complexity of each candidate path in the set of candidate paths for each learnable hybrid operation in the image detection network model. The path weight is used to characterize the probability that the hybrid operation selects the candidate path.

[0065] Specifically, the overparameterized network O of the image detection network model can be represented as follows:

[0066] O = Encoding Network ∪ Decoding Network ∪ Pooling Module ∪ First Intermediate Network ∪ Second Intermediate Network

[0067] In the image detection network model, each mixture operation O i All operations ∈ O are learnable. The i-th mixed operation is O. i Having a set of alternative paths And each alternative path Each corresponds to a path weight. To represent the selection of this alternative path The possibility of . Where j represents the i-th hybrid operation O. i The corresponding number of alternative paths. This embodiment can use FLOPs (Floating Point Operations) to characterize the path complexity corresponding to each alternative path.

[0068] S422. Based on the path weights and path complexities corresponding to each hybrid operation, determine the target path complexity for each hybrid operation.

[0069] Specifically, for each hybrid operation, the expected value of the path complexity of each alternative path corresponding to the hybrid operation can be used as the target path complexity of the hybrid operation.

[0070] For example, S422 may include: multiplying the path weight and path complexity corresponding to the current blending operation, and summing the multiplication results corresponding to the current blending operation, with the summed result serving as the target path complexity corresponding to the current blending operation. For example, the blending operation O i The corresponding target path complexity E FLOPs [O i The determination can be based on the following formula:

[0071]

[0072] S423. Determine the complexity weight for each hybrid operation. The complexity weight is used to characterize the importance of the target path complexity of the hybrid operation.

[0073] Specifically, in existing technologies, each hybrid operation has the same complexity importance, meaning all complexity weights are equal. However, this embodiment corrects the complexity importance of the path to which each hybrid operation belongs, thereby increasing the complexity of important hybrid operations and decreasing the complexity of unimportant hybrid operations, thus enabling the search for a better-performing network architecture. This embodiment can use the penalty weight in the loss function as the complexity weight; for example, the smaller the complexity weight corresponding to a hybrid operation, the smaller the penalty weight, indicating a higher importance of the target path complexity of the hybrid operation.

[0074] For example, S423 may include: detecting whether the current mixing operation is a mixing operation in a preset importance network in the image detection network model; if yes, then determining the complexity weight corresponding to the current mixing operation as a first preset value; if no, then determining the complexity weight corresponding to the current mixing operation as a second preset value; wherein the first preset value is less than the second preset value.

[0075] Here, the preset importance network refers to network modules in the image detection network model with high complexity importance. For example, the preset importance network may include, but is not limited to, the encoding network and pooling module; that is, the hybrid operation in the encoding network and pooling module has greater complexity importance than other network modules, thus tending to search for a network architecture with higher complexity in the encoding network and pooling module. The sum of the first preset value and the second preset value can be set to 1. In this case, the first preset value can be set to a value less than 0.5, and the second preset value can be set to a value greater than 0.5. For example, the hybrid operation O i The corresponding complexity weight β i It can be determined based on the following formula:

[0076]

[0077] S424. Based on the target path complexity and complexity weight corresponding to each hybrid operation, construct a preset search loss function.

[0078] Specifically, the preset search loss function corresponding to the overparameterized network O can be characterized using the FLOPs of all hybrid operations.

[0079] For example, S424 may include: multiplying the target path complexity and complexity weight corresponding to each hybrid operation, and summing the multiplication results corresponding to each hybrid operation to construct a preset search loss function. For example, the constructed preset search loss function E FLOPs [O] can be represented as follows:

[0080]

[0081] This embodiment utilizes a simple complexity importance correction method to search for a network architecture with better performance, thereby further improving the detection accuracy and efficiency of the searched salient target detection model.

[0082] The following are embodiments of the apparatus for determining a salient target detection model provided in this disclosure. This apparatus and the method for determining a salient target detection model described above belong to the same inventive concept. For details not described in detail in the embodiments of the apparatus for determining a salient target detection model, please refer to the method for determining a salient target detection model described above.

[0083] Example 3

[0084] Figure 5 This is a schematic diagram of a device for determining a salient target detection model according to Embodiment 3 of this disclosure. This embodiment is applicable to determining a network model for detecting salient targets in an image. Figure 5 As shown, the device specifically includes: an image detection network model construction module 510 and a salient target detection model determination module 520.

[0085] The image detection network model construction module 510 is used to construct an image detection network model with a U-shaped search structure. The image detection network model includes: an encoding network, a decoding network, a pooling module for connecting the encoding network and the decoding network, a first intermediate network located between the encoding network and the decoding network, and a second intermediate network located between the pooling module and the decoding network. The salient target detection model determination module 520 is used to search and train the image detection network model based on sample data, using salient targets in the detected image as search targets, to determine the salient target detection model.

[0086] The technical solution of this disclosure involves constructing an image detection network model with a U-shaped search structure. This model includes an encoding network, a decoding network, a pooling module connecting the encoding and decoding networks, a first intermediate network between the encoding and decoding networks, and a second intermediate network between the pooling module and the decoding network. By using salient targets in the detected image as the direct search target, the image detection network model is trained based on sample data. This automatically searches for and determines a salient target detection model without manual intervention. Furthermore, redundant calculations are removed during the search and training process, effectively ensuring the detection accuracy and efficiency of the searched salient target detection model.

[0087] Based on the above technical solution, the salient target detection model determination module 520 includes:

[0088] The network architecture search unit is used to search and train the network architecture parameters in the image detection network model based on a preset search loss function and the first sample data, so as to obtain the target network model after the search training is completed. The preset search loss function is constructed based on the complexity importance of each learnable mixture operation in the image detection network model.

[0089] The network weight training unit is used to train the network weight parameters in the target network model based on a preset training loss function and second sample data, so as to obtain a salient target detection model after training.

[0090] Based on the above technical solutions, the network architecture search unit is specifically used to: input the first sample data into the image detection network model, search and train the network architecture parameters of the image detection network model based on gradient descent; stop training when the preset search loss function reaches the minimum value, and obtain the target network model after training.

[0091] Based on the above technical solutions, the device further includes: a preset search loss function construction module, comprising:

[0092] The alternative path information acquisition unit is used to acquire the path weight and path complexity of each alternative path in the alternative path set of each learnable hybrid operation in the image detection network model during the search training process. The path weight is used to characterize the probability of the hybrid operation selecting the alternative path.

[0093] The target path complexity determination unit is used to determine the target path complexity for each hybrid operation based on the path weights and path complexities corresponding to each hybrid operation.

[0094] The complexity weight determination unit is used to determine the complexity weight corresponding to each hybrid operation. The complexity weight is used to characterize the importance of the target path complexity of the hybrid operation.

[0095] A preset search loss function construction unit is used to construct a preset search loss function based on the target path complexity and complexity weight corresponding to each hybrid operation.

[0096] Based on the above technical solutions, the target path complexity determination unit is specifically used to: multiply the path weight and path complexity corresponding to the current hybrid operation, and add up the multiplication results corresponding to the current hybrid operation, and use the sum as the target path complexity corresponding to the current hybrid operation.

[0097] Based on the above technical solutions, the complexity weight determination unit is specifically used to: detect whether the current mixing operation is a mixing operation in a preset important network in the image detection network model; if so, determine the complexity weight corresponding to the current mixing operation as a first preset value; if not, determine the complexity weight corresponding to the current mixing operation as a second preset value; wherein, the first preset value is less than the second preset value.

[0098] Based on the above technical solutions, a preset search loss function construction unit is specifically used to: multiply the target path complexity and complexity weight corresponding to each hybrid operation, and add the multiplication results corresponding to each hybrid operation to construct the preset search loss function.

[0099] Based on the above technical solutions, the preset training loss function is: BCE binary classification cross-entropy function.

[0100] Based on the above technical solutions, the encoding network includes: a stem convolutional module and a preset number of downsampling modules; the decoding network includes: a target detection module and a preset number of upsampling modules; the first intermediate network includes: a preset number of first short connection modules; the second intermediate network includes: a preset number of second short connection modules; wherein, the downsampling module, the upsampling module, the first short connection module, and the second short connection module are all in one-to-one correspondence.

[0101] Based on the above technical solutions, each downsampling module includes: a first number of downsampling operations; the alternative search space corresponding to the downsampling operations is: each MBConv inverted bottleneck convolutional layer obtained by combining the first preset number of preset convolutional kernel sizes and the second preset number of preset expansion ratios.

[0102] Each upsampling module includes: a second number of upsampling operations; the alternative search space corresponding to the upsampling operations is: each MBConv inverted bottleneck convolutional layer obtained by combining a third preset number of preset convolutional kernel sizes and a fourth preset number of preset expansion ratios;

[0103] Each first short connection module includes: a third number of first short connection operations; the alternative search space corresponding to the first short connection operation is: each MBConv inverted bottleneck convolutional layer obtained by combining a fifth preset number of preset convolutional kernel sizes and a sixth preset number of preset expansion ratios;

[0104] Each second short connection module includes: a fourth number of second short connection operations; the alternative search space corresponding to the second short connection operations is: each MBConv inverted bottleneck convolutional layer obtained by combining a seventh preset number of preset convolutional kernel sizes and an eighth preset number of preset expansion ratios.

[0105] Based on the above technical solutions, each downsampling module is used to: downsample the feature information output by the previous downsampling module and determine the feature information output by the current downsampling module;

[0106] Each first short connection module is used to: perform short connection processing on the feature information output by the previous sampling module, and determine the feature information output by the current first short connection module;

[0107] Each second short connection module is used to: perform short connection processing on the feature information output by the pooling module, and determine the feature information output by the current second short connection module;

[0108] Each upsampling module is used to: upsample the feature information output by the previous upsampling module, and add the upsampled feature information, the feature information output by the first short connection module corresponding to the current upsampling module, and the feature information output by the second short connection module corresponding to the current upsampling module to determine the feature information output by the current upsampling module.

[0109] Based on the above technical solutions, the pooling module is a lightweight pyramid pooling module.

[0110] Based on the above technical solutions, the device also includes:

[0111] The salient object detection module is used to: acquire the image to be detected after determining the salient object detection model; input the image to be detected into the salient object detection model; and obtain the salient objects in the image to be detected based on the output of the salient object detection model.

[0112] The apparatus for determining the salient target detection model provided in this disclosure can execute the method for determining the salient target detection model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method for determining the salient target detection model.

[0113] It is worth noting that in the embodiments of the above-mentioned salient target detection model determination device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of the present invention.

[0114] Example 4

[0115] The following is for reference. Figure 6 It shows a schematic diagram of the structure of an electronic device 900 suitable for implementing embodiments of the present disclosure. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0116] like Figure 6 As shown, electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 902 or a program loaded from storage device 908 into random access memory (RAM) 903. RAM 903 also stores various programs and data required for the operation of electronic device 900. Processing device 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0117] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0118] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of embodiments of this disclosure.

[0119] The electronic device provided in this disclosure and the method for determining the salient target detection model provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this disclosure can be found in the above embodiments, and this disclosure and the above embodiments have the same beneficial effects.

[0120] Example 5

[0121] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the method for determining a salient target detection model provided in the above embodiments.

[0122] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0123] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0124] The aforementioned computer-readable medium may be included in the aforementioned server; or it may exist independently and not assembled into the server.

[0125] The aforementioned computer-readable medium carries one or more programs, which, when executed by the server, cause the server to: construct an image detection network model with a U-shaped search structure, the image detection network model including: an encoding network, a decoding network, a pooling module for connecting the encoding network and the decoding network, a first intermediate network located between the encoding network and the decoding network, and a second intermediate network located between the pooling module and the decoding network; and, using salient targets in the detected image as search targets, perform search training on the image detection network model based on sample data to determine a salient target detection model.

[0126] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0128] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the units are not necessarily limiting in certain circumstances; for example, an editable content display unit can also be described as an "editing unit".

[0129] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0130] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0131] According to one or more embodiments of this disclosure, [Example 1] provides a method for determining a salient target detection model, including:

[0132] An image detection network model with a U-shaped search structure is constructed. The image detection network model includes: an encoding network, a decoding network, a pooling module for connecting the encoding network and the decoding network, a first intermediate network located between the encoding network and the decoding network, and a second intermediate network located between the pooling module and the decoding network.

[0133] Using salient targets in the detected image as search targets, the image detection network model is trained based on sample data to determine the salient target detection model.

[0134] According to one or more embodiments of this disclosure, [Example 2] provides a method for determining a salient target detection model, further comprising:

[0135] Optionally, the step of using salient targets in the detected image as search targets, and training the image detection network model based on sample data to determine the salient target detection model includes:

[0136] Based on a preset search loss function and first sample data, the network architecture parameters in the image detection network model are searched and trained to obtain the target network model after the search training is completed. The preset search loss function is constructed based on the complexity importance of each learnable hybrid operation in the image detection network model.

[0137] Based on a preset training loss function and second sample data, the network weight parameters in the target network model are trained to obtain a salient target detection model after training.

[0138] According to one or more embodiments of this disclosure, [Example 3] provides a method for determining a salient target detection model, further comprising:

[0139] Optionally, the step of searching and training the network architecture parameters in the image detection network model based on a preset search loss function and first sample data to obtain the target network model after search training includes:

[0140] The first sample data is input into the image detection network model, and the network architecture parameters of the image detection network model are searched and trained based on the gradient descent method.

[0141] Training stops when the preset search loss function reaches its minimum value, and the target network model after training is obtained.

[0142] According to one or more embodiments of this disclosure, [Example 4] provides a method for determining a salient target detection model, further comprising:

[0143] Optionally, the process of constructing the preset search loss function includes:

[0144] During the search training process, the path weight and path complexity of each candidate path in the candidate path set of each learnable hybrid operation in the image detection network model are obtained, wherein the path weight is used to characterize the probability of the hybrid operation selecting the candidate path.

[0145] Based on the path weights and path complexities corresponding to each of the hybrid operations, the target path complexity corresponding to each of the hybrid operations is determined;

[0146] Determine the complexity weight corresponding to each hybrid operation, wherein the complexity weight is used to characterize the importance of the target path complexity of the hybrid operation;

[0147] Based on the target path complexity and complexity weight corresponding to each of the hybrid operations, the preset search loss function is constructed.

[0148] According to one or more embodiments of this disclosure, [Example 5] provides a method for determining a salient target detection model, further comprising:

[0149] Optionally, determining the target path complexity for each of the hybrid operations based on the path weights and path complexities corresponding to each hybrid operation includes:

[0150] Multiply each path weight and path complexity corresponding to the current hybrid operation, and add the results of each multiplication corresponding to the current hybrid operation. The sum is used as the target path complexity corresponding to the current hybrid operation.

[0151] According to one or more embodiments of this disclosure, [Example Six] provides a method for determining a salient target detection model, further comprising:

[0152] Optionally, determining the complexity weight corresponding to each hybrid operation includes:

[0153] Detect whether the current blending operation is a blending operation in a preset importance network of the image detection network model;

[0154] If so, then the complexity weight corresponding to the current mixed operation is determined to be the first preset value;

[0155] If not, then determine the complexity weight corresponding to the current mixed operation as the second preset value;

[0156] Wherein, the first preset value is less than the second preset value.

[0157] According to one or more embodiments of this disclosure, [Example Seven] provides a method for determining a salient target detection model, further comprising:

[0158] Optionally, the step of constructing the preset search loss function based on the target path complexity and the complexity weight corresponding to each of the hybrid operations includes:

[0159] The target path complexity and complexity weight corresponding to each of the hybrid operations are multiplied together, and the multiplication results corresponding to each hybrid operation are added together to construct the preset search loss function.

[0160] According to one or more embodiments of this disclosure, [Example Eight] provides a method for determining a salient target detection model, further comprising:

[0161] Optionally, the preset training loss function is: BCE binary classification cross-entropy function.

[0162] According to one or more embodiments of this disclosure, [Example Nine] provides a method for determining a salient target detection model, further comprising:

[0163] Optionally, the encoding network includes: a stem convolutional module and a preset number of downsampling modules;

[0164] The decoding network includes: a target detection module and the preset number of upsampling modules;

[0165] The first intermediate network includes: the preset number of first short connection modules;

[0166] The second intermediate network includes: the preset number of second short connection modules;

[0167] The downsampling module, the upsampling module, the first short connection module, and the second short connection module are all one-to-one correspondences.

[0168] According to one or more embodiments of this disclosure, [Example 10] provides a method for determining a salient target detection model, further comprising:

[0169] Optionally, each of the downsampling modules includes: a first number of downsampling operations; the alternative search space corresponding to the downsampling operations is: each MBConv inverted bottleneck convolutional layer obtained by combining a first preset number of preset convolutional kernel sizes and a second preset number of preset expansion ratios;

[0170] Each of the upsampling modules includes: a second number of upsampling operations; the alternative search space corresponding to the upsampling operations is: each MBConv inverted bottleneck convolutional layer obtained by combining a third preset number of preset convolutional kernel sizes and a fourth preset number of preset expansion ratios;

[0171] Each of the first short connection modules includes: a third number of first short connection operations; the alternative search space corresponding to the first short connection operation is: each MBConv inverted bottleneck convolutional layer obtained by combining a fifth preset number of preset convolutional kernel sizes and a sixth preset number of preset expansion ratios;

[0172] Each of the second short connection modules includes: a fourth number of second short connection operations; the alternative search space corresponding to the second short connection operation is: each MBConv inverted bottleneck convolutional layer obtained by combining a seventh preset number of preset convolutional kernel sizes and an eighth preset number of preset expansion ratios.

[0173] According to one or more embodiments of this disclosure, [Example 11] provides a method for determining a salient target detection model, further comprising:

[0174] Optionally, each of the downsampling modules is used to: downsample the feature information output by the previous downsampling module to determine the feature information output by the current downsampling module;

[0175] Each of the first short connection modules is used to: perform short connection processing on the feature information output by the previous sampling module, and determine the feature information output by the current first short connection module;

[0176] Each of the second short connection modules is used to: perform short connection processing on the feature information output by the pooling module, and determine the feature information output by the current second short connection module;

[0177] Each of the upsampling modules is used to: upsample the feature information output by the previous upsampling module, and add the upsampled feature information, the feature information output by the first short connection module corresponding to the current upsampling module, and the feature information output by the second short connection module corresponding to the current upsampling module to determine the feature information output by the current upsampling module.

[0178] According to one or more embodiments of this disclosure, [Example Twelve] provides a method for determining a salient target detection model, further comprising:

[0179] Optionally, the pooling module is a lightweight pyramid pooling module.

[0180] According to one or more embodiments of this disclosure, [Example Thirteen] provides a method for determining a salient target detection model, further comprising:

[0181] Optionally, after determining the salient target detection model, the method further includes:

[0182] Acquire the image to be detected;

[0183] The image to be detected is input into the salient target detection model, and the salient targets in the image to be detected are obtained according to the output of the salient target detection model.

[0184] According to one or more embodiments of this disclosure, [Example Fourteen] provides an apparatus for determining a salient target detection model, comprising:

[0185] An image detection network model construction module is used to construct an image detection network model with a U-shaped search structure. The image detection network model includes: an encoding network, a decoding network, a pooling module for connecting the encoding network and the decoding network, a first intermediate network located between the encoding network and the decoding network, and a second intermediate network located between the pooling module and the decoding network.

[0186] The salient target detection model determination module is used to search and train the image detection network model based on sample data, using salient targets in the detected image as search targets, to determine the salient target detection model.

[0187] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0188] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0189] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for determining a salient target detection model, characterized in that, The method comprises the following steps: An image detection network model with a U-shaped search structure is constructed, which comprises an encoding network, a decoding network, a pooling module for connecting the encoding network and the decoding network, a first intermediate network for restoring details between the encoding network and the decoding network, and a second intermediate network for guiding global information output by the pooling module into a top-down path between the pooling module and the decoding network. The first intermediate network comprises a preset number of first short connection modules, each of which is used for short connection processing of feature information output by a previous downsampling module, and determines feature information output by a current first short connection module. The first first short connection module is used for short connection processing of feature information output by a stem convolution module, and determines feature information output by the first first short connection module. The encoding network comprises a stem convolution module and a preset number of downsampling modules. The decoding network comprises a target detection module and the preset number of upsampling modules. The second intermediate network comprises the preset number of second short connection modules. The downsampling modules, the upsampling modules, the first short connection modules, and the second short connection modules correspond one-to-one. Each downsampling module is used for downsampling feature information output by a previous downsampling module, and determining feature information output by a current downsampling module. Each second short connection module is used for short connection processing of feature information output by the pooling module, and determining feature information output by a current second short connection module. Each upsampling module is used for upsampling feature information output by a previous upsampling module, and adding the upsampled feature information, feature information output by a first short connection module corresponding to the current upsampling module, and feature information output by a second short connection module corresponding to the current upsampling module, to determine feature information output by the current upsampling module. The target detection module is used for converting feature information output by a last upsampling module into a single-channel feature map, and obtaining an output result by sigmoid activation function processing operation and outputting. A significant target in an image is detected as a search target, and the image detection network model is searched and trained based on sample data to determine a significant target detection model.

2. The method of claim 1, wherein, The method of detecting a significant target in an image as a search target and searching and training the image detection network model based on sample data to determine a significant target detection model comprises the following steps: Based on a preset search loss function and first sample data, network architecture parameters in the image detection network model are searched and trained to obtain a target network model after search training is completed. The preset search loss function is constructed based on the complexity importance of each hybrid operation that can be learned in the image detection network model. Based on a preset training loss function and second sample data, network weight parameters in the target network model are trained to obtain a significant target detection model after training is completed.

3. The method of claim 2, wherein, The searching training of the network architecture parameters in the image detection network model based on the preset search loss function and the first sample data is performed, and a target network model after the searching training is completed is obtained, including: The first sample data is input into the image detection network model, and the network architecture parameters of the image detection network model are searched and trained based on a gradient descent method; The training is stopped when the preset search loss function reaches a minimum value, and a target network model after the training is completed is obtained.

4. The method of claim 2, wherein, The preset search loss function is constructed, including: In the searching training process, the path weight and the path complexity corresponding to each candidate path in the candidate path set of each learnable mixed operation in the image detection network model are obtained, wherein the path weight is used to represent the probability of the mixed operation selecting the candidate path; Based on each path weight and each path complexity corresponding to each mixed operation, a target path complexity corresponding to each mixed operation is determined; A complexity weight corresponding to each mixed operation is determined, and the complexity weight is used to represent the importance of the target path complexity of the mixed operation; Based on the target path complexity and the complexity weight corresponding to each mixed operation, the preset search loss function is constructed.

5. The method of claim 4, wherein, The target path complexity corresponding to each mixed operation is determined based on each path weight and each path complexity corresponding to each mixed operation, including: Each path weight and the path complexity corresponding to the current mixed operation are multiplied, and each multiplication result corresponding to the current mixed operation is added, and the addition result is taken as the target path complexity corresponding to the current mixed operation.

6. The method of claim 4, wherein, The complexity weight corresponding to each mixed operation is determined, including: It is detected whether the current mixed operation is a mixed operation in a preset important network in the image detection network model; If yes, the complexity weight corresponding to the current mixed operation is determined as a first preset value; If not, the complexity weight corresponding to the current mixed operation is determined as a second preset value; Wherein, the first preset value is less than the second preset value.

7. The method of claim 4, wherein, The preset search loss function is constructed based on the target path complexity and the complexity weight corresponding to each mixed operation, including: Each target path complexity and the complexity weight corresponding to each mixed operation are multiplied, and each multiplication result corresponding to each mixed operation is added, and the preset search loss function is constructed.

8. The method of claim 2, wherein, The preset training loss function is a BCE binary classification cross-entropy function.

9. The method of claim 1, wherein Each down-sampling module includes a first number of down-sampling operations; and a candidate search space corresponding to the down-sampling operation is each MBConv inverted bottleneck convolution layer obtained by combining a first preset number of each preset convolution kernel size and a second preset number of each preset expansion ratio. Each of the up-sampling modules comprises a second operation number of up-sampling operations; the alternative search space corresponding to the up-sampling operations is each MBConv inverted bottleneck convolutional layer obtained by combining a third preset number of each preset convolutional kernel size and a fourth preset number of each preset expansion ratio; Each of the first short connection modules comprises a third operation number of first short connection operations; the alternative search space corresponding to the first short connection operations is each MBConv inverted bottleneck convolutional layer obtained by combining a fifth preset number of each preset convolutional kernel size and a sixth preset number of each preset expansion ratio. Each of the second short connection modules comprises a fourth operation number of second short connection operations; the alternative search space corresponding to the second short connection operations is each MBConv inverted bottleneck convolutional layer obtained by combining a seventh preset number of each preset convolutional kernel size and an eighth preset number of each preset expansion ratio.

10. The method of claim 1, wherein, The pooling module is a lightweight pyramid pooling module.

11. The method according to any one of claims 1 to 9, characterized in that, After determining the saliency target detection model, further comprising: obtaining a to-be-detected image; inputting the to-be-detected image into the saliency target detection model, and obtaining a saliency target in the to-be-detected image according to the output of the saliency target detection model. 12.A determination apparatus of a salient object detection model, characterized in that, Comprising: An image detection network model construction module is configured to construct an image detection network model with a U-shaped search structure, the image detection network model comprising: an encoding network, a decoding network, a pooling module for connecting the encoding network and the decoding network, a first intermediate network for restoring details between the encoding network and the decoding network, and a second intermediate network for guiding global information output by the pooling module to a top-down path between the pooling module and the decoding network, the first intermediate network comprising a preset number of first short connection modules, each of the first short connection modules being configured to perform short connection processing on feature information output by an upper down-sampling module to determine feature information output by a current first short connection module, wherein a first first short connection module is configured to perform short connection processing on feature information output by a stem convolutional module to determine feature information output by the first first short connection module; The encoding network comprises a stem convolution module and a preset number of down-sampling modules; the decoding network comprises a target detection module and the preset number of up-sampling modules; the second intermediate network comprises the preset number of second short connection modules; wherein the down-sampling modules, the up-sampling modules, the first short connection modules and the second short connection modules are one-to-one corresponding; each down-sampling module is configured to down-sample feature information output by a previous down-sampling module to determine feature information output by the current down-sampling module; each second short connection module is configured to perform short connection processing on feature information output by the pooling module to determine feature information output by the current second short connection module; each up-sampling module is configured to up-sample feature information output by a previous up-sampling module, and add the up-sampled feature information, feature information output by a first short connection module corresponding to the current up-sampling module and feature information output by a second short connection module corresponding to the current up-sampling module to determine feature information output by the current up-sampling module; and the target detection module is configured to convert feature information output by a last up-sampling module into a single-channel feature map, and obtain an output result by processing operation through a sigmoid activation function and output the output result. The significant target detection model determination module is configured to take a significant target in an image as a search target, search train the image detection network model based on sample data, and determine a significant target detection model.

13. An electronic device, comprising: The electronic device includes: one or more processors; memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the determination method of the significant target detection model as claimed in any one of claims 1-11.

14. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the determination method of the significant target detection model as claimed in any one of claims 1-11.

Images