Method, device, medium, program product for generating a bounding box of an image object

Abstract

Provided are a method for generating a bounding box of an image target, an electronic device, a non-transitory computer-readable storage medium, and a computer program product. The method includes: obtaining a feature map based on one or more of a self-attention feature, a cross-attention feature, and a latent vector obtained in a process of generating an image including an image target or in a process of adding noise to the image and regenerating the image; and obtaining a bounding box of the image target based on the feature map.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of computer vision, and more specifically to a method for generating bounding boxes of image targets, electronic devices, non-transitory computer-readable storage media, and computer program products. Background Technology

[0002] In the field of computer vision, the main purpose of object detection is to enable computers to automatically locate the position of objects in images or video frames and identify their categories. Bounding boxes can be drawn around the object to indicate its location, thus aiding in recognition, or a bounding box can be drawn around the object after it has been identified. Bounding box generation is a crucial step in object detection tasks.

[0003] Object detection technology has a wide range of applications in computer vision, such as facial recognition on smartphones, product detection in vending machines, and pedestrian detection in autonomous driving. However, object detection performance drops significantly when training data is limited. In recent years, diffusion models have been used to generate more diverse images, becoming an effective data augmentation technique. However, images generated by diffusion models cannot be directly used for object detection training because they lack corresponding bounding boxes for the objects, which is one of the main challenges currently facing the field of object detection. Summary of the Invention

[0004] According to one aspect of this disclosure, at least one embodiment provides a method for generating a bounding box of an image target, comprising: obtaining a feature map based on at least one of self-attention features and cross-attention features and one or more latent vectors obtained during the generation of an image including the image target or during the process of adding noise to the image and regenerating the image; and obtaining a bounding box of the image target based on the feature map.

[0005] According to one aspect of this disclosure, at least one embodiment provides a system for generating bounding boxes of image targets, comprising: a feature map obtaining device configured to obtain a feature map based on at least one of self-attention features, cross-attention features, and one or more latent vectors obtained during the generation of an image including the image targets or during the process of adding noise to the image and regenerating the image; and a bounding box obtaining device configured to obtain bounding boxes of the image targets based on the feature map.

[0006] According to another aspect of this disclosure, at least one embodiment provides an electronic device, including: a memory for storing computer instructions; and a processor for reading the computer instructions from the memory and performing a method according to at least one embodiment of this disclosure.

[0007] According to another aspect of this disclosure, at least one embodiment provides a non-transitory computer-readable storage medium having computer instructions stored thereon, wherein, when executed by a processor, the computer instructions cause the processor to perform a method according to at least one embodiment of this disclosure.

[0008] According to another aspect of this disclosure, at least one embodiment provides a computer program product including computer instructions, wherein, when executed by a processor, the computer instructions cause the processor to perform a method according to at least one embodiment of this disclosure.

[0009] According to various aspects and embodiments of this disclosure, it is possible to obtain more accurate target edge information by utilizing the self-attention features and / or cross-attention features of the image obtained simultaneously during the generation of an image including the target image or during the process of adding noise to the image and regenerating the image, as well as the latent vector features, thereby obtaining a feature map that more accurately describes the edge characteristics of the target, and generating the bounding box of the target more accurately based on the more accurate feature map. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 An example diagram of the bounding box of an image target according to at least one embodiment of the present disclosure is shown.

[0012] Figure 2 A schematic implementation diagram of generating the bounding box of an image target according to at least one embodiment of the present disclosure is shown.

[0013] Figure 3 A flowchart illustrating a method for generating bounding boxes of image targets according to at least one embodiment of the present disclosure is shown.

[0014] Figure 4 Example diagrams of self-attention feature maps, cross-attention feature maps, and latent vector feature maps of example images according to at least one embodiment of the present disclosure are shown.

[0015] Figure 5 An example latent vector map of one or more latent vectors according to at least one embodiment of the present disclosure is shown.

[0016] Figure 6AAn example diagram is shown illustrating the fusion of self-attention features and cross-attention features to generate fused attention features according to at least one embodiment of the present disclosure, and the input of the target latent vector and the fused attention features into one or more neural networks to obtain a feature map.

[0017] Figure 6B A schematic diagram is shown illustrating a method for obtaining a feature map by inputting a target latent vector and self-attention features into one or more neural networks according to at least one embodiment of the present disclosure.

[0018] Figure 6C A schematic diagram is shown illustrating a method for obtaining a feature map by inputting a target latent vector and cross-attention features into one or more neural networks according to at least one embodiment of the present disclosure.

[0019] Figure 6D A schematic diagram is shown illustrating a method for obtaining a feature map by inputting a target latent vector along with self-attention features and cross-attention features into one or more neural networks according to at least one embodiment of the present disclosure.

[0020] Figures 7A-7C An example diagram showing feature maps of one or more neural network outputs according to at least one embodiment of the present disclosure and the distribution of the number of corresponding pixel values ​​is illustrated.

[0021] Figure 8 The differences between bounding boxes obtained according to at least one embodiment of the present disclosure, true bounding boxes, and bounding boxes obtained by conventional methods are illustrated.

[0022] Figure 9 A block diagram of a system for generating bounding boxes of image targets according to at least one embodiment of the present disclosure is shown.

[0023] Figure 10 A block diagram of an exemplary electronic device according to at least one embodiment of the present disclosure is shown. Detailed Implementation

[0024] Referring now to specific embodiments of this disclosure, examples of which are illustrated in the accompanying drawings. Although this application will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit this application to the described embodiments. Rather, it is intended to cover variations, modifications, and equivalents included within the spirit and scope of this disclosure. It should be noted that the method steps described herein can be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of both.

[0025] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related targets, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following targets have an "or" relationship.

[0026] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0027] Figure 1 An example diagram of the bounding box of an image target according to at least one embodiment of the present disclosure is shown.

[0028] Figure 1 Images 1, 2, and 3 show targets including various types of antelope. (See also:) Figure 1 As shown, the goal is to generate bounding box 1 for the target in image 1, bounding box 2 for the target in image 2, and bounding box 3 for the target in image 3. However, in reality, complex backgrounds and varying lighting conditions in images can make it difficult to accurately generate bounding boxes. For example, the tip of the animal's long horn in image 1 might not be easily identified within the bounding box, and the limbs covered by grass in image 2 might not be easily identified within the bounding box, and so on. How to accurately generate bounding boxes for image targets is a problem that urgently needs to be solved.

[0029] According to at least one embodiment of this disclosure, self-attention features and / or cross-attention features of an image obtained simultaneously during the generation of an image including the target image or during the process of adding noise to the image and regenerating the image, and further utilizing latent vector features, are used to obtain more accurate target edge information, thereby obtaining a feature map that more accurately describes the edge characteristics of the target, and generating the target's bounding box more accurately based on the more accurate feature map. The generated image, the generated bounding box, and / or the guide word can be used as training data to train the target detector, so that the target detector can provide a recognition result and / or the bounding box of the recognition result.

[0030] Figure 2 A schematic implementation diagram of generating the bounding box of an image target according to at least one embodiment of the present disclosure is shown.

[0031] like Figure 2As shown, during the process of generating images using the generative neural network 202, text 201 (also called a guide word) can be input to generate the image. The generative neural network can generate an image 206 corresponding to the text based on the text. For example, inputting "squirrel" can generate an image 206 containing the squirrel target; inputting "antelope" can generate an image 206 containing the antelope target. Of course, the generative neural network can also generate images containing more than two targets based on two or more input texts; there is no limit to the number.

[0032] During the image generation process of the generative neural network 202, a self-attention layer 203, a cross-attention layer 204, and a latent vector layer 205 are passed through. The self-attention feature maps and / or cross-attention feature maps, and latent vector feature maps automatically generated during the passing through these layers can be used by a method (or system, etc.) for generating bounding boxes of the image according to at least one embodiment of this disclosure to output the bounding boxes of the generated image 206. Thus, during the image generation process of the generative neural network 202, both the image and its bounding boxes can be generated simultaneously. The generated image, its bounding boxes, and / or recognition result information can be used as training data to train the object detector.

[0033] Alternatively, in cases where it is necessary to generate a generated image associated with an existing input image, the existing image may include an antelope. The existing image, with added noise and text (e.g., antelope) 201' used to generate the image, can be used as input to the generative neural network 202 to generate an image 206 containing the antelope as a target. This image 206 may be related to the input 201' and may be the same or different. Because only some features obtained during the image generation process need to be considered, there is no limitation on what kind of image must be generated. That is, during the image generation process of the generative neural network 202, the self-attention feature maps and / or cross-attention feature maps automatically generated by the self-attention layer 203, cross-attention layer 204, and latent vector layer 205, and the latent vector feature maps, can still be used by a scheme (method or system, etc.) for generating bounding boxes of the image according to at least one embodiment of this disclosure to output the bounding box of the generated image 206.

[0034] Figure 3 A flowchart is shown of a method 300 for generating bounding boxes of image targets according to at least one embodiment of the present disclosure.

[0035] The method 300 for generating the bounding box of the image target may include steps 310 and 320.

[0036] In step 310, a feature map is obtained based on at least one of self-attention features and cross-attention features, and one or more latent vectors, obtained during the generation of the image including the image target or during the process of adding noise to the image and regenerating the image. In step 320, the bounding box of the image target is obtained based on the feature map.

[0037] In step 310, at least one of the self-attention features and cross-attention features of the image including the image target, and one or more latent vectors are obtained.

[0038] Self-attention and cross-attention mechanisms are techniques for improving deep learning models, especially when processing sequential and cross-modal data, they can focus on important information and ignore unimportant information.

[0039] Self-attention is a mechanism in generative models and can also be used in diffusion models (diffusion models first add noise stepwise through a forward diffusion process to transform the data distribution into an easily tractable prior distribution (usually a standard Gaussian distribution), and then train a neural network to learn the inverse process to gradually remove the noise, thereby generating real data from noise). It allows the model to pay attention to all other positions in the sequence when processing it, thus capturing the dependencies within the sequence. Specifically, in the calculation of self-attention features, for each position in the input image, a correlation score with all other positions is calculated, and then the self-attention features are obtained by weighting these scores. This mechanism helps capture global contextual information. Its core working principle can be divided into the following steps: Linear mapping: For each input vector in the sequence (e.g., a patch of an image in image processing), it is linearly mapped to a query vector (q), a key vector (k), and a value vector (v) through three different weight matrices (W^q, W^k, W^v). Similarity Calculation: The similarity between each query vector and the key vectors of all other elements is calculated using a dot product operation. This similarity represents the degree of attention between elements at different positions in the sequence. Normalization: The similarity is normalized using the softmax function, converting them into an effective probability distribution representing the relative attention weight of each element to other elements. Weighted Summation: The value vectors are weighted and summed using the normalized attention weights to obtain the output vector for each element. This output vector contains not only information about the element itself but also information about other related elements.

[0040] In this way, the self-attention mechanism can dynamically capture the dependency features between elements at different positions in the sequence as self-attention features.

[0041] Cross-attention is a key mechanism connecting the encoder and decoder in generative models, and it can also be used in diffusion models. It allows the decoder to focus on the encoder's output while generating the output sequence, thus achieving a deeper understanding of the input sequence. The difference between cross-attention and self-attention is that cross-attention operates between two different inputs. In image processing, this typically means that the model can focus on relevant parts of another input (such as the image itself) as cross-attention features based on information from one input (such as the textual guide words of an image).

[0042] The general workflow of the cross-attention mechanism is as follows: Feature extraction: Initial feature maps of the two inputs (e.g., image and text, such as the guide words to be generated for the image) are extracted using convolutional neural networks or other methods. Similarity calculation: The similarity matrix between the two initial feature maps is calculated, typically using methods such as dot product or cosine similarity. Normalization: The similarity matrix is ​​normalized to obtain the attention weight matrix. Weighted summation: The initial feature map of the second input is weighted and summed using the attention weight matrix to obtain the cross-attention feature representation.

[0043] In this paper, self-attention features and cross-attention features can be automatically obtained during the process of generating images using a diffusion model with the help of guide words (and images).

[0044] Specifically, in the diffusion model, to obtain self-attention features, the following steps can be performed: Initialization: Define convolutional kernels for the query, key, and value at each location in the input image, as well as an output projection convolutional kernel. These convolutional kernels are used to extract features from the input image and compute attention weights. Feature Extraction: Use convolution operations to extract features from the input image to obtain feature representations of the query, key, and value. Calculate Attention Weights: Obtain attention weights by calculating the similarity between the query and key. This typically involves matrix multiplication of the feature representations of the query and key, followed by normalization using a scaling factor and a softmax function. Weighted Summation: Apply the attention weights to the feature representation of the value at each location and perform a weighted summation to obtain the self-attention feature representation at each location.

[0045] In diffusion models, cross-attention mechanisms can be used to guide the gradual removal of noise and the gradual generation of data. For example, in image generation tasks, cross-attention mechanisms can be used to combine textual guide words and image data to generate images closely related to the text content. In this case, the textual guide words can be considered as an additional input sequence that interacts with the image data through the cross-attention mechanism to guide the image generation process.

[0046] To obtain cross-attention features in a diffusion model, the following steps can be performed: Feature extraction: Extract feature representations of the input sequence (e.g., textual guide words) and the noisy image (or latent representation), respectively. Calculate cross-attention weights: Using the feature representation of the input sequence as the query and the feature representation of the noisy image as the key and value, calculate the cross-attention weights. Weighted summation: Apply the cross-attention weights to the value representation at each position in the noisy image and perform a weighted summation to obtain the cross-attention feature representation at each position that incorporates information from the input sequence.

[0047] Figure 4 Example diagrams of self-attention feature maps, cross-attention feature maps, and latent vector feature maps of example images according to at least one embodiment of the present disclosure are shown. Figure 4 The self-attention feature map 410 corresponding to the image on the left is shown. Self-attention feature map 410 represents the dependencies between pixels at different locations in the image. For example... Figure 4 The cross-attention feature map 420 corresponding to the image on the left shows the dependency relationship between two images or between an image and text.

[0048] An image's latent vector is a vector in a latent representation space within a deep learning model. It is a low-dimensional, dense, and continuous vector typically used to represent images. This vector captures key features of the image and can be used to generate new images similar to the original image. The latent vector of an image can be obtained during the image generation process of the deep learning model.

[0049] For example, latent vectors can be obtained using a Latent Diffusion Model (LDM), where the LDM progressively converts an image into noise through a diffusion process and generates an image from the noise through a reverse diffusion process. In this process, the latent vector serves as the starting point or intermediate state for diffusion and reverse diffusion; that is, when generating an image, new latent vectors can be sampled from the latent distribution, and a new image can be generated through the reverse diffusion process. Specifically, at the beginning of the diffusion model, a random noise tensor is typically sampled from a standard normal distribution as the initial latent vector. The random noise tensor is transformed into a latent representation by an encoder (which can be a simple mapping function or a complex neural network). This latent representation is the latent vector, containing the initial information of the generated data. At each time step, the model progressively updates this representation based on the temporal embedding, the embedding of the textual guide word, and the current latent vector, making it increasingly closer to the latent representation of the target data. After multiple iterations, the final latent vector is obtained. In other words, latent vectors can be obtained during the image generation process through diffusion (e.g., during the generation of images from large-scale text models).

[0050] Alternatively, high-dimensional image data can be compressed into low-dimensional latent vectors using an autoencoder, which is trained on a large amount of image data to learn an efficient mapping from images to latent vectors. Another approach is to use a variational autoencoder (VAE), which adds a regularization term to the autoencoder to ensure the latent vectors follow a specific distribution (e.g., a unit Gaussian distribution). A VAE can be trained by optimizing the KL (Kullback-Leibler) divergence between the reconstruction error and the latent space distribution. Alternatively, a generative adversarial network (GAN) can be used. A GAN consists of a generator and a discriminator. The generator attempts to generate realistic images from random noise, while the discriminator tries to distinguish between generated and real images. By iteratively training the generator and discriminator, a GAN gradually learns the ability to generate realistic images, where random noise vectors are typically treated as latent vectors. By adjusting the values ​​of these vectors, images of different styles can be generated.

[0051] The different model structures listed above vary in their ability to generate and represent latent vectors. The parameters of these models (such as the dimension of the latent vectors and the weights of the regularization terms) also affect the generation of latent vectors. In practical applications, it is necessary to select appropriate models and methods to generate and utilize latent vectors based on the specific task and data characteristics. The specific process will not be described in detail here.

[0052] In this disclosure, latent vectors, self-attention features, and / or cross-attention features are mainly obtained through the diffusion process of the diffusion model. Thus, self-attention features and / or cross-attention features and latent vectors can be obtained in one image generation process by the diffusion model, thereby improving efficiency.

[0053] like Figure 4 The latent vector feature map 430 corresponding to the image on the left is shown. The latent vector feature map 430 represents the latent features or structure of the image.

[0054] As can be seen, if only one of the self-attention feature map 410 and the cross-attention feature map 420 is considered, or if the fused feature map of the self-attention feature map 410 and the cross-attention feature map 420 is considered (see...), Figure 4 In the fused feature map 440, the limbs of the target covered by grass may be ignored. If only one of the self-attention feature map 410 and the cross-attention feature map 420 is considered, or if the fused feature map of self-attention feature map 410 and the cross-attention feature map 420 is considered, the generated bounding box may be a reference... Figure 4 The smaller solid-lined box in the fusion feature map 440 ignores most of the limbs covered by grass. The true bounding box is likely a reference box. Figure 4 The larger solid-line box in the fusion feature map 440 shows that using only at least one of the attention feature map 410 and the cross-attention feature map 420 to generate the bounding box will result in an inaccurate bounding box, which in turn affects the object detection capability.

[0055] In some embodiments, the fused feature map of self-attention feature map 410 and cross-attention feature map 420 can be achieved through various methods, such as direct fusion, attention mechanism fusion, and hierarchical fusion. In direct fusion, self-attention feature map 410 and cross-attention feature map 420 can be concatenated or added together. This method can retain all the information of both features. In attention mechanism fusion, the association weights between the two features can be calculated, and these weights can be weighted and fused with the feature representation. This method allows for more flexible adjustment of the influence of different features on the final result and may improve the performance of the fused feature map. In hierarchical fusion, the two features can be fused at a lower level first, and then further fusion and processing can be performed at a higher level. This method can progressively extract and integrate feature information from different levels, thereby improving the performance of the fused feature map. Here, no restrictions are placed on the fusion method of the fused feature map of self-attention feature map 410 and cross-attention feature map 420.

[0056] However, in addition to at least one of the attention feature map 410 and cross-attention feature map 420 mentioned above, at least one embodiment of this application can further utilize the latent vector feature map 430 to capture latent features or structures of the image, such as basic content information in the image, such as the shape, color, and texture of objects, as well as stylistic features of the image, such as painting style, photographic style, and even the relationships between objects in the image and the atmosphere of the scene. Therefore, by considering latent vectors to construct the feature map of the image, it is possible to obtain more information about the boundaries (edges) of the image targets, thereby generating more accurate bounding boxes from the feature map.

[0057] In some embodiments, step 310 may include obtaining a feature map using one or more neural networks based on at least one of self-attention features, cross-attention features, and one or more latent vectors. The one or more neural networks may be trained, and how to train them will be described in detail later.

[0058] In some embodiments, obtaining a feature map through one or more neural networks based on at least one of self-attention features, cross-attention features, and one or more latent vectors may include: selecting at least one latent vector from one or more latent vectors as a target latent vector for obtaining the feature map according to a first predetermined rule; or combining at least two latent vectors from one or more latent vectors according to a second predetermined rule to generate a target latent vector for obtaining the feature map.

[0059] Figure 5 An example latent vector map of one or more latent vectors according to at least one embodiment of the present disclosure is shown.

[0060] During the generation of latent vectors, one or more latent vectors may be generated due to iteration or other reasons. Figure 5 The diagram illustrates the generation of four latent vector layers from the image on the left. Each latent vector layer may contain information about certain aspects of the object in the image (e.g., texture, style, detail, etc.). Latent vectors can be obtained by inputting the image into a variational autoencoder. Latent vectors represent the capture of multi-level features of the image at different network layers. For example, shallow latent vectors may contain low-level features of the image (such as edges and texture), mid-level latent vectors capture more complex patterns (such as the shape and structure of objects), while deep latent vectors contain high-level semantic information (such as style and detail).

[0061] For example, Figure 5As shown, four latent vectors are generated across four layers. When considering these four latent vectors, at least one can be selected as the target latent vector, or these four latent vectors can be combined to generate a single target latent vector. There are many ways to select or combine them. For example, selection can be done randomly, or it can be based on a first predetermined rule. Combination can be based on a second predetermined rule, which could be a direct summation of the four latent vector maps, or a weighted summation based on the second predetermined rule, to obtain the final target latent vector used to obtain the feature map.

[0062] In some embodiments, the first predetermined rule for selection may include random selection, or based on the ratio of the size of the foreground region to the size of the background region in the latent vector map of one or more latent vectors, and / or the concentration of the foreground region in the image. The second predetermined rule for combination may include direct summation, or based on the ratio of the size of the foreground region to the size of the background region in the latent vector map of one or more latent vectors, and / or the concentration of the foreground region in the image.

[0063] For example, the larger the ratio of the foreground region size to the background region size in the latent vector map, and the higher the concentration of the foreground region in the latent vector map, the more likely it is to be selected or combined with a higher weight. This is because a larger ratio of the foreground region size to the background region size and a higher concentration of the foreground region in the latent vector map indicate that more accurate boundary or edge information of the target can be obtained from one or more latent vectors, thereby supplementing the omissions or deficiencies in boundary or edge information of the self-attention features and / or cross-attention features described earlier.

[0064] like Figure 5 The best fourth-layer latent vector among the four latent vectors shown exhibits a large ratio of foreground to background size and a high concentration of the foreground region in the latent vector map. The second-best third-layer latent vector shows a relatively large ratio of foreground to background size and a relatively high concentration of the foreground region in the latent vector map. Therefore, we can consider selecting the best fourth-layer latent vector as the target latent vector for obtaining the feature map (or we can consider selecting the best fourth-layer latent vector and the second-best third-layer latent vector as the target latent vectors), or we can consider combining these four latent vectors by setting the weight of the fourth-layer latent vector to the highest, the weight of the third-layer latent vector to the second highest, and so on, to generate the target latent vector for obtaining the feature map.

[0065] Thus, selecting or combining latent vectors to obtain target latent vectors for obtaining feature maps can further increase the accuracy of obtaining bounding boxes of image targets in subsequent processes.

[0066] In some embodiments, at least one of self-attention features, cross-attention features, and one or more latent vectors can be fused to obtain a feature map. For example, the feature map can be obtained by adding, weighting, or other fusion methods involving at least one of self-attention features, cross-attention features, and one or more latent vectors.

[0067] In some embodiments, at least one of self-attention features and cross-attention features, and one or more latent vectors, can be input into one or more neural networks to obtain a feature map. This can include various embodiments: inputting a target latent vector and self-attention features into one or more neural networks to obtain a feature map; inputting a target latent vector and cross-attention features into one or more neural networks to obtain a feature map; inputting a target latent vector, self-attention features, and cross-attention features into one or more neural networks to obtain a feature map; or fusing self-attention features and cross-attention features to generate fused attention features, and then inputting the target latent vector and fused attention features into one or more neural networks to obtain a feature map.

[0068] Figure 6A The diagram illustrates a process according to at least one embodiment of the present disclosure, in which self-attention features and cross-attention features are fused to generate fused attention features, and a target latent vector and the fused attention features are input into one or more neural networks to obtain a feature map.

[0069] like Figure 6A As shown, the self-attention feature map 610 and the cross-attention feature map 620 are first fused to generate the fused attention feature map 640, and the target latent vector map 630 and the fused attention feature map 640 are input into one or more neural networks 660 to obtain the feature map 650.

[0070] Figure 6B A schematic diagram is shown illustrating a method for obtaining a feature map by inputting a target latent vector and self-attention features into one or more neural networks according to at least one embodiment of the present disclosure.

[0071] like Figure 6B As shown, the target latent vector map 630 and the self-attention feature map 610 are input into one or more neural networks 660 to obtain the feature map 650.

[0072] Figure 6C A schematic diagram is shown illustrating a method for obtaining a feature map by inputting a target latent vector and cross-attention features into one or more neural networks according to at least one embodiment of the present disclosure.

[0073] like Figure 6C As shown, the target latent vector map 630 and the cross-attention feature map 620 are input into one or more neural networks 660 to obtain the feature map 650.

[0074] Figure 6D A schematic diagram is shown illustrating a method for obtaining a feature map by inputting a target latent vector along with self-attention features and cross-attention features into one or more neural networks according to at least one embodiment of the present disclosure.

[0075] like Figure 6D As shown, the target latent vector map 630, along with the self-attention feature map 610 and the cross-attention feature map 620, are input into one or more neural networks 660 to obtain the feature map 650.

[0076] The aforementioned one or more neural networks can be trained based on their respective input training data. For example, such as Figure 6A The one or more neural networks shown can be trained by fusing self-attention feature samples and cross-attention feature samples to generate fused attention feature samples and target latent vector samples (referred to as latent vector samples). Figure 6B The one or more neural networks shown can be trained using self-attention feature samples and latent vector samples. Figure 6C The one or more neural networks shown can be trained using cross-attention feature samples and latent vector samples. Figure 6D One or more neural networks shown can be trained using self-attention feature samples, cross-attention feature samples, and latent vector samples.

[0077] In some embodiments, one or more neural networks may include edge enhancement functionality. This edge enhancement functionality can be implemented using various edge enhancement algorithms, such as the Sobel operator, Prewitt operator, Canny edge detection, Laplacian operator, LoG operator, SIFT algorithm, etc. Alternatively, the edge enhancement functionality can be implemented by a single neural network; however, this neural network may not need to be trained individually, but rather all neural networks can be trained so that the final output meets the desired edge enhancement.

[0078] In some embodiments, one or more neural networks can be trained using a heuristic training strategy.

[0079] Traditional neural network training methods that rely on loss calculation and backpropagation may fail to backpropagate the loss. This is because the neural network outputs a predicted feature map, while the bounding boxes are calculated using `cv2.threshold(OSTU)` (an adaptive method for determining the image binarization threshold) and `cv2.findContours` (used to find contours in the binarized image) through a series of non-differentiable operations (such as `torch.nonzero` (a PyTorch function that returns the coordinates of non-zero elements in the input tensor) and `coords.min / max` (properties used when processing coordinate tensors). At this point, the neural network's propagation process has ended, making it impossible to backpropagate the loss between the predicted and ground truth bounding boxes to optimize the neural network's (e.g., feature fusion networks, edge enhancement networks) parameters. However, heuristic training strategies can focus on improving the feature maps generated by the neural network during training, gradually resolving the problem of loss propagation failure in traditional methods that rely on loss calculation and backpropagation during the neural network's output process.

[0080] Specifically, in this scheme, the goal of the heuristic training strategy for the neural network is to generate well-fused feature maps for accurate bounding box generation. "Well-fused features" means that the feature map has a high degree of distinction between the foreground and background. When the feature map is typically a grayscale image, the pixel values ​​of the foreground target are mostly concentrated around 255, while the pixel values ​​of the background are mostly concentrated around 0. That is, the pixel value distribution curve has two peaks, concentrated around 0 or 255. Such feature maps are more distinct in black and white, with clearer boundaries, thus enabling more accurate bounding boxes to be obtained in the subsequent step 320 of obtaining image target bounding boxes based on the feature map.

[0081] Therefore, in some embodiments, the evaluation criteria for the heuristic training strategy include that the number of pixels in the image whose pixel values ​​are within a predetermined range around 0 exceeds a first threshold, and the number of pixels in the image whose pixel values ​​are within a predetermined range around 255 exceeds a second threshold. Here, "exceeding the first threshold" or "exceeding the second threshold" indicates a large number. The first threshold or the second threshold can be set to a high value, such as an approximation of the result of dividing the total number of pixels in the feature map by 2. The first threshold and the second threshold can be the same or different.

[0082] In some embodiments, a heuristic training strategy may include the following steps.

[0083] First, obtain the pixel value distribution of the feature maps output by one or more neural networks. The horizontal axis of the distribution represents pixel values, and the vertical axis represents the number of occurrences. Here, "number of occurrences" refers to the frequency of each pixel value in the image. For example, the number of occurrences of the pixel value 233 is 134,450, meaning that the pixel value 233 appears 134,450 times. The distribution map visually shows the distribution of pixel values ​​in an image, indicating which pixel values ​​are more common and which are less common.

[0084] Figures 7A-7C Schematic diagrams are shown of feature maps of one or more neural network outputs according to at least one embodiment of the present disclosure and several examples of the distribution of the number of corresponding pixel values.

[0085] Figure 7A The following is shown Figure 7A The feature map above shows a distribution of pixel values ​​with multiple (more than two) peaks, and the peak values ​​of these peaks are concentrated around 0. Figure 7A The feature map above is predominantly black, and the edges of the target are less distinct, which would lead to a bounding box generated based on this feature map (see...). Figure 7A The box within the feature map above is smaller than the actual true bounding box.

[0086] Figure 7B The following is shown Figure 7B The feature map above shows a distribution of pixel values ​​with multiple (more than two) peaks, with one peak concentrated around 0 and several others concentrated around 255. Figure 7B The feature map above is clearly dominated by white, and the edges of the target are too large and inaccurate, which will lead to the bounding box generated based on this feature map (see...). Figure 7B The box within the feature map above is larger than the actual true bounding box.

[0087] Figure 7C The following is shown Figure 7C The pixel value distribution in the feature map above has two peaks, and these two peaks are concentrated around 0 and 255. Figure 7C The edges of the feature map above are moderate, and the bounding box generated based on this feature map (see...) Figure 7C The boxes within the feature map above typically correspond to the actual ground truth bounding boxes.

[0088] Therefore, in order to achieve Figure 7C To achieve the desired effect, heuristic training strategies can further include the following steps.

[0089] Identify one or more waves with peaks in the quantity distribution. Here, "waves with peaks" primarily refers to waves with only peaks and no troughs, meaning only convex waves are considered, not concave waves, as convex waves represent the largest portion of the pixel quantity distribution. For example, each of these waves could be a wave with a corresponding peak pixel value of 0 and a decreasing trend to the right of that peak; a wave with a corresponding peak pixel value of 255 and an increasing trend to the left of that peak; or a wave with a corresponding peak pixel value between 0 and 255, an increasing trend to the left of that peak, and a decreasing trend to the right of that peak.

[0090] It is possible to calculate the mean, variance or standard deviation, x-axis amplitude (AmpX), and y-axis amplitude (AmpY) for each wave in one or more waves. The x-axis amplitude (AmpX) and y-axis amplitude (AmpY) can be calculated using parameters such as the wave's polarization state, propagation direction, and number of waves. The mean and variance or standard deviation can be calculated using many methods known in the art, and variance and standard deviation are interchangeable.

[0091] The total score can be calculated for the feature map output by the neural network based on the number of waves, the mean, variance or standard deviation of each wave, the x-axis amplitude, and the y-axis amplitude. A higher total score is obtained when the following conditions are met: the number of waves is 2, and the pixel values ​​of the two waves are concentrated at 0 and 255 respectively. Concentrating the pixel values ​​of the two waves at 0 and 255 can include the mean values ​​of the two waves being near 0 and 255 respectively, and the two waves having a tall and slender shape.

[0092] The specific scoring method could be to set an initial score, and deduct more points if the above conditions are not met.

[0093] For example, the initial score can be 100.

[0094] You can start by scoring based on the number of waves. Calculate the initial score based on the number of waves in one or more waves: initial score – abs(number of waves – 2) * a, where abs represents the absolute value. Abs means that more or fewer waves are not acceptable; the more or fewer waves, the more points are deducted.

[0095] Specifically, for example, scores can be assigned based on the wave's mean. Two waves with means falling within a predetermined range of 0 and 255 are selected from one or more waves. If no two are found, they are ignored, indicating the feature map is unsuitable. If two are found, a second score is calculated for the wave whose mean falls within the predetermined range of 0, given as an initial score – MSE(wave mean – 0) * b. A third score is calculated for the wave whose mean falls within the predetermined range of 255, given as an initial score – MSE(wave mean – 255) * b, where MSE represents the mean squared error. Here, the predetermined range can be set to 10 or other values ​​to ensure the mean is as close as possible to 0 or 255. That is, waves with a mean near 0 receive a lower score the further their mean deviates from 0, and waves with a mean near 255 receive a lower score the further their mean deviates from 255. Furthermore, a score can be calculated for each of the two waves individually.

[0096] Scoring can be based on the shape of the waves. As mentioned earlier, two waves with a mean around 0 or 255 can be found from all waves. If none are found, they are ignored, indicating that the feature map is unsuitable. If found, a fourth score can be calculated for each of the two waves based on their variance or standard deviation, x-axis amplitude, and y-axis amplitude. This fourth score is calculated as: initial score – MSE (wave aspect ratio or standard deviation – predetermined wave aspect ratio or standard deviation) * c, where the wave aspect ratio is the wave's y-axis amplitude divided by its x-axis amplitude, and the standard deviation is the square root of the variance. A fourth score is calculated for each wave. Here, the predetermined aspect ratio is, for example, 10, and the predetermined standard deviation is, for example, 0.2, but this is not a limitation. That is, each wave shape is preferably tall and thin; the greater the difference between the wave shape and the predetermined aspect ratio or standard deviation, the lower the score. Tall and thin waves indicate that pixel values ​​are concentrated around 0 or 255.

[0097] a, b, and c represent the weights. You can set a > b > c. Of course, you can also set other weight relationships based on importance considerations.

[0098] The total score can be obtained by adding the first score, the second score, the third score, and the fourth score of the two waves.

[0099] One or more neural networks can be trained a predetermined number of times (e.g., 100 times), and the trained one or more neural networks are determined by retaining the model parameters of the one or more neural networks with the highest total score.

[0100] Figures 7A-7C The curve quality scores (i.e., the total scores mentioned above) for some examples are also shown. The scores are negative because the initial scores were set too low.

[0101] Therefore, the above strategy can train a suitable neural network more accurately.

[0102] In this way, the optimal model parameters of a neural network can be found with lower cost and higher efficiency through heuristic training strategies, while avoiding the problems caused by traditional loss functions and backpropagation methods in neural network training strategies.

[0103] Next, in step 320, the bounding box of the image target can be obtained based on the feature map. In some embodiments, this can be achieved by using the feature map... Figure 2 Binary images are binarized and contours are found to obtain bounding boxes. For example, the `cv2.threshold(OSTU)` algorithm (an adaptive method for determining the image binarization threshold) or the `cv2.findContours` algorithm (used to find contours in a binary image) can be used, and the bounding boxes are determined based on the maximum bounding box extent of the contours. Examples of bounding boxes can be found [link to example]. Figure 7C The bounding boxes are shown in the feature map above. Of course, other edge detection methods can also be used to obtain the contour of the target, and then the bounding boxes can be determined according to the maximum bounding box range of the contour. The methods for obtaining bounding boxes will not be listed here.

[0104] Figure 8 The differences between bounding boxes obtained according to at least one embodiment of the present disclosure, true bounding boxes, and bounding boxes obtained by conventional methods are illustrated.

[0105] Figure 8 In the three images on the left, solid-line bounding boxes represent bounding boxes obtained using the traditional method, while dashed-line bounding boxes represent the true bounding boxes. It is evident that... Figure 8 The bounding boxes obtained in the three images on the left differ significantly from the true bounding boxes, indicating that the obtained bounding boxes cannot accurately reproduce the true bounding boxes.

[0106] Figure 8 In the three images on the right, solid-line bounding boxes represent bounding boxes obtained according to at least one embodiment of the present disclosure, while dashed-line bounding boxes represent true bounding boxes. It is evident that... Figure 8 The bounding boxes obtained in the three images on the right are not significantly different from the true bounding boxes, or at least smaller. Figure 8 The bounding box on the left is closer to the real bounding box, so the obtained bounding box more realistically reproduces the real bounding box.

[0107] Thus, according to at least one embodiment of this disclosure, self-attention features and / or cross-attention features of the image, as well as latent vector features, obtained simultaneously during the generation of an image including the target image or during the process of adding noise to the image and regenerating the image, are used to obtain more accurate target edge information, thereby obtaining a feature map that more accurately describes the edge characteristics of the target, and generating the target bounding box more accurately based on the more accurate feature map.

[0108] The bounding boxes generated according to at least one embodiment of this disclosure can also be used as training data, along with images (and guide words or recognition results), as training data for a neural network used for bounding box generation to further train the object detector, so that the input image can directly detect the object's recognition result and obtain the object's bounding box. Note that the image here can be a generative image generated simultaneously with obtaining self-attention features and / or cross-attention features, thus obtaining the generated bounding box while generating the image, and simultaneously training with the generative image. Alternatively, the image here can also be any input image, which, according to at least one embodiment of this disclosure, will be processed to obtain self-attention features and / or cross-attention features to generate bounding boxes, and simultaneously trained with the arbitrary image. Here, this disclosure does not limit various subsequent applications.

[0109] Compare Figure 8 As can be seen from the bounding boxes on the left and right sides, the bounding boxes obtained according to at least one embodiment of the present disclosure are more accurate and closer to the true bounding boxes than those obtained by conventional solutions.

[0110] Figure 9 A block diagram of a system 900 for generating bounding boxes of image targets according to at least one embodiment of the present disclosure is shown.

[0111] The image target bounding box generation system 900 may include: feature map acquisition device 910 and bounding box acquisition device 920.

[0112] The feature map acquisition device 910 can be configured to acquire a feature map based on at least one of self-attention features, cross-attention features, and one or more latent vectors obtained during the generation of an image including an image target or during the process of adding noise to an image and regenerating the image.

[0113] The bounding box acquisition device 920 can be configured to acquire the bounding box of an image target based on a feature map.

[0114] In some embodiments, the feature map acquisition device 910 may be configured to input at least one of self-attention features, cross-attention features, and one or more latent vectors into one or more neural networks to obtain feature maps.

[0115] In some embodiments, the feature map obtaining device 910 may be configured to select at least one latent vector from one or more latent vectors as a latent vector for obtaining a feature map according to a first predetermined rule, or to combine at least two latent vectors from one or more latent vectors according to a second predetermined rule to obtain a latent vector for obtaining a feature map.

[0116] In some embodiments, the first predetermined rule for selection may include random selection, or based on the ratio of the size of the foreground region to the size of the background region in the latent vector map of one or more latent vectors, and / or the concentration of the foreground region in the image. The second predetermined rule for combination may include direct summation, or based on the ratio of the size of the foreground region to the size of the background region in the latent vector map of one or more latent vectors, and / or the concentration of the foreground region in the image.

[0117] In some embodiments, the feature map acquisition device 910 may be configured to input a target latent vector and self-attention features into one or more neural networks to obtain a feature map; input a target latent vector and cross-attention features into one or more neural networks to obtain a feature map; input a target latent vector and self-attention features and cross-attention features into one or more neural networks to obtain a feature map; or fuse self-attention features and cross-attention features to generate fused attention features, and input the target latent vector and fused attention features into one or more neural networks to obtain a feature map.

[0118] In some embodiments, one or more neural networks may include features for edge enhancement.

[0119] In some embodiments, one or more neural networks may be trained using a heuristic training strategy.

[0120] In some embodiments, the evaluation criteria for the heuristic training strategy may include the number of pixels in the image whose pixel values ​​are within a predetermined range around 0 exceeding a first threshold, and the number of pixels in the image whose pixel values ​​are within a predetermined range around 255 exceeding a second threshold.

[0121] In some embodiments, the heuristic training strategy may include: obtaining the number distribution of pixel values ​​of feature maps output by one or more neural networks; determining one or more waves with peaks in the number distribution, wherein each of the one or more waves is a wave with a corresponding peak pixel value at 0 and a decreasing trend to the right of the corresponding peak, a wave with a corresponding peak pixel value at 255 and an increasing trend to the left of the corresponding peak, or a wave with a corresponding peak pixel value between 0 and 255, an increasing trend to the left of the corresponding peak and a decreasing trend to the right of the corresponding peak; calculating the mean, variance or standard deviation, x-axis amplitude, and y-axis amplitude of each of the one or more waves; calculating a total score for the feature maps output by the neural networks based on the number of waves, the mean, variance or standard deviation, x-axis amplitude, and y-axis amplitude of each of the one or more waves, wherein the higher the total score, the more the following conditions are met: the number of waves is 2, and the pixel values ​​of the two waves are concentrated at 0 and 255, respectively; training one or more neural networks for a predetermined number of times, and determining the one or more trained neural networks by retaining the model parameters of the one or more neural networks with the highest total score.

[0122] Thus, according to at least one embodiment of this disclosure, the self-attention features and / or cross-attention features of the image obtained simultaneously during the generation of the image including the target or during the process of adding noise to the image and regenerating the image, and further by means of latent vector features, are used to obtain more accurate target edge information, thereby obtaining a feature map that more accurately describes the edge characteristics of the target, and generating the target bounding box more accurately based on the more accurate feature map.

[0123] Note that the defects and problems existing in the above-mentioned prior art solutions are also the result of careful research after practical and creative labor. The discovery process of the above problems and the solutions proposed by at least one embodiment disclosed below for the above problems are all creative contributions made during the invention process.

[0124] Figure 10 A block diagram of an exemplary electronic device according to at least one embodiment of the present disclosure is shown.

[0125] The electronic device may include a processor 1010 and a memory 1020, the memory 1020 being coupled to the processor 1010 and storing computer instructions therein for performing steps of various methods of at least one embodiment of the present disclosure when executed by the processor 1010.

[0126] The processor 1010 may include, but is not limited to, one or more processors or microprocessors.

[0127] The memory 1020 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.).

[0128] In addition, the electronic device may also include (but is not limited to) a data bus 1030, an input / output (I / O) bus 1040, a display 1050, and input / output devices 1060 (e.g., keyboard, mouse, speaker, etc.).

[0129] The processor 1010 can communicate with external displays 1050 and input / output devices 1060 via the I / O bus 1040.

[0130] In one embodiment, the at least one computer instruction may also be compiled into or comprise a computer program product or software product, wherein one or more computer instructions, when executed by a processor, perform the steps of the various functions and / or methods in the embodiments described herein.

[0131] According to at least one embodiment of this disclosure, a non-transitory computer-readable storage medium is also provided. Instructions, such as computer instructions, are stored on the non-transitory computer-readable storage medium. When the computer instructions are executed by a processor, the various methods described above can be performed. This non-transitory computer-readable storage medium includes, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.). For example, the non-transitory computer-readable storage medium can be connected to a computing device such as a computer, and then, when the computing device executes the computer instructions stored on the computer-readable storage medium, the various methods described above can be performed.

[0132] This disclosure may also include a computer program product that can perform the methods, steps, and operations given herein. For example, such a computer program product may be a computer software package, computer code instructions, or a computer-readable tangible medium having computer instructions tangibly stored (and / or encoded) thereon, which can be executed by a processor to perform the operations described herein. The computer program product may include packaging materials.

[0133] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The term “such as / for example” as used herein refers to the phrase “such as / for example but not limited to,” and is used interchangeably with it.

[0134] The flowcharts and method descriptions in this disclosure are merely illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the given order. As those skilled in the art will recognize, the steps in the above embodiments can be performed in any order. Words such as "then," "next," etc., are not intended to limit the order of the steps; these words are only used to guide the reader through the description of these methods. Furthermore, any reference to a singular element, such as the use of the articles "a," "one," or "the," is not to be construed as limiting that element to the singular.

[0135] Furthermore, the steps and apparatus in the various embodiments herein are not limited to any one embodiment. In fact, new embodiments can be conceived by combining relevant steps and apparatus in the various embodiments herein based on the concepts of this disclosure, and these new embodiments are also included within the scope of this disclosure.

[0136] The above methods can be implemented in hardware, software, firmware, or any combination thereof.

[0137] Furthermore, modules and / or other suitable means for carrying out the methods and techniques described herein can be downloaded from a server wirelessly when appropriate. Alternatively, the various methods described herein can be provided via a storage component so that the various methods are available when coupled to the storage component. Additionally, any other suitable techniques for providing the methods and techniques described herein to the device can be utilized.

[0138] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit at least one embodiment of the present disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for generating the bounding box of an image target, comprising: A feature map is obtained based on at least one of self-attention features and cross-attention features, and one or more latent vectors, obtained during the generation of an image including an image target or during the process of adding noise to the image and regenerating the image. The bounding box of the image target is obtained based on the feature map.

2. The method according to claim 1, wherein, The method of obtaining a feature map based on at least one of self-attention features and cross-attention features, and one or more latent vectors obtained during the generation of an image including an image target or during the process of adding noise to the image and regenerating the image, includes: The feature map is obtained through one or more neural networks based on at least one of the self-attention features, the cross-attention features, and the one or more latent vectors.

3. The method according to claim 2, wherein, The method of obtaining a feature map based on at least one of self-attention features and cross-attention features, and one or more latent vectors obtained during the generation of an image including an image target or during the process of adding noise to the image and regenerating the image, includes: At least one latent vector from the one or more latent vectors is selected as the target latent vector for obtaining the feature map according to a first predetermined rule; or The target latent vector is generated by combining at least two latent vectors from the one or more latent vectors according to a second predetermined rule. The first predetermined rule includes random selection, or based on the ratio of the size of the foreground region to the size of the background region in the latent vector map of each of the one or more latent vectors, and / or the concentration of the foreground region in the image. The second predetermined rule includes direct summation, or based on the ratio of the size of the foreground region to the size of the background region in the latent vector map of each of the one or more latent vectors, and / or the concentration of the foreground region in the image.

4. The method according to claim 3, wherein, The feature map is obtained through one or more neural networks based on at least one of the self-attention features, the cross-attention features, and the one or more latent vectors, including: The target latent vector and the self-attention feature are input into the one or more neural networks to obtain the feature map; The target latent vector and the cross-attention feature are input into the one or more neural networks to obtain the feature map; The target latent vector, along with the self-attention feature and the cross-attention feature, are input into one or more neural networks to obtain the feature map; or The self-attention features and the cross-attention features are fused to generate fused attention features, and the target latent vector and the fused attention features are input into the one or more neural networks to obtain the feature map.

5. The method according to claim 2, wherein, The one or more neural networks include functionality for edge enhancement.

6. The method according to claim 2, wherein, The one or more neural networks are trained using a heuristic training strategy.

7. The method according to claim 6, wherein, The evaluation criteria for the heuristic training strategy include that the number of pixels in the image whose pixel values ​​are within a predetermined range around 0 exceeds a first threshold, and the number of pixels in the image whose pixel values ​​are within a predetermined range around 255 exceeds a second threshold.

8. The method according to claim 7, wherein, The heuristic training strategy includes: Obtain the distribution of the number of pixel values ​​in the feature map output by the one or more neural networks; Identify one or more waves with peaks in the quantity distribution, wherein each of the one or more waves is a wave with a corresponding peak pixel value of 0 and a decreasing trend to the right of the corresponding peak, a wave with a corresponding peak pixel value of 255 and an increasing trend to the left of the corresponding peak, or a wave with a corresponding peak pixel value between 0 and 255, an increasing trend to the left of the corresponding peak and a decreasing trend to the right of the corresponding peak. Calculate the mean, variance or standard deviation, x-axis amplitude, and y-axis amplitude for each of the one or more waves; The total score is calculated on the feature map output by the neural network based on the number of waves, the mean, variance or standard deviation of each wave, the x-axis amplitude and the y-axis amplitude. The total score is higher if the following conditions are met: the number of waves is 2, and the pixel values ​​of the two waves are concentrated at 0 and 255 respectively. The one or more neural networks are trained a predetermined number of times, and the trained one or more neural networks are determined by retaining the model parameters of the one or more neural networks with the highest total score.

9. An electronic device, comprising: Memory, which stores computer instructions; At least one processor is configured to execute the computer instructions in the memory to perform the method according to any one of claims 1-8.

10. A computer program product, comprising computer instructions, in, When the computer instructions are executed by a processor, the processor performs the method according to any one of claims 1-8.

Images