An image shadow detection processing method, device, equipment and medium

Abstract

The present application relates to the technical field of deep learning, and provides a processing method for image shadow detection, which comprises the following steps: acquiring an image data set; performing standardization processing on an image training data set to generate a standard image data set; performing label extraction processing on the standard image data set to generate a labeled image data set; inputting the labeled image data set into an initial convolutional neural network model to perform optimization processing and generate a target convolutional neural network model; inputting the labeled image data set into the target convolutional neural network model to perform training processing and generate a residual conversion network model; and inputting an image test data set into the residual conversion network model to perform shadow detection processing and generate a shadow detection image data. The present application can perform fine learning on the edges of a shadow feature map, so that the shadow feature map has good robustness.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, specifically to a processing method, apparatus, device, and medium for image shadow detection. Background Technology

[0002] In recent years, with the continuous development of intelligent driving technology, various image processing systems have been widely applied in the field of automotive intelligent driving. However, since visible light does not have significant penetrating properties, when there is occlusion, the occluded area will appear as a shadow in the image. The most intuitive manifestation of shadow coverage is that it covers the continuity of image grayscale, increasing the difficulty of perception applications such as object detection and semantic segmentation in the field of automotive intelligent driving perception. Existing image shadow detection methods only perform pooling processing on feature maps through attention mechanisms, failing to learn the edges of shadow feature maps in a refined manner, resulting in limited ability to represent global features and a lack of robustness. Summary of the Invention

[0003] In view of the shortcomings of the prior art described above, the present invention provides a processing method, apparatus, device and medium for image shadow detection to solve the above technical problems.

[0004] This invention provides a method for image shadow detection, the method comprising:

[0005] Obtain an image dataset, wherein the image dataset includes an image training dataset and an image test dataset;

[0006] The image training dataset is standardized to generate a standard image dataset;

[0007] The standard image dataset is subjected to label extraction processing to generate a labeled image dataset;

[0008] The labeled image dataset is input into the initial convolutional neural network model for optimization to generate the target convolutional neural network model;

[0009] The labeled image dataset is input into the target convolutional neural network model for training, generating a residual transformation network model; and

[0010] The image test dataset is input into the residual transformation network model for shadow detection processing to generate shadow detection image data.

[0011] In one embodiment of the present invention, the step of standardizing the image training dataset to generate a standard image dataset includes:

[0012] The images in the image training dataset are scaled to generate a standard-sized image dataset;

[0013] The standard-size image datasets are flipped to generate flipped image datasets; and

[0014] The flipped image dataset is segmented to generate a standard image dataset.

[0015] In one embodiment of the present invention, the labeled image dataset includes a shadow edge label dataset and a shadow region label dataset.

[0016] In one embodiment of the present invention, the step of inputting the labeled image dataset into an initial convolutional neural network model for optimization processing to generate a target convolutional neural network model includes:

[0017] The labeled image dataset is input into the first linear layer of the initial convolutional neural network model for linear fusion processing to generate attention image data;

[0018] The attention image data is input into the second linear layer of the initial convolutional neural network model for linear attention enhancement processing to generate global attention image data; and

[0019] The global attention image data is input into the convolutional layer of the initial neural network model to perform convolution operations, thereby obtaining the target convolutional neural network model.

[0020] In one embodiment of the present invention, the global attention data in the global attention image data includes a query vector, a key vector, and a key-value vector.

[0021] In one embodiment of the present invention, the step of inputting the labeled image dataset into the target convolutional neural network model for training processing to generate a residual transformation network model includes:

[0022] The visual conversion module is input into the target convolutional neural network model for feature extraction processing to generate global feature image data;

[0023] The global feature image data is input into the pooling channel for downsampling to generate high-dimensional feature image data.

[0024] The high-dimensional feature image data is processed by cross-entropy loss function to generate shadow region image data;

[0025] The global feature image data is input into the residual channel for convolution processing to generate low-dimensional feature image data.

[0026] The low-dimensional feature image data is processed using a weighted cross-entropy loss function to evaluate the loss, thereby generating shadow label image data; and

[0027] The image data of the shadowed region and the image data of the shadowed label are input into the target neural network model for training to generate a residual transformation network model.

[0028] In one embodiment of the present invention, the step of inputting the image test dataset into the residual transformation network model for shadow detection processing to generate shadow detection image data includes:

[0029] The image test dataset is input into the residual transformation network model to calculate the shadow probability, generating the shadow probability and non-shadow probability of each pixel in the test image;

[0030] Determine the magnitudes of the shadow probability and the non-shadow probability;

[0031] If the shadow probability is greater than or equal to the non-shadow probability, then the pixel region of the test image is set to white to generate a shadow region detection image;

[0032] If the shadow probability is less than the non-shadow probability, then the pixel region of the test image is set to black to generate a shadow edge detection image;

[0033] The shadow region detection image and the shadow edge detection image are fused together to generate shadow detection image data.

[0034] The present invention also provides an image shadow detection processing apparatus, the apparatus comprising:

[0035] A data acquisition module is used to acquire an image dataset, wherein the image dataset includes an image training dataset and an image test dataset;

[0036] The data standardization module is used to standardize the image training dataset to generate a standard image dataset;

[0037] The label extraction module is used to perform label extraction processing on the standard image dataset to generate a labeled image dataset;

[0038] An optimization processing module is used to input the labeled image dataset into the initial convolutional neural network model for optimization processing to generate a target convolutional neural network model;

[0039] The residual network generation module is used to input the labeled image dataset into the target convolutional neural network model for training and to generate a residual transformation network model.

[0040] The detection image generation module is used to input the image test dataset into the residual transformation network model for shadow detection processing and generate shadow detection image data.

[0041] The present invention also provides an electronic device, the electronic device comprising:

[0042] One or more processors;

[0043] A storage device for storing one or more programs that, when executed by one or more processors, cause the electronic device to implement the image shadow detection processing method as described in any of the preceding claims.

[0044] The present invention also provides a computer-readable storage medium, characterized in that it stores a computer program thereon, which, when executed by a computer processor, enables the computer to perform the image shadow detection processing method as described in any of the preceding claims.

[0045] The beneficial effects of this invention are as follows: This invention optimizes the initial convolutional neural network model to obtain a target convolutional neural network model, then inputs the labeled image dataset into the target convolutional neural network model for training to obtain a residual transformation network model, and finally inputs the image test dataset into the residual transformation network model for shadow detection processing to generate shadow detection image data. This invention enables refined learning of the edges of the shadow feature map, giving the shadow feature map better robustness.

[0046] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0047] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0048] Figure 1 This is a schematic diagram illustrating the implementation environment of an image shadow detection processing method according to an exemplary embodiment of this application;

[0049] Figure 2 This is a schematic flowchart illustrating an exemplary embodiment of the image shadow detection processing method of this application;

[0050] Figure 3 yes Figure 2 Step S220 in the illustrated embodiment is a flowchart of an exemplary embodiment;

[0051] Figure 4 yes Figure 2 The flowchart of step S240 in the illustrated embodiment is shown in an exemplary embodiment;

[0052] Figure 5 This is a schematic diagram illustrating the structure of a visual conversion module according to an exemplary embodiment of this application;

[0053] Figure 6 yes Figure 2 The flowchart of step S250 in the illustrated embodiment is shown in an exemplary embodiment;

[0054] Figure 7 yes Figure 2 The flowchart of step S260 in the illustrated embodiment is shown in an exemplary embodiment;

[0055] Figure 8 This is a test image from an image test dataset shown in an exemplary embodiment of this application;

[0056] Figure 9 This is an exemplary embodiment of the present application illustrating a shadow region detection image and a shadow edge detection image;

[0057] Figure 10 This is a shadow detection image illustrated in an exemplary embodiment of this application;

[0058] Figure 11 This is a schematic diagram of the structure of an image shadow detection processing apparatus according to an exemplary embodiment of this application;

[0059] Figure 12 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0060] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0061] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0062] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0063] First, it's important to note that with the development of the big data era, artificial intelligence technology has advanced rapidly, with deep learning technology being applied to many fields such as image segmentation, object detection, and speech segmentation. Image shadow detection methods have already been widely used in the field of intelligent driving. However, because visible light does not have significant penetrating properties, when there is occlusion, the occluded area will appear as a shadow in the image. The most intuitive manifestation of shadow coverage is that it obstructs the continuity of image grayscale, increasing the difficulty of perception applications such as object detection and semantic segmentation in the field of intelligent driving perception. Deep learning (DL) is a new research direction in the field of machine learning (ML). It has been introduced into machine learning to bring it closer to its original goal—artificial intelligence (AI). Deep learning learns the inherent patterns and representational layers of sample data. The information gained in this learning process greatly helps in interpreting data such as text, images, and sound. Its ultimate goal is to enable machines to have analytical and learning capabilities like humans, and to be able to recognize data such as text, images, and sound. Existing image shadow detection methods merely pool feature maps using attention mechanisms, failing to perform refined learning of shadow feature map edges. This results in limited representation of global features and a lack of robustness. The proposed image shadow detection method involves acquiring an image dataset, standardizing the training dataset to obtain a standard image dataset, and then extracting labels from the standard dataset to obtain a labeled image dataset. This labeled image dataset is then input into a target convolutional neural network model for training, resulting in a residual transformation network model. Finally, the image test dataset is input into the residual transformation network model for shadow detection, yielding shadow detection image data to complete the image shadow detection. In other application scenarios, the image shadow detection method for an object can be configured according to the actual situation; the embodiments of this application do not impose such limitations.

[0064] Figure 1 This is a schematic diagram illustrating the implementation environment of an image shadow detection processing method according to an exemplary embodiment of this application. For example... Figure 1 As shown, image information can be collected through intelligent sensing sensors installed on the smart terminal 110. The image information is then standardized and labeled sequentially to generate a standard image dataset and a labeled image dataset. The labeled image dataset is then input into a target convolutional neural network model for training. Finally, the image test data is input into the server 120 for shadow detection processing to generate shadow detection image data. Figure 1The smart terminal 110 shown can be a smartphone, a smart car, a tablet computer, a laptop computer, or any terminal device that supports the installation of smart sensing sensors, but is not limited to these. Figure 1 The server 120 shown is a server, which can be a standalone display screen, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. No limitation is imposed here. The smart terminal 110 can communicate with the server 120 via wireless networks such as 3G (third-generation mobile information technology), 4G (fourth-generation mobile information technology), and 5G (fifth-generation mobile information technology). No limitation is imposed here either. Because existing image shadow detection methods only perform pooling processing on feature maps through attention mechanisms and fail to perform refined learning of the edges of shadow feature maps, their ability to represent global features is limited, resulting in poor robustness. To solve these problems, embodiments of this application propose an image shadow detection processing method, an image shadow detection processing device, an electronic device, and a computer-readable storage medium, which will be described in detail below.

[0065] Please see Figure 2 , Figure 2 This is a flowchart illustrating an exemplary embodiment of the image shadow detection processing method of this application. It should be understood that the method can also be applied to other exemplary implementation environments and specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which the method is applicable.

[0066] like Figure 2 As shown, in an exemplary embodiment, the image shadow detection processing method includes at least steps S210 to S260, which are described in detail below:

[0067] Step S210: Obtain the image dataset, which includes an image training dataset and an image test dataset.

[0068] Step S220: Standardize the image training dataset to generate a standard image dataset.

[0069] Step S230: Perform label extraction processing on the standard image dataset to generate a labeled image dataset.

[0070] Step S240: Input the labeled image dataset into the initial convolutional neural network model for optimization processing to generate the target convolutional neural network model.

[0071] Step S250: Input the labeled image dataset into the target convolutional neural network model for training processing to generate a residual transformation network model.

[0072] Step S260: Input the image test dataset into the residual transformation network model for shadow detection processing to generate shadow detection image data.

[0073] like Figure 2 As shown, in an exemplary embodiment, when step S210 is executed, an image dataset is obtained, wherein the image dataset includes an image training dataset and an image test dataset. It should be noted that the image training dataset may include 1330 training images with a size of 480×640, but is not limited to this; the number of images in the training dataset can be other than this, and the size of the image training dataset can be other than this.

[0074] like Figure 3 As shown, in an exemplary embodiment, when step S220 is performed, the image training dataset is standardized to generate a standard image dataset. Specifically, step S220 may include steps S221 to S223, which are described in detail below:

[0075] Step S221: Scale the images in the image training dataset to generate a standard-size image dataset.

[0076] Step S222: Flip the standard-size image dataset to generate a flipped image dataset.

[0077] Step S223: Segment the flipped image dataset to generate a standard image dataset.

[0078] In one exemplary embodiment, each image in the standard-size image dataset is of standard size. Flipping the standard-size images may include sequentially performing horizontal flipping, vertical flipping, and rotation operations on the standard-size images. Segmenting the flipped image dataset may involve dividing each image in the flipped image dataset into non-overlapping 256×256 pixel squares in a top-to-bottom, left-to-right order.

[0079] like Figure 2As shown, in an exemplary embodiment, when step S230 is executed, label extraction processing is performed on the standard image dataset to generate a labeled image dataset. Specifically, the label extraction processing may include first using image labeling software (labelme tool) to add shadow edge labels to the shadow edge pixel regions in the standard image. The labeled image dataset may include a shadow edge label dataset and a shadow region label dataset. Specifically, for each shadow region image in the shadow region label dataset, regions with a pixel value of 0 can represent the image background, and regions with a pixel value of 1 can represent the shadow region of the image. For images in the shadow edge label dataset, regions with a pixel value of 0 can represent the image background, and regions with a pixel value of 1 can represent shadow edges.

[0080] like Figure 4 and Figure 5 As shown, in an exemplary embodiment, when step S240 is executed, the labeled image dataset is input into the initial convolutional neural network model for optimization processing to generate the target convolutional neural network model. Specifically, step S240 may include steps S241 to S243, which are described in detail below:

[0081] Step S241: Input the labeled image dataset into the first linear layer of the initial convolutional neural network model for linear fusion processing to generate attention image data.

[0082] Step S242: Input the attention image data into the second linear layer of the initial convolutional neural network model for linear attention enhancement processing to generate global attention image data.

[0083] Step S243: Input the global attention image data into the convolutional layer of the initial neural network model to perform convolution operations and obtain the target convolutional neural network model.

[0084] In an exemplary embodiment, the initial convolutional neural network model is a residual network model (ResNet50). The ResNet50 structure can incorporate an attention mechanism and a Gaussian matrix. The attention mechanism can include spatial attention units and channel attention units. The ResNet50 structure can include 49 convolutional layers and 1 fully connected layer. The ResNet50 network structure can be divided into seven parts. The first part does not contain residual layers and mainly performs convolution, regularization, activation function, and max pooling calculations on the input image. The second, third, fourth, and fifth parts all contain residual layers. In the residual network structure, each residual layer can include three convolutional layers. Optimization can be achieved by incorporating an attention mechanism into the initial convolutional neural network model to enhance the learning of target features while reducing redundancy in network parameters. The first linear layer can be denoted as linear1, and the second linear layer can be denoted as linear2. The global attention data in the global attention image data can include query vectors, key vectors, and key-value vectors. The linear attention enhancement process satisfies the following formula:

[0085] Att = Softmax(QK) T V

[0086] Where Att represents the global attention of the global attention image, Softmax represents the activation function, which can be used to normalize a numerical vector into a probability distribution vector, and the sum of the probabilities is 1. The Softmax function can be used as the last layer of a neural network for the output of multi-class classification problems. Q can represent the query vector in the global attention image data, K can represent the key vector in the global attention image data, V can represent the key-value vector in the global attention image data, and T can represent the transpose of the key vector K in the global attention image data. However, it is not limited to this; the activation function in the target convolutional neural network model can also satisfy the following formula:

[0087]

[0088] Here, x can be represented as the input in the target convolutional neural network model, and the SiLU function can be represented as the activation of the input data.

[0089] like Figure 5 and Figure 6 As shown, in an exemplary embodiment, when step S250 is executed, the labeled image dataset is input into the target convolutional neural network model for training processing to generate a residual transformation network model. Specifically, step S250 may include steps S251 to S256, which are described in detail below:

[0090] Step S251: Input the visual conversion module into the target convolutional neural network model for feature extraction processing to generate global feature image data.

[0091] Step S252: Input the global feature image data into the pooling channel for downsampling processing to generate high-dimensional feature image data.

[0092] Step S253: Perform loss evaluation processing on the high-dimensional feature image data using the cross-entropy loss function to generate shadow region image data.

[0093] Step S254: Input the global feature image data into the residual channel for convolution processing to generate low-dimensional feature image data.

[0094] Step S255: Apply a weighted cross-entropy loss function to the low-dimensional feature image data to perform loss evaluation processing, so as to generate shadow label image data.

[0095] Step S256: Input the shadow area image data and shadow label image data into the target neural network model for training to generate a residual transformation network model.

[0096] In an exemplary embodiment, the visual transformation module may specifically include four convolutional layers and one transformation layer. Specifically, among the four convolutional layers, the first layer may be a regular convolution with a 3×3 kernel, the second layer may be a dilated convolution with a 1×1 kernel, the third layer may be the same as the second convolutional layer, and the fourth layer may be the same as the first convolutional layer. The visual transformation module can be used to extract features from the residual layers and convolutional layers in the target convolutional neural network model to generate global feature image data. The cross-entropy function can be expressed as L... shadow Cross-entropy can be used as a loss function in a target convolutional neural network. The cross-entropy loss function measures the similarity between the distribution of the true labels and the distribution of the predicted labels from the trained model. Convolution processing satisfies the following formula:

[0097] x′=vit_conv3(vit_trans(vit_conv2(vit_conv1(x))))

[0098] y=vit_conv4(concate(x,x′))

[0099] Where x can represent global feature image data, vit_conv1 represents the first ordinary convolution with a kernel size of 3×3, vit_conv2 represents the second dilated convolution with a kernel size of 1×1, vit_trans represents the transformation of the result of the second dilated convolution with a kernel size of 1×1, vit_conv3 represents the third dilated convolution with a kernel size of 1×1, vit_conv4 represents the first ordinary convolution with a kernel size of 3×3, x′=vit_conv3(vit_trans(vit_conu2(vit_conv1(x)))) represents the feature map data generated after performing three convolutions on the global feature image data x, concate can represent the feature fusion processing of the global feature image data and the feature map data generated after three convolutions, and y can represent low-dimensional feature image data. The weighted cross-entropy loss function can be expressed as L contour The weighted cross-entropy loss function can weight the loss, allowing for weighting according to the weights of different samples. The training loss can satisfy the following formula:

[0100] L = L shadow +L contour

[0101] Among them, L shadow This can be represented as the cross-entropy function, L contour This can be represented as a weighted cross-entropy loss function, where L represents the total loss of the feature image data during training. The initial learning rate for training the target neural network model can be 0.01, the number of training epochs can be 70, the batch size can be 8, and the training method can be stochastic gradient descent (SGD). The stochastic gradient descent training method can include linearly increasing the learning rate of the target convolutional neural network model from 0.001 to 0.01 in the first 5 training epochs, then linearly decreasing it from 0.01 to 0.001 in the following 40 training epochs, then linearly decreasing it from 0.001 to 0.0001 in the following 20 training epochs, and finally linearly decreasing it to 0.00001 in the last 5 training epochs. However, this is not limited to these methods; the initial learning rate can also be other values, and the number of training epochs is not limited to 70.

[0102] like Figure 7 , Figure 8 , Figure 9 and Figure 10As shown, in an exemplary embodiment, when step S260 is executed, the image test dataset is input into the residual transformation network model for shadow detection processing to generate shadow detection image data. Specifically, step S260 may include steps S261 to S260, which are described in detail below:

[0103] Step S261: Input the image test dataset into the residual transformation network model to calculate the shadow probability and generate the shadow probability and non-shadow probability of each pixel in the test image.

[0104] Step S262: Determine the relative probabilities of shadow and non-shadow. If the shadow probability is greater than or equal to the non-shadow probability, set the corresponding pixel region of the test image to white to generate a shadow region detection image. If the shadow probability is less than the non-shadow probability, set the corresponding pixel region of the test image to black to generate a shadow edge detection image.

[0105] Step S263: Perform fusion processing on the shadow region detection image and the shadow edge detection image to generate shadow detection image data.

[0106] In an exemplary embodiment, the shadow probability calculation for the image test dataset can be performed using the Softmax activation function. The Softmax activation function can map the output layer values ​​to the 0-1 interval through the activation function, constructing a probability distribution from the output of the residual transformation network. In multi-class problems, the larger the value mapped by the Softmax activation function, the greater the probability of the true class.

[0107] Figure 11 This is a schematic diagram illustrating the structure of an image shadow detection processing apparatus according to an exemplary embodiment of this application. This apparatus can be applied in... Figure 1 The device is shown in the implementation environment and is specifically configured in the smart terminal 110. This device can also be applied to other exemplary implementation environments and specifically configured in other devices; this embodiment does not limit the implementation environment to which the device is applicable.

[0108] The image shadow detection processing device may include a data acquisition module 710, a data standardization module 720, a label extraction module 730, an optimization processing module 740, a residual network generation module 750, and a detection image generation module 760.

[0109] In one exemplary embodiment, the data acquisition module 710 can be used to acquire an image dataset, wherein the image dataset includes an image training dataset and an image test dataset. It should be noted that the image training dataset may include 1330 training images with a size of 480×640, but is not limited to this; the number of images in the training dataset can be other numbers, and the size of the image training dataset can be other sizes.

[0110] In an exemplary embodiment, the data standardization module 720 can be used to standardize the image training dataset to generate a standard image dataset. Specifically, the data standardization module 720 can also be used to scale the images in the image training dataset to generate a standard-sized image dataset. The data standardization module 720 can also be used to flip the standard-sized image dataset to generate a flipped image dataset. The data standardization module 720 can also be used to segment the flipped image dataset to generate a standard image dataset. It should be noted that each image in the standard-sized image dataset has a standard size. Flipping the standard-sized images can include sequentially flipping horizontally, vertically, and rotating the standard-sized images. Segmenting the flipped image dataset can involve dividing each image in the flipped image dataset into non-overlapping 256×256 pixel squares in a top-to-bottom, left-to-right order.

[0111] In an exemplary embodiment, the label extraction module 730 can be used to perform label extraction processing on a standard image dataset to generate a labeled image dataset. Specifically, the label extraction processing may include first using image labeling software (labelme tool) to add shadow edge labels to the shadow edge pixel regions in the standard image. The labeled image dataset may include a shadow edge label dataset and a shadow region label dataset. Specifically, for each shadow region image in the shadow region label dataset, regions with a pixel value of 0 can represent the image background, and regions with a pixel value of 1 can represent the shadow region of the image. For images in the shadow edge label dataset, regions with a pixel value of 0 can represent the image background, and regions with a pixel value of 1 can represent shadow edges.

[0112] In an exemplary embodiment, the optimization processing module 740 can be used to input the labeled image dataset into the initial convolutional neural network model for optimization processing to generate the target convolutional neural network model. It should be noted that the initial convolutional neural network model is a residual network model (ResNet50). The residual network structure (ResNet50) can incorporate an attention mechanism and a Gaussian matrix. The attention mechanism can include spatial attention units and channel attention units. The residual network structure (ResNet50) can include 49 convolutional layers and 1 fully connected layer. The ResNet50 network structure can be divided into seven parts. The first part does not contain residual layers and mainly performs convolution, regularization, activation function, and max pooling calculations on the input image. The second, third, fourth, and fifth parts all contain residual layers. In the residual network structure, each residual layer can include three convolutional layers. The optimization processing can incorporate an attention mechanism into the initial convolutional neural network model to enhance the learning of target features while reducing the redundancy of network parameters. The first linear layer can be denoted as linear1, and the second linear layer can be denoted as linear2. Global attention data in global attention image data can include query vectors, key vectors, and key-value vectors. The linear attention enhancement process satisfies the following formula:

[0113] Att = Softmax(QK) T V

[0114] Where Att represents the global attention of the global attention image, Softmax represents the activation function, which can be used to normalize a numerical vector into a probability distribution vector, and the sum of the probabilities is 1. The Softmax function can be used as the last layer of a neural network for the output of multi-class classification problems. Q can represent the query vector in the global attention image data, K can represent the key vector in the global attention image data, V can represent the key-value vector in the global attention image data, and T can represent the transpose of the key vector K in the global attention image data. However, it is not limited to this; the activation function in the target convolutional neural network model can also satisfy the following formula:

[0115]

[0116] Here, x can be represented as the input in the target convolutional neural network model, and the SiLU function can be represented as the activation of the input data.

[0117] In an exemplary embodiment, the residual network generation module 750 can be used to input a labeled image dataset into a target convolutional neural network model for training processing, generating a residual transformation network model. Specifically, the visual transformation module may include four convolutional layers and one transformation layer. Among the four convolutional layers, the first layer can be a regular convolution with a 3×3 kernel, the second layer can be a dilated convolution with a 1×1 kernel, the third layer can be the same as the second convolutional layer, and the fourth layer can be the same as the first convolutional layer. The visual transformation module can be used to extract features from the residual layers and convolutional layers in the target convolutional neural network model to generate global feature image data. The cross-entropy function can be expressed as L... shadow Cross-entropy can be used as a loss function in a target convolutional neural network. The cross-entropy loss function measures the similarity between the distribution of the true labels and the distribution of the predicted labels from the trained model. Convolution processing satisfies the following formula:

[0118] x′=vit_conv3(vit_trans(vit_conv2(vit_conv1(x))))

[0119] y=vit_conv4(concate(x,x′))

[0120] Where x can represent global feature image data, vit_conv1 represents the first ordinary convolution with a kernel size of 3×3, vit_conv2 represents the second dilated convolution with a kernel size of 1×1, vit_trans represents the transformation of the result of the second dilated convolution with a kernel size of 1×1, vit_conv3 represents the third dilated convolution with a kernel size of 1×1, vit_conv4 represents the first ordinary convolution with a kernel size of 3×3, x′=vit_conv3(vit_trans(vit_conv2(vit_conv1(x)))) represents the feature map data generated after performing three convolutions on the global feature image data x, concatenate can represent the feature fusion processing of the global feature image data and the feature map data generated after three convolutions, and y can represent low-dimensional feature image data. The weighted cross-entropy loss function can be expressed as L contour The weighted cross-entropy loss function can weight the loss, allowing for weighting according to the weights of different samples. The training loss can satisfy the following formula:

[0121] L = L shadow +L contour

[0122] Among them, L shadow This can be represented as the cross-entropy function, L contourThis can be represented as a weighted cross-entropy loss function, where L represents the total loss of the feature image data during training. The initial learning rate for training the target neural network model can be 0.01, the number of training epochs can be 70, the batch size can be 8, and the training method can be stochastic gradient descent (SGD). The stochastic gradient descent training method can include linearly increasing the learning rate of the target convolutional neural network model from 0.001 to 0.01 in the first 5 training epochs, then linearly decreasing it from 0.01 to 0.001 in the following 40 training epochs, then linearly decreasing it from 0.001 to 0.0001 in the following 20 training epochs, and finally linearly decreasing it to 0.00001 in the last 5 training epochs. However, this is not limited to these methods; the initial learning rate can also be other values, and the number of training epochs is not limited to 70.

[0123] In an exemplary embodiment, the detection image generation module 760 can be used to input an image test dataset into a residual transformation network model for shadow detection processing, generating shadow detection image data. It should be noted that the shadow probability calculation for the image test dataset can employ the Softmax activation function. The Softmax activation function maps the output layer values ​​to the 0-1 range, constructing a probability distribution from the output of the residual transformation network. In multi-class classification problems, the larger the Softmax activation function mapping value, the greater the likelihood of the true class.

[0124] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0125] Leveraging the powerful learning capabilities of machine learning, a machine learning process using a large number of historical trajectories can achieve displacement bias estimation for comprehensive features of a navigation object, including its speed, direction, movement habits, and dynamic / static state. This ensures that the estimated real-time position of the navigation object is more accurate and reliable. For example, the machine learning model can include a supervised model based on neural networks, such as a binary classification machine learning model. By training the machine learning model with a large number of historical trajectories, the model parameters can be adjusted during training, resulting in a more comprehensive predictive performance for the navigation object's speed, direction, movement habits, and dynamic / static state.

[0126] It should be noted that the image shadow detection processing device and the image shadow detection processing method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the traffic condition refresh device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0127] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the image shadow detection processing method provided in the above embodiments.

[0128] Figure 12 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 12 The computer system 1200 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0129] like Figure 12As shown, the computer system 1200 includes a Central Processing Unit (CPU) 1201, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1202 or programs loaded from storage portion 1208 into Random Access Memory (RAM) 1203, such as performing the methods described in the above embodiments. The RAM 1203 also stores various programs and data required for system operation. The CPU 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An Input / Output (I / O) interface 1205 is also connected to the bus 1204.

[0130] The following components are connected to I / O interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to I / O interface 1205 as needed. Removable media 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1210 as needed so that computer programs read from them can be installed into storage section 1208 as needed.

[0131] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1209, and / or installed from removable medium 1211. When the computer program is executed by central processing unit (CPU) 1201, it performs various functions defined in the system of this application.

[0132] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0134] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0135] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the image shadow detection processing method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0136] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for image shadow detection, characterized in that, The method includes: Obtain an image dataset, wherein the image dataset includes an image training dataset and an image test dataset; The image training dataset is standardized to generate a standard image dataset; The standard image dataset is subjected to label extraction processing to generate a labeled image dataset; The labeled image dataset is input into the initial convolutional neural network model for optimization to generate the target convolutional neural network model; The labeled image dataset is input into the target convolutional neural network model for training, generating a residual transformation network model; and The image test dataset is input into the residual transformation network model for shadow detection processing to generate shadow detection image data; The step of inputting the labeled image dataset into the target convolutional neural network model for training to generate a residual transformation network model includes: The visual conversion module is used as the feature extraction layer in the target convolutional neural network model to perform feature extraction processing and generate global feature image data. The global feature image data is input into the pooling channel for downsampling to generate high-dimensional feature image data. The high-dimensional feature image data is processed by cross-entropy loss function to generate shadow region image data; The global feature image data is input into the residual channel for convolution processing to generate low-dimensional feature image data. The low-dimensional feature image data is processed using a weighted cross-entropy loss function to generate shadow edge label image data; and The shadow region image data and the shadow label image data are input into the target neural network model for training to generate a residual transformation network model; The step of inputting the image test dataset into the residual transformation network model for shadow detection processing to generate shadow detection image data includes: The image test dataset is input into the residual transformation network model to calculate the shadow probability, generating the shadow probability and non-shadow probability of each pixel in the test image. Determine the magnitudes of the shadow probability and the non-shadow probability; If the shadow probability is greater than or equal to the non-shadow probability, then the pixel region of the test image is set to white to generate a shadow region detection image; If the shadow probability is less than the non-shadow probability, then the pixel region of the test image is set to black to generate a shadow edge detection image; The shadow region detection image and the shadow edge detection image are fused together to generate shadow detection image data.

2. The image shadow detection processing method according to claim 1, characterized in that, The step of standardizing the image training dataset to generate a standard image dataset includes: The images in the image training dataset are scaled to generate a standard-sized image dataset; The standard-size image datasets are flipped to generate flipped image datasets; and The flipped image dataset is segmented to generate a standard image dataset.

3. The image shadow detection processing method according to claim 1, characterized in that, The labeled image dataset includes a shadow edge label dataset and a shadow region label dataset.

4. The image shadow detection processing method according to claim 1, characterized in that, The step of inputting the labeled image dataset into the initial convolutional neural network model for optimization processing to generate the target convolutional neural network model includes: The labeled image dataset is input into the first linear layer of the initial convolutional neural network model for linear fusion processing to generate attention image data; The attention image data is input into the second linear layer of the initial convolutional neural network model for linear attention enhancement processing to generate global attention image data; and The global attention image data is input into the convolutional layer of the initial neural network model to perform convolution operations, thereby obtaining the target convolutional neural network model.

5. The image shadow detection processing method according to claim 4, characterized in that, The global attention data in the global attention image data includes query vector, key vector, and key-value vector.

6. An image shadow detection processing apparatus, employing the image shadow detection processing method as described in any one of claims 1 to 5, characterized in that, The device includes: A data acquisition module is used to acquire an image dataset, wherein the image dataset includes an image training dataset and an image test dataset; The data standardization module is used to standardize the image training dataset to generate a standard image dataset; The label extraction module is used to perform label extraction processing on the standard image dataset to generate a labeled image dataset; An optimization processing module is used to input the labeled image dataset into the initial convolutional neural network model for optimization processing to generate a target convolutional neural network model; The residual network generation module is used to input the labeled image dataset into the target convolutional neural network model for training processing, and generate a residual transformation network model; and The detection image generation module is used to input the image test dataset into the residual transformation network model for shadow detection processing and generate shadow detection image data.

7. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the image shadow detection processing method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by the computer's processor, causes the computer to perform the image shadow detection processing method according to any one of claims 1 to 5.

Images