Thickness detection method, detection device, training method and training device

Abstract

The application provides a thickness detection method, a detection device, a training method and a training device. The thickness detection method comprises the following steps: obtaining a to-be-recognized image; extracting image feature information of the to-be-recognized image; processing the image feature information of the to-be-recognized image through a neural network to recognize a target object shape; correcting each pixel point of the image of the target object shape based on parameters when the to-be-recognized image is obtained to obtain a shape size of the target object in reality; filtering the image of the target object shape to smooth a contour edge of the target object; extracting a contour edge shape of the target object, extracting a maximum inscribed circle in the contour edge shape, calculating a diameter of the maximum inscribed circle, and obtaining a thickness of the target object. The target object can be observed automatically and accurately on the basis of meeting the real-time identification, and the robustness and accuracy of the identification are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to thickness detection methods, detection devices, training methods and training devices, especially for detecting the thickness of ice and snow surfaces. Background Technology

[0002] When ships navigate in icy waters, such as during polar expeditions or transport voyages, they come into contact with sea ice. When the ice is thick, icebreaking operations are necessary. Under these navigation conditions, accurately measuring the thickness of sea ice and snow around the ship is crucial for rational route planning and reducing navigation risks. Currently, measuring sea ice and snow thickness mainly relies on on-site measurements by crew members, which is complex, labor-intensive, unable to obtain real-time snow thickness data, and susceptible to subjective interference. Summary of the Invention

[0003] This application provides a thickness detection method, detection device, training method, and training device to solve the current technical problems of complex operation and inability to automatically, accurately, and in real-time obtain measurement results for thickness measurement, especially ice and snow thickness measurement.

[0004] In a first aspect, embodiments of this application provide a thickness detection method, particularly a thickness detection method for detecting the thickness of snow on ice surfaces, comprising:

[0005] Step S1: Obtain the image to be recognized;

[0006] Step S2: Extract image feature information from the image to be identified;

[0007] Step S3: Process the image feature information of the image to be identified through a neural network, identify each pixel, determine whether the pixel is a target object or a non-target object, and collect the pixels identified as target objects to obtain an image of the shape of the target object.

[0008] In this process, the convolutional blocks in the decoding section receive the outputs of the convolutional pooling module above them and the corresponding convolutional blocks in the encoding section. After processing by the spatial attention mechanism module, these outputs are then fed into path one and path two for further processing. The spatial attention mechanism module receives the outputs of the convolutional pooling module above the decoding section and the outputs from the encoding section. It performs convolution on both outputs and the outputs from the encoding section, sums the results to obtain an integrated result, activates the integrated result using a linear rectified function, performs convolution, and then passes it through a sigmoid activation function to obtain spatial attention weights. Multiplying the outputs from the encoding section by these spatial attention weights yields the spatially attention-processed encoding features. Finally, these spatially attention-processed encoding features are added to the received outputs of the convolutional pooling module above the decoding section to obtain the output S of the spatial attention mechanism module.

[0009] ;

[0010] Where S is the output of the spatial attention mechanism module, x e g represents the raw, unprocessed features from the encoded portion. p For spatial attention weights, x d This is the output of the previous layer in the decoding section after upsampling, where σ is the ReLU function, δ is the Sigmoid function, and g... p It is spatial attention weight. These are all weights of the 1×1 convolutional kernel in the spatial attention mechanism module, where... Matrix multiplication is represented by linear convolution operations;

[0011] Step S4: Based on the parameters obtained when acquiring the image to be recognized, each pixel of the target object shape image is mapped to its position coordinates in the world coordinate system in order to correct each pixel of the target object shape image and obtain the shape and size of the target object in reality.

[0012] Step S5: Filter the image of the target object shape to smooth the outline edges of the target object;

[0013] Step S6: Extract the outline edge shape of the target object, extract the largest inscribed circle within the outline edge shape, calculate the diameter of the largest inscribed circle, and obtain the thickness of the target object.

[0014] In one possible implementation, the neural network in step S3 includes an encoding part and a decoding part. The encoding part includes convolutional pooling modules, each of which includes a convolutional block and a max-pooling layer. The decoding part also includes convolutional pooling modules, each of which includes a convolutional block and a max-pooling layer. Each convolutional block in the encoding part corresponds one-to-one with the convolutional blocks in the corresponding layer of the decoding part. The convolutional blocks in the decoding part receive the output of the convolutional pooling module above the convolutional block in the decoding part and the output of the convolutional block in the encoding part corresponding to the convolutional block.

[0015] In one possible implementation, after each convolutional block of the encoding and decoding parts receives the input, the input is processed in the convolutional block via path one and path two, respectively. The input is processed by convolution in path one to obtain a first output, and then processed by batch normalization, linear rectified function activation, and convolution in path two to obtain a second output. The output of the convolutional block is obtained by adding the first output and the second output.

[0016] In one possible implementation, a channel attention mechanism module is set in path two. The channel attention mechanism module receives the result of the first convolution processing as input, processes it through a global average pooling layer, and then sequentially processes it through dimensionality reduction, batch normalization, linear rectified function activation, and dimensionality increase. After that, the channel attention weights are obtained through the Sigmoid activation function. The channel attention weights are multiplied by the result of the first convolution processing to obtain the output result of the channel attention mechanism module.

[0017] In one possible implementation, the parameters for acquiring the image to be identified in step S4 include internal parameters and external parameters. The internal parameters include the focal length, pixel size, and optical axis position of the image capturing device, while the external parameters include the water outlet height and the optical axis pointing angle.

[0018] In one possible implementation, the method for filtering the image of the target object shape in step S5 is as follows: First, perform image morphological erosion on the image of the target object shape to eliminate isolated points, remove noise and small connected components; then, operate on the image of the target object shape using morphological dilation to restore the area of ​​the target object, fill the holes in the image of the target object shape, and smooth the contour edges of the target object, thereby achieving the denoising and filtering processing of the image of the target object shape.

[0019] In one possible implementation, step S6, the method for calculating the thickness of the target object includes:

[0020] Step S6.1 Extract the contour edge shape of the target object: For the image of the target object shape after noise reduction and filtering, scan and search in a predetermined order. When a pixel with a value of 1 is found and its neighboring pixels with a value of 0, it is considered that the initial contour point has been found. Search for other unsearched contour points among the neighboring pixels of the current contour point. Connect these contour points to form the contour. Repeat the search and connection of the neighboring pixels of the current contour point to continuously expand the contour boundary until the starting contour point is returned. All the connected contour points are considered the contour edge shape. Extract the contour of the entire image in the above manner until the entire image is scanned and the contours of all target objects in the image are obtained.

[0021] Step S6.2 Extract the maximum inscribed circle: For all pixels inside the contour of a target object in the image, calculate the minimum distance from each pixel to the contour of the target object. The pixel with the maximum value at the minimum distance is the center of the maximum inscribed circle in the contour of the target object, and the maximum value is the radius of the maximum inscribed circle. The thickness of the target object is twice the maximum value.

[0022] Secondly, embodiments of this application provide an image recognition-based thickness detection system for implementing the thickness detection method described in the first aspect. The system includes an image acquisition device and an image recognition system. The image acquisition device captures an image including a target object and sends the image to the image recognition system for processing. The image recognition system includes:

[0023] The neural network module is used to process images and identify whether each pixel in the image belongs to the target object.

[0024] The projection transformation module, based on the parameters obtained when acquiring the image to be recognized, eliminates image distortion and maps the target object in the image to the actual coordinates of the target object, thus obtaining the state of the target object in the world coordinate system.

[0025] The filtering and denoising module processes the image of the target object that has been transformed to the world coordinate system, removing noise from the image and smoothing the edges of the target object in the image;

[0026] The thickness calculation module extracts the contour edge of each target object in the image after filtering and denoising, calculates the maximum inscribed circle within the contour edge of each target object, and uses the diameter of the maximum inscribed circle as the thickness of the target object.

[0027] Thirdly, embodiments of this application provide a method for training a neural network module in a thickness detection system, used to train a neural network module in the thickness detection system as described in the second aspect, comprising the following steps:

[0028] Step S11: Prepare a training image sample set: Obtain images containing target objects, label the category of each pixel in the image according to the features of the target objects in each image, and use the category information as a label, thereby obtaining a training image sample set containing pixel information and corresponding label information;

[0029] Step S12 divides the training image sample set into a training set, a validation set, and a test set according to a predetermined ratio, preferably 3:1:1;

[0030] Step S13 uses the training set to train the neural network model. After each round of training, the loss function on the validation set is calculated using the neural network model until the loss function on the validation set converges, at which point training stops.

[0031] Step S14 uses test set data to validate the neural network model. If the validation fails, proceed to step S13 to continue training; if the validation passes, training is complete.

[0032] In one possible implementation, the loss function is composed of a weighted dice loss and a binary cross-entropy added in a predetermined ratio, preferably 5:1, wherein the formula for the weighted dice loss is:

[0033] ;

[0034] ;

[0035] in, Weighted_Dice_Loss For weighted dice losses, and The Dice coefficients represent y values ​​of 0 and 1, and w0 and w1 represent the proportion of pixels with y values ​​of 0 and 1 in the total number of pixels; for Dice coefficient, y i The label value representing pixel i. The predicted value represents pixel i. N This represents the number of pixels to be verified;

[0036] The formula for binary cross-entropy is:

[0037] ;

[0038] in, BCE Represents the binary cross-entropy, where N represents the total number of data points, and y... i This is a binary tag, with a value of 0 or 1. This represents the probability that the output belongs to the label.

[0039] Fourthly, embodiments of this application provide a training apparatus for a neural network module in a thickness detection system, used to train a neural network module in an image recognition-based thickness detection system as described in the second aspect, comprising:

[0040] The acquisition module is used to acquire a training image sample set, which includes images and corresponding labels, and to divide the training image sample set into a training set, a validation set and a test set according to a predetermined ratio.

[0041] The training module trains the neural network model based on the training set and its corresponding labels until the objective function of the neural network model converges on the validation set, thus obtaining the trained neural network model.

[0042] Test module: Validates the neural network model using test set data.

[0043] Fifthly, embodiments of this application provide an electronic device, including: a processor and a memory;

[0044] The memory is used to store computer programs; the processor is configured to implement the thickness detection method as in the first aspect and / or the training method of the neural network module in the image recognition-based thickness detection system as in the third aspect by executing the computer programs.

[0045] Sixthly, embodiments of this application provide a readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the thickness detection method of the first aspect and / or the training method of the neural network module in the thickness detection system of the third aspect.

[0046] Implementing this invention has the following beneficial effects: This application provides a thickness detection method, detection device, training method, and training device. The thickness detection method includes: acquiring an image to be identified; extracting image feature information from the image to be identified; processing the image feature information of the image to be identified using a neural network to identify the shape of a target object; correcting each pixel of the target object shape image based on parameters obtained when acquiring the image to be identified, to obtain the shape and size of the target object in reality; filtering the target object shape image to smooth the contour edge of the target object; extracting the contour edge shape of the target object, extracting the largest inscribed circle within the contour edge shape, calculating the diameter of the largest inscribed circle, and obtaining the thickness of the target object; it can automatically and accurately complete the observation of the target object while meeting the real-time requirements of recognition, improving the robustness and accuracy of recognition. Attached Figure Description

[0047] To more clearly illustrate the technical solution of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of the thickness detection system provided in an embodiment of this application;

[0049] Figure 2 This is a flowchart of the thickness detection method based on image recognition provided in the embodiments of this application;

[0050] Figure 3 This is a schematic diagram of the neural network structure in the thickness detection method based on image recognition provided in the embodiments of this application;

[0051] Figure 4 This is a schematic diagram of the structure of the convolutional block in the decoding part of the neural network provided in the embodiments of this application;

[0052] Figure 5 This is a schematic diagram of the spatial attention mechanism module in the convolutional block of the neural network provided in this application embodiment;

[0053] Figure 6 This is a schematic diagram of the channel attention mechanism module in the convolutional block of the neural network provided in this application embodiment;

[0054] Reference numerals: 1. Ship; 2. Image acquisition device; 3. Sea surface. Detailed Implementation

[0055] The technical solutions in the embodiments of the invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] To address the current technical problems of complex operation and inability to automatically, accurately, and in real-time obtain measurement results for thickness measurement, especially ice and snow thickness measurement, this application proposes an automatic identification and measurement system for ice and snow thickness. By using a neural network with residual connections and attention mechanisms, the system significantly improves identification accuracy while meeting automation and real-time requirements. This provides effective assistance and support for ships navigating and planning routes on icy seas, and also provides strong support for icebreaker operations and planning.

[0057] This application provides an image recognition-based thickness detection method, specifically for measuring the thickness of ice and snow in the area near a ship. For example... Figure 1 As shown, ship 1 is sailing on sea surface 3. When encountering ice on sea surface 3, it is necessary to measure the thickness of the sea ice and snow around ship 1. This application provides a thickness detection system, which is installed on ship 1. The thickness detection system includes an image acquisition device 2 and an image recognition system (not shown). Specifically, the image acquisition device is a camera, suspended outside the ship's side, used to capture images of the sea surface conditions in the area surrounding ship 1 and generate images; the images are then sent to the image recognition system for processing.

[0058] This application provides an image recognition-based thickness detection method, particularly a thickness detection method for detecting the thickness of ice and snow surfaces, such as... Figure 2 As shown, it includes:

[0059] Step S1: Obtain the image to be recognized.

[0060] When the sea surface is covered with ice, the movement of ships crushes and compresses the ice along their route, causing it to overturn. During this overturning process, the thickness of the sea ice and the amount of snow on the ice surface can be observed from a bird's-eye view. By capturing images of the sea surface around the ship using cameras, it is possible to obtain images of the overturned sea ice and the thickness of the snow on the ice surface.

[0061] Step S2: Extract image feature information of the image to be identified.

[0062] Here, it is preferable to take pictures of the sea surface using a digital camera to obtain digital images; preferably, a camera using a CCD or CMOS sensor is used, and each image produced includes multiple pixels.

[0063] Step S3: The image feature information of the image to be identified is processed by the neural network. Each pixel is identified and determined to be either a target object or a non-target object. The pixels identified as target objects are collected to obtain an image of the shape of the target object.

[0064] In this embodiment, the neural network is based on the U-net model, such as Figure 3As shown, the neural network includes an encoding section at the input and a decoding section at the output. Both the encoding and decoding sections include multiple convolutional pooling modules; in this embodiment, there are four convolutional pooling modules stacked together. The lower-level convolutional pooling module receives the output of the upper-level module as its input. Each convolutional pooling module in the encoding section consists of a convolutional block and a max-pooling layer, while each convolutional pooling module in the decoding section consists of a convolutional block and an upsampling layer. The encoding and decoding sections are symmetrically arranged, with each of the four convolutional layers in the encoding section corresponding one-to-one with the four convolutional layers in the decoding section. These corresponding convolutional layers have the same resolution level.

[0065] The input image to the neural network is first processed by the encoding part, which consists of four convolutional blocks. Each convolutional block is followed by a max-pooling layer to reduce the dimensionality of the image, increase the number of channels, and make the image features more numerous and complex, thus obtaining shallow, simple features. The high-dimensional feature map extracted by the encoder is fed into the decoding part for feature reconstruction. The decoding part uses the shallow features extracted by the encoding part for depth mining, replaces the max-pooling layer of the encoding part with an upsampling layer, and enlarges the length and width of the image through nearest-neighbor interpolation to obtain deep, abstract features.

[0066] like Figure 3 As shown, the neural network in this embodiment has a jumper connection structure. A jumper connection structure is set between the convolutional layer of the decoding part and the corresponding convolutional layer of the encoding part with the same resolution level. This jumper connection structure can use the features extracted by the encoding part as a supplement to the decoding part, which to some extent makes up for the information loss problem in the upsampling and downsampling process. It can fully associate shallow simple features and deep abstract features, and supplement some lost information to restore fine edges by feature superposition and integration.

[0067] To address the degradation problem in neural network models, this invention incorporates a residual connection mechanism into the convolutional blocks of the encoding and decoding parts of the neural network. The residual connection adds the output of the convolutional block to the original input without convolution processing, i.e., it connects them through an identity mapping. This allows the model to learn only the residual between the output and the original input, reducing the degradation problem in deep networks and mitigating potential gradient vanishing and exploding, thus accelerating convergence and improving model accuracy.

[0068] This explanation uses the convolutional block in the decoding section as an example. Figure 4 As shown, the output x of the convolutional layer corresponds to the encoding part of the convolutional block. e And the output x from the previous layer convolutional pooling module of the decoding part. d After processing by the spatial attention mechanism module PA, x is obtained. res x resThe convolutional block is processed via path one and path two, respectively. Path one consists of only a 1×1 convolutional kernel, used to obtain the direct linear mapping result. W res × x res ; W res Let x be the weights of the 1×1 convolution kernel. res Upon entering path two, the process proceeds sequentially through batch normalization (BN), rectified linear unit (ReLU) activation, convolution, and so on, to obtain the nonlinear mapping result F(x). res ).

[0069] The ReLU function adds non-linearity to the data. The Batch Normalization (BN) layer maintains the statistical distribution of data in the network, allowing the network to capture feature information more quickly and achieve faster convergence. This alleviates the "vanishing gradient problem" in deep networks to some extent and improves the network's generalization ability.

[0070] Finally, the results from path one and path two are added together to obtain the output M(x) of the convolutional block. res ), that is:

[0071] M(x res )= W res ×x res + F(x res ) .

[0072] When a deep network containing multiple convolutional blocks processes data, then for the Lth convolutional block, its input... for:

[0073] ;

[0074] in, , respectively, are the weights of the 1×1 convolutional kernel in path one of the s-th and v-th convolutional blocks, which are the original inputs of the first residual convolutional block. The linear mapping of the information in this original input can still be preserved during forward propagation. This allows for mitigation of the gradient vanishing problem while deepening the network, ensuring the network's reliability to a certain extent.

[0075] Sea ice images are characterized by numerous and complex features, as well as significant amounts of irrelevant interference. Furthermore, after sea ice flips, the features of the ice surface and snow layer become quite similar, making the model susceptible to interference from irrelevant features and leading to a decrease in snow thickness recognition accuracy. To address this issue, this invention incorporates a spatial attention mechanism (PA) and a channel attention mechanism (CA) into the convolutional block. By applying weights to different regions and channels of the image, the model can concentrate more resources on key areas such as the difficult-to-distinguish boundaries between sea ice and snow layers. This reduces the impact of irrelevant interference in the image, improves the model's robustness, and enhances the accuracy of snow thickness recognition.

[0076] Spatial attention mechanism module PA, such as Figure 6 As shown, the PA receives the input feature xe and the upsampled output xd from the previous layer of the decoder via jumpers, and performs convolution with a 1×1 kernel. The results from both parts are then summed. The result is then processed sequentially through batch normalization (BN), ReLU activation, and a second 1×1 kernel convolution, followed by batch normalization (BN). Finally, the spatial attention weights gp are obtained through a sigmoid activation function. The unprocessed original feature xe from the input is then multiplied by each corresponding element of the obtained spatial attention weights gp to obtain the final result.

[0077]

[0078] Where σ is the ReLU function, δ is the Sigmoid function, gp is the spatial attention weight, and W p W e W d All are the weights of the 1×1 convolutional kernel in the spatial attention mechanism module PA, where W p W e W d The process of matrix multiplication represents a linear convolution operation.

[0079] Then the encoded features that have undergone spatial attention processing are... and the unprocessed decoded part of the input x d The sum is used as the output S of the spatial attention module for subsequent calculations, that is:

[0080] .

[0081] Therefore, the introduction of the spatial attention mechanism module PA can enable neural network models to allocate more attention to important regions and reduce attention to irrelevant and interfering regions.

[0082] In addition to adding a spatial attention mechanism module PA, the neural network model of this invention also incorporates a channel attention mechanism module CA, such as... Figure 4 , 6 As shown, a channel attention mechanism module (CA) is set up in path two. The channel attention mechanism module CA receives the result of the first convolutional processing as input, and obtains Z after processing by a global average pooling layer. m At this point, the feature map size will be reduced from the original H×W×C to 1×1×C. Then, it undergoes sequential processing including convolution for dimensionality reduction (preferably using a 1×1 convolution kernel), batch normalization, linear rectified function activation, and convolution for dimensionality increase. Finally, the channel attention weights are obtained through the Sigmoid activation function. Multiplying these channel attention weights by the result of the first convolution process yields the output of the channel attention mechanism module. ,Right now:

[0083] ;

[0084] in, All are 1×1 convolution kernels.

[0085] Therefore, the Channel Attention (CA) module allows the model to assign different weights to different channels of the feature map, which can focus attention on important features for more effective and accurate recognition.

[0086] Through the above processing, it is possible to identify whether a pixel in the image belongs to the snow layer on the surface of the floating ice, and to collect the pixels that belong to the snow layer on the surface of the floating ice to form a mask image corresponding to the snow layer on the surface of the floating ice.

[0087] Step S4: Based on the parameters obtained when acquiring the image to be recognized, each pixel of the target object shape image is mapped to its position coordinates in the world coordinate system in order to correct each pixel of the target object shape image and obtain the shape and size of the target object in reality.

[0088] Specifically, the intrinsic parameters of the camera used for overhead shots of snow thickness are measured in advance, and the focal length, pixel size, and optical axis position are determined using the Zhang Zhengyou calibration method. The extrinsic parameters of the camera used for overhead shots of snow thickness are measured in real time, specifically including the water level and optical axis pointing angle. By combining the intrinsic and extrinsic parameters and establishing a homography matrix, perspective distortion during the shooting process is corrected. Mathematically, any pixel in the mask image is selected, its pixel coordinates are located in the image, and then, using the intrinsic and extrinsic parameters, these pixel coordinates are restored to sea level coordinates in the world coordinate system, thus determining the pixel's position and size in the real world. By transforming all pixels one by one, the captured mask image can be subjected to an inverse perspective transformation, projected 1:1 onto the real-world sea level, achieving the effect of restoring the mask image's true size and shape on the sea level.

[0089] Step S5: Filter the image of the target object shape to smooth the outline edges of the target object.

[0090] Specifically, the mask image is operated on using morphological opening operations. First, the mask image is subjected to image morphological erosion to eliminate isolated points, remove mask noise and small connected components. Then, the mask image is operated on using morphological dilation to restore the area of ​​the target snow thickness, fill the holes in the mask, and smooth the contour edge of the target snow thickness, thereby achieving the effect of denoising and filtering the mask image of the ice surface with thick snow.

[0091] Step S6: Extract the outline edge shape of the target object, extract the largest inscribed circle within the outline edge shape, calculate the diameter of the largest inscribed circle, and obtain the thickness of the target object.

[0092] Specifically, firstly, the contour polygons of each ice and snow thickness are extracted. For the mask binary image of the ice and snow thickness, a scan is performed from left to right and top to bottom. When a pixel with a value of 1 is found while its neighboring pixels are 0 (i.e., the boundary between the flipped ice and snow thickness and other areas), it is considered to have found the initial contour point. Other unsearched contour points are then searched among the neighboring pixels of this contour point. Connecting these contour points forms the contour. This process of searching and connecting the neighboring pixels of the current contour point is repeated, continuously expanding the contour boundary until the starting contour point is reached. All connected contour points constitute the contour of a snow thickness mask polygon. Contour extraction is performed on the entire mask binary image in the same way until the pixel in the lower right corner of the image is scanned, at which point the contours of all snow thicknesses in the image are considered complete.

[0093] Secondly, extract the maximum inscribed circle within all snow-thickness contour polygons. Traverse all pixels inside the current snow-thickness mask and calculate the minimum distance from these points to the current polygon contour points, which is set as a; take the maximum value in set a, that is, the point inside the polygon farthest from the contour, which is the center of the maximum inscribed circle. This maximum value is the radius of the center, and the diameter of the maximum inscribed circle can be used as the observed value of the current snow thickness.

[0094] Finally, the maximum or average value of all flipped snow thicknesses obtained within the current image is taken as the observed snow thickness for the current region.

[0095] Furthermore, this application provides an image recognition-based thickness detection system for implementing the aforementioned image recognition-based thickness detection method. The system includes an image acquisition device and an image recognition system. The image acquisition device captures an image including the target object and sends the image to the image recognition system for processing. The image recognition system includes:

[0096] The neural network module is used to process images and identify whether each pixel in the image belongs to the target object.

[0097] The projection transformation module, based on the parameters obtained when acquiring the image to be recognized, eliminates image distortion and maps the target object in the image to the actual coordinates of the target object, thus obtaining the state of the target object in the world coordinate system.

[0098] The filtering and denoising module processes the image of the target object that has been transformed to the world coordinate system, removing noise from the image and smoothing the edges of the target object in the image;

[0099] The thickness calculation module extracts the contour edge of each target object in the image after filtering and denoising, calculates the maximum inscribed circle within the contour edge of each target object, and uses the diameter of the maximum inscribed circle as the thickness of the target object.

[0100] This application provides a training method for a neural network module in an image recognition-based thickness detection system, used to train the neural network module in the aforementioned image recognition-based thickness detection system, including the following steps:

[0101] Step S11: Prepare a training image sample set: Obtain images containing the target object, label the category of each pixel in the image according to the features of the target object in each image, and use the category information as a label, thereby obtaining a training image sample set containing pixel information and corresponding label information.

[0102] Step S12 divides the training image sample set into a training set, a validation set, and a test set according to a predetermined ratio, preferably 3:1:1.

[0103] Step S13 uses the training set to train the neural network model. After each round of training, the loss function on the validation set is calculated using the neural network model. Training stops when the loss function on the validation set converges.

[0104] The loss function is composed of a weighted dice loss and a binary cross-entropy sum in a predetermined ratio, preferably 5:1. The formula for the weighted dice loss is as follows:

[0105] ;

[0106] ;

[0107] in, Weighted_Dice_Loss For weighted dice losses, and The Dice coefficients represent y values ​​of 0 and 1, and w0 and w1 represent the proportion of pixels with y values ​​of 0 and 1 in the total number of pixels; for Dice coefficient, y i The label value representing pixel i. The predicted value represents pixel i. N This represents the number of pixels to be verified;

[0108] The formula for binary cross-entropy is:

[0109] ;

[0110] in, BCE Represents the binary cross-entropy, where N represents the total number of data points, and y... i This is a binary tag, with a value of 0 or 1. This represents the probability that the output belongs to the label.

[0111] Step S14 uses test set data to validate the neural network model. If the validation fails, proceed to step S13 to continue training; if the validation passes, training is complete.

[0112] This application provides a training device for a neural network module in an image recognition-based thickness detection system, used to train the neural network module in the aforementioned image recognition-based thickness detection system, including:

[0113] The acquisition module is used to acquire a training image sample set, which includes images and corresponding labels, and to divide the training image sample set into a training set, a validation set and a test set according to a predetermined ratio.

[0114] The training module trains the neural network model based on the training set and its corresponding labels until the objective function of the neural network model converges on the validation set, thus obtaining the trained neural network model.

[0115] Test module: Validates the neural network model using test set data.

[0116] This application provides an electronic device that may include a processor and a memory. The memory stores programs. Specifically, the programs may include program code, which includes computer operation instructions. The memory may include high-speed RAM (Random Access Memory) and may also include non-volatile memory, such as multiple disk drives. The processor executes the computer execution instructions stored in the memory to implement the artificial intelligence-based image processing and training methods provided in the above method embodiments.

[0117] The processor may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0118] Optionally, in practical implementation, if the communication interface, memory, and processor are implemented independently, they can be interconnected via a bus to complete communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0119] Optionally, in a specific implementation, if the communication interface, memory, and processor are integrated on a single chip, then the communication interface, memory, and processor can communicate through an internal interface.

[0120] This application also provides a chip, including a processor and an interface. The interface is used to input and output data or instructions processed by the processor. The processor is used to execute the artificial intelligence-based image processing and training methods provided in the above method embodiments.

[0121] This application also provides a computer-readable storage medium, which may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a disk, or an optical disk. Specifically, the computer-readable storage medium stores program information, which is used for the artificial intelligence-based image processing method and training method in the above method embodiments.

[0122] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the artificial intelligence-based image processing method and training method as described in the above method embodiments.

[0123] This application also provides a computer program that enables a computer to execute the artificial intelligence-based image processing method and training method in the above method embodiments.

[0124] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0125] Implementing this invention has the following beneficial effects: This application provides a thickness detection method, detection device, training method, and training device based on image recognition. The thickness detection method based on image recognition includes: acquiring an image to be identified; extracting image feature information from the image to be identified; processing the image feature information of the image to be identified through a neural network to identify the shape of a target object; correcting each pixel of the target object shape image based on parameters acquired when acquiring the image to be identified, to obtain the shape and size of the target object in reality; filtering the target object shape image to smooth the contour edge of the target object; extracting the contour edge shape of the target object, extracting the largest inscribed circle within the contour edge shape, calculating the diameter of the largest inscribed circle, and obtaining the thickness of the target object; it can automatically and accurately complete the observation of the target object while meeting the real-time requirements of recognition, improving the robustness and accuracy of recognition.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A thickness detection method for detecting the thickness of snow on an ice surface, comprising: Step S1: obtaining an image to be recognized; Step S2: extracting image feature information of the image to be recognized; Step S3: processing the image feature information of the image to be recognized through a neural network, identifying each pixel point, determining whether the pixel point is a target object or a non-target object, collecting the pixel points identified as target objects, and obtaining an image of the shape of the target object; wherein the neural network comprises an encoding part and a decoding part, the encoding part comprises convolution pooling modules, each convolution pooling module comprises a convolution block and a max pooling layer; the decoding part comprises convolution pooling modules, each convolution pooling module comprises a convolution block and a max pooling layer; after each convolution block of the encoding part and the decoding part receives an input, the input is processed in the convolution block via path one and path two, respectively, the input is processed via a 1x1 convolution kernel in path one to obtain a first output, and the input is processed via batch normalization, linear rectifier function activation, and convolution in path two to obtain a second output, the first output and the second output are added to obtain the output of the convolution block; The convolution block of the decoding part receives the output of the convolutional pooling module of the previous layer of the decoding part and the output of the convolution block of the corresponding encoding part, and then is processed by the spatial attention mechanism module, and is sent to path one and path two for processing respectively; the spatial attention mechanism module receives the output of the convolutional pooling module of the previous layer of the decoding part and the output from the encoding part, and respectively performs convolution processing on the output of the convolutional pooling module of the previous layer of the decoding part and the output from the encoding part, and adds the processed results to obtain an integrated result, then performs convolution processing on the integrated result after linear rectification function activation, and obtains spatial attention weight through a Sigmoid activation function, and multiplies the output from the encoding part by the spatial attention weight to obtain the encoding part feature processed by the spatial attention, and then adds the encoding part feature processed by the spatial attention and the received The output result S of the spatial attention mechanism module is obtained by adding the output of the convolutional pooling module of the previous layer of the decoding part and the output from the encoding part, that is: where S is the output of the spatial attention mechanism module, x e is the original feature from the encoding part without processing, x d is the output of the last layer of the decoding part after up-sampling, σ is the ReLU function, δ is the Sigmoid function, g p is the spatial attention weight, 、 、 are the weights of the 1x1 convolution kernel in the spatial attention mechanism module, where and 、 、 The matrix multiplication process represents a linear convolution operation; Step S4: based on parameters when the image to be recognized is obtained, corresponding each pixel point of the image of the shape of the target object to a position coordinate in a world coordinate system to correct each pixel point of the image of the shape of the target object, and obtaining the shape and size of the target object in reality; Step S5: filtering the image of the shape of the target object to smooth the contour edge of the target object; Step S6: extracting the contour edge shape of the target object, extracting the largest inscribed circle within the contour edge shape, calculating the diameter of the largest inscribed circle, and obtaining the thickness of the target object; wherein the parameters when the image to be recognized is obtained in step S4 include internal parameters and external parameters, the internal parameters include the focal length, pixel size, and optical axis position of the image capturing device, and the external parameters include the water outlet height and optical axis pointing angle.

2. The thickness detection method according to claim 1, wherein In step S3, each layer of convolution blocks of the encoding part corresponds to a convolution block of a corresponding layer of the decoding part, and the convolution block of the decoding part receives the output of the convolution pooling module of the previous layer of the convolution block in the decoding part and the output of the convolution block of the corresponding encoding part.

3. The thickness detection method according to claim 1, wherein The input is processed via a 1x1 convolution kernel in path one to obtain a first output; and the input is sequentially processed via batch normalization, linear rectifier function activation, first convolution, batch normalization, linear rectifier function activation, and second convolution in path two to obtain a second output.

4. The thickness detection method according to claim 3, wherein A channel attention mechanism module is arranged in path two, the channel attention mechanism module receives the result of the first convolution as input, processes it via a global average pooling layer, and then sequentially processes it via dimension reduction, batch normalization, linear rectifier function activation, dimension increase, and Sigmoid activation function to obtain channel attention weights, and multiplies the channel attention weights with the result of the first convolution to obtain the output result of the channel attention mechanism module.

5. The thickness detection method according to claim 4, wherein In step S5, the method of filtering the image of the target object shape is as follows: first, the image of the target object shape is subjected to image morphological erosion to eliminate isolated points therein, remove noise and small connected domains; then the image of the target object shape is subjected to morphological dilation to restore the area of the target object, fill the holes in the image of the target object shape, and smooth the contour edges of the target object, thereby achieving the denoising filtering of the image of the target object shape.

6. The thickness detection method according to claim 4, wherein In step S6, the method of calculating the thickness of the target object includes: In step S6.1, the contour edge shape of the target object is extracted: for the image of the target object shape after denoising filtering, a scanning search is performed in a predetermined order, and when a pixel point value of 1 and its adjacent pixel points of 0 are searched, it is determined that the starting contour point is searched; other contour points that have not been searched are searched between the adjacent pixels of the contour point, and the contour points are connected to form a contour. The adjacent pixels of the current contour point are repeatedly searched and connected, and the contour boundary is continuously expanded until the starting contour point is returned. All connected contour points are regarded as the contour edge shape. The contour of the entire image is extracted according to the above method until the scanning of the entire image is completed, and the contours of all target objects in the image are obtained. In step S6.2, the largest inscribed circle is extracted: for all pixel points inside the contour of a target object in the image, the minimum distance from each pixel point to the contour of the target object is calculated, wherein the pixel point with the maximum value in the minimum distance is the center of the largest inscribed circle in the contour of the target object, and the maximum value is the radius of the largest inscribed circle, and the thickness of the target object is twice the maximum value.

7. A thickness detection system for implementing the thickness detection method according to any one of claims 1 to 6, comprising an image acquisition device and an image recognition system, the image acquisition device being configured to capture an image comprising a target object and transmit the image to the image recognition system for processing. The image recognition system includes: A neural network module for processing the image and identifying whether each pixel point in the image belongs to the target object; A projection transformation module for eliminating image distortion based on the parameters obtained when the image to be recognized is acquired, and for one-to-one correspondence between the target object in the image and the actual coordinates of the target object, to obtain the state of the target object in the world coordinate system; A filter denoising module for processing the image of the target object converted to the world coordinate system, removing noise in the image and smoothing the edges of the target object in the image; A thickness calculation module for extracting the contour edge of each target object in the image after filter denoising processing, finding the largest inscribed circle inside the contour edge of each target object, and taking the diameter of the largest inscribed circle as the thickness of the target object.

8. A training method of a neural network module in a thickness detection system, for training the neural network module in the thickness detection system according to claim 7, comprising the following steps: Step S11: preparing a training image sample set: obtaining an image containing a target object, marking the class of each pixel point in the image according to the characteristics of the target object in each image, and taking the class information as a label, thereby obtaining a training image sample set containing pixel points and corresponding label information; Step S12 divides the training image sample set into a training set, a validation set and a test set according to a predetermined ratio, and the predetermined ratio is 3:1:1; Step S13 trains the neural network model using the training set, calculates the loss function on the validation set using the neural network model after each round of training, and stops training until the loss function on the validation set converges; Step S14 verifies the neural network model using the test set data, and if the verification fails, goes to step S13 to continue training; if the verification passes, the training is completed.

9. The training method of claim 8, wherein, The loss function is composed of weighted dice loss and binary cross entropy according to a predetermined ratio, and the predetermined ratio is 5:1, wherein the formula of the weighted dice loss is: where, Weighted_Dice_Loss is the weighted dice loss, and represents the Dice coefficient for y taking 0 and 1 values, and represents the proportion of total pixels that are pixels with y taking 0 and 1 values; for Dice the coefficient, represents the label value of pixel i, represents the predicted value of pixel i, N represents the number of pixels to be verified; The formula of binary cross-entropy is: wherein, BCE represents binary cross-entropy, N represents the total number of data, is a binary label, taking a value of 0 or 1, represents the probability that the output belongs to the label.

10. A training device of a neural network module in a thickness detection system, used for training the neural network module in the thickness detection system according to claim 7, comprising: an acquisition module, configured to acquire a training image sample set, wherein the training image sample set contains images and corresponding labels, and divide the training image sample set into a training set, a validation set and a test set according to a predetermined ratio; a training module, configured to train a neural network model according to the training set and its corresponding labels until the objective function of the neural network model on the validation set converges, and obtain a trained neural network model; a test module, configured to verify the neural network model through the data of the test set.

11. An electronic device comprising: a processor and a memory; the memory is configured to store a computer program; the processor is configured to implement the thickness detection method according to any one of claims 1 to 6 and / or the training method of the neural network module in the thickness detection system according to claim 8 by executing the computer program.

12. A readable storage medium, wherein the readable storage medium stores computer execution instructions, and when the processor executes the computer execution instructions, the thickness detection method according to any one of claims 1 to 6 and / or the training method of the neural network module in the thickness detection system according to claim 8 is implemented.

Images