Intelligent sensing method for automatic driving of small ship

Abstract

The invention belongs to the technical field of intelligent ships, and discloses an intelligent sensing method for automatic driving of a small ship, and the method comprises the following specific steps: S1, preparing a navigation scene image sample, generating various navigation scene image data through a DCGAN network model, and constructing a data set; s2, training a YOLOv9s network model based on the data set, and performing test evaluation optimization; s3, acquiring image or video information in a navigation process through a visual sensor, and substituting the image or video information into the trained YOLOv9s network model for detection and identification; and S4, obtaining a target identification result of the image or video information in the navigation process in real time. According to the method, various navigation scene image data are generated by using the DCGAN network model, and the data set is constructed based on the data to train the YOLOv9s network model, so that the accuracy of small ship environment perception is improved, the model is light, comprehensive perception and analysis of the surrounding environment of the ship are realized, and the method is more suitable for variable marine environments.

Description

Technical Field

[0001] The present invention belongs to the technical field of smart ships, and in particular relates to an intelligent perception method for automatic driving of small ships. Background Art

[0002] With the continuous development of ship autonomous driving technology, small ships are becoming increasingly widely used in many fields such as commercial, transportation, monitoring, etc. The advantages of small ships are small size, light weight and strong flexibility. They are suitable for narrow and complex navigation environments and have lower construction and operation costs.

[0003] In the field of ship autonomous driving technology, intelligent perception of the marine environment is the primary issue for small ships to achieve autonomous driving. At present, existing perception systems on the market have problems such as the inability to obtain massive and variable environmental data and the inability to deploy algorithm models in a lightweight manner. Especially in complex sea conditions, at night or in low visibility, existing perception systems are prone to blind spots or misjudgments, that is, they cannot effectively detect changes in the surrounding environment under certain conditions, misjudge obstacles, or fail to detect potential collision risks in time; these problems will undoubtedly affect the safety and reliability of small ship autonomous driving, and greatly limit the practical application of autonomous driving technology. Therefore, there is a need for an intelligent perception system that can provide accurate perception and decision support in a variable environment to achieve autonomous navigation and obstacle avoidance of small ships. Summary of the invention

[0004] In view of the shortcomings of the background technology and taking into account the characteristics of remote sensing ship images, the present invention proposes an intelligent perception method for automatic driving of small ships, the steps of which include: Step S1, collecting navigation scene image sample data, generating various navigation scene image data through the DCGAN network model, and constructing a data set; Step S2, training a YOLOv9s network model based on the data set of navigation scene images, and performing test evaluation and optimization training on the model; Step S3, obtaining image or video information during the navigation process through a visual sensor, and substituting it into the trained YOLOv9s network model for detection and recognition; Step S4, obtaining target recognition results of the image or video information during the navigation process in real time.

[0005] Preferably, the process of step S1 includes: Step S11, collecting navigation scene image samples and preprocessing them; Step S12, training the DCGAN network model; Step S13, using the DCGAN network model to obtain diverse image data and construct a data set.

[0006] Preferably, the DCGAN network model in step S1 includes three SE modules and SN regularization.

[0007] Preferably, the SE module is composed of three processes: compression, excitation and recalibration; 1) Compression process: The input feature map of the navigation scene image is operated through global average pooling, and the spatial dimension of each channel is compressed to obtain the global feature description of each channel; 2) Excitation process: The obtained global feature description information is processed through a pair of fully connected layers. The first fully connected layer is used for dimensionality reduction, and the second layer is used for dimensionality increase. A weight vector in the range of [0,1] is generated through the Sigmoid function to obtain the weight value of a channel. 3) Recalibration process: Multiply the obtained channel weight value with each channel feature of the input feature map of the navigation scene image, and pass the weighted feature map to the subsequent convolutional layer or network.

[0008] Preferably, the SN regularization performs a normalization operation on the network weight matrix so that its maximum singular value does not exceed 1, and the specific process is: Given a weight matrix , The singular value decomposition formula is:

[0009] in, and is the unit orthogonal matrix, ∑ is a diagonal matrix containing The singular values ​​of The formula for spectral normalization is:

[0010] The maximum singular value of the normalized spectrum is 1: , where the calculation of the maximum singular value is achieved through the power iteration method.

[0011] Preferably, in step S2, the YOLOv9s network model is trained and the process of testing, evaluating and optimizing the training is as follows: S21. Prepare a rich and diverse dataset of navigation scene images and annotate the dataset with target recognition labels; S22. Train the YOLOv9s network model; S23. Evaluate the trained YOLOv9s network model and optimize the model.

[0012] Preferably, the YOLOv9s network model in step S2 includes a GSConv module, an EMA attention module and an Inner_CIOU loss function.

[0013] Preferably, the GSConv module is composed of a standard convolution (SC) and a depthwise separable convolution (DSC). The first two layers of AConv modules in the backbone of the YOLOv9s network model are replaced with GSConv modules to achieve lightweighting. The specific process is as follows: 1) The process of grouping the navigation scene image input feature map includes: dividing the navigation scene image input feature map into a plurality of groups, each group being responsible for processing a portion of the channels of the navigation scene image input feature map; 2) The process of grouping convolution kernels includes: the convolution kernels also need to be grouped, each group corresponds to a small convolution kernel, and these convolution kernels only perform convolution operations on the input channels of the corresponding group; 3) The process of optimizing the spatial features includes: the GSConv module further groups or restricts the spatial dimension; wherein, during the convolution operation, the GSConv module divides the spatial region of the input feature map of the navigation scene image into a plurality of small blocks and adjusts the spatial region; 4) The process of obtaining the output feature map of the navigation scene image includes: after the convolution operations of all groups are completed, the output feature maps obtained by all groups will be spliced ​​together to form the final output feature map of the navigation scene image.

[0014] Preferably, the EMA attention module is implemented as follows: in the process of calculating the attention score, exponentially weighted averaging the past attention weights; The attention weight is calculated by the inner product between the query and the key, and then the value is weighted summed according to the attention weight. The mathematical expression formula of the standard self-attention mechanism is:

[0015] in, is the query vector, is the key vector, is a vector of values, is the dimension of the key vector, is the similarity measure between the query and the key; The attention weights calculated at each step are exponentially weighted averaged. The mathematical expression formula of the EMA attention mechanism is:

[0016] in, is the exponentially weighted decay coefficient, represents the original attention weight calculated at step t.

[0017] Preferably, the calculation formula of the Inner_CIOU loss function is:

[0018]

[0019] in, is the intersection-and-union ratio, is the square of the Euclidean distance between the center point of the predicted box and the true box, is the square of the diagonal length of the minimum bounding box containing the predicted box and the true box, is a measure of the difference in box aspect ratio, is an adjustable weight parameter, is a function of the box center and aspect ratio.

[0020] Compared with the prior art, the present invention proposes an intelligent perception method for automatic driving of small ships, which has the following advantages: (1) The present invention uses the DCGAN network model to generate rich and diverse navigation scene image data and constructs a navigation scene data set; on this basis, the YOLOv9s network model is optimized using the GSConv module, the EMA attention module and the Inner_CIOU loss function, wherein the GSConv module is an efficient convolution operation that enhances the network's ability to capture global context information; the EMA attention module dynamically adjusts the weight of the feature map so that the model focuses more on key information and suppresses background noise, thereby improving the accuracy of target recognition; in addition, the Inner_CIOU loss function is an improved regression loss function that can more accurately adjust the position of the prediction box, thereby improving the positioning accuracy and convergence speed of the detection box; through the above model, the present invention realizes target recognition in a variable marine environment, significantly improves the accuracy and real-time performance of environmental perception, and the network model is lightweight and can be deployed on small ships; (2) The method of the present invention can adapt to the changing marine environment and navigation scenarios, has high recognition accuracy and a lightweight model, and can achieve comprehensive perception and analysis of the ship's surrounding environment, thereby improving the autonomous driving capability and safety of small ships in various sea conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of the method of the present invention; Figure 2 This is a DCGAN network model structure diagram of the present invention; Figure 3 It is the structural diagram of the SE module of the present invention; Figure 4It is a YOLOv9s network model structure diagram of the present invention; Figure 5 GSConv module structure diagram of the present invention; Figure 6 This is a structural diagram of the EMA module of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to further clearly and completely describe the technical solutions in the embodiments of the present application. It should be noted that the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0023] In order to make the invention objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application are further described in detail in conjunction with the drawings in the specification: In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the advantages of the present invention will be further illustrated by comparing the embodiments in conjunction with the drawings and specific implementation methods.

[0024] The present invention proposes an intelligent perception method for automatic driving of a small ship. The flowchart of the method is as follows: Figure 1 As shown, the steps of this method are described in detail: Step S1, collecting navigation scene image sample data, generating various navigation scene image data through the DCGAN network model, and constructing a data set; Furthermore, the process of step S1 includes: Step S11, collecting navigation scene image samples and performing preprocessing on them, such as cropping; Step S12, training the DCGAN network model; Step S13, using the DCGAN network model to obtain diverse image data and construct a data set.

[0025] Specifically, the DCGAN network model in step S1 includes three SE modules and SN regularization, such as Figure 2 shown.

[0026] Specifically, the SE module consists of three processes: Squeeze, Excitation, and Recalibration. Figure 3 As shown; 1) Compression process: The navigation scene image input feature map is operated through global average pooling (GAP) to compress the spatial dimension of each channel to obtain the global feature description of each channel; 2) Excitation process: The obtained global feature description information is processed through a pair of fully connected (FC) layers. The first fully connected layer is used for dimensionality reduction, and the second layer is used for dimensionality increase. A weight vector in the range of [0,1] is generated through the Sigmoid function to obtain the weight value of a channel. 3) Recalibration process: Multiply the obtained channel weight value with each channel feature of the input feature map of the navigation scene image to achieve re-weighting of the feature map, and pass the weighted feature map to the subsequent convolutional layer or network, so that the network can give higher weights to more important channel features and suppress unimportant channel features.

[0027] Specifically, the SN regularization performs a normalization operation on the network weight matrix so that its maximum singular value does not exceed 1, that is, the SN regularization constrains the weight matrix of each layer to ensure that its maximum singular value remains within a stable range, which can prevent gradient explosion during training; the specific process is: Given a weight matrix , The singular value decomposition formula is:

[0028] in, and is the unit orthogonal matrix, ∑ is a diagonal matrix containing The singular values ​​of The goal of spectral normalization is to find The maximum singular value of is used to normalize the weight matrix. The formula for spectral normalization is:

[0029] The maximum singular value of the normalized spectrum is 1: , the calculation of the maximum singular value is achieved by the power iteration method.

[0030] Step S2, training a YOLOv9s network model based on the navigation scene image dataset, and testing, evaluating and optimizing it; Furthermore, the process of step S2 includes: S21. Prepare a rich and diverse dataset of navigation scene images and annotate the dataset with target recognition labels; S22. Train the YOLOv9s network model; S23. Evaluate the trained YOLOv9s network model and optimize the model.

[0031] Specifically, the YOLOv9s network model in step S2 includes a GSConv module, an EMA attention module, and an Inner_CIOU loss function, such as Figure 4 shown.

[0032] Specifically, Figure 5 As shown in the figure, in order to realize the lightweight module, the first two layers of AConv modules in the backbone of YOLOv9s are replaced with GSConv modules to realize lightweight. The GSConv module is composed of standard convolution (SC) and depth-wise separable convolution (DSC). Its working principle is: the navigation scene image data is grouped according to the channel, and each group is subjected to SC with the corresponding convolution kernel; on this basis, DSC is performed on the output of each group, and then the results of DSC are combined by point-by-point convolution; point-by-point convolution usually uses 1x1 convolution for merging, which can effectively compress the calculation amount of channel dimension and spatial dimension; the specific process is as follows: 1) The process of grouping the input feature map of the navigation scene image includes: In the GSConv module, the input feature map of the navigation scene image (usually a multi-channel matrix) first needs to be divided into multiple groups, the purpose of which is to improve efficiency by reducing the calculation range of each convolution operation; assuming that the size of the input feature map of the navigation scene image is ,in, and are the height and width of the image, is the number of input channels, the number of channels is divided into multiple groups, each of which is responsible for processing a portion of the channels of the input feature map of the navigation scene image; for example, Divided into G groups, each group contains channels; 2) The process of grouping the convolution kernels includes: in the GSConv module, the convolution kernels also need to be grouped to match the grouping of the input feature map of the navigation scene image; each group corresponds to a small convolution kernel, and these convolution kernels only perform convolution operations on the input channels of the corresponding group; ①Assume that the size of the convolution kernel is ,in and is the height and width of the convolution kernel, is the number of input channels, is the number of output channels; ②The number of input channels of the convolution kernel Divide into G groups, each with channels, and the convolution kernel size corresponding to each group is , that is, each convolution kernel only processes part of the channels; ③The convolution operation of each group only involves input channels and output channels, reducing the amount of computation for each convolution operation; 3) The process of optimizing the spatial features includes: optimizing the spatial features in the GSConv module; the traditional convolution operation performs convolution on the entire navigation scene image or feature map, while the GSConv module further groups or restricts the spatial dimension to improve the computational efficiency; ① During the convolution operation, the GSConv module divides the spatial area of ​​the input feature map of the navigation scene image into multiple small blocks; for example, the image is divided into several local areas, and each area uses a separate convolution kernel for feature extraction. This approach helps to reduce computational redundancy and improve the accuracy of local feature extraction; ②The GSConv module can adjust the spatial area so that the convolution operation of each small block can focus more on the features of the local area, thereby improving the ability to capture fine-grained features; 4) The process of obtaining the output feature map of the navigation scene image includes: after the convolution operation of all groups is completed, the output feature maps obtained by all groups will be spliced ​​together to form the final output feature map of the navigation scene image; assuming that the output size of each group is , the final output feature map size is ; The outputs of each group are spliced ​​along the channel dimension to obtain the output feature map of the final navigation scene image; The GSConv module effectively reduces the computational complexity while improving the network's expressiveness and efficiency.

[0033] Specifically, the EMA attention module is implemented as follows: in the process of calculating the attention score, the past attention weights are exponentially weighted averaged, and then the current attention weight calculation is adjusted; the specific process is as follows: Add an EMA attention module to the Neck network of YOLOv9s, embedded between the RepNCSPELAN module and P5, such as Figure 6 As shown in Figure 2, the working principle of the EMA attention module is: 1) The process of preparing queries, keys, and values ​​includes: , , ;in, represents the sequence length, are the dimensions of the query and key, is the dimension of the value; 2) The process of calculating standard attention includes: first calculating the attention weight by the inner product between the query and the key, and then weighted summing the values ​​according to the attention weight; The formula for weighted summation is:

[0034] in, is the similarity measure between the query and the key; Then the normalized weights are obtained through softmax; 3) The process of smoothing the historical weights of EMA includes: exponentially weighted averaging the attention weights calculated at each step, so that the attention weights farther away gradually decay; The calculation formula of the EMA attention module is:

[0035] in, represents the smoothed attention weight at step t, represents the original attention weight calculated at step t; is a smoothing factor, usually set to a constant close to 1 (e.g. 0.9), Determines the influence of the most recent attention weight on the final result; 4) The process of calculating the weighted sum value includes: the EMA attention module uses the smoothed EMA attention weights to weight the sum value; The formula for the weighted sum is:

[0036] in, is the smoothed attention weight, is a matrix of values; The weighted value after EMA smoothing will be used as the final output; this output contains a smoothed and weighted representation that can effectively reflect historical information while avoiding over-reliance on information at a certain moment; 5) The process of updating the EMA weight includes: after each step of calculation is completed, the EMA weight is updated to prepare for the next round of calculation; The calculation formula for EMA weight update is:

[0037] In this way, the model is able to dynamically smooth and adjust weights at each time step, thereby avoiding over-reliance on calculations at a single moment and enhancing the ability to model long-term dependencies.

[0038] Specifically, the calculation formula of the Inner_CIOU loss function is:

[0039]

[0040] in, is the intersection-and-union ratio, is the square of the Euclidean distance between the center point of the predicted box and the true box, is the square of the diagonal length of the minimum bounding box containing the predicted box and the true box, is a measure of the difference in box aspect ratio, is an adjustable weight parameter, is a function of the box center and aspect ratio.

[0041] Step S3, obtaining image or video information during the navigation process through a visual sensor, and substituting it into the trained YOLOv9s network model for detection and recognition; Specifically, in step S3, the detection and recognition process is: using visual sensors to obtain real-time image or video information of the navigation process, using the trained YOLOv9s model for detection and recognition, so as to realize real-time information processing and environmental perception of the surrounding environment by small ships. The model adapts to the changing ocean environment and navigation scenes, and has high recognition accuracy.

[0042] Step S4, obtaining target recognition results of the image or video information during the navigation process in real time.

[0043] Specifically, in step S4, the process of obtaining the target recognition results of the image or video information during the navigation process in real time is: the trained YOLOv9s network model directly outputs the target detection and recognition results, performs information processing and environmental perception of the navigation scene in real time, and has high recognition accuracy, thereby improving the automatic driving capability and safety of small ships in various sea conditions.

[0044] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0045] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An intelligent perception method for automatic driving of small ships, characterized in that: include: Step S1, collecting navigation scene image sample data, generating various navigation scene image data through the DCGAN network model, and constructing a data set; Step S2, training a YOLOv9s network model based on the data set of navigation scene images, and performing test evaluation and optimization training on the model; Step S3, obtaining image or video information during the navigation process through a visual sensor, and substituting it into the trained YOLOv9s network model for detection and recognition; Step S4, obtaining target recognition results of the image or video information during the navigation process in real time.

2. According to claim 1, the intelligent perception method for automatic driving of a small ship is characterized in that: In step S1, the process of constructing a data set includes: Step S11, collecting navigation scene image sample data and preprocessing it; Step S12, training the DCGAN network model; Step S13, using the DCGAN network model to obtain diverse image data and construct a data set.

3. The intelligent perception method for automatic driving of a small ship according to claim 1 is characterized in that: The DCGAN network model in step S1 includes three SE modules and SN regularization.

4. The intelligent perception method for automatic driving of a small ship according to claim 3 is characterized in that: The SE module is composed of three processes: compression, excitation and recalibration; 1) Compression process: The input feature map of the navigation scene image is operated through global average pooling, and the spatial dimension of each channel is compressed to obtain the global feature description of each channel; 2) Excitation process: The obtained global feature description information is processed through a pair of fully connected layers. The first fully connected layer is used for dimensionality reduction, and the second layer is used for dimensionality increase. A weight vector in the range of [0,1] is generated through the Sigmoid function to obtain the weight value of a channel. 3) Recalibration process: Multiply the obtained channel weight value with each channel feature of the input feature map of the navigation scene image, and pass the weighted feature map to the subsequent convolutional layer or network.

5. The intelligent perception method for automatic driving of a small ship according to claim 3 is characterized in that: The SN regularization performs a normalization operation on the network weight matrix so that its maximum singular value does not exceed 1. The specific process is as follows: Given a weight matrix , The singular value decomposition formula is: ； in, and is the unit orthogonal matrix, ∑ is a matrix containing The diagonal matrix of the singular values ​​of ; The formula for spectral normalization is: ； The maximum singular value of the normalized spectrum is 1: , where the calculation of the maximum singular value is achieved through the power iteration method.

6. The intelligent perception method for automatic driving of a small ship according to claim 1, characterized in that: In step S2, the YOLOv9s network model is trained and the process of testing, evaluating and optimizing the training is as follows: S21. Prepare a rich and diverse dataset of navigation scene images and annotate the dataset with target recognition labels; S22. Train the YOLOv9s network model; S23. Evaluate the trained YOLOv9s network model and optimize the model.

7. The intelligent perception method for automatic driving of a small ship according to claim 1 is characterized in that: The YOLOv9s network model in step S2 includes a GSConv module, an EMA attention module, and an Inner_CIOU loss function.

8. The intelligent perception method for automatic driving of a small ship according to claim 7, characterized in that: The GSConv module is composed of standard convolution and depth-separable convolution. The first two layers of AConv modules in the backbone of the YOLOv9s network model are replaced with GSConv modules to achieve lightweight. The specific process is as follows: 1) The process of grouping the input feature map of the navigation scene image includes: dividing the input feature map of the navigation scene image into a plurality of groups, each group being responsible for processing a part of the channels of the input feature map of the navigation scene image; 2) The process of grouping convolution kernels includes: the convolution kernels also need to be grouped, each group corresponds to a small convolution kernel, and these convolution kernels only perform convolution operations on the input channels of the corresponding group; 3) The process of optimizing the spatial features includes: the GSConv module further groups or restricts the spatial dimension; wherein, during the convolution operation, the GSConv module divides the spatial region of the input feature map of the navigation scene image into multiple small blocks and adjusts the spatial region; 4) The process of obtaining the output feature map of the navigation scene image includes: after the convolution operations of all groups are completed, the output feature maps obtained by all groups will be spliced ​​together to form the final output feature map of the navigation scene image.

9. The intelligent perception method for automatic driving of a small ship according to claim 7, characterized in that: The EMA attention module is implemented as follows: in the process of calculating the attention score, the past attention weights are exponentially weighted averaged; The attention weight is calculated by the inner product between the query and the key, and then the value is weighted summed according to the attention weight. The mathematical expression formula of the standard self-attention mechanism is: ； in, is the query vector, is the key vector, is a vector of values, is the dimension of the key vector, is the similarity measure between the query and the key; The attention weights calculated at each step are exponentially weighted averaged. The mathematical expression formula of the EMA attention mechanism is: ； in, is the exponentially weighted decay coefficient, represents the original attention weight calculated at step t.

10. The intelligent perception method for automatic driving of a small ship according to claim 7, characterized in that: The calculation formula of the Inner_CIOU loss function is: ； ； in, is the intersection-and-union ratio, is the square of the Euclidean distance between the center point of the predicted box and the true box, is the square of the diagonal length of the minimum bounding box containing the predicted box and the true box, is a measure of the difference in box aspect ratio, is an adjustable weight parameter, is a function of the box center and the aspect ratio.

Images