Ship modeling method, intelligent vehicle for automatic scanning and modeling and readable medium

Abstract

The embodiment of the invention discloses a ship modeling method, an intelligent vehicle for automatic scanning and modeling and a readable medium. A specific embodiment of the method comprises the following steps: in response to received scanning starting information, controlling a horizontal laser radar arranged on an intelligent vehicle to collect point cloud data of a target ship in a horizontal direction as first point cloud data, a vertical laser radar arranged on the intelligent vehicle is controlled to collect point cloud data of the target ship in the vertical direction to serve as second point cloud data, and the intelligent vehicle is provided with a horizontal laser radar and the vertical laser radar; performing coordinate conversion on the collected second point cloud data to obtain third point cloud data; generating fusion point cloud data according to the collected first point cloud data and the converted third point cloud data; and constructing a three-dimensional ship model corresponding to the target ship based on the fused point cloud data. According to the embodiment, the completeness of scanning the ship and the completeness of the three-dimensional ship model are improved, and the delay time of the ship manufacturing construction period is shortened.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a ship modeling method, a smart car for automatic scanning and modeling, and a readable medium. Background Art

[0002] With the rapid development of computer technology, the integrated automation technology of ships has been increasingly developed and widely used on ships. Scanning and modeling ships is the key technology of integrated automation technology of ships. At present, the laser radars equipped on smart cars are all for the purpose of obstacle avoidance, and the hull is scanned manually.

[0003] However, when the above method is used to scan and model a ship, the following technical problems often occur: the laser radar used for obstacle avoidance can only scan obstacles within the height area of ​​the vehicle body and form a map at most, and the integrity of the scan of the surrounding environment is poor, resulting in poor integrity of the modeled three-dimensional ship model. In addition, ships are usually tall, and manual scanning takes a long time, resulting in long delays in the ship manufacturing schedule.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose a ship modeling method, an intelligent vehicle for automatic scanning and modeling, and a computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a ship modeling method, which is applied to an intelligent vehicle for automatic scanning and modeling, comprising: in response to receiving a scan start information, controlling a horizontal laser radar provided on the above-mentioned intelligent vehicle to collect point cloud data of the target ship in the horizontal direction as the first point cloud data, and controlling a vertical laser radar provided on the above-mentioned intelligent vehicle to collect point cloud data of the above-mentioned target ship in the vertical direction as the second point cloud data, wherein the above-mentioned horizontal laser radar and the above-mentioned vertical laser radar are provided on the above-mentioned intelligent vehicle; performing coordinate conversion on the collected second point cloud data to obtain third point cloud data; generating fused point cloud data according to the collected first point cloud data and the converted third point cloud data; and constructing a three-dimensional ship model corresponding to the above-mentioned target ship based on the above-mentioned fused point cloud data.

[0008] Optionally, the horizontal laser radar is arranged on a horizontal rotating device, and the horizontal rotating device is arranged on the top of the smart car; and the horizontal laser radar arranged on the smart car is controlled to collect the point cloud data of the target ship in the horizontal direction as the first point cloud data, including: controlling the horizontal rotating device to drive the horizontal laser radar to rotate according to a preset first rotation mode; controlling the horizontal laser radar to collect the point cloud data of the target ship in the horizontal direction as the first point cloud data during the rotation process.

[0009] Optionally, the vertical laser radar is arranged on a vertical rotation device, and the vertical rotation device is arranged on the front side or the rear side of the smart car; and the vertical laser radar arranged on the smart car is controlled to collect the point cloud data of the target ship in the vertical direction as the second point cloud data, including: controlling the vertical rotation device to drive the vertical laser radar to rotate according to a preset second rotation mode; controlling the vertical laser radar to collect the point cloud data of the target ship in the vertical direction as the second point cloud data during the rotation process.

[0010] Optionally, the above-mentioned coordinate transformation of the collected second point cloud data to obtain third point cloud data includes: extracting the installation position information corresponding to the above-mentioned vertical laser radar from the configuration file corresponding to the above-mentioned smart car as the first installation position information; according to the above-mentioned first installation position information, converting the collected second point cloud data to the vehicle coordinate system corresponding to the above-mentioned smart car to obtain third point cloud data.

[0011] Optionally, the above-mentioned generation of fused point cloud data based on the collected first point cloud data and the converted third point cloud data includes: obtaining the installation position information corresponding to the above-mentioned horizontal laser radar from the configuration file corresponding to the above-mentioned smart car as the second installation position information; according to the above-mentioned second installation position information, converting the collected first point cloud data to the vehicle coordinate system corresponding to the above-mentioned smart car to obtain fourth point cloud data; combining the above-mentioned third point cloud data and the above-mentioned fourth point cloud data into initial fused point cloud data; performing point cloud deduplication processing on the above-mentioned initial fused point cloud data to obtain the initial fused point cloud data after deduplication processing as the fused point cloud data.

[0012] Optionally, the three-dimensional ship model corresponding to the target ship is constructed based on the fused point cloud data, including: denoising the fused point cloud data to obtain denoised point cloud data; performing ground recognition processing on the denoised point cloud data to obtain a ground recognition result; determining whether the ground recognition result includes point data of a corresponding recognition category representing the ground; in response to determining that the ground recognition result includes point data of a corresponding recognition category representing the ground, determining each point data of the corresponding recognition category representing the ground included in the ground recognition result as a ground point set; deleting each point data included in the ground point set from the denoised point cloud data. ground points to obtain initial ship point cloud data; perform background recognition processing on the initial ship point cloud data to obtain a background recognition result; determine whether the background recognition result includes point data of a corresponding recognition category representing the background; in response to determining that the background recognition result includes point data of a corresponding recognition category representing the background, determine each point data of the corresponding recognition category representing the background included in the background recognition result as a background point set; delete each background point included in the background point set from the initial ship point cloud data to obtain ship point cloud data; and construct a three-dimensional ship model corresponding to the target ship based on the ship point cloud data.

[0013] Optionally, the above-mentioned three-dimensional ship model corresponding to the above-mentioned target ship is constructed based on the above-mentioned ship point cloud data, including: performing key point detection on the above-mentioned ship point cloud data to obtain a key point set; performing surface fitting processing on the above-mentioned ship point cloud data according to the above-mentioned key point set to obtain an initial three-dimensional ship model; performing detail enhancement processing on the above-mentioned initial three-dimensional ship model to obtain a refined three-dimensional ship model; performing vertex simplification processing on the above-mentioned refined three-dimensional ship model to obtain a simplified three-dimensional ship model; and performing texture mapping processing on the above-mentioned simplified three-dimensional ship model according to color information corresponding to each point data in the above-mentioned ship point cloud data to obtain a three-dimensional ship model.

[0014] In a second aspect, some embodiments of the present disclosure provide an intelligent vehicle for automatic scanning and modeling, comprising: a horizontal laser radar configured to collect point cloud data of a target ship in a horizontal direction; a vertical laser radar configured to collect point cloud data of the target ship in a vertical direction; one or more processors; a storage device storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect.

[0015] In a third aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

[0016] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the ship modeling method of some embodiments of the present disclosure, the integrity of scanning the ship and the integrity of the three-dimensional ship model are improved, and the delay time of the ship manufacturing period is shortened. Specifically, the reasons for the poor integrity of the modeled three-dimensional ship model and the long delay time of the ship manufacturing period are: the laser radar for the purpose of obstacle avoidance can only scan the obstacles within the vehicle height area at most and form a map, and the integrity of scanning the surrounding environment is poor, resulting in poor integrity of the modeled three-dimensional ship model. In addition, the ship is usually high, and the manual scanning takes a long time, resulting in a long delay time of the ship manufacturing period. Based on this, the ship modeling method of some embodiments of the present disclosure, first, in response to receiving the scanning start information, controls the horizontal laser radar set on the above-mentioned smart car to collect the point cloud data of the target ship in the horizontal direction as the first point cloud data, and controls the vertical laser radar set on the above-mentioned smart car to collect the point cloud data of the above-mentioned target ship in the vertical direction as the second point cloud data, wherein the above-mentioned horizontal laser radar and the above-mentioned vertical laser radar are set on the above-mentioned smart car. Thus, when scanning starts, the horizontal laser radar and the vertical laser radar can be controlled to collect point cloud data in the horizontal and vertical directions of the ship respectively. Then, the coordinate conversion is performed on the collected second point cloud data to obtain the third point cloud data. Thus, the point cloud data in the vertical direction can be converted to a unified coordinate system. Then, fused point cloud data is generated according to the collected first point cloud data and the converted third point cloud data. Thus, the point cloud data in the horizontal and vertical directions can be fused into the overall point cloud data. Finally, based on the above-mentioned fused point cloud data, a three-dimensional ship model corresponding to the above-mentioned target ship is constructed. Thus, a complete three-dimensional ship model can be constructed based on the overall point cloud data. Also because the intelligent car is provided with a horizontal laser radar for collecting point cloud data in the horizontal direction, and a vertical laser radar for collecting point cloud data in the vertical direction is added, the point cloud data of a relatively complete ship can be scanned, thereby improving the integrity of the scanning of the ship and the integrity of the three-dimensional ship model. And because it is automatically scanned by laser radars in two directions, there is no need to scan manually, which improves the scanning speed, thereby shortening the delay time of the ship manufacturing period. As a result, the integrity of the ship scan and the integrity of the 3D ship model are improved, and delays in ship manufacturing are shortened. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0018] Figure 1is a flow chart of a ship modeling method according to some embodiments of the present disclosure;

[0019] Figure 2 is a schematic diagram of a structure of a smart car for automatic scanning and modeling according to some embodiments of the present disclosure;

[0020] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0022] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0023] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0024] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0025] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0026] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0027] Figure 1 It shows a process 100 of some embodiments of the ship modeling method. The process 100 of the ship modeling method is applied to an intelligent vehicle for automatic scanning and modeling, and includes the following steps:

[0028] Step 101, in response to receiving the scanning start information, controlling the horizontal laser radar set on the smart car to collect the point cloud data of the target ship in the horizontal direction as the first point cloud data, and controlling the vertical laser radar set on the smart car to collect the point cloud data of the target ship in the vertical direction as the second point cloud data.

[0029] In some embodiments, the execution subject of the ship outer plate spraying control method (for example, the vehicle terminal installed on the smart car) can control the horizontal laser radar set on the above-mentioned smart car to collect the point cloud data of the target ship in the horizontal direction as the first point cloud data, and control the vertical laser radar set on the above-mentioned smart car to collect the point cloud data of the target ship in the vertical direction as the second point cloud data in response to receiving the scanning start information. Among them, the above-mentioned smart car can be an unmanned vehicle, which can be used for automatic scanning and modeling. The above-mentioned smart car can be provided with a horizontal laser radar and a vertical laser radar. The horizontal laser radar can be placed horizontally on the roof of the above-mentioned smart car. The above-mentioned vertical laser radar can be placed vertically at the front or rear of the above-mentioned smart car. The above-mentioned horizontal laser radar can be a laser radar for 360-degree scanning in the horizontal direction. The above-mentioned vertical laser radar can be a laser radar for 360-degree scanning in the vertical direction. The above-mentioned scanning start information can be information representing the start of scanning. For example, the scanning start information can be "start scanning". The scanning start information can be sent by the operator through a terminal device. The terminal device can be but not limited to: a mobile phone, a tablet computer, a desktop computer, and a vehicle terminal on the above-mentioned smart car. The terminal device is connected to the smart car in communication. In practice, the execution subject can synchronously control the horizontal laser radar and the vertical laser radar to collect point cloud data of the target ship in the horizontal direction and the vertical direction respectively.

[0030] As an example, a smart car for automatic scanning and modeling can be found in Figure 2 . Figure 2 In the embodiment, the horizontal laser radar 21 is arranged on the top of the smart car, and the vertical laser radar 22 is arranged on the rear of the smart car.

[0031] Optionally, the horizontal laser radar can be arranged on a horizontal rotating device. The horizontal rotating device can be arranged on the top of the smart car. The surface of the horizontal rotating device in contact with the horizontal laser radar is horizontal. The horizontal rotating device can drive the horizontal laser radar to rotate in the horizontal direction.

[0032] In some optional implementations of some embodiments, the execution subject may control the horizontal laser radar provided on the smart vehicle to collect point cloud data of the target ship in the horizontal direction as the first point cloud data through the following steps:

[0033] The first step is to control the horizontal rotation device to drive the horizontal laser radar to rotate according to a preset first rotation mode. For example, the preset first rotation mode can be clockwise rotation or counterclockwise horizontal rotation.

[0034] The second step is to control the horizontal laser radar to collect the point cloud data of the target ship in the horizontal direction as the first point cloud data during the rotation process. Therefore, even if the top of the smart car is not horizontal, the horizontal laser radar can be placed in a horizontal state through the horizontal rotating device.

[0035] Optionally, the vertical laser radar can be arranged on a vertical rotating device. The vertical rotating device can be arranged on the front side or the rear side of the smart car.

[0036] In some optional implementations of some embodiments, the execution subject may control the vertical laser radar provided on the smart vehicle to collect point cloud data of the target ship in the vertical direction as the second point cloud data by:

[0037] The first step is to control the vertical rotation device to drive the vertical laser radar to rotate according to a preset second rotation mode. For example, the preset second rotation mode can be clockwise rotation or counterclockwise vertical rotation.

[0038] The second step is to control the vertical laser radar to collect the point cloud data of the target ship in the vertical direction as the second point cloud data during the rotation process. Therefore, even if the front side or the rear side of the smart car is not vertical, the vertical laser radar can be placed in a vertical state through the vertical rotation device.

[0039] Step 102: performing coordinate transformation on the collected second point cloud data to obtain third point cloud data.

[0040] In some embodiments, the execution subject may perform coordinate transformation on the collected second point cloud data to obtain third point cloud data. In practice, the execution subject may transform the second point cloud data into the coordinate system of the horizontal laser radar by coordinate transformation to obtain the third point cloud data.

[0041] In some optional implementations of some embodiments, the execution subject may perform coordinate transformation on the collected second point cloud data through the following steps to obtain third point cloud data:

[0042] The first step is to extract the installation position information corresponding to the vertical laser radar from the configuration file corresponding to the smart car as the first installation position information. The configuration file may be a file for describing the geometric structure, connection structure, sensor configuration and other information of the smart car. For example, the configuration file may be a URDF (Unified Robot Description Format) file. The first installation position information may be information for describing the installation position of the vertical laser radar. For example, the first installation position information may include the coordinates of the vertical laser radar relative to the vehicle reference system of the smart car.

[0043] In the second step, according to the first installation position information, the collected second point cloud data is converted to the vehicle coordinate system corresponding to the smart car to obtain the third point cloud data. In practice, the execution subject can convert the second point cloud data to the vehicle coordinate system by using the first installation position information through coordinate transformation to obtain the third point cloud data.

[0044] Step 103: Generate fused point cloud data according to the collected first point cloud data and the converted third point cloud data.

[0045] In some embodiments, the execution entity may generate fused point cloud data based on the collected first point cloud data and the converted third point cloud data. In practice, the execution entity may combine the first point cloud data and the third point cloud data into fused point cloud data.

[0046] In some optional implementations of some embodiments, the execution subject may generate fused point cloud data according to the collected first point cloud data and the converted third point cloud data through the following steps:

[0047] The first step is to obtain the installation position information corresponding to the horizontal laser radar from the configuration file corresponding to the smart car as the second installation position information. The second installation position information may be information used to describe the installation position of the horizontal laser radar. For example, the second installation position information may include the coordinates of the horizontal laser radar relative to the vehicle reference system of the smart car.

[0048] In the second step, according to the second installation position information, the first point cloud data collected is converted to the vehicle coordinate system corresponding to the smart car to obtain the fourth point cloud data. In practice, the execution subject can convert the first point cloud data to the vehicle coordinate system by using the second installation position information through coordinate transformation to obtain the third point cloud data.

[0049] The third step is to combine the third point cloud data and the fourth point cloud data into initial fused point cloud data.

[0050] The fourth step is to perform point cloud deduplication processing on the above initial fused point cloud data to obtain the deduplication-processed initial fused point cloud data as the fused point cloud data. In practice, the above execution subject can remove duplicate points through an approximate nearest neighbor search algorithm (such as FLANN) to obtain fused point cloud data. In this way, the amount of data after fusion can be reduced.

[0051] Step 104: construct a three-dimensional ship model corresponding to the target ship based on the fused point cloud data.

[0052] In some embodiments, the execution entity may construct a three-dimensional ship model corresponding to the target ship based on the fused point cloud data.

[0053] In some optional implementations of some embodiments, the execution subject may construct a three-dimensional ship model corresponding to the target ship based on the fused point cloud data through the following steps:

[0054] The first step is to perform denoising on the fused point cloud data to obtain denoised point cloud data. In practice, the execution subject may use statistical filtering, distance filtering and other methods to remove outliers and noise in the fused point cloud data to obtain denoised point cloud data.

[0055] The second step is to perform ground recognition processing on the denoised point cloud data to obtain a ground recognition result. In practice, the execution entity may use a segmentation algorithm to perform ground segmentation processing on the denoised point cloud data to obtain a ground recognition result. The ground recognition result may include each point data and the recognition category corresponding to each point data. For example, the segmentation algorithm may be a RANSAC algorithm.

[0056] The third step is to determine whether the above ground recognition result includes point data of the corresponding recognition category representing the ground.

[0057] In the fourth step, in response to determining that the ground recognition result includes point data of corresponding recognition categories representing the ground, each point data of corresponding recognition categories representing the ground included in the ground recognition result is determined as a ground point set.

[0058] The fifth step is to delete each ground point included in the above ground point set from the above denoised point cloud data to obtain initial ship point cloud data.

[0059] Step 6: Perform background recognition processing on the initial ship point cloud data to obtain background recognition results. In practice, the execution entity may use a segmentation algorithm to perform background segmentation processing on the initial ship point cloud data to obtain background recognition results. The background recognition results may include each point data and the recognition category corresponding to each point data. For example, the segmentation algorithm may be a RANSAC algorithm.

[0060] The seventh step is to determine whether the background recognition result includes point data of the corresponding recognition category representing the background.

[0061] In the eighth step, in response to determining that the background recognition result includes point data of the corresponding recognition category representing the background, each point data of the corresponding recognition category representing the background included in the background recognition result is determined as a background point set.

[0062] In the ninth step, each background point included in the background point set is deleted from the initial ship point cloud data to obtain the ship point cloud data. Thus, the point cloud data used to construct the three-dimensional ship model can be pre-processed in advance.

[0063] The tenth step is to construct a three-dimensional ship model corresponding to the target ship based on the ship point cloud data.

[0064] In some optional implementations of some embodiments, the execution subject may construct a three-dimensional ship model corresponding to the target ship based on the ship point cloud data through the following steps:

[0065] The first step is to perform key point detection on the above-mentioned ship point cloud data to obtain a key point set. In practice, the above-mentioned execution entity can use a key point extraction algorithm to perform key point detection on the above-mentioned ship point cloud data to obtain a key point set. For example, the key point extraction algorithm may include but is not limited to at least one of the following: SHOT (Shape Context on Tangent Space), FPFH (Fast Point Feature Histograms), Harris-Laplace, SIFT (Scale-Invariant Feature Transform).

[0066] The second step is to perform surface fitting processing on the ship point cloud data according to the key point set to obtain an initial three-dimensional ship model. In practice, the execution subject can use a Delaunay triangulation algorithm or a Poisson surface reconstruction algorithm to perform surface fitting processing on the ship point cloud data according to the key point set to obtain an initial three-dimensional ship model.

[0067] The third step is to perform detail enhancement processing on the initial three-dimensional ship model to obtain a refined three-dimensional ship model. In practice, the execution subject may perform detail enhancement processing on the initial three-dimensional ship model using a local smoothing algorithm to obtain a refined three-dimensional ship model. For example, the local smoothing algorithm may be a Laplace smoothing algorithm.

[0068] The fourth step is to simplify the vertices of the refined three-dimensional ship model to obtain a simplified three-dimensional ship model. In practice, the execution subject can simplify the vertices of the refined three-dimensional ship model by using methods such as vertex clustering or quadtree to obtain a simplified three-dimensional ship model.

[0069] In the fifth step, according to the color information corresponding to each point data in the above-mentioned ship point cloud data, the above-mentioned simplified three-dimensional ship model is texture mapped to obtain a three-dimensional ship model. In practice, the above-mentioned execution subject can map the color information corresponding to each point data in the above-mentioned ship point cloud data to the above-mentioned simplified three-dimensional ship model to obtain a three-dimensional ship model.

[0070] In the process of adopting technical solutions to solve the technical problems mentioned in the background technology, the following technical problems are often accompanied: during scanning, the laser radar collects point cloud data with fixed acquisition accuracy parameters. When the battery of the smart car is low on power, the scanning task will be interrupted. If the power is still insufficient after charging and running for a period of time, the scanning needs to be interrupted again for charging, resulting in a long number of scanning interruptions and interruption times, which leads to a long delay in the shipbuilding period. In response to this technical problem, the conventional solution is generally: before starting the scan, set the acquisition accuracy of the laser radar according to the remaining power and experience; or directly charge it after an interruption. However, the above conventional solutions still have the following problems: the method of setting the laser radar acquisition accuracy based on experience has relatively single factors that can be considered and has poor accuracy; the method of directly charging it after an interruption requires waiting for the battery to be fully charged, resulting in a longer scanning interruption time, which leads to a longer delay in the shipbuilding period.

[0071] Considering the problems of the above conventional solutions, facing the above technical problems: when scanning, the laser radar collects point cloud data with fixed acquisition accuracy parameters. When the battery of the smart car is low, the scanning task will be interrupted. When the scanning is started again, the continuity of the collected point cloud data is poor. Combining the mature application of sensor technology and the convenience of obtaining historical data series, it can be decided to adopt the following solution.

[0072] Optionally, the smart car may also be provided with a light intensity sensor and a temperature sensor. The light intensity sensor may be used to collect light intensity in the environment. The temperature sensor may be used to collect temperature in the environment.

[0073] Optionally, the above execution entity may further perform the following steps:

[0074] The first step is to control the above-mentioned smart car to rotate in place at a preset angle at the scanning start position in response to receiving the scanning start information, and to collect light intensity and temperature through the above-mentioned light intensity sensor and the above-mentioned temperature sensor during the rotation of the above-mentioned smart car to obtain an environmental information set, wherein each environmental information in the above-mentioned environmental information set includes the light intensity and temperature collected at the same time.

[0075] The second step is to determine the rotation angle corresponding to each piece of environmental information in the environmental information set. The rotation angle corresponding to the environmental information may be the rotation angle of the smart car at the time of collecting the environmental information.

[0076] In the third step, for each piece of environmental information in the environmental information set, the rotation angle corresponding to the environmental information is added to the environmental information to obtain comprehensive environmental information.

[0077] The fourth step is to determine the obtained pieces of comprehensive environmental information as a comprehensive environmental information set.

[0078] The fifth step is to obtain the remaining power information sequence of the smart car in a preset historical time period, thereby obtaining the historical records of the smart car at different power levels.

[0079] The sixth step is to obtain the acquisition parameter sequence collected by the horizontal laser radar of the above-mentioned smart car within a preset historical time period as the horizontal acquisition parameter sequence. In this way, the historical acquisition parameters of the horizontal laser radar of the smart car can be obtained.

[0080] Step 7: Obtain the acquisition parameter sequence collected by the vertical laser radar of the smart car in a preset historical time period as the vertical acquisition parameter sequence. Thus, the historical acquisition parameters of the vertical laser radar of the smart car can be obtained.

[0081] In the eighth step, the remaining power information sequence, the comprehensive environment information set, the horizontal acquisition parameter sequence and the vertical acquisition parameter sequence are input into the input layer of the acquisition parameter information generation model to obtain the initial remaining power information sequence and the initial illumination information set. The acquisition parameter information generation model includes an input layer, an environment feature extraction network, an acquisition feature extraction network, a sequence processing network, a multimodal fusion layer, a feature fully connected layer and an output layer. The input layer is used to receive the remaining power information sequence, the comprehensive environment information set, the horizontal point cloud data sequence and the vertical point cloud data sequence. The environment feature extraction network is used to extract the features of the comprehensive environment information set. The environment feature extraction network may include a convolution layer, a pooling layer and a fully connected layer. The acquisition feature extraction network may be used to extract the temporal features of the horizontal acquisition parameter sequence and the vertical acquisition parameter sequence. The acquisition feature extraction network includes a long short-term memory network and a fully connected layer. The sequence processing network is used to extract the temporal features of the remaining power information sequence. The sequence processing network includes a bidirectional long short-term memory network and a fully connected layer. The multimodal fusion layer is used to fuse the features of the multimodal. The multimodal fusion layer includes an attention layer. The feature fully connected layer is used to convert the features after multimodal fusion into a higher-level representation. The output layer is used to output two acquisition accuracy information. The two acquisition accuracy information include acquisition accuracy information for the horizontal laser radar and acquisition accuracy information for the vertical laser radar. The activation function corresponding to the feature fully connected layer may be a ReLU activation function. The output layer may include two sub-output layers. One of the sub-output layers is used to output the acquisition accuracy information for the horizontal laser radar, and the corresponding activation function may be a linear activation function. The other sub-output layer is used to output the acquisition accuracy information for the vertical laser radar, and the corresponding activation function may be a linear activation function.

[0082] In the ninth step, the initial illumination information set is input into the convolution layer included in the environmental feature extraction network to obtain initial environmental feature information. Thus, local features can be extracted through the convolution layer.

[0083] In the tenth step, the initial environmental feature information is input into the pooling layer included in the environmental feature extraction network to obtain reduced-dimensional environmental feature information. Thus, the feature dimension can be reduced by the pooling layer.

[0084] In the eleventh step, the dimension-reduced environmental feature information is input into the fully connected layer of the environmental feature extraction network to obtain environmental feature information, thereby extracting a higher-level representation of environmental features.

[0085] In the twelfth step, the horizontal acquisition parameter sequence and the acquisition parameter sequence are respectively input into the acquisition feature extraction network to obtain horizontal acquisition feature information and vertical acquisition feature information. In practice, the long short-term memory network included in the acquisition feature extraction network can be used to capture the temporal dependency of historical acquisition parameters. The fully connected layer included in the acquisition feature extraction network can be used to further extract sequence features of the horizontal acquisition parameter sequence and the vertical acquisition parameter sequence.

[0086] In the thirteenth step, the remaining power information sequence is input into the sequence processing network to obtain the remaining power characteristic information. In practice, the bidirectional long short-term memory network included in the sequence processing network can be used to capture the temporal dependencies of the remaining power information sequence. The fully connected layer included in the sequence processing network can be used to further extract the sequence characteristics of the remaining power information sequence.

[0087] In step 14, the environmental feature information, the horizontal acquisition feature information, the vertical acquisition feature information and the remaining power feature information are input into the multimodal fusion layer to obtain fused feature information. The attention layer included in the multimodal fusion layer can assign weights according to the importance of the input sequence, so that the model can focus on more relevant parts.

[0088] In the fifteenth step, the fused feature information is input into the feature fully connected layer to obtain decision feature information. Thus, the multimodal fused features can be transformed into a higher-level representation.

[0089] In the sixteenth step, the decision feature information is input into the output layer to obtain the acquisition accuracy information for the horizontal laser radar and the acquisition accuracy information for the vertical laser radar. The acquisition accuracy information for the horizontal laser radar is used to control the horizontal laser radar to collect the point cloud data of the target ship in the horizontal direction. The acquisition accuracy information for the vertical laser radar is used to control the vertical laser radar to collect the point cloud data of the target ship in the vertical direction. The acquisition accuracy information may include but is not limited to at least one of the following: acquisition frequency, resolution.

[0090] The above-mentioned first step to the sixteenth step, as an inventive point of an embodiment of the present disclosure, solves the technical problem that "during scanning, the laser radar collects point cloud data with fixed acquisition accuracy parameters. When the battery of the smart car is insufficient, the scanning task will be interrupted. If the battery is still insufficient after charging and running for a period of time, the scanning needs to be interrupted again for charging, resulting in a long number of scanning interruptions and a long interruption time, thereby causing a long delay in the shipbuilding construction period." The factors that cause a long delay in the shipbuilding construction period are often as follows: during scanning, the laser radar collects point cloud data with fixed acquisition accuracy parameters. When the battery of the smart car is insufficient, the scanning task will be interrupted. If the battery is still insufficient after charging and running for a period of time, the scanning needs to be interrupted again for charging, resulting in a long number of scanning interruptions and a long interruption time. If the above factors are solved, the effect of shortening the delay in the shipbuilding construction period can be achieved. In order to achieve this effect, the present disclosure obtains data of the environment dimension, the remaining power dimension, and the historical acquisition parameter dimension before the horizontal laser radar and the vertical laser radar start scanning, which are used to send to the pre-trained acquisition parameter information generation model for feature extraction and multi-task decision-making, and finally output the acquisition parameter information adapted to the current smart car. The acquisition parameter information adapted to the current smart car is generated based on the comprehensive characteristics of the current smart car's environment, remaining power, and historical acquisition parameters of the installed horizontal laser radar and vertical laser radar. The generated acquisition accuracy information for the horizontal laser radar and the vertical laser radar have high accuracy, so the horizontal laser radar and the vertical laser radar are controlled to collect point cloud data. This can support the smart car to have more power to support the operation of the horizontal laser radar and the vertical laser radar. Even if scanning interruptions occur, the power required to support the end of the scan is less, and the charging time is shorter, thereby shortening the delay time of the shipbuilding construction period.

[0091] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the ship modeling method of some embodiments of the present disclosure, the integrity of scanning the ship and the integrity of the three-dimensional ship model are improved, and the delay time of the ship manufacturing period is shortened. Specifically, the reasons for the poor integrity of the modeled three-dimensional ship model and the long delay time of the ship manufacturing period are: the laser radar for the purpose of obstacle avoidance can only scan the obstacles within the vehicle height area at most and form a map, and the integrity of scanning the surrounding environment is poor, resulting in poor integrity of the modeled three-dimensional ship model. In addition, the ship is usually high, and the manual scanning takes a long time, resulting in a long delay time of the ship manufacturing period. Based on this, the ship modeling method of some embodiments of the present disclosure, first, in response to receiving the scanning start information, controls the horizontal laser radar set on the above-mentioned smart car to collect the point cloud data of the target ship in the horizontal direction as the first point cloud data, and controls the vertical laser radar set on the above-mentioned smart car to collect the point cloud data of the above-mentioned target ship in the vertical direction as the second point cloud data, wherein the above-mentioned horizontal laser radar and the above-mentioned vertical laser radar are set on the above-mentioned smart car. Thus, when scanning starts, the horizontal laser radar and the vertical laser radar can be controlled to collect point cloud data in the horizontal and vertical directions of the ship respectively. Then, the coordinate conversion is performed on the collected second point cloud data to obtain the third point cloud data. Thus, the point cloud data in the vertical direction can be converted to a unified coordinate system. Then, fused point cloud data is generated according to the collected first point cloud data and the converted third point cloud data. Thus, the point cloud data in the horizontal and vertical directions can be fused into the overall point cloud data. Finally, based on the above-mentioned fused point cloud data, a three-dimensional ship model corresponding to the above-mentioned target ship is constructed. Thus, a complete three-dimensional ship model can be constructed based on the overall point cloud data. Also because the intelligent car is provided with a horizontal laser radar for collecting point cloud data in the horizontal direction, and a vertical laser radar for collecting point cloud data in the vertical direction is added, the point cloud data of a relatively complete ship can be scanned, thereby improving the integrity of the scanning of the ship and the integrity of the three-dimensional ship model. And because it is automatically scanned by laser radars in two directions, there is no need to scan manually, which improves the scanning speed, thereby shortening the delay time of the ship manufacturing period. As a result, the integrity of the ship scan and the integrity of the 3D ship model are improved, and delays in ship manufacturing are shortened.

[0092] Reference below Figure 3 , which shows a schematic structural diagram of an electronic device 300 (a vehicle-mounted terminal installed on a smart car for automatic scanning and modeling) suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0093] like Figure 3 As shown, the electronic device 300 may include a processing device 301 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0094] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a horizontal laser radar, a vertical laser radar, an ultrasonic sensor, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The horizontal laser radar is configured to collect point cloud data of the target ship in the horizontal direction. The vertical laser radar is configured to collect point cloud data of the above-mentioned target ship in the vertical direction. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or wired to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0095] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0096] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0097] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0098] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to receiving the scanning start information, controls the horizontal laser radar provided on the smart car to collect the point cloud data of the target ship in the horizontal direction as the first point cloud data, and controls the vertical laser radar provided on the smart car to collect the point cloud data of the target ship in the vertical direction as the second point cloud data, wherein the smart car is provided with the horizontal laser radar and the vertical laser radar; performs coordinate conversion on the collected second point cloud data to obtain the third point cloud data; generates fused point cloud data according to the collected first point cloud data and the converted third point cloud data; and constructs a three-dimensional ship model corresponding to the target ship based on the fused point cloud data.

[0099] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0100] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0101] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0102] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A ship modeling method, applied to an intelligent vehicle for automatic scanning and modeling, comprising: In response to receiving the scanning start information, controlling the horizontal laser radar arranged on the smart car to collect the point cloud data of the target ship in the horizontal direction as the first point cloud data, and controlling the vertical laser radar arranged on the smart car to collect the point cloud data of the target ship in the vertical direction as the second point cloud data, wherein the horizontal laser radar and the vertical laser radar are arranged on the smart car; Performing coordinate transformation on the collected second point cloud data to obtain third point cloud data; generating fused point cloud data according to the collected first point cloud data and the converted third point cloud data; Based on the fused point cloud data, a three-dimensional ship model corresponding to the target ship is constructed.

2. The method according to claim 1, wherein: The horizontal laser radar is arranged on a horizontal rotating device, and the horizontal rotating device is arranged on the top of the smart car; And the controlling the horizontal laser radar arranged on the smart vehicle to collect point cloud data of the target ship in the horizontal direction as the first point cloud data comprises: Control the horizontal rotation device to drive the horizontal laser radar to rotate according to a preset first rotation mode; The horizontal laser radar is controlled to collect point cloud data of the target ship in the horizontal direction as first point cloud data during the rotation process.

3. The method according to claim 1, wherein: The vertical laser radar is arranged on a vertical rotating device, and the vertical rotating device is arranged on the front side or the rear side of the smart car; And the controlling the vertical laser radar arranged on the smart vehicle to collect the point cloud data of the target ship in the vertical direction as the second point cloud data comprises: Controlling the vertical rotating device to drive the vertical laser radar to rotate according to a preset second rotation mode; The vertical laser radar is controlled to collect point cloud data of the target ship in the vertical direction as second point cloud data during the rotation process.

4. The method according to claim 1, wherein: The step of performing coordinate transformation on the collected second point cloud data to obtain third point cloud data includes: Extracting installation position information corresponding to the vertical laser radar from a configuration file corresponding to the smart car as first installation position information; According to the first installation position information, the collected second point cloud data is converted into a vehicle coordinate system corresponding to the smart car to obtain third point cloud data.

5. The method according to claim 1, wherein: The step of generating fused point cloud data according to the collected first point cloud data and the converted third point cloud data includes: Obtaining installation position information corresponding to the horizontal laser radar from a configuration file corresponding to the smart car as second installation position information; According to the second installation position information, the collected first point cloud data is converted into a vehicle coordinate system corresponding to the smart car to obtain fourth point cloud data; Combining the third point cloud data and the fourth point cloud data into initial fused point cloud data; The initial fused point cloud data is subjected to point cloud deduplication processing to obtain the initial fused point cloud data after deduplication processing as the fused point cloud data.

6. The method according to claim 1, wherein: The step of constructing a three-dimensional ship model corresponding to the target ship based on the fused point cloud data includes: Performing denoising processing on the fused point cloud data to obtain denoised point cloud data; Performing ground recognition processing on the denoised point cloud data to obtain a ground recognition result; Determining whether the ground recognition result includes point data of a corresponding recognition category representing the ground; In response to determining that the ground recognition result includes point data of a corresponding recognition category representing the ground, determining each point data of a corresponding recognition category representing the ground included in the ground recognition result as a ground point set; Deleting each ground point included in the ground point set from the denoised point cloud data to obtain initial ship point cloud data; Performing background recognition processing on the initial ship point cloud data to obtain a background recognition result; Determine whether the background recognition result includes point data of the corresponding recognition category representing the background; In response to determining that the background recognition result includes point data of a corresponding recognition category representing the background, determining each point data of a corresponding recognition category representing the background included in the background recognition result as a background point set; Deleting each background point included in the background point set from the initial ship point cloud data to obtain ship point cloud data; Based on the ship point cloud data, a three-dimensional ship model corresponding to the target ship is constructed.

7. The method according to claim 6, wherein: The step of constructing a three-dimensional ship model corresponding to the target ship based on the ship point cloud data includes: Performing key point detection on the ship point cloud data to obtain a key point set; According to the key point set, the ship point cloud data is subjected to surface fitting processing to obtain an initial three-dimensional ship model; Performing detail enhancement processing on the initial three-dimensional ship model to obtain a refined three-dimensional ship model; Performing vertex simplification processing on the refined three-dimensional ship model to obtain a simplified three-dimensional ship model; According to the color information corresponding to each point data in the ship point cloud data, texture mapping processing is performed on the simplified three-dimensional ship model to obtain a three-dimensional ship model.

8. A smart car for automatic scanning and modeling, comprising: A horizontal laser radar is configured to collect point cloud data of the target ship in the horizontal direction; A vertical laser radar configured to collect point cloud data of the target ship in a vertical direction; one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

9. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images