A method, system, device and medium for generating ship assisted driving screen

Abstract

The present invention discloses a method, system, device and medium for generating a ship-assisted driving picture, wherein the method obtains a first positioning data set and a second positioning data set, wherein the first positioning data set is a collection of position information and shape information obtained by the target ship, and the second positioning data set is a collection of position information and shape information obtained by the communication target object of the target ship; based on the first positioning data set, the second positioning data set is subjected to data set fusion to obtain a positioning fusion data set; the positioning fusion data set is subjected to video rendering generation through a picture rendering model to obtain an original video picture; based on the second positioning data set, the original video picture is subjected to video frame insertion to obtain a target video picture. This method can improve the reliability of the generated ship-assisted driving picture and improve the safety of ship navigation. The present invention relates to the field of ship-assisted driving technology.

Description

Technical Field

[0001] The present invention relates to the technical field of ship assisted driving, and in particular to a method, system, device and medium for generating a ship assisted driving picture. Background Art

[0002] With the development of water transportation, water shipping has become increasingly busy. During the navigation of ships, the ship's assisted driving screen has attracted much attention because it can enable the ship driver to clearly understand the specific location of the waterway and surrounding related information.

[0003] At present, the traditional method of generating ship assisted driving images usually obtains a visual image base map through a visual sensor or an infrared sensor, and then generates a ship assisted driving image by superimposing the visual image base map with other monitoring signals. This method is highly dependent on environmental conditions. In complex scene environments, it is often unable to fully perceive the environment around the ship, which poses certain safety hazards. The reliability of the generated ship assisted driving images is unsatisfactory.

[0004] Therefore, the problems existing in the existing technology still need to be solved and optimized. Summary of the Invention

[0005] The purpose of the present invention is to solve one of the technical problems existing in the related art to at least a certain extent.

[0006] To this end, an object of an embodiment of the present invention is to provide a method, system, device and medium for generating a ship assisted driving picture, wherein the method can improve the reliability of the generated ship assisted driving picture and improve the safety of ship navigation.

[0007] In order to achieve the above technical objectives, the technical solutions adopted in the embodiments of the present application include:

[0008] In a first aspect, an embodiment of the present application provides a method for generating a ship assisted driving screen, comprising:

[0009] Acquire a multi-source ship navigation dataset of a target ship, the multi-source ship navigation dataset comprising a first positioning dataset and a second positioning dataset, the first positioning dataset being a set of position information and shape information acquired for the target ship, and the second positioning dataset being a set of position information and shape information acquired for a communication target object of the target ship;

[0010] performing data set fusion on the second positioning data set according to the first positioning data set to obtain a positioning fusion data set;

[0011] Performing video rendering on the positioning fusion data set through a picture rendering model to generate an original video picture;

[0012] According to the second positioning data set, video frames are inserted into the original video picture to obtain a target video picture.

[0013] In addition, the method according to the above embodiment of the present application may also have the following additional technical features:

[0014] Furthermore, in one embodiment of the present application, obtaining the navigation data set of the target ship includes:

[0015] Acquire the first positioning data set, as well as a first communication detection range and a second communication detection range, where the first communication detection range is a communication detection range of a shore-based target, and the second communication detection range is a communication detection range of a target adjacent to a ship;

[0016] performing a first range comparison on the first positioning data set according to the first communication detection range to obtain a first range comparison result;

[0017] If the result of the first range comparison is that the target ship is within the first communication detection range, a shore-based hybrid positioning dataset of the shore-based target object is obtained, and the shore-based hybrid positioning dataset is determined as the second positioning dataset; alternatively, if the result of the first range comparison is that the target ship is not within the first communication detection range, the second positioning dataset is obtained based on the second communication detection range.

[0018] Furthermore, in one embodiment of the present application, acquiring the second positioning data set according to the second communication detection range includes:

[0019] performing a second range comparison on the first positioning data set according to the second communication detection range to obtain a second range comparison result;

[0020] If the result of the second range comparison is that the target ship is within the second communication detection range, then the neighboring hybrid positioning data set of the neighboring ship target is obtained, and the neighboring hybrid positioning data set is determined as the second positioning data set; or, if the result of the second range comparison is that the target ship is not within the second communication detection range, then return to the step of obtaining the first positioning data set, and the first communication detection range and the second communication detection range corresponding to the target ship.

[0021] Furthermore, in one embodiment of the present application, performing dataset fusion on the second positioning dataset based on the first positioning dataset to obtain a positioning fused dataset includes:

[0022] performing first data set preprocessing on the first positioning data set to obtain a first intermediate data set, and performing second data set preprocessing on the second positioning data set to obtain a second intermediate data set and a third intermediate data set, wherein the second intermediate data set is used to record a set of shape information obtained for the communication target object, and the third intermediate data set is used to record a set of position information obtained for the communication target object;

[0023] Prioritizing the third intermediate data set to obtain a fourth intermediate data set;

[0024] The first intermediate dataset is subjected to complementary dataset fusion based on the second intermediate dataset and the fourth intermediate dataset to obtain the positioning fusion dataset.

[0025] Furthermore, in one embodiment of the present application, prioritizing the third intermediate data set to obtain a fourth intermediate data set includes:

[0026] Performing data set age division on the third intermediate data set to obtain a highest age data set and a delayed age data set;

[0027] Performing weighted evaluation and division on the delayed aging dataset to obtain a weighted aging dataset;

[0028] The data sets are sorted according to the weighted time-sensitive data set and the highest time-sensitive data set to obtain the fourth intermediate data set.

[0029] Furthermore, in one embodiment of the present application, the image rendering model includes a three-dimensional grid framework and a point cloud framework, and the image rendering model is used to render the positioning fusion dataset to generate a video image to obtain an original video image, including:

[0030] Get the preset reference distance;

[0031] According to the reference distance, the positioning fusion dataset is divided into dataset distances to obtain a first positioning fusion dataset and a plurality of second positioning fusion datasets, wherein the first positioning fusion dataset is a positioning fusion dataset in which the target interval distance is less than the reference distance, and the second positioning fusion dataset is a positioning fusion dataset in which the target interval distance is greater than or equal to the reference distance, where the target interval distance is the interval distance between the communication target object and the target ship;

[0032] Inputting the first positioning fusion data set into a three-dimensional grid framework to perform three-dimensional shape rendering to obtain a close-range shape image;

[0033] Inputting all the second positioning fusion data sets into the point cloud framework for point cloud rendering to obtain a plurality of long-distance point cloud images, each of which corresponds to one of the second positioning fusion data sets, and the level of detail corresponding to the long-distance point cloud images is negatively correlated with the target interval distance;

[0034] According to all the long-distance point cloud images, the short-distance shape images are dynamically loaded to obtain the original video images.

[0035] Furthermore, in the embodiment of the present application, performing video frame insertion on the original video picture according to the second positioning data set to obtain the target video picture includes:

[0036] Performing missing frame analysis on the original video image to obtain missing frame information;

[0037] Performing timestamp screening on the second positioning data set according to the missing frame information to obtain target positioning data corresponding to the timestamp in the missing frame information;

[0038] The original video picture is interpolated according to the target positioning data to obtain the target video picture.

[0039] In a second aspect, an embodiment of the present application provides a system for generating a ship assisted driving screen, comprising:

[0040] a first processing unit, configured to obtain a multi-source ship navigation dataset of a target ship, the multi-source ship navigation dataset comprising a first positioning dataset and a second positioning dataset, the first positioning dataset being a collection of position information and shape information obtained for the target ship, and the second positioning dataset being a collection of position information and shape information obtained for a communication target object of the target ship;

[0041] a second processing unit, configured to perform data set fusion on the second positioning data set based on the first positioning data set to obtain a positioning fused data set;

[0042] A third processing unit is configured to perform video rendering on the positioning fusion dataset using a picture rendering model to generate an original video picture;

[0043] The fourth processing unit is configured to perform video frame insertion on the original video picture according to the second positioning data set to obtain a target video picture.

[0044] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0045] at least one processor;

[0046] at least one memory for storing at least one program;

[0047] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0048] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a program executable by a processor, and the program executable by the processor is used to implement the above method when executed by the processor.

[0049] The advantages and benefits of this application will be partially given in the following description, and partially become apparent from the following description, or learned through practice of this application:

[0050] The embodiments of the present application disclose a method, system, device, and medium for generating a ship-assisted driving image. The method obtains a multi-source ship navigation dataset of a target ship, wherein the multi-source ship navigation dataset includes a first positioning dataset and a second positioning dataset, wherein the first positioning dataset is a set of position information and shape information obtained by the target ship, and the second positioning dataset is a set of position information and shape information obtained by a communication target of the target ship; based on the first positioning dataset, the second positioning dataset is subjected to dataset fusion to obtain a positioning fusion dataset; the positioning fusion dataset is subjected to video rendering generation through a picture rendering model to obtain an original video image; based on the second positioning dataset, the original video image is subjected to video interpolation to obtain a target video image. The method fuses the first positioning dataset and the second positioning dataset from the target ship and the communication target, respectively, and generates a video image rendering. Through the complementary fusion of the data of the target ship and the communication target, the method achieves a comprehensive perception of the surrounding environment of the target ship, effectively improves the reliability of the generated ship-assisted driving image, and thus effectively improves the safety of ship navigation; in addition, the method also interpolates the original video image through the second positioning dataset, which can effectively ensure the accuracy and reliability of the ship-assisted driving image, thereby effectively improving the safety of ship navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present application or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly expressing some embodiments of the technical solutions of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0052] Figure 1A simplified network diagram of a communication topology network provided in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of a flow chart of a method for generating a ship assisted driving screen provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of the structural framework of a system for generating a ship-assisted driving image provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. For the step numbers in the following embodiments, they are provided only for the convenience of explanation and are not intended to limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0058] At present, the traditional method of generating ship assisted driving images usually obtains a visual image base map through a visual sensor or an infrared sensor, and then generates a ship assisted driving image by superimposing the visual image base map with other monitoring signals (such as signals obtained by ARPA radar). This method is highly dependent on environmental conditions. For example, in severe weather conditions such as low vision, night, heavy rain, heavy fog, or obstructed areas (such as bridge areas or lock areas in narrow waterways, bridges, buildings or other ships, etc.), this method often cannot obtain regional information of the entire waterway area, that is, it often cannot fully perceive the environment around the ship, there are certain safety hazards, and the reliability of the generated ship assisted driving images is unsatisfactory.

[0059] In addition, there is also a method in the prior art for generating ship assisted driving images based on visual images provided by shore-based equipment and visual images obtained by the ship itself. This method is still highly dependent on environmental conditions. In severe weather conditions such as low vision, night, heavy rain, and heavy fog, the visual images provided by shore-based equipment and the visual images obtained by the ship itself are usually unable to obtain regional information of the entire waterway area, and the visual images provided by shore-based equipment and the visual images obtained by the ship itself are also affected by the blocked areas, which poses certain safety hazards. The reliability of the generated ship assisted driving images is unsatisfactory.

[0060] In view of this, an embodiment of the present invention provides a method, system, device and medium for generating a ship-assisted driving picture, wherein the generation method generates a ship-assisted driving picture by fusing a first positioning data set and a second positioning data set respectively from a target ship and a communication target object, and rendering a video picture. Through the complementary fusion of data between the target ship and each communication target object (such as a shore-based target object and a neighboring ship target object), a comprehensive perception of the surrounding environment of the target ship is achieved, and the reliability of the generated ship-assisted driving picture is effectively improved, thereby effectively improving the safety of ship navigation; in addition, the generation method also performs video interpolation on the original video picture through the second positioning data set, which can effectively ensure the accuracy and reliability of the ship-assisted driving picture, thereby effectively improving the safety of ship navigation.

[0061] The following is an explanation of the communication topology network involved in the embodiments of the present application.

[0062] Reference Figure 1 The communication topology network involved in the embodiment of the present application can be obtained by heterogeneous networking. In one feasible implementation, the communication topology network of the embodiment of the present application can include a shore-based mobile sub-network and a ship wireless sub-network, wherein the shore-based mobile sub-network is mainly based on technologies such as WiMAX and LTE, and its coverage (i.e. Figure 1 The red solid line range in the figure is mainly limited to offshore and inland areas and is highly dependent on the distribution of land base stations. The ship wireless sub-network is mainly based on the medium-frequency NAVTEX system, the high-frequency NAVDAT system and the very high-frequency VDES system. These systems are mainly used for positioning navigation and providing maritime safety information. The data transmission rate of these systems is relatively low and the coverage range (i.e. Figure 1 The blue solid line in the figure is limited.

[0063] Reference Figure 2 In an embodiment of the present application, a method for generating a ship-assisted driving screen includes:

[0064] Step 110: Acquire a multi-source ship navigation dataset of a target ship, wherein the multi-source ship navigation dataset includes a first positioning dataset and a second positioning dataset, wherein the first positioning dataset is a set of position information and shape information acquired for the target ship, and the second positioning dataset is a set of position information and shape information acquired for a communication target object of the target ship;

[0065] In an embodiment of the present application, a multi-source ship navigation dataset of a target ship can be obtained through a heterogeneous network. Specifically, the target ship can determine the shape information and position information acquired in real time as a first positioning dataset. The position information includes the position data of the target ship and the position data of targets other than the target ship. The targets can be communication targets such as neighboring ships and shore-based base stations, as well as non-communication targets such as fixed obstacles around the target ship. In addition, the shape information acquired by the target ship can include the geometric structure information of the target ship itself and the geometric structure information of targets other than the target ship. The geometric structure information acquired by the target ship itself has a higher detail accuracy than the geometric structure information of the targets.

[0066] It is understood that the second positioning dataset may be a collection of position information and shape information acquired from the communication target object. The content of this second positioning dataset is similar to the first positioning dataset described above and can be derived by analogy. Furthermore, various methods exist for acquiring positioning datasets for target vessels or communication targets, and this application will not elaborate on these methods here.

[0067] It should be noted that, for non-communication targets such as fixed obstacles around the target ship that cannot communicate, the position information and shape information of the non-communication target may exist in the first positioning data set, and specifically may be obtained by the target ship through the corresponding sensor; or, the position information and shape information of the non-communication target may also exist in the second positioning data set, and specifically may be obtained by a neighboring ship through the corresponding sensor, by a shore-based base station through the corresponding sensor or by pre-storing the position information and shape information of the non-communication target, or by a neighboring ship forwarding the position information and shape information of the non-communication target provided by the shore-based base station, etc. The examples in this application are for illustration only.

[0068] In some embodiments, the step 110 of obtaining a navigation dataset of a target ship includes:

[0069] A1. Acquire the first positioning data set, as well as a first communication detection range and a second communication detection range, where the first communication detection range is a communication detection range of a shore-based target, and the second communication detection range is a communication detection range of a target adjacent to a ship;

[0070] A2. Perform a first range comparison on the first positioning data set according to the first communication detection range to obtain a first range comparison result;

[0071] A3. If the first range comparison result indicates that the target vessel is within the first communication detection range, obtaining a shore-based hybrid positioning dataset of the shore-based target object, and determining the shore-based hybrid positioning dataset as the second positioning dataset;

[0072] In this embodiment of the present application, step A1 involves acquiring a first positioning dataset of the target vessel and the communication detection range of a communication target object capable of communicating with the target vessel. In practical applications, multiple first communication detection ranges may be acquired, each of which indicates the communication detection range of a corresponding shore-based target object (i.e., the communication detection range of a shore-based mobile network), which may be a shore-based base station. Furthermore, the second communication detection range (i.e., the communication detection range of the ship wireless subnetwork of the neighboring vessel) is similar to the first communication detection range described above and can be derived by analogy.

[0073] It can be understood that step A2 can be to determine whether the real-time position of the target ship is within one or more first communication detection ranges. Specifically, it can be based on the position information of the target ship in the current first positioning data set of the target ship and the first communication detection range. A range comparison is performed to obtain a first range comparison result, and the first range comparison result is used to characterize whether the target ship enters the first communication detection range, that is, whether the target ship is within the first communication detection range.

[0074] It should be noted that if the target ship is within the first communication detection range, it means that the target ship has entered the communication detection range of at least one shore-based target. At this time, the target ship can exchange data with each shore-based target. Specifically, a shore-based hybrid positioning data set of each shore-based target can be obtained, and all the obtained shore-based hybrid positioning data sets can be determined as the second positioning data set, wherein the shore-based hybrid positioning data set includes the position information and shape information of the shore-based target itself, as well as the position information and shape information of other targets. The other targets can be other shore-based targets that communicate with the current shore-based target, other ships traveling on the route and communicating with the shore-based target, etc., or fixed obstacles, etc. The examples in this application are for illustrative purposes only.

[0075] Alternatively, if the first range comparison result is that the target ship is not within the first communication detection range, A4 obtains the second positioning data set according to the second communication detection range.

[0076] Furthermore, the step A4 of acquiring the second positioning data set according to the second communication detection range includes:

[0077] A41. Perform a second range comparison on the first positioning data set according to the second communication detection range to obtain a second range comparison result.

[0078] A42. If the second range comparison result indicates that the target vessel is within the second communication detection range, obtaining a proximity hybrid positioning dataset of the adjacent vessel target object, and determining the proximity hybrid positioning dataset as the second positioning dataset;

[0079] Alternatively, A43, if the second range comparison result is that the target ship is not within the second communication detection range, return to the step of obtaining the first positioning data set, and the first communication detection range and the second communication detection range corresponding to the target ship.

[0080] In the embodiment of the present application, the contents of steps A41 to A42 are similar to the contents of the aforementioned steps A2 to A3 and can be simply deduced by analogy, so this application will not repeat them here. In addition, if the second range comparison result shows that the target vessel is not within the second communication detection range, it means that there are no neighboring ships or shore-based base stations in the surrounding environment of the target vessel. At this time, the process can return to step A1 and implement real-time data exchange between the target vessel and the neighboring ships or shore-based base stations by looping through steps A1 to A4. In addition, after the target vessel obtains the second positioning data set, if the target vessel leaves the first communication detection range or the second communication detection range, the process can also return to step A1.

[0081] It should be noted that the embodiment of the present application, through steps A1 to A4, specifically through the combined communication strategy of the target ship to the shore-based target objects and the neighboring ships, can achieve efficient data transmission in offshore areas with high communication demand, and can also ensure that basic communication capabilities are still maintained in the open sea or communication blind spots, so that the target ship can comprehensively consider the position information and shape information obtained by itself, as well as the position information and shape information obtained by other targets. It can also continue to provide reliable complex ship driving images in complex scene environments (such as obstructed vision, large environmental interference, etc.), thereby improving the safety of ship navigation.

[0082] For example, continue to refer to Figure 1 ,by Figure 1 Any ship in the target ship is used as the target ship. The target ship may first obtain its own first positioning data set, which may be obtained through the target ship's own related sensors and satellite positioning.

[0083] Then, the target ship detects in turn whether it has entered the coverage range of the shore-based mobile subnetwork (i.e., the first communication detection range) and the coverage range of the ship wireless subnetwork (i.e., the second communication detection range). Specifically, if the target ship enters the first communication detection range, the shore-based hybrid positioning data set provided by the shore-based target object is obtained through the shore-based mobile subnetwork. The shore-based hybrid positioning data set may specifically include the position information and shape information of the shore-based target object, the position information and shape information of the fixed obstacle (such as a lighthouse), the position information and shape information of the target ship obtained by the shore-based target object, and the position information and shape information of the remaining neighboring ships within the first communication detection range.

[0084] Alternatively, if the target ship has not entered the first communication detection range and has entered the second communication detection range of a neighboring ship, the neighboring hybrid positioning data set provided by the neighboring ship can be obtained through the ship wireless subnetwork between the target ship and the neighboring ship. The neighboring hybrid positioning data set can specifically include the shape information and position information of the neighboring ship, the position information and shape information of the target ship obtained by the neighboring ship, and the position information and shape information of related targets, etc., wherein the related targets can be other ships or fixed obstacles within the second communication detection range of the neighboring ship, etc., or they can be the neighboring positioning data set provided by the shore-based target corresponding to the neighboring ship, and the neighboring positioning data set is the second positioning data set obtained by the neighboring ship.

[0085] Step 120: Perform dataset fusion on the second positioning dataset based on the first positioning dataset to obtain a positioning fused dataset;

[0086] In an embodiment of the present application, data set fusion is used to fuse and summarize the position information and shape information of the target ship, as well as the position information and shape information of other communication target objects, so as to obtain a positioning fusion data set.

[0087] In some embodiments, step 120 of performing dataset fusion on the second positioning dataset based on the first positioning dataset to obtain a positioning fused dataset includes:

[0088] B1. Performing first data set preprocessing on the first positioning data set to obtain a first intermediate data set, and performing second data set preprocessing on the second positioning data set to obtain a second intermediate data set and a third intermediate data set;

[0089] In an embodiment of the present application, the preprocessing of the first data set can first be based on statistical methods (such as standard deviation or box plot) to identify and remove outliers in the first positioning data set to obtain the first positioning data set after the outliers are removed; then, the missing values ​​in the first positioning data set after the outliers are removed are identified, and according to the data distribution characteristics, the missing values ​​are processed based on interpolation, mean replacement or deletion of missing records to obtain the first positioning data set after the missing values ​​are filled; then, the data format of the first positioning data set after the missing values ​​are filled is regularized to ensure that all data use the same unit of measurement, and all location data in the first positioning data set are converted into a unified geographic coordinate system to obtain a first intermediate data set.

[0090] It can be understood that the content of the second intermediate data set is similar to that of the aforementioned first intermediate data set, and after obtaining the second intermediate data set with a regular data format, the second intermediate data set and the third intermediate data set are respectively divided based on the shape information condition and the position information condition, wherein the second intermediate data set is used to record the set of shape information obtained by the communication target object, that is, the shape information obtained by each communication target object about itself, and the shape information of other targets obtained by the communication target object; the third intermediate data set is used to record the set of position information obtained by the communication target object, that is, the position information obtained by each communication target object about itself, and the position information of other targets obtained by the communication target object.

[0091] B2. Prioritize the third intermediate data set to obtain a fourth intermediate data set;

[0092] Furthermore, the step B2 of prioritizing the third intermediate data set to obtain a fourth intermediate data set includes:

[0093] B21. Performing dataset age division on the third intermediate dataset to obtain a maximum age dataset and a delayed age dataset;

[0094] B22. Perform weighted evaluation and division on the delayed aging dataset to obtain a weighted aging dataset;

[0095] B23. Sort the datasets according to the weighted time-sensitive dataset and the highest time-sensitive dataset to obtain the fourth intermediate dataset.

[0096] In an embodiment of the present application, priority sorting can be based on the timeliness of the location information, and each location information in the third intermediate data set is sorted to obtain a fourth intermediate data set, in which the location information with a higher ranking in the fourth intermediate data set has higher reliability.

[0097] It can be understood that step B21 can first obtain a preset time difference threshold, and the specific numerical value of the time difference threshold can be flexibly set according to actual conditions, such as any one of 5 seconds, 30 seconds, 60 seconds, etc., and then obtain the timestamp corresponding to each location information in the fourth intermediate data set, and then based on the time difference threshold and the current moment, perform a difference comparison on all timestamps to screen out several location information whose time difference with the current moment is less than the time difference threshold, and determine these location information as the highest timeliness data set; then, determine the remaining location information as the delayed timeliness data set.

[0098] It should be noted that the weighted evaluation division can be a weighted evaluation and division of each location information in the delayed aging data set, so as to obtain a weighted aging data set. Specifically, step B22 can first be to obtain the time difference between the timestamp of each location information in the delayed aging data set and the current moment, the data source credibility score and the data processing credibility score, etc., wherein the data source credibility score can be calculated based on the reliability of the data source. For example, when a certain location information is obtained by a high-precision sensor, the higher the reliability of its data source, conversely, when a certain location information is obtained by a low-precision sensor, the lower the reliability of its data source. The data processing credibility can be obtained based on the number of compression, encryption and other processing times that the location information undergoes during transmission. Specifically, since these compression and encryption processing links may damage the originality of the data, the fewer the processing times, the higher the credibility of the data processing, and conversely, the lower the credibility score of the data processing.

[0099] It is worth mentioning that after obtaining the time difference between the timestamp of each location information in the delay time-sensitive data set and the current moment, the data source credibility score and the data processing credibility score, each location information can be weighted and sorted to obtain the weighted time-sensitive data set. There are many specific weight evaluation methods, which will not be repeated in this application.

[0100] It should be added that the data set sorting in step B23 can be based on the weighted time-effectiveness data set and the highest time-effectiveness data set. Specifically, the weighted time-effectiveness data set can be spliced ​​after the highest time-effectiveness data set, that is, the time-effectiveness priority of the highest time-effectiveness data set is greater than the time-effectiveness priority of the weighted time-effectiveness data set, thereby obtaining the fourth intermediate data set.

[0101] B3. Perform data set complementary fusion on the first intermediate data set based on the second intermediate data set and the fourth intermediate data set to obtain the positioning fusion data set.

[0102] In this embodiment of the present application, the complementary fusion of datasets is used to update the position information and shape information acquired by the communication target object to the first intermediate dataset, thereby obtaining a time-based positioning fusion dataset. Specifically, step B3 may first add the shape information acquired by each communication target object in the second intermediate dataset to the first intermediate dataset, and aggregate the shape information of non-communication targets acquired by several communication targets and add it to the first intermediate dataset; then, based on the sorting relationship of the fourth intermediate dataset, the position information corresponding to the targets with higher time priority is added to the first intermediate dataset, thereby obtaining a time-based positioning fusion dataset.

[0103] Step 130: Render the positioning fusion dataset using a picture rendering model to generate a video picture, thereby obtaining an original video picture.

[0104] In an embodiment of the present application, step 130 may be to input the positioning fusion data set into a picture rendering model, and generate a video picture including the target ship and surrounding environment objects through rendering by the picture rendering model, thereby obtaining the original video picture.

[0105] In some embodiments, the image rendering model in step 130 includes a three-dimensional grid framework and a point cloud framework, and the image rendering model is used to render the positioning fusion dataset to generate a video image to obtain an original video image, including:

[0106] C1. Obtain the preset reference distance;

[0107] C2. Dividing the positioning fusion dataset by dataset distance according to the reference distance to obtain a first positioning fusion dataset and multiple second positioning fusion datasets, wherein the first positioning fusion dataset is a positioning fusion dataset in which the target interval distance is less than the reference distance, and the second positioning fusion dataset is a positioning fusion dataset in which the target interval distance is greater than or equal to the reference distance, where the target interval distance is the interval distance between the communication target object and the target ship;

[0108] C3. Inputting the first positioning fusion data set into a three-dimensional grid framework to perform three-dimensional shape rendering to obtain a close-range shape image;

[0109] C4. Inputting all the second positioning fusion data sets into the point cloud framework for point cloud rendering to obtain a plurality of long-distance point cloud images, each of which corresponds to one of the second positioning fusion data sets, and wherein the level of detail corresponding to the long-distance point cloud images is negatively correlated with the target interval distance;

[0110] C5. Dynamically load the close-range shape image based on all the long-range point cloud images to obtain the original video image.

[0111] In the embodiments of the present application, the specific value of the reference distance can be set based on actual conditions. Step C2 can be performed by comparing the target separation distances between each target object and the target vessel in the positioning fusion dataset based on the reference distance, and determining the position information and shape information of all targets with target separation distances less than the reference distance as the first positioning fusion dataset. Furthermore, after obtaining the position information and shape information of all targets with target separation distances less than the reference distance, multiple second positioning fusion datasets can be generated based on the target separation distances. Specifically, the position information and shape information of targets with the same or similar target separation distances can be divided into a second positioning fusion dataset, each corresponding to a level of detail. The level of detail of each second positioning fusion dataset is negatively correlated with the target separation distance. That is, the higher the level of detail corresponding to the second positioning fusion dataset, the smaller the target separation distance. Conversely, the lower the level of detail corresponding to the second positioning fusion dataset, the larger the target separation distance. Alternatively, after obtaining the position information and shape information of all targets with target separation distances less than the reference distance, multiple second positioning fusion datasets can be generated based on multiple preset distance division thresholds. This example is provided for illustrative purposes only.

[0112] It is understood that step C3 can be implemented by inputting the first positioning fusion dataset into a 3D grid framework, loading the position and shape information of the target vessel and adjacent objects in the first positioning fusion dataset through the 3D grid framework, and constructing a simulated image of a 3D rigid structure with the highest level of detail, possessing the most geometric details and complex textures, thereby obtaining a close-range shape image. Step C4 can be implemented by inputting all second positioning fusion datasets into a point cloud framework for point cloud image rendering, and generating simulated images of point cloud structures at different levels of detail for each second positioning fusion dataset through the point cloud framework, thereby obtaining multiple long-range point cloud images.

[0113] It should be noted that, since the level of detail corresponding to each long-distance point cloud image is negatively correlated with the target interval distance, the long-distance point cloud images with different levels of detail have different positions in the simulation image. Therefore, step C5 can be to dynamically load the long-distance point cloud images with different levels of detail into the close-range shape image, and generate the original video image through real-time updating.

[0114] It is worth mentioning that the embodiment of the present application can display the target objects closer to the target ship as a high-precision three-dimensional rigid body structure by loading long-distance point cloud images of different detail levels into the close-range shape image, and display the hierarchical details of targets at different target intervals. This can reduce the required computing and rendering resources while still maintaining the target ship's comprehensive perception of the surrounding environment, thereby improving the user experience of the ship driver and the safety of the ship's navigation.

[0115] Step 140: Perform video frame insertion on the original video image according to the second positioning data set to obtain a target video image.

[0116] In an embodiment of the present application, step 140 may be to perform a frame filling operation on the missing video frames in the original video picture based on the second positioning data set, so as to obtain a target video picture with better fluency and reliability.

[0117] In some embodiments, step 140 of performing video frame insertion on the original video image according to the second positioning data set to obtain the target video image includes:

[0118] D1. Analyze missing frames on the original video to obtain missing frame information;

[0119] D2. Perform timestamp screening on the second positioning data set according to the missing frame information to obtain target positioning data corresponding to the timestamp in the missing frame information;

[0120] D3. Perform frame interpolation on the original video image according to the target positioning data to obtain the target video image.

[0121] In an embodiment of the present application, the missing frame analysis in step D1 can be performed by determining whether the shape information or position information of the target vessel or object is missing in the video frame at a specific timestamp in the original video image, thereby obtaining the missing frame information. Then, based on the timestamp in the obtained missing frame information, the position information and shape information of the target object at that timestamp in the second positioning data set can be obtained, and the obtained position information and shape information of the target object can be determined as the target positioning data. Then, based on the position information and shape information of the target positioning data, the missing position information and shape information of the video frame at the corresponding timestamp can be filled in to obtain the target video image.

[0122] A system for generating a ship assisted driving image according to an embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0123] Reference Figure 3 , a system for generating a ship assisted driving picture proposed in an embodiment of the present application includes:

[0124] A first processing unit 101 is configured to obtain a multi-source ship navigation dataset of a target ship, wherein the multi-source ship navigation dataset includes a first positioning dataset and a second positioning dataset, wherein the first positioning dataset is a set of position information and shape information obtained for the target ship, and the second positioning dataset is a set of position information and shape information obtained for a communication target object of the target ship;

[0125] A second processing unit 102 is configured to perform data set fusion on the second positioning data set based on the first positioning data set to obtain a positioning fused data set;

[0126] The third processing unit 103 is configured to perform video rendering on the positioning fusion dataset using a picture rendering model to generate an original video picture;

[0127] The fourth processing unit 104 is configured to perform video frame insertion on the original video picture according to the second positioning data set to obtain a target video picture.

[0128] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0129] Reference Figure 4 , an embodiment of the present application further provides an electronic device, including:

[0130] at least one processor 201;

[0131] At least one memory 202, configured to store at least one program;

[0132] When the at least one program is executed by the at least one processor 201 , the at least one processor 201 implements the above method embodiment.

[0133] Similarly, it can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0134] An embodiment of the present application further provides a computer-readable storage medium, in which a program executable by the processor 201 is stored. The program executable by the processor 201 is used to implement the above-mentioned method embodiment when executed by the processor 201.

[0135] Similarly, the contents of the above method embodiments are applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0136] In some optional embodiments, the functions / operations mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, the two boxes shown in succession may actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flow chart of the present application are provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logic flows presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0137] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present application. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present application as set forth in the claims using ordinary techniques without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the appended claims and their equivalents.

[0138] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0139] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0140] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0141] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0142] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples.

[0143] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0144] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present application, and these equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for generating a ship assisted driving picture, characterized in that: include: Acquire a multi-source ship navigation dataset of a target ship, the multi-source ship navigation dataset comprising a first positioning dataset and a second positioning dataset, the first positioning dataset being a set of position information and shape information acquired for the target ship, and the second positioning dataset being a set of position information and shape information acquired for a communication target object of the target ship; performing data set fusion on the second positioning data set according to the first positioning data set to obtain a positioning fusion data set; Performing video rendering on the positioning fusion data set through a picture rendering model to generate an original video picture; Performing video frame insertion on the original video picture according to the second positioning data set to obtain a target video picture; The image rendering model includes a three-dimensional grid framework and a point cloud framework. The image rendering model is used to render the positioning fusion data set to generate a video image to obtain an original video image, including: Get the preset reference distance; According to the reference distance, the positioning fusion dataset is divided into dataset distances to obtain a first positioning fusion dataset and a plurality of second positioning fusion datasets, wherein the first positioning fusion dataset is a positioning fusion dataset in which the target interval distance is less than the reference distance, and the second positioning fusion dataset is a positioning fusion dataset in which the target interval distance is greater than or equal to the reference distance, where the target interval distance is the interval distance between the communication target object and the target ship; Inputting the first positioning fusion data set into a three-dimensional grid framework to perform three-dimensional shape rendering to obtain a close-range shape image; Inputting all the second positioning fusion data sets into the point cloud framework for point cloud rendering to obtain a plurality of long-distance point cloud images, each of which corresponds to one of the second positioning fusion data sets, and the level of detail corresponding to the long-distance point cloud images is negatively correlated with the target interval distance; According to all the long-distance point cloud images, the short-distance shape images are dynamically loaded to obtain the original video images.

2. The generation method according to claim 1, characterized in that The step of obtaining the navigation data set of the target ship includes: Acquire the first positioning data set, as well as a first communication detection range and a second communication detection range, where the first communication detection range is a communication detection range of a shore-based target, and the second communication detection range is a communication detection range of a target adjacent to a ship; performing a first range comparison on the first positioning data set according to the first communication detection range to obtain a first range comparison result; If the result of the first range comparison is that the target ship is within the first communication detection range, a shore-based hybrid positioning dataset of the shore-based target object is obtained, and the shore-based hybrid positioning dataset is determined as the second positioning dataset; alternatively, if the result of the first range comparison is that the target ship is not within the first communication detection range, the second positioning dataset is obtained based on the second communication detection range.

3. The generation method according to claim 2, characterized in that The acquiring, according to the second communication detection range, the second positioning data set includes: performing a second range comparison on the first positioning data set according to the second communication detection range to obtain a second range comparison result; If the result of the second range comparison is that the target ship is within the second communication detection range, then the neighboring hybrid positioning data set of the neighboring ship target is obtained, and the neighboring hybrid positioning data set is determined as the second positioning data set; or, if the result of the second range comparison is that the target ship is not within the second communication detection range, then return to the step of obtaining the first positioning data set, and the first communication detection range and the second communication detection range corresponding to the target ship.

4. The generation method according to claim 1, characterized in that The step of performing dataset fusion on the second positioning dataset based on the first positioning dataset to obtain a positioning fused dataset includes: performing first data set preprocessing on the first positioning data set to obtain a first intermediate data set, and performing second data set preprocessing on the second positioning data set to obtain a second intermediate data set and a third intermediate data set, wherein the second intermediate data set is used to record a set of shape information obtained for the communication target object, and the third intermediate data set is used to record a set of position information obtained for the communication target object; Prioritizing the third intermediate data set to obtain a fourth intermediate data set; The first intermediate dataset is subjected to complementary dataset fusion based on the second intermediate dataset and the fourth intermediate dataset to obtain the positioning fusion dataset.

5. The generation method according to claim 4, characterized in that Prioritizing the third intermediate data set to obtain a fourth intermediate data set includes: Performing data set age division on the third intermediate data set to obtain a highest age data set and a delayed age data set; Performing weighted evaluation and division on the delayed aging dataset to obtain a weighted aging dataset; The data sets are sorted according to the weighted time-sensitive data set and the highest time-sensitive data set to obtain the fourth intermediate data set.

6. The generation method according to claim 1, characterized in that The performing video frame insertion on the original video picture according to the second positioning data set to obtain the target video picture includes: Performing missing frame analysis on the original video image to obtain missing frame information; Performing timestamp screening on the second positioning data set according to the missing frame information to obtain target positioning data corresponding to the timestamp in the missing frame information; The original video picture is interpolated according to the target positioning data to obtain the target video picture.

7. A system for generating a ship-assisted driving picture, characterized in that: include: a first processing unit, configured to obtain a multi-source ship navigation dataset of a target ship, the multi-source ship navigation dataset comprising a first positioning dataset and a second positioning dataset, the first positioning dataset being a collection of position information and shape information obtained for the target ship, and the second positioning dataset being a collection of position information and shape information obtained for a communication target object of the target ship; a second processing unit, configured to perform data set fusion on the second positioning data set based on the first positioning data set to obtain a positioning fused data set; A third processing unit is configured to perform video rendering on the positioning fusion dataset using a picture rendering model to generate an original video picture; a fourth processing unit, configured to perform video frame insertion on the original video picture according to the second positioning data set to obtain a target video picture; The image rendering model includes a three-dimensional grid framework and a point cloud framework. The image rendering model is used to render the positioning fusion data set to generate a video image to obtain an original video image, including: Get the preset reference distance; According to the reference distance, the positioning fusion dataset is divided into dataset distances to obtain a first positioning fusion dataset and a plurality of second positioning fusion datasets, wherein the first positioning fusion dataset is a positioning fusion dataset in which the target interval distance is less than the reference distance, and the second positioning fusion dataset is a positioning fusion dataset in which the target interval distance is greater than or equal to the reference distance, where the target interval distance is the interval distance between the communication target object and the target ship; Inputting the first positioning fusion data set into a three-dimensional grid framework to perform three-dimensional shape rendering to obtain a close-range shape image; Inputting all the second positioning fusion data sets into the point cloud framework for point cloud rendering to obtain a plurality of long-distance point cloud images, each of which corresponds to one of the second positioning fusion data sets, and the level of detail corresponding to the long-distance point cloud images is negatively correlated with the target interval distance; According to all the long-distance point cloud images, the short-distance shape images are dynamically loaded to obtain the original video images.

8. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to any one of claims 1 to 6 when executed by the processor.

Images