Customs port comprehensive monitoring and command system and method based on 5G communication transmission

Abstract

The present invention discloses a comprehensive monitoring and command system and method for customs ports based on 5G communication transmission, including a multi-site video acquisition module for collecting panoramic video and regional video at the customs port; a customs business database module for storing, managing and reading customs business data; a monitoring and command module for receiving and displaying panoramic video and regional video, forming a panoramic map of the customs port, adding video switching labels, displaying corresponding panoramic video or regional video according to operations, and displaying customs business data; and an intelligent individual soldier dispatching module for sending assembly instructions to the staff's terminal. The present invention has the advantages of large bandwidth, low latency and high speed, realizes large-scale and three-dimensional monitoring of the customs port, and through linkage with the customs business database module, enables the customs agency to view various customs business data in real time on the monitoring and command module, and can carry out administrative law enforcement activities with better quality and efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated information systems, and specifically to a customs port comprehensive monitoring and command system and method based on 5G communication transmission. Background Art

[0002] Customs is a national administrative agency that exercises import and export supervision and management powers in accordance with laws and administrative regulations. Customs has both the authority and the responsibility to implement supervision and management over imported and exported goods and inbound and outbound means of transport. Goods generated in international trade activities must enter and exit the country through customs ports. Therefore, customs ports are important places for customs to carry out administrative law enforcement activities.

[0003] Customs ports such as container terminals and land transport hubs are characterized by huge areas, high cargo mobility, and frequent exchanges of vehicles and personnel. It is obviously difficult to adapt to the needs of customs administrative law enforcement activities at customs ports by relying on manual inspections and management.

[0004] The fifth generation of mobile communication technology (referred to as "5G technology") is at the forefront of the current era, and has an increasingly profound impact on social production and people's lifestyles. 5G technology has three major network capabilities: enhanced mobile broadband (eMBB, referred to as large bandwidth), massive machine type communication (mMTC, referred to as large connection) and low latency high reliability communication (uRLLC, referred to as low latency). Accelerating the construction of new infrastructure such as 5G technology networks and data centers has become an important development project; with the continuous advancement of the national 5G technology development strategy and the continuous optimization and improvement of 5G technology network coverage in regulatory work places, the demand for 5G technology applications in customs places will also grow rapidly. China is the world's largest trading country, generating nearly a quarter of the world's trade. As of the end of 2018, the country's ports had 2,444 berths of 10,000 tons and above. There is a large market application space for 5G technology in my country, which can improve the work efficiency of ports, the management efficiency and safety of port parks, and enhance the coordination between ports and ships and cargo transportation.

[0005] With the improvement of informatization, video surveillance systems have played an important role in the daily activities of customs. Through the video surveillance system, the business departments of customs can conduct centralized video network monitoring of key places, key links, and emergency nodes in customs ports; in addition, some customs agencies have also formed a visual management system for individual positioning, voice calling, video scheduling, etc. with two-dimensional electronic maps as the core.

[0006] However, the video surveillance systems in the prior art often focus on close-up shooting of local and detailed images, lacking large-scale, three-dimensional monitoring of customs ports, and lack of linkage between the video surveillance system and customs business data, making it impossible to achieve data interoperability and real-time display; in addition, the above-mentioned two-dimensional electronic map is still not intuitive enough for the display of customs ports, and it is difficult to grasp the on-site situation of the customs ports in real time on the two-dimensional electronic map. Summary of the invention

[0007] The purpose of the present invention is to provide a comprehensive monitoring and command system and method for customs ports based on 5G communication transmission, which can realize large-scale and three-dimensional monitoring of customs ports and can be linked with the customs business database.

[0008] To achieve the above objectives, the present invention provides the following technical solutions: a customs port integrated monitoring and command system based on 5G communication transmission, comprising:

[0009] A multi-site video acquisition module, used to acquire panoramic video in at least one panoramic area of ​​a customs port, acquire regional video in at least one local area of ​​the customs port, and output the panoramic video and the regional video in a 5G communication transmission manner;

[0010] A customs business database module, used to store, manage and read customs business data passing through the customs port;

[0011] a monitoring and command module, for receiving and displaying the panoramic video and the regional video in real time, using the panoramic video and the regional video to form a panoramic map of the customs port, adding a video switching label to the panoramic map, displaying the corresponding panoramic video or regional video according to the user's operation on the video switching label, and overlaying and displaying the customs business data on the panoramic map;

[0012] a container identification module, for supporting a user to select a container in the panoramic image or the regional image displayed by the monitoring and command module, to track and focus the selected container in a dynamic manner, to identify the identity of the selected container, and, based on the identity of the container, to overlay and display customs business data associated with the container on the panoramic image and / or the regional image; and

[0013] The intelligent individual soldier dispatching module is used to send assembly instructions, location information associated with the assembly instructions, and event information associated with the assembly instructions to the terminal of the customs staff.

[0014] In the above technical scheme, the customs port comprehensive monitoring and command system based on 5G communication transmission of the present invention also includes a label adding and management module; the label adding and management module is used to add custom labels to the video objects in the panoramic video and the regional video, manage the custom labels, associate the custom labels with the coordinates of the panoramic map, search for the custom labels, and locate the custom labels on the panoramic video or the regional video according to the coordinates of the custom labels on the panoramic map.

[0015] In the above technical solution, the label adding and management module is also used to add regional labels to the panoramic video, the regional video, a partial area of ​​the panoramic video and a partial area of ​​the regional video, combine the regional labels into a region list, and switch to the panoramic video or regional video corresponding to the region label according to the user's operation on the region list.

[0016] In the above technical solution, the custom tag is associated with the coordinates of the panoramic map, and the specific method includes:

[0017] S1, determining the video object to which the custom tag is added;

[0018] S2, obtaining the ptz value and the field of view angle of the video object in the panoramic image or the regional image;

[0019] S3, calculating and obtaining the xy value of the video object in the panoramic image or the regional image according to the ptz value and the viewing angle of the video object;

[0020] S4. Associate the xy values ​​of the video object in the panoramic image or the regional image as coordinates of the panoramic map.

[0021] In the above technical solution, the custom tag is at least one of a text tag, a picture tag, a document tag, a voice tag, a video tag and a mixed tag.

[0022] In the above technical solution, the identity of the selected container is identified, and the specific method includes:

[0023] Sa. Acquire an image of the container according to the container selected by the user;

[0024] Sb. Using an image semantic segmentation model, roughly locating the container surface of the container image to segment out the container number in the image, and preprocessing the image to obtain a container type image;

[0025] Sc, using the CPTN model to perform text detection on the box-shaped image;

[0026] Sd, use the CRNN model to perform OCR recognition on all the results of text detection;

[0027] Se, regularize the recognition results according to the container identity format and output the final container identity.

[0028] In the above technical solution, the multi-site video acquisition module includes:

[0029] A panoramic video acquisition device is arranged in at least one panoramic area of ​​the customs port, and is used to acquire panoramic video of the panoramic area and transmit it to the monitoring and command module in real time by means of 5G communication transmission;

[0030] The regional video acquisition equipment is deployed in at least one local area of ​​the customs port, and is used to collect the regional video of the local area and transmit it to the monitoring and command module in real time by means of 5G communication transmission.

[0031] In the above technical solution, the monitoring and command module is also used to control the panoramic video acquisition device and the regional video acquisition device to rotate, zoom and switch the day vision / night vision mode respectively, so as to rotate, zoom and switch the day vision / night vision mode of the panoramic video and the regional video respectively.

[0032] In the above technical scheme, the customs port comprehensive monitoring and command system based on 5G communication transmission of the present invention also includes a video acquisition device positioning module; the video acquisition device positioning module is used to locate the corresponding video switching label on the panoramic video or the regional video according to the name of the panoramic video acquisition device or the regional video acquisition device input by the user.

[0033] A comprehensive monitoring and command method for customs ports based on 5G communication transmission, comprising:

[0034] Collecting a panoramic video in at least one panoramic area of ​​a customs port, collecting a regional video in at least one local area of ​​the customs port, and outputting the panoramic video and the regional video in a 5G communication transmission manner;

[0035] receiving and displaying the panoramic video and the regional video in real time;

[0036] Using the panoramic image and the regional image to form a panoramic map of the customs port;

[0037] Adding a video switching label to the panoramic map;

[0038] Displaying corresponding panoramic video or regional video according to the user's operation on the video switching tag;

[0039] Overlaying and displaying the customs business data in the customs business database on the panoramic map;

[0040] Supporting a user to select a container in the panoramic video or the regional video, dynamically tracking and focusing on the selected container, identifying the identity of the selected container, and, based on the identity of the container, overlaying and displaying customs business data associated with the container on the panoramic video and / or the regional video; and,

[0041] The user is supported to send assembly instructions, location information associated with the assembly instructions, and event information associated with the assembly instructions to the terminal of the customs staff.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows: the comprehensive monitoring and command system and method for customs ports based on 5G communication transmission of the present invention can collect panoramic images and regional images through a multi-site video acquisition module, and realize the display of panoramic images and regional images, the construction of panoramic maps, the switching between different panoramic images and regional images, the display of customs business data, the addition and management of custom labels, the identification of containers and the reading of customs business data through the monitoring and command module; the use of 5G communication transmission makes the comprehensive monitoring and command system and method for customs ports have the advantages of large bandwidth, low latency, stable signal distribution, high reliability and high rate, and can meet the transmission requirements of high-definition panoramic images and regional images. Based on the 5G communication transmission network, high-definition panoramic images and regional images are transmitted back, and structured analysis is performed in combination with AR technology to display the real-time panoramic situation of the venue. It realizes large-scale and three-dimensional monitoring of customs ports, making it easy for customs agencies to grasp the on-site situation of customs ports in real time, and through linkage with the customs business database module, customs agencies can view various customs business data in real time on the monitoring and command module, thereby enabling customs agencies to carry out administrative law enforcement activities with better quality and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a schematic diagram of the system structure of Embodiment 1 of the present invention.

[0044] Figure 2 This is one of the network topology diagrams after the customs port comprehensive monitoring and command system based on 5G communication transmission of embodiment 1 of the present invention establishes a connection with the established customs business management system.

[0045] Figure 3 This is the second network topology diagram after the customs port integrated monitoring and command system based on 5G communication transmission of embodiment 1 of the present invention is connected with the established customs business management system.

[0046] Figure 4 This is a schematic diagram of a typical application case of 5G+smart government affairs. Implementation

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Embodiment 1

[0048] This embodiment provides a comprehensive monitoring and command system for customs ports based on 5G communication transmission, which can be used in customs ports such as container terminals and land transportation hubs to assist customs agencies in conducting real-time and around-the-clock monitoring and management of customs ports.

[0049] 5G technology has three major network capabilities: enhanced mobile broadband (eMBB, referred to as large bandwidth), massive machine type communications (mMTC, referred to as large connections) and low latency and high reliability communications (uRLLC, referred to as low latency). Applying 5G technology to the customs port comprehensive monitoring and command system can enable the customs port comprehensive monitoring and command system and method to have the advantages of large bandwidth, low latency, stable signal distribution, high reliability and high speed, meeting the transmission requirements of high-definition panoramic video and regional video.

[0050] See also Figure 4 The network built with 5G technology, combined with ultra-high-definition video technology and VR / AR technologies, can effectively improve the level of smart government remote services and user experience, allowing people to run less errands and handle affairs more easily. At present, all regions are planning and developing 5G+ smart government affairs, and personalized application pilot projects have also been carried out in customs ports and other institutions.

[0051] See also Figure 1 The customs port integrated monitoring and command system based on 5G communication transmission of this embodiment includes:

[0052] A multi-site video acquisition module, used to acquire panoramic video in at least one panoramic area of ​​the customs port, acquire regional video in at least one local area of ​​the customs port, and output the panoramic video and regional video in a 5G communication transmission mode;

[0053] The customs business database module is used to store, manage and read the customs business data passing through the customs port;

[0054] A monitoring and command module is used to receive and display panoramic video and regional video in real time, use the panoramic video and regional video to form a panoramic map of the customs port, add a video switching label to the panoramic map, display the corresponding panoramic video or regional video according to the user's operation on the video switching label, and overlay the customs business data on the panoramic map;

[0055] A container identification module, for supporting a user to select a container in the panoramic image or the regional image displayed by the monitoring and command module, dynamically track and focus on the selected container, identify the identity of the selected container, and, based on the identity of the container, overlay and display customs business data associated with the container on the panoramic image and / or the regional image; and

[0056] The intelligent individual soldier dispatching module is used to send assembly instructions, location information associated with the assembly instructions, and event information associated with the assembly instructions to the terminals of customs staff.

[0057] Among them, the panoramic area can be a location with a good view in the customs port, such as high-position monitoring points, the top of high-rise buildings, and the installation location of communication base stations. The panoramic video can reflect the panoramic view, overall scene or large-scale scene of the customs port; the local area is the node positions such as the checkpoint functional area, weighing scale functional area, berth functional area, inspection field functional area and yard functional area distributed in various parts of the customs port, and the regional video can reflect the scene of the above-mentioned area; the panoramic video and the regional video use a unified coordinate system, and after being associated with the map coordinates, a panoramic map composed of the panoramic video and the regional video can be constructed.

[0058] In fact, the location information associated with the assembly instruction may be location coordinates, location details or location codes, etc., and the event information associated with the assembly instruction may include textual event information, related pictures and risk abnormality instructions, etc.

[0059] Specifically, the multi-site video acquisition module includes:

[0060] A panoramic video acquisition device is deployed in at least one panoramic area of ​​the customs port, and is used to collect panoramic video of the panoramic area and transmit it to the monitoring and command module in real time by means of 5G communication transmission;

[0061] The regional video acquisition equipment is deployed in at least one local area of ​​the customs port, and is used to collect the regional video of the local area and transmit it to the monitoring and command module in real time by means of 5G communication transmission.

[0062] The transmission of panoramic and regional videos has two obvious characteristics: first, it has high requirements for network transmission quality, such as high bandwidth, low latency, and wide connection, which requires simplifying the network architecture to better leverage the low latency characteristics of 5G; second, it requires processing large amounts of data such as high-definition video and maps, which puts great pressure on the existing customs system network and may create new information security risks.

[0063] The use of 5G communication transmission enables this customs port comprehensive monitoring and command system and method to have the advantages of large bandwidth (traffic density can reach ≥20Tbps / Km²), low latency (average response time is 0.87 seconds, end-to-end latency ≤1ms), stable signal distribution, high reliability and high rate (peak rate ≥0.6Gbps), which can meet the transmission requirements of high-definition panoramic video and regional video.

[0064] In fact, in order to collect panoramic videos and regional videos at customs ports, it is often necessary to set up panoramic video acquisition equipment and regional video acquisition equipment in locations with good visibility, such as rooftops, high platforms, nearby buildings, communication towers, etc. These locations are difficult to lay signal lines or optical cables for various reasons. Therefore, the real-time transmission of panoramic videos and regional videos in the form of 5G communication transmission enables the panoramic video acquisition equipment and regional video acquisition equipment to overcome the problem of difficulty in laying signal lines or optical cables and ensure the bandwidth for high-definition video transmission.

[0065] To be more specific, the panoramic video acquisition device can be a heavy-duty pan / tilt camera or other camera equipment that can capture panoramic video; the regional video acquisition device can be a facial recognition camera, a high-definition gun-type surveillance camera, a high-definition ball-type surveillance camera, or other camera equipment that can capture regional video.

[0066] Specifically, the customs business database module can be deployed as a single database, multiple databases, or a distributed database. Similarly, the database server of the customs business database module can be arranged as one, multiple, or a remote server system; the customs business database module is specifically used to store, manage, and read customs business data such as customs port information data, customs clearance data, weighbridge management data, and loading and unloading management data.

[0067] Specifically, the monitoring and command module is composed of a back-end server and a front-end terminal that are communicatively connected to each other; the back-end server is used to process, forward, manage and store data from the multi-site video acquisition module and the customs business database module, such as panoramic video, regional video and customs business data; the front-end terminal is a human-computer interaction device, such as a personal computer, a multi-screen monitoring and command system, and portable terminals such as smart phones or tablet computers; in fact, the front-end terminal is used to display panoramic video, regional video and panoramic maps, overlay and display customs business data, and support various user interactive operations.

[0068] Specifically, it supports users to select containers in the panoramic video or regional video displayed by the monitoring and command module. Specifically, it supports users to select a container in the panoramic video or regional video; the customs business data associated with the container specifically includes the customs clearance data, weighbridge management data, and loading and unloading management data of the container.

[0069] Furthermore, the monitoring and command module is also used to control the panoramic video acquisition device and the regional video acquisition device to rotate, zoom and switch the day vision / night vision mode respectively, so as to rotate, zoom and switch the day vision / night vision mode of the panoramic video and the regional video respectively; in fact, the rotation and zoom of the panoramic video and the regional video can be integrated in the interactive interface of the monitoring and command module in the form of a control compass, and superimposed on the panoramic video and the regional video; the switching of the day vision / night vision mode can be integrated in the interactive interface of the monitoring and command module in the form of a control button, and superimposed on the panoramic video and the regional video, or it can be automatically switched according to factors such as the current time and ambient brightness.

[0070] Furthermore, the customs port comprehensive monitoring and command system based on 5G communication transmission of this embodiment also includes a label adding and management module; in fact, the label adding and management module is a software module, which can be deployed on the hardware on the monitoring and command module; the label adding and management module is used to add custom tags to video objects in panoramic video and regional video, manage custom tags, associate custom tags with the coordinates of the panoramic map, search for custom tags, and locate custom tags on the panoramic video or regional video according to the coordinates of the custom tags on the panoramic map.

[0071] Specifically, adding custom tags means supporting users to add custom tags to a certain video object (such as containers, vehicles, personnel, etc.) in panoramic videos and regional videos, managing custom tags means modifying or deleting custom tags, and searching for custom tags means searching for corresponding custom tags according to the name of the custom tags.

[0072] Specifically, the custom tag is at least one of a text tag, a picture tag, a document tag, a voice tag, a video tag and a mixed tag; a text tag refers to a custom tag with text, a picture tag refers to a custom tag with an inserted picture, a document tag refers to a custom tag with an inserted document file such as a text document, a spreadsheet document and a presentation document, a voice tag refers to a custom tag with an inserted voice recorded by a user, a video tag refers to a custom tag with an inserted video recorded, edited or captured by a user, and a mixed tag refers to a custom tag with any combination of the above-mentioned text, picture, document, voice and video elements.

[0073] Furthermore, the label adding and management module is also used to add regional labels to the panoramic video, regional video, partial areas of the panoramic video and partial areas of the regional video, combine the regional labels into a region list, and switch to the panoramic video or regional video corresponding to the region label according to the user's operation on the region list; in fact, the coordinates of each region label on the panoramic map have been pre-stored, and when locating the corresponding video switching label on the panoramic video or regional video, it is only necessary to align the center of the panoramic video or regional video with the coordinates of the video switching label.

[0074] Specifically, the custom label is associated with the coordinates of the panoramic map, and the specific method includes:

[0075] S1. Determine the video object to which the custom tag is added.

[0076] In this step, the video object is specifically a video object in a panoramic image or a regional image (such as a container, a vehicle, a person, etc.), which is obtained through image recognition technology.

[0077] S2. Obtain the ptz value and field of view angle of the video object in the panoramic video or the regional video.

[0078] Among them, ptz stands for Pan / Tilt / Zoom, which represents the horizontal angle, up and down angle and zoom ratio of the panoramic video acquisition device and the regional video acquisition device.

[0079] S3. Calculate and obtain the xy value of the video object in the panoramic image or the regional image according to the ptz value and the viewing angle of the video object.

[0080] S4. Associating the xy values ​​of the video object in the panoramic image or the regional image with the coordinates of the panoramic map.

[0081] Specifically, the identity of the selected container is identified, and the specific method includes:

[0082] Sa. Acquire an image of the container according to the container selected by the user.

[0083] In this step, the image selected by the user in the panoramic video or the regional video can be captured as a frame and directly used as the image of the container.

[0084] Sb. Use the image semantic segmentation model to roughly locate the container surface of the container image to segment the container number in the image, and preprocess the image to obtain the container type image.

[0085] In this step, the image semantic segmentation model is used to achieve classification at the pixel level, which can be achieved by using semantic segmentation methods such as TextonForest, random forest classifier or deep learning method. This embodiment uses an image semantic segmentation model based on convolutional neural network.

[0086] Sc. Use the CPTN model to perform text detection on box-type images.

[0087] The trained CPTN model can detect text sequences of indefinite length. The container number on the container surface can be detected through the CPTN model.

[0088] Sd. Use the CRNN model to perform OCR recognition on all results of text detection.

[0089] CRNN is an end-to-end text recognition model that does not require character segmentation and can recognize sequences of any length.

[0090] Se, regularize the recognition results according to the container identity format and output the final container identity.

[0091] The comprehensive monitoring and command system for customs ports based on 5G communication transmission of this embodiment can collect panoramic videos and regional videos through multi-point video acquisition modules, and realize the display of panoramic videos and regional videos, the construction of panoramic maps, the switching between different panoramic videos and regional videos, the display of customs business data, the addition and management of custom labels, the identification of containers and the reading of customs business data through the monitoring and command modules; it realizes large-scale and three-dimensional monitoring of customs ports, making it easy for customs agencies to grasp the on-site situation of customs ports in real time, and through the linkage with the customs business database module, the customs agencies can view various customs business data in real time on the monitoring and command modules, so that the customs agencies can carry out administrative law enforcement activities with better quality and efficiency.

[0092] like Figure 2 or Figure 3 As shown, the customs port comprehensive monitoring and command system based on 5G communication transmission in this embodiment, since its monitoring and command module is composed of a back-end server and a front-end terminal that are communicatively connected to each other, it can establish a connection with the established customs business management system through the customs internal network, such as the existing video surveillance system, vehicle identification system, individual management system, weighbridge management system, loading and unloading management system, operation and maintenance management system, and the Internet of Things sensing system including a container sensing module and a weighbridge sensing module. Embodiment 2

[0093] This embodiment provides a comprehensive monitoring and command method for customs ports based on 5G communication transmission, which applies the comprehensive monitoring and command system for customs ports based on 5G communication transmission recorded in Example 1.

[0094] The comprehensive monitoring and command method for customs ports based on 5G communication transmission in this embodiment includes:

[0095] Collecting panoramic video in at least one panoramic area of ​​the customs port, collecting regional video in at least one local area of ​​the customs port, and outputting the panoramic video and regional video in a 5G communication transmission manner;

[0096] Receive and display panoramic and regional images in real time;

[0097] Use panoramic images and regional images to construct a panoramic map of the customs port;

[0098] Added visual switching labels in the panoramic map;

[0099] Displaying corresponding panoramic video or regional video according to the user's operation on the video switching label;

[0100] Overlay and display customs business data in the customs business database on the panoramic map;

[0101] Supporting the user to select a container in the panoramic or regional video, dynamically track and focus on the selected container, identify the selected container, and, based on the container identity, overlay and display customs business data associated with the container on the panoramic and / or regional video; and,

[0102] Support users to send assembly instructions, location information associated with the assembly instructions, and event information associated with the assembly instructions to the customs staff's terminal.

[0103] Among them, the panoramic area can be a location with a good view in the customs port, such as high-position monitoring points, the top of high-rise buildings, and the installation location of communication base stations. The panoramic video can reflect the panoramic view, overall scene or large-scale scene of the customs port; the local area is the node positions such as the checkpoint functional area, weighing scale functional area, berth functional area, inspection field functional area and yard functional area distributed in various parts of the customs port, and the regional video can reflect the scene of the above-mentioned area; the panoramic video and the regional video use a unified coordinate system, and after being associated with the map coordinates, a panoramic map composed of the panoramic video and the regional video can be constructed.

[0104] Specifically, customs business data can be deployed in a single database, in multiple databases, or in a distributed database. Similarly, the database servers of the above databases can be arranged as one, multiple, or remote server systems; customs business data specifically include customs port information data, customs clearance data, weighbridge management data, loading and unloading management data, and other customs business data.

[0105] The real-time reception and display of panoramic video and regional video, the use of panoramic video and regional video to form a panoramic map of the customs port, the addition of video switching tags in the panoramic map, the display of corresponding panoramic video or regional video according to the user's operation on the video switching tag, and the superimposition and display of customs business data in the customs business database on the panoramic map can be undertaken by a monitoring and command module, which is composed of a back-end server and a front-end terminal that are interconnected; the back-end server is used to process, forward, manage and store data from a multi-site video acquisition module and a customs business database module, such as panoramic video, regional video and customs business data; the front-end terminal is a human-computer interaction device, such as a personal computer, a multi-screen monitoring and command system, and a portable terminal such as a smart phone or a tablet computer; in fact, the front-end terminal is used to display panoramic video, regional video and panoramic maps, superimpose and display customs business data, and support various user interactive operations.

[0106] The comprehensive monitoring and command method for customs ports based on 5G communication transmission in this embodiment realizes large-scale, three-dimensional monitoring of customs ports, making it easy for customs agencies to grasp the on-site conditions of customs ports in real time, and through linkage with the customs business database module, customs agencies can view various customs business data in real time on the monitoring and command module, enabling customs agencies to carry out administrative law enforcement activities with better quality and efficiency.

[0107] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A customs port integrated monitoring and command system based on 5G communication transmission, characterized in that: include: A multi-point video acquisition module is used to acquire panoramic video in at least one panoramic area of ​​a customs port, to acquire regional video in at least one local area of ​​the customs port, and to output the panoramic video and the regional video in a 5G communication transmission mode; the panoramic area is a high-position monitoring point, a top of a high-rise building, and a communication base station installation position; the local area is a checkpoint functional area, a weighbridge functional area, a berth functional area, an inspection field functional area, and a storage yard functional area distributed throughout the customs port; A customs business database module, used to store, manage and read customs business data passing through the customs port; a monitoring and command module, for receiving and displaying the panoramic video and the regional video in real time, using the panoramic video and the regional video to form a panoramic map of the customs port, adding a video switching label to the panoramic map, displaying the corresponding panoramic video or regional video according to the user's operation on the video switching label, and overlaying and displaying the customs business data on the panoramic map; a container identification module, for supporting a user to select a container in the panoramic image or the regional image displayed by the monitoring and command module, to track and focus the selected container in a dynamic manner, to identify the identity of the selected container, and, based on the identity of the container, to overlay and display customs business data associated with the container on the panoramic image and / or the regional image, wherein the customs business data associated with the container includes customs clearance data, weighbridge management data, and loading and unloading management data of the container; and An intelligent individual soldier dispatching module, used to send assembly instructions, location information associated with the assembly instructions, and event information associated with the assembly instructions to a terminal of a customs staff member; It also includes a tag adding and management module; The tag adding and managing module is used to add custom tags to video objects in the panoramic video and the regional video, manage the custom tags, associate the custom tags with coordinates of the panoramic map, search for the custom tags, and locate the custom tags on the panoramic video or the regional video according to the coordinates of the custom tags on the panoramic map; The custom tag is associated with the coordinates of the panoramic map, and the specific method includes: S1, determining the video object to which the custom tag is added; S2, obtaining the ptz value and the field of view angle of the video object in the panoramic image or the regional image; S3, calculating and obtaining the xy value of the video object in the panoramic video or the regional video according to the ptz value and the field of view angle of the video object in the panoramic video or the regional video; S4. Associate the xy values ​​of the video object in the panoramic image or the regional image as coordinates of the panoramic map.

2. The customs port integrated monitoring and command system based on 5G communication transmission according to claim 1 is characterized in that: The label adding and management module is also used to add regional labels to the panoramic video, the regional video, a partial area of ​​the panoramic video, and a partial area of ​​the regional video, combine the regional labels into a regional list, and switch to the panoramic video or regional video corresponding to the regional label according to the user's operation on the regional list.

3. The customs port integrated monitoring and command system based on 5G communication transmission according to claim 1 is characterized in that: The custom tag is at least one of a text tag, a picture tag, a document tag, a voice tag, a video tag, and a mixed tag.

4. The customs port integrated monitoring and command system based on 5G communication transmission according to claim 1 is characterized in that: Identify the selected container, the specific methods include: Sa. Acquire an image of the container according to the container selected by the user; Sb. Using an image semantic segmentation model, roughly locating the container surface of the container image to segment out the container number in the image, and preprocessing the image to obtain a container type image; Sc, using the CPTN model to perform text detection on the box-shaped image; Sd, use the CRNN model to perform OCR recognition on all the results of text detection; Se, regularize the recognition results according to the container identity format and output the final container identity.

5. According to the 5G communication transmission-based customs port integrated monitoring and command system of claim 1, it is characterized by: The multi-site video acquisition module includes: A panoramic video acquisition device is arranged in at least one panoramic area of ​​the customs port, and is used to acquire panoramic video of the panoramic area and transmit it to the monitoring and command module in real time by means of 5G communication transmission; The regional video acquisition equipment is deployed in at least one local area of ​​the customs port, and is used to collect the regional video of the local area and transmit it to the monitoring and command module in real time by means of 5G communication transmission.

6. The customs port integrated monitoring and command system based on 5G communication transmission according to claim 5 is characterized in that: The monitoring and command module is also used to control the panoramic video acquisition device and the regional video acquisition device to rotate, zoom and switch the day vision / night vision mode respectively, so as to rotate, zoom and switch the day vision / night vision mode of the panoramic video and the regional video respectively.

7. The customs port integrated monitoring and command system based on 5G communication transmission according to claim 6 is characterized in that: It also includes a video acquisition device positioning module; The video acquisition device positioning module is used to locate the corresponding video switching label on the panoramic video or the regional video according to the name of the panoramic video acquisition device or the regional video acquisition device input by the user.

8. A comprehensive monitoring and command method for customs ports based on 5G communication transmission, characterized in that: include: Collecting a panoramic video in at least one panoramic area of ​​a customs port, collecting a regional video in at least one local area of ​​the customs port, and outputting the panoramic video and the regional video in a 5G communication transmission manner; Store, manage and read customs business data passing through the customs port; receiving and displaying the panoramic video and the regional video in real time, forming a panoramic map of the customs port using the panoramic video and the regional video, adding a video switching label to the panoramic map, displaying the corresponding panoramic video or regional video according to the user's operation on the video switching label, and superimposing and displaying the customs business data on the panoramic map; Supporting the user to select a container in the panoramic video or the regional video, dynamically track and focus on the selected container, identify the identity of the selected container, and, based on the identity of the container, overlay and display customs business data associated with the container on the panoramic video and / or the regional video, the customs business data associated with the container including customs clearance data, weighbridge management data, and loading and unloading management data of the container; as well as, Supporting users to send assembly instructions, location information associated with the assembly instructions, and event information associated with the assembly instructions to the terminal of customs staff; Adding custom tags to video objects in the panoramic video and the regional video, managing the custom tags, associating the custom tags with coordinates of the panoramic map, searching for the custom tags, and locating the custom tags on the panoramic video or the regional video according to the coordinates of the custom tags on the panoramic map; The custom tag is associated with the coordinates of the panoramic map, and the specific method includes: S1, determining the video object to which the custom tag is added; S2, obtaining the ptz value and the field of view angle of the video object in the panoramic image or the regional image; S3, calculating and obtaining the xy value of the video object in the panoramic video or the regional video according to the ptz value and the field of view angle of the video object in the panoramic video or the regional video; S4. Associate the xy values ​​of the video object in the panoramic image or the regional image as coordinates of the panoramic map.

Images