A monitoring method applied to drone mother vehicle parking area

Abstract

The present invention provides a monitoring method applied to a drone mother car parking area, which at least includes: a camera installed on the roof collects images around the roof, compresses the images and sends them to the TSN gateway; the TSN gateway sends the images to a preset image processing host for image stitching to form a panoramic image; the panoramic image is sent to a preset target terminal for display; the roof camera sets a variety of different position parameter arrays, corresponding to the roof surrounding environment monitoring in different application scenarios. The monitoring of the drone mother car parking lot provided by the present invention can be used for drones to monitor the parking area during takeoff and landing, obtain panoramic images in real time, and ensure that the drone can take off and land safely at the drone mother car parking area.

Description

Technical Field

[0001] The present invention relates to the field of monitoring of a drone mother vehicle parking area, and in particular to a monitoring method of a drone mother vehicle parking area. Background Art

[0002] In recent years, with the development and progress of technology, the application of drones has gradually extended from the military field to the civilian field, and the scope of application has been continuously broadened, and it has become increasingly mature in the consumer, plant protection, electricity, security, surveying and mapping and other industries. The application of drones plays an increasingly important role in the national economy and social production and life. However, drones themselves also have their inherent defects, such as the problem of endurance, which has always plagued the scope of drone applications. If the drone flies far to perform a mission, it is likely that the drone will be unable to return after running out of power, and it will also be unable to perform long-distance applications. For example, when encountering a prominent event, such as a forest fire or a flash flood, the drone cannot be executed to the designated location due to the long distance. In the existing technology, there is a method of using a car to install a drone to perform a flight mission, but during the take-off or landing of the drone, people are required to leave the car and place the drone's parking area on the open roadside for take-off or landing, which is very inconvenient. The existing technology also places the drone airport on the roof, but it cannot achieve automatic take-off or landing. Even if it can achieve take-off or landing, the technology is very immature. There is a lack of monitoring devices on the roof, and the people in the car cannot know whether the parking environment on the roof is suitable for parking. Even if it is suitable for parking, there may be interference from obstacles, making it impossible for the drone to stop smoothly. In order to solve the problems of the existing technology, it is necessary to provide a monitoring method using the roof of the drone mother vehicle to ensure that the drone can be safely docked at the preset position of the drone mother vehicle. Summary of the invention

[0003] Based on the defects existing in the prior art, the present invention provides a monitoring method applied to a drone mother vehicle parking area, which at least includes: a camera installed on the roof collects images around the roof, compresses the images and sends them to a TSN gateway;

[0004] The TSN gateway sends the image to a preset image processing host for image stitching to form a panoramic image;

[0005] Send the panoramic image to a preset target terminal for display;

[0006] Among them, the roof camera is set with a variety of different position parameter arrays, corresponding to the roof environment monitoring in different application scenarios, and each position parameter is set with a corresponding panoramic stitching image overlapping stitching feature point; when the panoramic image is stitched, the position parameter array of the current roof camera is obtained and the corresponding overlapping stitching feature points are called according to the position parameter array to stitch the image to obtain a panoramic image.

[0007] A monitoring method applied to a drone mother vehicle parking area, further, the roof camera is a fisheye camera, and before use, the fisheye camera is subjected to distortion correction; the distortion correction method includes:

[0008] Establish the image coordinate system of the roof camera, map the pixel coordinates of the roof camera to the polynomial in the unit sphere coordinate system; project the adjacent image matching point pairs onto the unit sphere, and take the spherical median or mean of the matching point pairs as the optimization target; use the Euclidean distance between the median or mean and the matching points as the cost function of the target; solve the cost function according to the gradient descent method or the least squares method to obtain the distortion correction parameters of the roof camera.

[0009] A monitoring method applied to a drone mother vehicle parking area, further, the position parameter array includes the position parameters of each roof camera, the position parameters are represented by [θij,hij], wherein θij represents the angle of the jth roof camera corresponding to the i-th set of position parameter arrays relative to a reference direction, and hij represents the height of the jth roof camera corresponding to the ii-th set of position parameter arrays relative to the reference direction;

[0010] The position parameters of the multiple roof cameras form a position parameter array.

[0011] A monitoring method applied to a drone mother vehicle parking area, further optionally, in the process of image stitching, after adjacent graphics are registered, pixel coordinates (xi, yi) of an average segmentation curve of the overlapped area are obtained according to the overlapped area of ​​the images, and the coordinates of several points are selected from the segmentation curve and stored as feature points for subsequent image stitching;

[0012] Get the position parameters of the current roof camera, determine whether the position parameters of all roof cameras match one of the preset position parameter arrays, and if so, call the stitching feature points matched by the reference coordinates to perform image stitching to form a panoramic image.

[0013] A monitoring method applied to a drone mother vehicle parking field, further optionally, a pixel coordinate calculation method of an average segmentation curve includes:

[0014] Acquire multiple panoramic images, and in the multiple panoramic images, obtain the overlapping area of ​​adjacent images in each panoramic image, and calculate the mean curve Ai corresponding to the overlapping area of ​​adjacent images in each panoramic image; sort the mean curves corresponding to the overlapping area of ​​adjacent images in each panoramic image in the order of first the x direction and then the y direction from small to large to obtain the area surrounded by the mean curve combination

[0015] According to the given endpoints, the average curve coordinates in the enclosed area are obtained as feature points for stitching adjacent images in image stitching.

[0016] A monitoring method applied to a drone mother vehicle parking area. Further optionally, a roof camera is provided with two first vehicle-mounted Ethernet PHY interfaces and a second vehicle-mounted Ethernet PHY interface; a plurality of roof cameras are connected in sequence through the first vehicle-mounted Ethernet PHY interface and the second vehicle-mounted Ethernet PHY interface; and a terminal roof camera is connected to a TSN gateway through a vehicle-mounted Ethernet bus.

[0017] A monitoring method applied to a drone mother vehicle parking area, further optionally, when the system is initialized, the panoramic service module sends a panoramic image service to the vehicle network by broadcasting, and after receiving the service, the node target in the network determines whether the service is needed. If the service is needed, a connection is established with the panoramic service module to subscribe to the panoramic service;

[0018] The panoramic service module records the node targets subscribing to the panoramic service through the panoramic list, and initializes the image cache management area at the same time;

[0019] The panoramic service module monitors the circular buffer space in the system, and when there is data stored in the circular buffer space, the data is sent to the node targets in the panoramic list respectively.

[0020] A monitoring method applied to a drone mother vehicle parking area. Further optionally, panoramic images are stored in a circular cache space opened by the system in a first-in-first-out order; the circular cache space is provided with a header file management area and a data area, the header file management area records information of the data area, and the data area includes multiple data blocks for storing data.

[0021] A monitoring method applied to a drone mother vehicle parking field, further optionally, the panoramic service module also includes an image buffer management, the image buffer management is used to synchronize header file management area data and to record the target increment of the panoramic image transmission;

[0022] Each time the panoramic service module obtains a data block from the circular cache space, a corresponding target increment is generated; each time a data block is sent to a node target, the target increment corresponding to the data block is reduced by 1.

[0023] A monitoring method applied to a drone mother vehicle parking lot. Further optionally, when a node target requiring panoramic service is interrupted, an instruction that no panoramic image service is required is sent to a panoramic service module. After receiving the instruction, the panoramic service module deletes the corresponding target from the panoramic list.

[0024] A monitoring method applied to a drone mother vehicle parking area, further optionally, obtains a node target terminal that needs to be synchronized, adjusts the system clocks of the node device where the panoramic image is located and the node target terminal to a common reference time, and sends time information on the network at a preset period, the time information includes:

[0025] Panoramic image timestamp and system timestamp;

[0026] After receiving the time information, the target terminal calculates the time synchronization information required for synchronization between devices.

[0027] Time synchronization of node devices and node target terminals includes exchanging timestamps, calculating time offsets based on round-trip delays between the node devices and the target terminals, and adjusting system time.

[0028] Beneficial effects:

[0029] 1. In the technical solution provided by the present invention, position parameter arrays of roof cameras with multiple different viewing angles are designed, each set of position parameter arrays corresponds to different application scenarios, and each set of parameter arrays stores pixel coordinate feature points of the overlapping area of ​​the position parameter array and the corresponding panoramic image stitching, and stitching is directly completed through the corresponding pixel coordinate feature points. This allows a set of roof cameras to adapt to different application scenarios, and panoramic image stitching can be completed only based on pixel feature points, with high real-time performance.

[0030] 2. In the technical solution provided by the present invention, images are collected under different lighting environments and different scenes, and the images collected under different lighting environments and different scenes are registered and then spliced ​​into multiple panoramic images. The average curve coordinates are obtained by calculating multiple different mean curves. The matching error of adjacent pixels can be reduced.

[0031] 3. In the technical solution provided by the present invention, the vehicle Ethernet display screen displays according to the content of the display control list by means of a display control list. The display control list has a display list library, which is used to select the display control list of interest and set it as the current display, so that the display content control of the vehicle Ethernet display screen can be easily realized.

[0032] 4. In the technical solution provided by the present invention, a panoramic service module is designed in the system so that multiple target terminals with service requirements can establish subscription services with the panoramic service module, and a circular buffer is designed at the same time to confirm the image sending through target increments, thereby reducing the number of users calling simultaneously and frequently establishing connections. The processing of data packets by the system process will increase the system load and occupy system resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The following drawings are only used to schematically illustrate and explain the present invention, and do not limit the scope of the present invention.

[0034] Figure 1 It is a structural schematic diagram of a monitoring system for a drone mother vehicle parking area in another embodiment of the present invention.

[0035] Figure 2 Schematic diagram of the connection between a roof camera and a TSN gateway in another embodiment of the present invention.

[0036] Figure 3 This is a schematic diagram of another connection between a roof camera and a TSN gateway in another embodiment of the present invention.

[0037] Figure 4 Schematic diagram of a monitoring system for a roof camera and a drone mother vehicle parking area in another embodiment of the present invention.

[0038] Figure 5 Schematic diagram of a method for transmitting a panoramic image to a preset target terminal in another embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to have a clearer understanding of the technical features, purposes and effects of this invention, the specific implementation methods of the present invention are now described with reference to the accompanying drawings, and the same reference numerals in each figure represent the same parts. In order to make the drawings concise, the parts related to the present invention are schematically shown in each figure, and do not represent the actual structure of the product. In addition, in order to make the drawings concise and easy to understand, in some figures, only one of the parts with the same structure or function is schematically drawn or marked.

[0040] Regarding control systems, functional modules, and application programs (APP), it is well known to those skilled in the art that they can take any appropriate form, either hardware or software, either multiple discrete functional modules or multiple functional units integrated into one hardware. In the simplest form, the control system can be a controller, such as a combinational logic controller, a microprogram controller, etc., as long as the operations described in this application can be implemented. Of course, the control system can also be integrated into a physical device as different modules, which does not deviate from the basic principles and protection scope of the present invention.

[0041] The term “connection” in the present invention may include direct connection, indirect connection, communication connection, and electrical connection, unless otherwise specified.

[0042] The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that when used in the specification, the terms "include" and / or "comprise" refer to the presence of stated features, values, steps, operations, elements and / or components, but do not exclude the presence or additional addition of one or more other features, values, steps, operations, elements, components and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the listed related items.

[0043] It should be understood that the term "vehicle" or "vehicular" or other similar terms used herein generally include motor vehicles, such as passenger cars including sport utility vehicles (SUVs), buses, trucks, various commercial vehicles, ships including various boats, marine vessels, aircraft, etc., and include hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles, and other alternative fuel vehicles (e.g., fuels derived from non-petroleum energy sources). As referred to herein, a hybrid vehicle is a vehicle having two or more power sources, such as both gasoline-powered and electric-powered vehicles.

[0044] In addition, the controller of the present disclosure may be embodied as a non-transitory computer readable medium on a computer readable medium containing executable program instructions executed by a processor, a controller, or the like. Examples of computer readable media include, but are not limited to, ROM, RAM, compact disk (CD)-ROM, magnetic tape, floppy disk, flash drive, smart card, and optical data storage device. The computer readable recording medium may also be distributed in a computer system coupled via a network so that the computer readable medium is stored and executed in a distributed manner, for example, via a telematics server or a controller area network (CAN).

[0045] The present invention provides a monitoring system applied to a drone mother vehicle parking field, specifically comprising: a roof camera, a TSN gateway, an on-board Ethernet display screen, and a central computing platform, wherein the roof camera is connected to the TSN gateway, and the central computing platform is connected to the TSN gateway;

[0046] Specifically, the roof camera includes one or two of a plurality of fisheye cameras and spherical cameras, and the roof camera includes four cameras. The first camera is installed at the rear of the vehicle body to monitor the rear and upper rear of the vehicle body, the second camera is installed at the front of the vehicle body to monitor the front and upper front of the vehicle body, and the third camera and the fourth camera are installed at the left and right sides of the vehicle body respectively to monitor the parking area set on the roof;

[0047] The angles of all roof cameras are adjustable. Specifically, the third camera can be adjusted upward to monitor the surrounding environment when the drone takes off or lands, and the fourth camera monitors the roof parking environment.

[0048] The images acquired by the first camera, the second camera, the third camera and the fourth camera are stitched together to acquire a panoramic image during landing or take-off;

[0049] In order to ensure that the lens data after imaging is not distorted, the roof camera needs to be rectified. The specific distortion correction methods include:

[0050] Establish an image coordinate system for the roof camera and map the pixel coordinates of the roof camera to a polynomial in the unit sphere coordinate system;

[0051] Project the adjacent image matching point pairs onto the unit sphere, and take the spherical median or mean of the matching point pairs as the optimization target;

[0052] The cost function uses the median or mean and the Euclidean distance between matching points as the target;

[0053] The cost function is solved according to the gradient descent method or the least squares method to obtain the distortion correction parameters of the roof camera.

[0054] In the process of calculating the parameters of the fisheye lens, the forward mapping method is used to map the feature points, that is, to map from the fisheye image to the panoramic image;

[0055] This embodiment is different from the prior art in that, during distortion correction, the spherical median or mean of the matching point pair is used as an optimization target, which can improve accuracy.

[0056] Specifically, most of the existing roof cameras are fixed, such as the on-board surround-view camera, which has a fixed angle and is used when reversing or observing the surrounding environment. However, if the roof camera is installed at a fixed angle, it can only observe the environment of the sky and the parking area during the take-off or landing of the drone, and cannot be used at other times, which undoubtedly causes waste. If a set of cameras is added to observe other areas of the roof, such as observing the overview of the road in the distance while moving forward to remind the driver to prepare for emergencies in advance, or using the on-board camera to observe the scenery around the road while moving forward. In order to solve this problem, this embodiment adopts a position parameter array of a roof camera with multiple different viewing angles, and the position parameter array includes multiple position coordinates, each position parameter corresponds to a camera, and the position parameters of multiple cameras form a position coordinate array, such as: the first set of position parameter arrays is used when the drone is parked or landed, and the second set of position parameter arrays is used to observe the environment around the car body while the car is driving;

[0057] Specifically:

[0058] Represent each set of position parameter arrays with a matrix, assuming there are four roof cameras;

[0059] The first set of positional parameter arrays:

[0060] Array1=[[θ11,h11],[θ12,h12],[θ13,h13],[θ14,h14]]

[0061] The first set of positional parameter arrays:

[0062] Array2=[[θ21,h21],[θ22,h22],[θ23,h23],[θ24,h24]]

[0063] The position parameters of the roof camera are defined as [θij,hij], where θij represents the angle of the j-th roof camera corresponding to the ith set of position parameter array relative to the reference direction, and hij represents the height of the j-th roof camera corresponding to the ii-th set of position parameter array relative to the reference direction.

[0064] Performing image stitching on images captured by multiple roof cameras under different position parameter arrays of the roof cameras;

[0065] In the process of image stitching, after the adjacent graphics are registered, the pixel coordinates (xi, yi) of the average segmentation curve of the overlapping area are obtained according to the image overlap area, and the coordinates of several points are selected from the segmentation curve and stored as feature points for subsequent image stitching;

[0066] After the roof camera takes an image, the coordinate parameters of the current roof camera are obtained respectively to determine whether the parameters of all roof cameras match the preset reference coordinates. If they match, the stitching feature points matching the reference coordinates are called to stitch the images to form a panoramic image.

[0067] Specifically, since the calculation of coincidence is obtained after adjacent image registration, there are errors in the matching of adjacent pixels during the image registration process. If the matching feature point error is large, it will affect the imaging effect of the panoramic image. Therefore, this embodiment provides a method for obtaining image feature points:

[0068] According to the coordinate parameters [θij,hij] of the given roof camera, adjust the camera used for stitching to the corresponding angle and height;

[0069] Collect images under different lighting environments and scenes respectively, and align the images collected under different lighting environments and scenes and then stitch them into multiple panoramic images;

[0070] In multiple panoramic images, the overlapping area of ​​adjacent images in each panoramic image is obtained, and the mean curve Ai corresponding to the overlapping area of ​​adjacent images in each panoramic image is calculated. The coordinates corresponding to the mean curve are Ai(uj,vj), where Ai identifies the overlapping area of ​​the i-th panoramic image A, and (uj,vj) identifies the pixel coordinates corresponding to the j-th pixel point of the mean curve on the x-axis and y-axis coordinates;

[0071] Merge the corresponding mean curves of the overlapping areas of adjacent images in each panoramic image [A1(uj,vj),A2(uj,vj),A3(uj,vj),…], and sort them from small to large in the order of x direction first and then y direction to obtain the area surrounded by the mean curve set;

[0072] According to the given endpoints, the average curve coordinates in the enclosed area are obtained as feature points for stitching adjacent images in image stitching.

[0073] Specifically, the above-mentioned average curve coordinates are obtained by calculating multiple different mean curves, and the obtained results may not be integers, but the integers required by the pixel coordinates. In the prior art, the calculated results are usually rounded. However, the applicant found in the process of rounding that if the rounding calculation is used, the image distortion and registration error near the image splicing area will increase. In order to solve this problem, this embodiment uses the following method for calculation:

[0074] If the calculated pixel Q coordinate is not an integer, the four coordinate points of the nearest neighbor of the pixel coordinate are obtained respectively, M11 (x1, y1), M12 (x2, y2), M21 (x3, y3), M22 (x4, y4), where the coordinate of Q is in the area surrounded by M11, M12, M21, and M22; M11 and M22 represent the upper left and lower right respectively, and M21 and M12 represent the lower and upper right respectively;

[0075] A functional relationship is established based on the distance between Q and the four points in the X-axis and Y-axis directions to obtain the pixel value of Q(x,y):

[0076] Q(x,y)=M11*w11+M12*w12+M21*w21+M22*w22

[0077] Among them, w11=(y2-y)(x2-x), w12=(x-x1)(y-y1)

[0078] W21=(y4-y)(x4-x),w22=(x-x3)(y-y3)

[0079] Specifically, the roof camera is connected to the TSN gateway using the vehicle Ethernet bus;

[0080] Specifically, in this embodiment, if the wiring harness cost is not considered, each vehicle-mounted camera can be connected to the TSN gateway via the vehicle-mounted Ethernet bus.

[0081] Specifically, in this embodiment, for safety reasons, if the TSN gateway is set outside the car, the shockproof, waterproof and bad weather will have a serious impact on the TSN gateway, which reduces the reliability of the TSN gateway. Therefore, in this embodiment, the TSN gateway is set inside the car, but after being set inside the car, if each roof camera is directly connected to the TSN gateway, the length of the wiring harness will increase, which undoubtedly increases the cost. Therefore, in order to reduce the wiring harness of the roof camera, in this embodiment, the structure of the roof camera is improved:

[0082] The roof camera is provided with two first vehicle Ethernet PHY interfaces and a second vehicle Ethernet PHY interface;

[0083] A plurality of roof cameras are connected in sequence via a first vehicle Ethernet PHY interface and a second vehicle Ethernet PHY interface;

[0084] The terminal roof camera is connected to the TSN gateway via the vehicle Ethernet bus;

[0085] Specifically, after the roof camera captures the image, it transmits the image to the central computing platform through the TSN gateway for image stitching processing. After the image stitching processing is completed, the panoramic image is sent to the on-board Ethernet display for display.

[0086] Specifically, the panoramic image of the roof camera may be called and displayed on multiple display screens, such as the on-board host display screen in the cab of the drone mother vehicle, and the display screen of the console in the vehicle. At the same time, the cloud server also needs to store the panoramic image and display it on the on-board Ethernet display screens in different command rooms and control rooms according to user needs.

[0087] The in-vehicle Ethernet display screen is provided with a display list library for storing a display control list;

[0088] The display control list includes at least one;

[0089] Each display control list includes at least one display content, and each display content includes at least: a service provider MAC address, an IP address, a Message ID, a Request ID, and an entry array.

[0090] Among them, Message ID is used to uniquely identify the message. When the message is of Method type, it is composed of Service ID and Method ID. When the message is of Event type, it is composed of Service ID and Event ID.

[0091] The service provider is Service, the service caller is Client, Service ID and Client ID are used for distinction, and the values ​​of these IDs are configured in the SOA architecture of this embodiment, and the IDs are unique.

[0092] The entry array contains at least the service instance and the event group to be subscribed, which is divided into two types: Service and Eventgroup. An SD message may contain multiple entry arrays, each of which is 16 bytes in size. An entry array may contain 0-2 optional contents for transmitting additional information, such as port number, communication channel, encryption method, etc.

[0093] A service instance includes at least Type, service ID, and instance ID. Specifically, TYPE indicates service discovery, service provision, and service notification. Service ID indicates the name of the provided service. Instance ID indicates the service instance content corresponding to the service ID. One service ID can provide multiple different instance contents.

[0094] The event group includes at least: Type, service ID, instance ID, event group ID, and subscription count. Specifically, Type represents subscribing to events and stopping subscribing to event groups; subscription confirmation and non-subscription event group response. The subscription count is used to distinguish subscription requests from the same subscriber.

[0095] Event group ID, that is, the granularity of SOME / IP event subscription and unsubscription to an event group.

[0096] Through the display control list method, the vehicle Ethernet display screen displays according to the content of the display control list. The display control list has a display list library, which can be used to select the display control list of interest and set it as the current display, which can easily realize the display content control of the vehicle Ethernet display screen.

[0097] Specifically, multiple users calling at the same time frequently establish connections and the system process processes data packets, which increases the system load and occupies system resources. In order to solve this problem, this example provides a method for calling a panoramic camera data packet, which is as follows:

[0098] Obtain the image collected by the roof camera and call the image processing algorithm to generate a panoramic image;

[0099] The panoramic image is stored in the circular buffer space opened by the system in a first-in-first-out order;

[0100] The circular cache space is provided with a header file management area and a data area. The data area includes a plurality of data blocks for storing data. The header file management area records the information of the data area, such as whether the plurality of data blocks in the data area store data, the address of each data block, and whether data writing is allowed;

[0101] The panoramic service module includes image buffer management and panoramic list, wherein the panoramic list records the target address to which the panoramic image data needs to be sent, and the image buffer management is used to synchronize the header file management area data and to record the target increment of the panoramic image sending. For example, if the corresponding panoramic image is currently called by three node targets, the target increment is 3 at the beginning. For the data area corresponding to the panoramic image to be sent, the panoramic service module generates a target increment each time it obtains a block from the circular buffer space; each time a data block is sent to a node target, the target increment corresponding to the data block is reduced by 1. When the target increment is 0, the block of the circular buffer corresponding to the panoramic image can be written with new image data;

[0102] When the system is initialized, the panoramic service module broadcasts panoramic image services to the entire network. After receiving the service, the node target in the network determines whether the service is needed. If the service is needed,

[0103] Then establish a connection with the panoramic service module to subscribe to the panoramic service;

[0104] The panoramic service module adds the target to the panoramic list and initializes the image cache management area at the same time;

[0105] When the target is interrupted, an instruction that no panoramic image service is required is sent to the panoramic service module. After receiving the instruction, the panoramic service module deletes the corresponding target from the panoramic list.

[0106] Specifically, when the drone mother vehicle and the drone are taking off or landing, real-time monitoring is carried out through the roof camera, and the images are shared with preset devices. However, if mobile terminals that are not equipped with time synchronization, such as mobile phones, tablets and other devices, need to synchronize images, because mobile phones and tablets are not based on the transmission of vehicle Ethernet protocols, there is a transmission delay when the monitoring system images are transmitted through the drone mother vehicle, and multiple screen sharing is not synchronized. This will cause the user to be unable to handle the emergency situation caused by the transmission delay when operating the drone takeoff and landing with the mobile terminal in the car. In order to solve this technical problem, the present invention provides an embodiment to solve the following method, which specifically includes:

[0107] Get the node target terminal that needs to be synchronized, take the node device where the panoramic image processing is located as the master clock, and the other target terminals as slave time, and send time information on the network at a preset period. The time information includes:

[0108] Panoramic image timestamp and system timestamp;

[0109] After receiving the time information, the target terminal calculates the time synchronization information required for synchronization between devices.

[0110] The system clocks of the node device and the target terminal where the panoramic image is located are adjusted to a common reference time, and clock synchronization is performed regularly to overcome the image jitter when playing on the device.

[0111] Time synchronization of node devices and node target terminals includes exchanging timestamps, calculating time offsets based on round-trip delays between the node devices and the target terminals, and adjusting system time.

[0112] Specifically, a camera for monitoring the interior of the vehicle is installed in the drone mother vehicle. The camera inside the vehicle is connected to the TSN gateway to monitor the situation inside the vehicle. According to the actual situation, the image obtained by the camera inside the vehicle is transmitted to the cloud server for observation by other authorized users.

[0113] At the same time, the in-car camera is used for video conferencing, live streaming, etc. between different users.

[0114] In addition, a USB cable is reserved in the car, and users can connect the smart terminal through the USB cable to control the drone;

[0115] Specifically, it is also possible to access the drone through app pairing and directly control the drone;

[0116] Another method is to connect to the vehicle's central control host, project the drone control program of the vehicle's central control host to the smart terminal for display, and then directly operate the drone through the smart terminal.

[0117] The above is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiment. It is clear to those skilled in the art that the form in the embodiment is not limited thereto, and the adjustable manner is not limited thereto. It is understood that other improvements and changes directly derived or associated with those skilled in the art without departing from the basic concept of the present invention should be considered to be included in the protection scope of the present invention.

Claims

1. A monitoring method applied to a drone mother vehicle parking area, characterized in that: At least including: a camera installed on the roof collects images around the roof, compresses the images and sends them to the TSN gateway; The TSN gateway sends the image to a preset image processing host for image stitching to form a panoramic image; Send the panoramic image to a preset target terminal for display; Among them, the roof camera is set with a variety of different position parameter arrays, corresponding to the roof environment monitoring in different application scenarios, and each position parameter is set with a corresponding panoramic stitching image overlap stitching feature point; when the panoramic image is stitched, the position parameter array of the current roof camera is obtained and the corresponding overlap stitching feature point is called according to the position parameter array to stitch the image to obtain the panoramic image; In the process of image stitching, after the adjacent graphics are registered, the pixel coordinates (xi, yi) of the average segmentation curve of the overlapping area are obtained according to the image overlap area, and the coordinates of several points are selected from the segmentation curve and stored as feature points for subsequent image stitching; Obtain the position parameters of the current roof camera, determine whether the position parameters of all roof cameras match one of the preset position parameter arrays, and if so, call the stitching feature points matched by the reference coordinates to stitch the images to form a panoramic image; The pixel coordinate calculation method of the average segmentation curve includes: Acquire multiple panoramic images, and in the multiple panoramic images, obtain the overlapped area of ​​adjacent images in each panoramic image, and calculate the mean curve Ai corresponding to the overlapped area of ​​adjacent images in each panoramic image; sort the mean curves corresponding to the overlapped area of ​​adjacent images in each panoramic image in the order of first the x direction and then the y direction from small to large to obtain the area surrounded by the mean curve combination; According to the given endpoints, the coordinates of the mean curve in the enclosed area are obtained as the feature points for stitching adjacent images in image stitching.

2. A monitoring method applied to a drone mother vehicle parking area as claimed in claim 1, characterized in that: The position parameter array includes the position parameters of each roof camera, and the position parameters are represented by [θij,hij], where θij represents the angle of the j-th roof camera corresponding to the i-th set of position parameter array relative to the reference direction, and hij represents the height of the j-th roof camera corresponding to the i-th set of position parameter array relative to the reference direction; The position parameters of the multiple roof cameras form a position parameter array.

3. A monitoring method applied to a drone mother vehicle parking area as claimed in claim 1, characterized in that: The roof camera is provided with a first vehicle-mounted Ethernet PHY interface and a second vehicle-mounted Ethernet PHY interface; a plurality of roof cameras are connected end to end in sequence through the first vehicle-mounted Ethernet PHY interface and the second vehicle-mounted Ethernet PHY interface; and the terminal roof camera is connected to the TSN gateway through the vehicle-mounted Ethernet bus.

4. A monitoring method applied to a drone mother vehicle parking area as claimed in claim 1, characterized in that: When the system is initialized, the panoramic service module broadcasts the panoramic image service to the vehicle network. After receiving the service, the node target in the network determines whether the service is needed. If the service is needed, it establishes a connection with the panoramic service module and subscribes to the panoramic service. The panoramic service module records the node targets subscribing to the panoramic service through the panoramic list, and initializes the image cache management area at the same time; The panoramic service module monitors the circular buffer space in the system, and when there is data stored in the circular buffer space, the data is sent to the node targets in the panoramic list respectively.

5. A monitoring method applied to a drone mother vehicle parking area as claimed in claim 4, characterized in that: The panoramic image is stored in a circular cache space opened by the system in a first-in-first-out order; the circular cache space is provided with a header file management area and a data area, the header file management area records the information of the data area, and the data area includes a plurality of data blocks for storing data.

6. A monitoring method applied to a drone mother vehicle parking area as claimed in claim 4, characterized in that: The panoramic service module also includes an image buffer management, which is used to synchronize the header file management area data and to record the target increment of the panoramic image transmission; Each time the panoramic service module obtains a block from the circular buffer space, a target increment is generated accordingly; Each time a data block is sent to a node target, the target increment corresponding to the data block is reduced by 1.

7. A monitoring method applied to a drone mother vehicle parking area as claimed in claim 4, characterized in that: When the node target that requires panoramic service is interrupted, an instruction that no panoramic image service is required is sent to the panoramic service module. After receiving the instruction, the panoramic service module deletes the corresponding target in the panoramic list.

Images