Split flying car docking control method and related devices

Abstract

The embodiment of the application provides a kind of split type flying car docking control method and related device, wherein the method comprises: in the case where land body is in docking position, first positioning image and second positioning image are acquired;Wherein, the first positioning image includes the first positioning mark configured in the land body, and the second positioning image includes the second positioning mark configured in the flying body;According to the first positioning image and the second positioning image, the pose deviation of the ring lock in the land body and the pull ring in the flying body is determined;In the case where the pose deviation does not satisfy set deviation condition, the pose of the land body is adjusted according to the pose deviation, and the step of acquiring first positioning image and second positioning image is returned to execute;In the case where the pose deviation satisfies the deviation condition, the ring lock is controlled to perform docking operation.

Description

Technical Field

[0001] This invention relates to the field of flying car technology, and more specifically, to a docking control method and related device for a split-type flying car. Background Technology

[0002] With the development of pure electric vertical take-off and landing vehicles, in addition to integrated flying cars, related technologies have also proposed the research direction of split flying cars, which divides the function of flying cars into land-based and flight-based parts, including various different configurations such as two-part and three-part.

[0003] Regardless of the configuration, the land-based component of a flying car typically refers to the land-based body, while the flying component typically refers to the flying body. The land-based and flying bodies can be combined and separated. When combined, the land-based body can charge the flying body and also transport it. When separated, both the land-based and flying bodies can operate independently.

[0004] The integration of land-based and air-based vehicles involves a docking process, which is crucial for the safe and reliable connection between the two. Therefore, achieving successful docking between land-based and air-based vehicles is a pressing technical challenge that needs to be addressed. Summary of the Invention

[0005] In view of this, in order to achieve accurate docking between land vehicles and flying vehicles, and thus improve the safety and reliability of the combination between land vehicles and flying vehicles, the purpose of this invention is to provide a docking control method and related device for split-type flying cars.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0007] A first aspect of the present invention provides a docking control method for a split-type flying car, the split-type flying car comprising a land-based body and a flying body, the method comprising:

[0008] When the land vehicle is in the docking position, a first positioning image and a second positioning image are acquired; wherein, the first positioning image includes a first positioning identifier disposed on the land vehicle, and the second positioning image includes a second positioning identifier disposed on the aircraft.

[0009] Based on the first positioning image and the second positioning image, determine the positional deviation of the ring lock in the land-based body and the pull ring in the flying body;

[0010] If the pose deviation does not meet the set deviation condition, the pose of the land vehicle is adjusted according to the pose deviation, and the process returns to the step of acquiring the first positioning image and the second positioning image.

[0011] If the pose deviation meets the deviation condition, control the ring lock to perform a docking operation.

[0012] In an optional implementation, the step of determining the pose deviation between the ring lock in the land vehicle and the pull ring in the air vehicle based on the first positioning image and the second positioning image includes:

[0013] Based on the first positioning image and the second positioning image, determine the image coordinates of the first positioning identifier and the image coordinates of the second positioning identifier, respectively;

[0014] Based on the image coordinates of the first positioning identifier, the set transformation matrix, and the physical positional relationship between the first positioning identifier and the set interlocking docking point in the land body, the first geographical coordinates of the interlocking docking point are determined.

[0015] Based on the image coordinates of the second positioning identifier, the transformation matrix, and the physical positional relationship between the second positioning identifier and the pull ring docking point set in the flight body, the second geographical coordinates of the pull ring docking point are determined;

[0016] Based on the first geographic coordinates and the second geographic coordinates, the pose deviation between the ring lock and the pull ring is determined; wherein, the pose deviation includes the X-axis distance deviation, Y-axis distance deviation, Z-axis distance deviation and angle deviation between the first geographic coordinates and the second geographic coordinates.

[0017] In an optional implementation, before acquiring the first positioning image and the second positioning image, the method further includes: when the land vehicle is at a laser ranging position, determining the docking position based on the laser ranging result and a third positioning image including the second positioning identifier; wherein the distance between the laser ranging position and the flying vehicle is greater than the distance between the docking position and the flying vehicle, and the laser ranging result is used to determine the distance between the land vehicle and the flying vehicle;

[0018] Control the land vehicle to travel to the docking position.

[0019] In an optional implementation, before determining the docking position, the method further includes:

[0020] When the land vehicle is in a visual switching position, the laser ranging position is determined based on a fourth positioning image including the second positioning identifier; wherein the distance between the visual switching position and the flying vehicle is greater than the distance between the laser ranging position and the flying vehicle;

[0021] Control the land vehicle to travel to the laser ranging position.

[0022] In an optional implementation, before determining the laser ranging position, the method further includes:

[0023] With the land vehicle in its initial position, the visual switching position is determined based on a fifth positioning image including the flying vehicle; wherein the distance between the initial position and the flying vehicle is greater than the distance between the visual switching position and the flying vehicle;

[0024] Control the land vehicle to travel to the vision switching position.

[0025] In an optional implementation, during the process of controlling the land vehicle to travel to the vision switching position, the method further includes:

[0026] Acquire a sixth positioning image, including the flying object, in real time or at regular intervals;

[0027] If the sixth positioning image includes the second positioning identifier, the visual switching position is updated according to the fifth positioning image;

[0028] In the step of controlling the land vehicle to travel to the visual switching position, the visual switching position is the updated visual switching position.

[0029] In an optional implementation, the step of determining the docking position based on the laser ranging result and the third positioning image including the second positioning identifier includes:

[0030] Determine the image coordinates of the second positioning identifier based on the third positioning image;

[0031] Based on the image coordinates of the second positioning identifier, the transformation matrix, and the physical positional relationship between the second positioning identifier and the pull ring docking point set in the flight body, the third geographical coordinates of the pull ring docking point are determined;

[0032] Based on the laser ranging results and the third geographic coordinates, determine the positioning coordinates of the pull ring docking point;

[0033] The docking position is obtained based on the positioning coordinates of the pull ring docking point and the set first distance threshold.

[0034] A second aspect of the present invention provides a docking control device for a split-type flying car, the split-type flying car comprising a land-based body and a flying body, the device comprising:

[0035] The image acquisition module is configured to acquire a first positioning image and a second positioning image when the land vehicle is in the docking position; wherein the first positioning image includes a first positioning identifier disposed on the land vehicle, and the second positioning image includes a second positioning identifier disposed on the flight vehicle;

[0036] The pose deviation determination module is configured to: determine the pose deviation of the ring lock in the land vehicle and the pull ring in the flying vehicle based on the first positioning image and the second positioning image;

[0037] The pose adjustment module is configured to: adjust the pose of the land vehicle according to the pose deviation when the pose deviation does not meet the set deviation condition, and return to the step of obtaining the first positioning image and the second positioning image.

[0038] The docking control module is configured to control the ring lock to perform a docking operation when the pose deviation meets the deviation condition.

[0039] A third aspect of the present invention provides a terrestrial vehicle, comprising:

[0040] The land vehicle body is equipped with a storage compartment, which contains a ring-locking mechanism and a first positioning mark;

[0041] An image acquisition module is installed on the vehicle body; and

[0042] The control module is used to realize the docking between the ring-locking mechanism and the flying body based on the image acquired by the image acquisition module, including the first positioning mark and the second positioning mark in the flying body, through the split-type flying car docking control method provided in any of the first aspects above.

[0043] In an optional implementation, the image acquisition module includes:

[0044] The surround-view camera unit includes multiple surround-view camera modules installed at different locations on the land vehicle body, used to acquire the fourth positioning image, the fifth positioning image, and the sixth positioning image; and

[0045] A positioning camera module is installed in the accommodating compartment and is used to acquire the first positioning image, the second positioning image, and the third positioning image.

[0046] In an optional embodiment, the accommodating compartment is further provided with a slide rail; the first positioning mark is disposed on the top surface of the slide rail; and the ring locking mechanism can slide along the slide rail.

[0047] A fourth aspect of the present invention provides a split-type flying car, including the land vehicle provided in any of the third aspects above.

[0048] A fifth aspect of the present invention provides an electronic device including a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor executing the machine-executable instructions to implement the split-type flying car docking control method provided in any of the first aspects above.

[0049] A sixth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the split-type flying car docking control method provided in any of the first aspects above.

[0050] The split-type flying car docking control method and related apparatus provided in this invention improve the accuracy of the pose deviation by determining the positional deviation of the ring lock in the land vehicle and the pull ring in the flying vehicle based on a first positioning image including a first positioning marker and a second positioning image including a second positioning marker when the land vehicle is in the docking position. This is because the pose deviation is based on the positions of the first and second positioning markers, and since the image features of the positioning markers are clearly distinguishable from other objects in the environment, the positioning markers can be quickly and accurately identified from the positioning images, thus obtaining the accurate position of the positioning markers. Furthermore, since the positional relationship between the first positioning marker and the ring lock, as well as the positional relationship between the second positioning marker and the pull ring, can be accurately measured in advance, and this positional relationship generally does not change due to changes in the surrounding environment, after obtaining the positions of the first and second positioning markers, the accurate positions of the ring lock and the pull ring can be obtained respectively based on their corresponding positional relationships, thus obtaining the accurate pose deviation. Next, if the attitude deviation does not meet the set deviation conditions, the attitude of the land vehicle is adjusted until the attitude deviation meets the set deviation conditions. Only then is the ring lock controlled to perform the docking operation. In this way, the accurate attitude deviation enables the land vehicle and the flying vehicle to dock accurately, improving the docking success rate and reliability, and thus improving the safety and reliability of subsequent docking between the land vehicle and the flying vehicle.

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This diagram illustrates a structural block diagram of an electronic device provided by an embodiment of the present invention.

[0054] Figure 2 This diagram illustrates the location of a first positioning identifier provided in an embodiment of the present invention.

[0055] Figure 3 This diagram shows a partial perspective top view of a land-based structure provided in an embodiment of the present invention.

[0056] Figure 4 A flowchart of a docking control method for a split-type flying car provided by an embodiment of the present invention is shown;

[0057] Figure 5 This diagram illustrates a top-view view of a land vehicle in a docking position, according to an embodiment of the present invention.

[0058] Figure 6 The diagram shows a functional block diagram of a split-type flying car docking control device provided in an embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0060] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0061] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0062] To address the technical problem in related technologies where accurate docking of land-based and air-based vehicles is impossible, thus affecting the safety and reliability of subsequent docking, this invention provides a docking control method for a split-type flying car. By determining the pose deviations of the ring lock in the land-based vehicle and the pull ring in the air-based vehicle based on a first positioning image (including a first positioning marker) and a second positioning image (including a second positioning marker) when the land-based vehicle is in the docking position, the pose deviation accuracy can be improved. This is because the pose deviation is based on the positions of the first and second positioning markers, and since the image features of the positioning markers are clearly distinguishable from other objects in the environment, the positioning markers can be quickly and accurately identified from the positioning images, thus obtaining the accurate position of the positioning markers. Furthermore, since the positional relationships between the first positioning marker and the ring lock, and between the second positioning marker and the pull ring, can be accurately measured in advance, and these relationships generally do not change due to changes in the surrounding environment, after obtaining the positions of the first and second positioning markers, the accurate positions of the ring lock and the pull ring can be obtained based on their respective positional relationships, thus obtaining the accurate pose deviation. Next, if the attitude deviation does not meet the set deviation conditions, the attitude of the land vehicle is adjusted until the attitude deviation meets the set deviation conditions. Only then is the ring lock controlled to perform the docking operation. In this way, the accurate attitude deviation enables the land vehicle and the flying vehicle to dock accurately, improving the docking success rate and reliability, and thus improving the safety and reliability of subsequent docking between the land vehicle and the flying vehicle.

[0063] The split-type flying car docking control method provided by this invention can be applied to electronic devices. Please refer to [reference needed]. Figure 1 This is a structural block diagram of an electronic device. The electronic device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, processor 120, and communication module 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0064] The memory is used to store programs or data. The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0065] The processor is used to read / write data or programs stored in memory and to perform the corresponding functions.

[0066] The communication module is used to establish communication connections between electronic devices and other communication terminals via a network, and to send and receive data via the network.

[0067] It should be understood that, Figure 1 The structure shown is only a schematic diagram of an electronic device; the electronic device may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0068] In some embodiments, the electronic device can be configured in the land body of the split-type flying car, for example, it can serve as the control module of the land body to respond to the docking request sent by the user terminal and realize the accurate docking of the land body and the flying body by executing the split-type flying car docking control method provided in the embodiments of the present invention.

[0069] As described above, the land-based component of a split-type flying car may include:

[0070] The land vehicle body is equipped with a storage compartment, which contains a ring-locking mechanism and a first positioning mark;

[0071] An image acquisition module is installed on the vehicle body; and

[0072] The control module is used to realize the docking between the ring-locking mechanism and the flying body based on the image acquired by the image acquisition module, including the first positioning mark and the second positioning mark in the flying body, through the split-type flying car docking control method provided in the embodiments of the present invention.

[0073] For details on how the ring-lock mechanism is implemented, please refer to relevant technologies.

[0074] In some embodiments, to facilitate accurate calibration of the physical positional relationship between the ring lock docking point and the first positioning mark in the ring lock mechanism, please refer to... Figure 2 , Figure 2 This is a schematic diagram of the position of a first positioning mark provided in an embodiment of the present invention. A slide rail 12 may also be provided in the accommodating compartment; the first positioning mark A may be disposed on the top surface of the slide rail 12, and the ring locking mechanism 11 may slide along the slide rail 12. Although Figure 2 The first positioning identifier A shown is a positioning code, but in other embodiments, the first positioning identifier A may not be limited to this. For example, it may be other specific patterns or specific shapes of markers, as long as they can be distinguished from other objects in the environment.

[0075] Similarly, the second positioning identifier in the aircraft can be a positioning code, or it can be a marker with a specific pattern or shape.

[0076] In some embodiments, during the docking process between the land vehicle and the air vehicle, in order to facilitate the land vehicle to accurately collect the second positioning mark in the air vehicle, the second positioning mark in the air vehicle can be set on the side of the air vehicle where the pull ring is provided.

[0077] In addition to the second positioning mark on the aforementioned side of the aircraft, other positioning marks can also be set on the sides of the aircraft in other directions. This allows the land vehicle to accurately locate the aircraft by using the positioning marks on the sides of the aircraft in other directions before it has reached the aircraft's pull ring, which is more conducive to the accurate execution of subsequent docking operations.

[0078] In some embodiments, to obtain the laser ranging results described below and to simplify the calculation process for determining the distance between land-based and air-based objects based on the laser ranging results, please continue to refer to... Figure 2 A laser rangefinder 13 can be installed at the ring-lock mechanism 11.

[0079] In some embodiments, to enable the land-based vehicle to understand the surrounding obstacles during its movement toward the flying vehicle, thereby ensuring safe obstacle avoidance and ensuring safety during docking, the configuration of the image acquisition module in the land-based vehicle can be optimized. See [link to relevant documentation]. Figure 3 , Figure 3 This is a partial perspective view of the top structure of a land vehicle provided in an embodiment of the present invention. The above-mentioned image acquisition module may include a surround-view camera unit and a positioning camera module 16.

[0080] The surround-view camera unit includes multiple surround-view camera modules installed at different locations on the land vehicle body, used to acquire the fourth, fifth and sixth positioning images described below.

[0081] In some examples, the multiple surround-view camera modules may include: a front parking camera module 141 mounted on the front of the land vehicle body, a right parking camera module 142 mounted on the right side rearview mirror of the land vehicle body, a left parking camera module 143 mounted on the left side rearview mirror of the land vehicle body, and a rear parking camera 144 mounted on the door of the land vehicle body's storage compartment.

[0082] The positioning camera module 16 is installed in the accommodating compartment and is used to acquire the first positioning image, the second positioning image and the third positioning image described below.

[0083] In some embodiments, to reduce the cost of configuring multiple surround-view camera modules while ensuring that the land vehicle can acquire higher-precision images at appropriate times (as described below when the land vehicle is in a vision switching position) to achieve more accurate positioning and docking, please refer to [further details]. Figure 3 The image acquisition module may further include a rear-view camera module 15, the image resolution and measurement accuracy of which are higher than those of the plurality of surround-view camera modules.

[0084] The following combination Figure 4 The docking control method for a split-type flying car provided in the embodiments of the present invention will be described. Figure 4 This is a flowchart of a docking control method for a split-type flying car provided by an embodiment of the present invention. The split-type flying car docking control method includes:

[0085] In step S100, when the land vehicle is in the docking position, a first positioning image and a second positioning image are acquired; wherein, the first positioning image includes a first positioning identifier configured on the land vehicle, and the second positioning image includes a second positioning identifier configured on the flight vehicle;

[0086] In step S200, the pose deviations of the ring lock in the land vehicle and the pull ring in the flying vehicle are determined based on the first positioning image and the second positioning image;

[0087] In step S300, if the pose deviation does not meet the set deviation condition, the pose of the land vehicle is adjusted according to the pose deviation, and the process returns to the step of acquiring the first positioning image and the second positioning image.

[0088] In step S400, if the pose deviation meets the deviation condition, the ring lock is controlled to perform a docking operation.

[0089] The following describes the working principle of steps S100 to S400, using a land-based vehicle as the executing entity of the split-type flying car docking control method provided in this embodiment of the invention:

[0090] When the land vehicle receives a docking request from the user terminal, the driver can control the land vehicle to travel to the docking position with the aircraft, or the land vehicle can be automatically controlled to travel to the docking position. The docking position can be a location at a first distance threshold directly in front of the side where the pull ring is located in the aircraft. Figure 5 As shown, Figure 5 This is a top-view schematic diagram of a land vehicle in the docking position according to an embodiment of the present invention. Figure 5 In the diagram, L1 represents the first distance threshold, 10 represents a land-based vehicle, and 20 represents a flying vehicle.

[0091] In the above, the first distance threshold can be obtained based on experience or experimentation, for example, it can be taken in the range of [70mm, 80mm].

[0092] Before the land-based vehicle automatically travels to the docking position, its geographical coordinates can be obtained from images captured by multiple surround-view camera modules and the transformation matrix between image coordinates and the geographic coordinate system. Simultaneously, its geographical coordinates can be obtained from the GPS or RTK positioning modules within the land-based vehicle. Next, the docking position can be calculated based on the land-based vehicle's geographical coordinates and the aforementioned first distance threshold. Then, a route planning algorithm from relevant technologies can be used to generate a path for the land-based vehicle to reach the docking position, based on the docking position and the land-based vehicle's geographical coordinates. The land-based vehicle can then automatically travel along this path to the docking position.

[0093] Furthermore, to ensure that the cargo compartment door has sufficient space to open safely, the cargo compartment door can be opened during the process of the land vehicle traveling from its initial position upon receiving the docking request to the docking position. Based on this, when the land vehicle reaches the docking position, the cargo compartment is in an open state. Since the rear parking camera 144 is installed at the cargo compartment door, its field of view changes after the door is opened, and it may be unable to capture images of the side where the pull ring is located in the flying vehicle. Therefore, in order to obtain the first and second positioning images, the positioning camera module 16 can be controlled to perform image acquisition in this situation.

[0094] Based on this, when the land vehicle is in the docking position, step S100 can be executed to obtain the first positioning image and the second positioning image captured by the positioning camera module 16.

[0095] Next, step S200 can be executed to determine the pose deviation of the ring lock in the land-based vehicle and the pull ring in the air-based vehicle based on the first and second positioning images. Since the transformation matrix between the image coordinate system and the geographic coordinate system of the positioning camera module can be obtained through pre-calibration, the positional relationship between the first positioning marker and the ring lock can also be pre-calibrated, as can the positional relationship between the second positioning marker and the pull ring. Therefore, the geographic coordinates of the ring lock can be calculated using the first positioning image containing the first positioning marker, the transformation matrix, and the positional relationship between the first positioning marker and the ring lock. Based on the same principle, the geographic coordinates of the pull ring can also be calculated. Subsequently, the pose deviation between the two can be calculated based on the geographic coordinates of the ring lock and the pull ring.

[0096] In some embodiments, to improve the calculation accuracy of pose deviation, this embodiment of the invention provides a scheme for calculating pose deviation. Specifically, step S200 above, which involves determining the pose deviation of the ring lock in the land vehicle and the pull ring in the air vehicle based on the first positioning image and the second positioning image, may include:

[0097] In step S210, the image coordinates of the first positioning identifier and the image coordinates of the second positioning identifier are determined based on the first positioning image and the second positioning image, respectively.

[0098] In step S220, the first geographic coordinates of the interlocking docking point are determined based on the image coordinates of the first positioning identifier, the set transformation matrix, and the physical positional relationship between the first positioning identifier and the set interlocking docking point in the land body.

[0099] In step S230, the second geographic coordinates of the pull ring docking point are determined based on the image coordinates of the second positioning identifier, the transformation matrix, and the physical positional relationship between the second positioning identifier and the pull ring docking point set in the flying body.

[0100] In step S240, the pose deviation between the ring lock and the pull ring is determined based on the first geographic coordinates and the second geographic coordinates; wherein, the pose deviation includes the X-axis distance deviation, Y-axis distance deviation, Z-axis distance deviation and angle deviation between the first geographic coordinates and the second geographic coordinates.

[0101] Understandably, during the execution of step S200, step S210 can be executed first to obtain the first positioning identifier and its image coordinates based on the first positioning image using image processing methods in related technologies, and to obtain the second positioning identifier and its image coordinates based on the second positioning image.

[0102] After obtaining the image coordinates of the first positioning marker, step S220 is executed. Based on the image coordinates of the first positioning marker, the transformation matrix, and the physical positional relationship between the first positioning marker and the ring-lock docking point set in the land vehicle, the first geographic coordinates of the ring-lock docking point are obtained. The transformation matrix can be found in the relevant description above. The ring-lock docking point is the midpoint of the end of the ring lock in the land vehicle that connects to the pull ring of the flying vehicle when the ring-lock mechanism reaches its maximum extension stroke. Figure 2 As shown, the extension direction of the ring lock mechanism is as follows: Figure 2 The direction indicated by Y in the middle, the ring-lock docking point is Figure 2 Point O is shown in the diagram.

[0103] After obtaining the image coordinates of the second positioning identifier, step S230 is executed. Based on the image coordinates of the second positioning identifier, the transformation matrix, and the physical positional relationship between the second positioning identifier and the ring docking point set in the flight body, the second geographical coordinates of the ring docking point are obtained. The ring docking point is the midpoint of the end of the ring in the flight body used for connection with the ring lock.

[0104] In the above, the execution order of steps S220 and S230 is not limited.

[0105] After obtaining the first geographic coordinates of the ring lock docking point and the second geographic coordinates of the pull ring docking point, step S240 is executed to calculate the X-axis distance deviation, Y-axis distance deviation, Z-axis distance deviation and angle deviation between the first geographic coordinates and the second geographic coordinates.

[0106] To better understand the various distance deviations mentioned above, the following example is provided:

[0107] Assuming the X-axis is parallel to the width direction of the landlocked structure, the Y-axis is parallel to the length direction, and the Z-axis is parallel to the height direction, the first geographic coordinates of the ring-lock docking point are (x1, y1, z1), and the second geographic coordinates of the pull ring docking point are (x2, y2, z2). Based on this, the X-axis distance deviation = x1 - x2, the Y-axis distance deviation = y1 - y2, and the Z-axis distance deviation = z1 - z2. The angular deviation can be obtained by calculating the angle between the line connecting points (x1, y1, z1) and (x2, y2, z2) and the Y-axis direction using the angle calculation formula in related technologies. It is evident that the above deviations may have positive or negative values.

[0108] Based on the above example, the X-axis distance deviation can be understood as the left and right deviation of the ring lock docking point relative to the pull ring docking point. The left and right deviations can be distinguished by positive and negative values. If the direction of the X-axis is from left to right, then if the calculated X-axis distance deviation is -0.1m, it means that the ring lock docking point is 0.1m to the left relative to the pull ring docking point. Conversely, if the X-axis distance deviation is +0.1m, it means that the ring lock docking point is 0.1m to the right relative to the pull ring docking point.

[0109] Correspondingly, the Y-axis distance deviation can be understood as the forward / backward deviation of the ring lock docking point relative to the pull ring docking point. The Z-axis distance deviation can be understood as the height deviation of the ring lock docking point relative to the pull ring docking point. The angle deviation can be understood as the rotational angle deviation of the ring lock docking point relative to the pull ring docking point.

[0110] Therefore, by calculating the pose deviation using the ring lock docking point and the pull ring docking point, the deviation at the mating point between the ring lock and the pull ring can be accurately obtained, which helps to improve the accuracy of subsequent docking between the ring lock and the pull ring.

[0111] After obtaining the pose deviation between the ring lock and the pull ring, the pose deviation is compared with the set deviation conditions to determine whether the pose deviation meets the deviation conditions. For example, the set deviation conditions may include X-axis deviation threshold range, Y-axis deviation threshold range, Z-axis deviation threshold range, and angle deviation threshold range. Each threshold range can be determined based on experience or experiments. For example, the X-axis deviation threshold range can be [-75mm, +75mm], the Y-axis deviation threshold range can be [-57.5mm, +57.5mm], the Z-axis deviation threshold range can be [-50mm, -5mm], and the angle deviation threshold range can be [-1°, +1°], but is not limited to these.

[0112] Based on this, if one of the pose deviations does not fall within the corresponding threshold range, it is determined that the pose deviation does not meet the set deviation condition, and step S300 is executed to adjust the pose of the land vehicle according to the pose deviation. For example, assuming that the X-axis distance deviation does not fall within the X-axis deviation threshold range, the land vehicle is controlled to move in the direction of reducing the X-axis distance deviation; the adjustment in other directions is the same, and will not be elaborated here.

[0113] After adjusting the pose of the land vehicle based on the current pose deviation, in order to determine whether the adjusted pose deviation meets the deviation conditions, the process returns to steps S100 to S200, and after step S200 is completed, it is determined again whether the current pose deviation meets the deviation conditions.

[0114] Therefore, if the pose deviation does not meet the set deviation conditions, the docking operation is not performed. Instead, the pose of the land vehicle is adjusted so that the adjusted pose deviation meets the set deviation conditions before the docking operation is performed.

[0115] If all deviations in the pose deviation are within their respective threshold ranges, the pose deviation is determined to meet the set deviation conditions, and step S400 is executed to control the ring lock to perform the docking operation. For example, the ring lock mechanism can be controlled to extend along the slide rail so that the ring lock of the ring lock mechanism reaches the destination. Figure 2 Align the position indicated by the dashed line with the pull ring of the flying vehicle, and then control the ring lock to tighten the pull ring, thus locking the ring lock and the pull ring together. Subsequently, control the ring lock mechanism to retract along the slide rail, which can pull the flying vehicle gradually into the accommodating compartment, thereby enabling the flying vehicle to be integrated into the land vehicle.

[0116] For ease of description, the control links corresponding to steps S100 to S400 above will be referred to as docking control links in the following text.

[0117] It is evident that the aforementioned docking control mechanism can ensure accurate docking of the land-based vehicle and the flying vehicle, thereby improving the safety and reliability of their combination. Furthermore, to ensure that the interlocking ring and pull ring have smaller pose deviations when the land-based vehicle arrives at the docking position, thus reducing the magnitude of pose deviation adjustment and improving docking efficiency, in some embodiments, before acquiring the first and second positioning images, the split-type flying car docking control method provided by this invention may further include:

[0118] In step S050, when the land vehicle is at the laser ranging position, the docking position is determined based on the laser ranging result and the third positioning image including the second positioning mark; wherein, the distance between the laser ranging position and the flying vehicle is greater than the distance between the docking position and the flying vehicle, and the laser ranging result is used to determine the distance between the land vehicle and the flying vehicle;

[0119] In step S060, the land vehicle is controlled to travel to the docking position.

[0120] Understandably, before executing the docking control step, the docking position determination step shown in steps S050 to S060 is executed first. The docking position determination step can obtain a more accurate docking position, thereby reducing the adjustment range or frequency of position deviation and improving docking efficiency.

[0121] As the land vehicle moves from its initial position to the laser ranging position, step S050 is executed, fusing the laser ranging result with a third positioning image including the second positioning marker to calculate the docking position. Since step S050 requires the use of a positioning camera module and a laser ranging sensor to achieve precise positioning of the docking position, the compartment door can be opened even when the land vehicle is at the laser ranging position. Alternatively, the compartment door can be opened before the land vehicle reaches the laser ranging position.

[0122] The laser ranging position is located at the second distance threshold directly in front of the side where the pull ring is located on the flying object. The second distance threshold can be obtained based on experience or experiments, for example, it can be taken in the range of [0.98m, 1.02m], but is not limited to this.

[0123] In some embodiments, the present invention provides a scheme for determining the docking position, that is, the step S050 above, which determines the docking position based on the laser ranging result and the third positioning image including the second positioning identifier, may include:

[0124] In step S051, the image coordinates of the second positioning identifier are determined based on the third positioning image;

[0125] In step S052, the third geographic coordinates of the pull ring docking point are determined based on the image coordinates of the second positioning identifier, the transformation matrix, and the physical positional relationship between the second positioning identifier and the pull ring docking point set in the flight body.

[0126] In step S053, the positioning coordinates of the pull ring docking point are determined based on the laser ranging result and the third geographic coordinates;

[0127] In step S054, the docking position is obtained based on the positioning coordinates of the pull ring docking point and the first distance threshold.

[0128] The technical principle of determining the docking position by executing steps S051 to S054 can be found in the relevant descriptions above, and will not be repeated here. The meaning of the first distance threshold recorded in step S054 can also be found in the relevant descriptions of the first distance threshold above, and will not be repeated here.

[0129] This allows for a more accurate docking position, which in turn reduces the positional deviation between the ring lock docking point and the pull ring docking point when the land vehicle arrives at the docking position.

[0130] After obtaining a more accurate docking position, step S060 is executed to control the land vehicle to travel to the docking position.

[0131] Similarly, to enable the land vehicle to reach a more accurate laser ranging position, thereby improving the accuracy of docking position determination and further reducing pose deviation, in some embodiments, before determining the docking position, the split-type flying car docking control method provided by the present invention may further include:

[0132] In step S030, when the land vehicle is in a visual switching position, the laser ranging position is determined based on a fourth positioning image including the second positioning identifier; wherein the distance between the visual switching position and the flying vehicle is greater than the distance between the laser ranging position and the flying vehicle;

[0133] In step S040, the land vehicle is controlled to travel to the laser ranging position.

[0134] Understandably, before performing the docking position determination step, the laser ranging position determination step shown in steps S030 to S040 is performed first to obtain a more accurate laser ranging position, further improving the accuracy of the docking position, thereby better reducing the adjustment range or frequency of the pose deviation and improving the docking efficiency.

[0135] When the land vehicle moves from its initial position to the visual switching position, step S030 is executed, and the laser ranging position is calculated based on the fourth positioning image including the second positioning identifier. The calculation principle can be found in the relevant description above, and will not be repeated here.

[0136] The visual switching position is the position at the third distance threshold directly in front of the side where the pull ring is located in the flying object; the third distance threshold can be obtained based on experience or experimentation, for example, it can be taken in the range of [4.5m, 5.5m].

[0137] After obtaining a more accurate laser ranging position, step S040 is executed to control the land vehicle to travel to the laser ranging position.

[0138] Similarly, to enable the land vehicle to reach a more accurate visual switching position, thereby improving the accuracy of docking position determination and further reducing pose deviation, in some embodiments, before determining the laser ranging position, the split-type flying car docking control method provided by the present invention may further include:

[0139] In step S010, when the land vehicle is in its initial position, the visual switching position is determined based on a fifth positioning image including the flying vehicle; wherein the distance between the initial position and the flying vehicle is greater than the distance between the visual switching position and the flying vehicle;

[0140] In step S020, the land vehicle is controlled to travel to the vision switching position.

[0141] Understandably, before performing the laser ranging position determination step, the visual switching position determination step shown in steps S010 to S020 is performed first. This is to obtain a more accurate visual switching position through the visual switching position determination step, further improve the accuracy of the docking position, and thus better reduce the adjustment range or frequency of the pose deviation, thereby improving the docking efficiency.

[0142] When the land vehicle is in its initial position, the surrounding cameras typically cannot capture images of the second positioning marker on the flight vehicle due to the distance between the initial position and the flight vehicle. To ensure the land vehicle can smoothly move towards the flight vehicle, step S010 is executed first. Coarse positioning of the flight vehicle is performed based on the fifth positioning image, and a visual switching position is obtained by combining this with the aforementioned third distance threshold. This ensures that the land vehicle can acquire images of the second positioning marker after reaching this visual switching position, guaranteeing that other stages can also acquire images of the second positioning marker, thus enabling fine positioning in subsequent stages.

[0143] After obtaining the visual switching position, execute step S020 to control the land vehicle to travel to the visual switching position.

[0144] Furthermore, since the land vehicle may capture images of the second positioning marker during its journey from the initial position to the visual switching position, to improve the accuracy of the visual switching position, the visual switching position can be further updated using the second positioning marker. Based on this, in some embodiments, during the execution of step S020 above, the split-type flying car docking control method provided by this invention may further include:

[0145] In step S021, a sixth positioning image including the flying object is acquired in real time or at regular intervals;

[0146] In step S022, if the sixth positioning image includes the second positioning identifier, the visual switching position is updated according to the fifth positioning image.

[0147] Accordingly, in step S020, the visual switching position in the step of controlling the land vehicle to travel to the visual switching position is the updated visual switching position.

[0148] Therefore, through steps S021 and S022, the distance deviation between the visual switching position and the flying object can be reduced by 0.3m. For example, assuming that the distance between the visual switching position and the flying object before the update is in the range of [4.5m, 5.5m], the distance between the visual switching position and the flying object after the update can be reduced to the range of [4.8m, 5.2m].

[0149] It is worth noting that the technical features or technical solutions in any of the above embodiments of the present invention can be combined with each other, as long as there is no contradiction in the combination.

[0150] As a combined example, the split-type flying car docking control method provided in this embodiment of the invention may further include a visual switching position determination stage, a laser ranging position determination stage, a docking position determination stage, and a docking control stage. The execution order of these four stages is as follows: visual switching position determination stage, laser ranging position determination stage, docking position determination stage, and docking control stage. By combining these four stages, more precise docking control can be achieved, thereby improving the safety and reliability of subsequent docking.

[0151] To perform the corresponding steps in the above embodiments and various possible methods, an implementation method of a split-type flying car docking control device is given below. Optionally, the split-type flying car docking control device can adopt the above-described... Figure 1 The device structure of the electronic device is shown. Further, please refer to... Figure 6 , Figure 6 This is a functional block diagram of a split-type flying car docking control device provided in an embodiment of the present invention. It should be noted that the basic principle and technical effects of the split-type flying car docking control device provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The split-type flying car docking control device 600 includes:

[0152] The image acquisition module 610 is configured to acquire a first positioning image and a second positioning image when the land vehicle is in the docking position; wherein the first positioning image includes a first positioning identifier disposed on the land vehicle, and the second positioning image includes a second positioning identifier disposed on the flight vehicle.

[0153] The pose deviation determination module 620 is configured to: determine the pose deviation of the ring lock in the land vehicle and the pull ring in the flying vehicle based on the first positioning image and the second positioning image;

[0154] The pose adjustment module 630 is configured to: adjust the pose of the land vehicle according to the pose deviation when the pose deviation does not meet the set deviation condition, and return to the step of acquiring the first positioning image and the second positioning image.

[0155] The docking control module 640 is configured to control the ring lock to perform a docking operation when the pose deviation meets the deviation condition.

[0156] In some embodiments, the process by which the pose deviation determination module 620 determines the pose deviation of the ring lock in the land vehicle and the pull ring in the air vehicle based on the first positioning image and the second positioning image is configured as follows:

[0157] Based on the first positioning image and the second positioning image, determine the image coordinates of the first positioning identifier and the image coordinates of the second positioning identifier, respectively;

[0158] Based on the image coordinates of the first positioning identifier, the set transformation matrix, and the physical positional relationship between the first positioning identifier and the set interlocking docking point in the land body, the first geographical coordinates of the interlocking docking point are determined.

[0159] Based on the image coordinates of the second positioning identifier, the transformation matrix, and the physical positional relationship between the second positioning identifier and the pull ring docking point set in the flight body, the second geographical coordinates of the pull ring docking point are determined;

[0160] Based on the first geographic coordinates and the second geographic coordinates, the pose deviation between the ring lock and the pull ring is determined; wherein, the pose deviation includes the X-axis distance deviation, Y-axis distance deviation, Z-axis distance deviation and angle deviation between the first geographic coordinates and the second geographic coordinates.

[0161] In some embodiments, the split-type flying car docking control device 600 provided in this invention may further include:

[0162] The location determination module is configured to: before acquiring the first and second positioning images, when the land vehicle is in a laser ranging position, determine the docking position based on the laser ranging result and a third positioning image including the second positioning identifier; wherein the distance between the laser ranging position and the flying vehicle is greater than the distance between the docking position and the flying vehicle, and the laser ranging result is used to determine the distance between the land vehicle and the flying vehicle;

[0163] The driving control module is configured to control the land vehicle to drive to the docking position.

[0164] In some embodiments, the position determination module is further configured to: before determining the docking position, when the land vehicle is in a visual switching position, determine the laser ranging position based on a fourth positioning image including the second positioning identifier; wherein the distance between the visual switching position and the flight vehicle is greater than the distance between the laser ranging position and the flight vehicle;

[0165] Accordingly, the driving control module is configured to control the land vehicle to drive to the laser ranging position.

[0166] In some embodiments, the position determination module is further configured to: before determining the laser ranging position, with the land vehicle in its initial position, determine the visual switching position based on a fifth positioning image including the flying vehicle; wherein the distance between the initial position and the flying vehicle is greater than the distance between the visual switching position and the flying vehicle;

[0167] Accordingly, the driving control module is configured to control the land vehicle to drive to the vision switching position.

[0168] In some embodiments, during the process of the driving control module controlling the land vehicle to drive to the vision switching position, it is further configured to:

[0169] Acquire a sixth positioning image, including the flying object, in real time or at regular intervals;

[0170] If the sixth positioning image includes the second positioning identifier, the visual switching position is updated according to the fifth positioning image;

[0171] In the step of controlling the land vehicle to travel to the visual switching position, the visual switching position is the updated visual switching position.

[0172] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory shown is either stored in or embedded in the operating system (OS) of the electronic device, and can be... Figure 1 The processor executes the commands. Meanwhile, the data and program code required to execute these modules can be stored in memory.

[0173] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0174] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0175] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0176] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A docking control method for a split-type flying car, characterized in that, The split-type flying car includes a land-based body and a flying body, and the method includes: When the land vehicle is in the docking position, a first positioning image and a second positioning image are acquired; wherein, the first positioning image includes a first positioning identifier disposed on the land vehicle, and the second positioning image includes a second positioning identifier disposed on the aircraft. Based on the first positioning image and the second positioning image, determine the positional deviation of the ring lock in the land-based body and the pull ring in the flying body; If the pose deviation does not meet the set deviation condition, the pose of the land vehicle is adjusted according to the pose deviation, and the process returns to the step of acquiring the first positioning image and the second positioning image. If the pose deviation meets the deviation condition, control the ring lock to perform a docking operation.

2. The method according to claim 1, characterized in that, The step of determining the pose deviation of the ring lock in the land-based body and the pull ring in the air-based body based on the first positioning image and the second positioning image includes: Based on the first positioning image and the second positioning image, determine the image coordinates of the first positioning identifier and the image coordinates of the second positioning identifier, respectively; Based on the image coordinates of the first positioning identifier, the set transformation matrix, and the physical positional relationship between the first positioning identifier and the set interlocking docking point in the land body, the first geographical coordinates of the interlocking docking point are determined. Based on the image coordinates of the second positioning identifier, the transformation matrix, and the physical positional relationship between the second positioning identifier and the pull ring docking point set in the flight body, the second geographical coordinates of the pull ring docking point are determined; Based on the first geographic coordinates and the second geographic coordinates, the pose deviation between the ring lock and the pull ring is determined; wherein, the pose deviation includes the X-axis distance deviation, Y-axis distance deviation, Z-axis distance deviation and angle deviation between the first geographic coordinates and the second geographic coordinates.

3. The method according to claim 1 or 2, characterized in that, Before acquiring the first and second positioning images, the method further includes: When the land vehicle is at the laser ranging position, the docking position is determined based on the laser ranging result and a third positioning image including the second positioning mark; wherein, the distance between the laser ranging position and the flying vehicle is greater than the distance between the docking position and the flying vehicle, and the laser ranging result is used to determine the distance between the land vehicle and the flying vehicle; Control the land vehicle to travel to the docking position.

4. The method according to claim 3, characterized in that, Before determining the docking position, the method further includes: When the land vehicle is in a visual switching position, the laser ranging position is determined based on a fourth positioning image including the second positioning identifier; wherein the distance between the visual switching position and the flying vehicle is greater than the distance between the laser ranging position and the flying vehicle; Control the land vehicle to travel to the laser ranging position.

5. The method according to claim 4, characterized in that, Before determining the laser ranging position, the method further includes: With the land vehicle in its initial position, the visual switching position is determined based on a fifth positioning image including the flying vehicle; wherein the distance between the initial position and the flying vehicle is greater than the distance between the visual switching position and the flying vehicle; Control the land vehicle to travel to the vision switching position.

6. The method according to claim 5, characterized in that, During the process of controlling the land vehicle to travel to the vision switching position, the method further includes: Acquire a sixth positioning image, including the flying object, in real time or at regular intervals; If the sixth positioning image includes the second positioning identifier, the visual switching position is updated according to the fifth positioning image; In the step of controlling the land vehicle to travel to the visual switching position, the visual switching position is the updated visual switching position.

7. The method according to claim 3, characterized in that, The step of determining the docking position based on the laser ranging result and the third positioning image including the second positioning identifier includes: Determine the image coordinates of the second positioning identifier based on the third positioning image; Based on the image coordinates of the second positioning identifier, the transformation matrix, and the physical positional relationship between the second positioning identifier and the pull ring docking point set in the flight body, the third geographical coordinates of the pull ring docking point are determined; Based on the laser ranging results and the third geographic coordinates, determine the positioning coordinates of the pull ring docking point; The docking position is obtained based on the positioning coordinates of the pull ring docking point and the set first distance threshold.

8. A split-type flying car docking control device, characterized in that, The split-type flying car includes a land-based body and a flying body, and the device includes: The image acquisition module is configured to acquire a first positioning image and a second positioning image when the land vehicle is in the docking position; wherein the first positioning image includes a first positioning identifier disposed on the land vehicle, and the second positioning image includes a second positioning identifier disposed on the flight vehicle; The pose deviation determination module is configured to: determine the pose deviation of the ring lock in the land vehicle and the pull ring in the flying vehicle based on the first positioning image and the second positioning image; The pose adjustment module is configured to: adjust the pose of the land vehicle according to the pose deviation when the pose deviation does not meet the set deviation condition, and return to the step of obtaining the first positioning image and the second positioning image. The docking control module is configured to control the ring lock to perform a docking operation when the pose deviation meets the deviation condition.

9. A terrestrial vehicle, characterized in that, include: The land vehicle body is equipped with a storage compartment, which contains a ring-locking mechanism and a first positioning mark; The image acquisition module is installed on the vehicle body; as well as The control module is used to realize the docking between the ring-locking mechanism and the flight body based on the image acquired by the image acquisition module, including the first positioning mark and the second positioning mark in the flight body, using the method of any one of claims 1 to 7.

10. The terrestrial vehicle according to claim 9, characterized in that, The image acquisition module includes: The surround-view camera unit includes multiple surround-view camera modules installed at different locations on the land vehicle body, used to acquire the fourth positioning image, the fifth positioning image, and the sixth positioning image when the control module performs the method described in claim 6; and A positioning camera module is installed in the accommodating compartment and is used to acquire the first positioning image and the second positioning image, and is also used to acquire the third positioning image when the control module performs the method of any one of claims 3 to 7.

11. The terrestrial vehicle according to claim 9 or 10, characterized in that, The accommodating compartment is also equipped with a slide rail; the first positioning mark is set on the top surface of the slide rail; the ring locking mechanism can slide along the slide rail.

12. A split-type flying car, characterized in that, The land vehicle includes any one of claims 9 to 11.

13. An electronic device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor to implement the method of any one of claims 1 to 7.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images