Automatic docking positioning method and device of hovercar and related equipment

Abstract

The invention discloses an automatic docking and positioning method and device of a hovercar and related equipment, the hovercar comprises a road vehicle and an aircraft, the method is applied to the road vehicle, the method comprises the steps that a first image is determined, the first image comprises the aircraft and a positioning pattern, the positioning pattern is arranged on the aircraft, and the first image is a first image; the road vehicle is provided with a first image acquisition device; cutting an area corresponding to the aircraft in the first image to obtain a second image; identifying a positioning pattern in the second image to obtain a to-be-positioned pattern; and determining the relative position between the road vehicle and the aircraft according to the to-be-positioned pattern. According to the embodiment of the invention, positioning between the road vehicle and the aircraft can be realized through image positioning.

Description

Technical Field

[0001] The present application relates to the field of image positioning technology, and in particular to an automatic docking positioning method, device and related equipment for a flying car. Background Art

[0002] Flying cars have always been a key research direction. With the development of pure electric vertical take-off and landing aircraft, flying cars have ushered in a new research boom. In addition to integrated flying cars, some studies have proposed the research direction of split flying cars, which is to split the functions of flying cars into a land part and a flying part, including two-part, three-part and other different types.

[0003] In the case of a flying car that is split into two parts, the flying car can be divided into a road-going vehicle and an aircraft. In order to realize the automatic combination of the road-going vehicle and the aircraft, how to determine the position between the road-going vehicle and the aircraft during the docking process is crucial for the automatic docking path control during the docking process, so higher requirements are placed on the positioning technology of the two. At present, there is no docking positioning solution for the two. Summary of the invention

[0004] The embodiments of the present application disclose an automatic docking positioning method, device and related equipment of a flying car, which are used to realize the positioning between a road-going vehicle and an aircraft through image positioning.

[0005] In a first aspect, an embodiment of the present application discloses an automatic docking and positioning method for a flying car, wherein the flying car comprises a road-going vehicle and an aircraft, and the method is applied to the road-going vehicle, and the method comprises:

[0006] Determine a first image, wherein the first image includes the aircraft and a positioning pattern, the positioning pattern is set on the aircraft, and a device for collecting the first image is installed on the road vehicle;

[0007] Cropping a region corresponding to the aircraft in the first image to obtain a second image;

[0008] identifying a positioning pattern in the second image to obtain a pattern to be positioned;

[0009] The relative position between the road vehicle and the aircraft is determined according to the pattern to be positioned.

[0010] In a second aspect, an embodiment of the present application discloses an automatic docking and positioning device for a flying car, wherein the flying car includes a road-going vehicle and an aircraft, and the device is applied to the road-going vehicle, and the device includes:

[0011] A first determining unit is used to determine a first image, wherein the first image includes the aircraft and a positioning pattern, the positioning pattern is arranged on the aircraft, and a device for collecting the first image is installed on the road vehicle;

[0012] A cropping unit, used for cropping a region corresponding to the aircraft in the first image to obtain a second image;

[0013] an identification unit, configured to identify a positioning pattern in the second image to obtain a pattern to be positioned;

[0014] The second determining unit is used to determine the relative position between the road vehicle and the aircraft according to the pattern to be positioned.

[0015] As a possible implementation manner, the second determining unit is specifically configured to:

[0016] Extracting a sub-pixel corner point corresponding to the pattern to be located in the second image to obtain a first sub-pixel corner point;

[0017] The relative position between the road vehicle and the aircraft is determined according to the first sub-pixel corner point.

[0018] As a possible implementation manner, the second determining unit determining the relative position between the road vehicle and the aircraft according to the first sub-pixel corner point includes:

[0019] Determine a sub-pixel corner point corresponding to the first sub-pixel corner point in the first image to obtain a second sub-pixel corner point;

[0020] The relative position between the road vehicle and the aircraft is determined according to the second sub-pixel corner point.

[0021] As a possible implementation manner, the second determining unit is further configured to determine a pixel corner point of the pattern to be located in the second image;

[0022] The second determining unit extracts the sub-pixel corner point corresponding to the pattern to be located in the second image to obtain the first sub-pixel corner point, including:

[0023] According to the pixel corner point, sub-pixel corner point detection is performed on the second image to obtain a first sub-pixel corner point.

[0024] As a possible implementation manner, the second determining unit determining the relative position between the road vehicle and the aircraft according to the second sub-pixel corner point includes:

[0025] The second sub-pixel corner point is converted from a two-dimensional coordinate to a three-dimensional coordinate to obtain a relative position between the road vehicle and the aircraft.

[0026] As a possible implementation manner, a plurality of positioning patterns are arranged on the aircraft, and the plurality of positioning patterns are different.

[0027] As a possible implementation manner, the device further includes:

[0028] a third determining unit, configured to determine a first path according to the relative position;

[0029] A driving unit is used to drive toward the aircraft according to the first path.

[0030] As a possible implementation manner, the device further includes:

[0031] a fourth determining unit, configured to determine a second path between the road vehicle and a target position according to the relative position, wherein the target position is a position between the road vehicle and the aircraft;

[0032] A driving unit is used to drive toward the target location according to the second path.

[0033] As a possible implementation manner, the device further includes:

[0034] An adjusting unit is used to adjust the position of the vehicle on the road according to the relative position.

[0035] In a third aspect, an embodiment of the present application discloses a road vehicle, comprising a processor and a memory, wherein the processor calls a computer program stored in the memory to execute the method disclosed in the first aspect.

[0036] In a fourth aspect, an embodiment of the present application discloses a flying car, comprising the road-going vehicle and the aircraft disclosed in the third aspect.

[0037] In a fifth aspect, an embodiment of the present application discloses a computer-readable storage medium, on which a computer program or computer instructions are stored. When the computer program or computer instructions are executed by a processor, the method disclosed in the first aspect above is implemented.

[0038] In a sixth aspect, an embodiment of the present application discloses a computer program product, which includes a computer program code. When the computer program code is executed by a processor, the above method is executed.

[0039] In the embodiment of the present application, the road-going vehicle in the flying car determines the first image, the area corresponding to the aircraft in the first image is cropped to obtain the second image, the positioning pattern in the second image is identified to obtain the pattern to be positioned, and the relative position between the road-going vehicle and the aircraft is determined according to the pattern to be positioned. It can be seen that the relative position between the road-going vehicle and the aircraft can be determined according to the first image, so the positioning between the road-going vehicle and the aircraft can be achieved through image positioning. In addition, the area corresponding to the aircraft in the first image is first cropped to obtain the second image, and then the positioning pattern in the second image is identified. The two-stage target detection can greatly improve the accuracy of the positioning pattern detection, and thus improve the accuracy of the image positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 It is a structural schematic diagram of a flying car disclosed in an embodiment of the present application;

[0042] Figure 2 It is a schematic diagram of a road vehicle and an aircraft in a separated state disclosed in an embodiment of the present application;

[0043] Figure 3 is a structural schematic diagram of another flying car disclosed in an embodiment of the present application;

[0044] Figure 4 It is a schematic diagram of automatic docking, energy replenishment and separation of a flying car disclosed in an embodiment of the present application;

[0045] Figure 5 It is a flow chart of an automatic docking and positioning method of a flying car disclosed in an embodiment of the present application;

[0046] Figure 6 is a schematic diagram of an X pattern disclosed in an embodiment of the present application;

[0047] Figure 7 is a schematic diagram of a positioning pattern set on a target object disclosed in an embodiment of the present application;

[0048] Figure 8 is a schematic diagram of determining a relative position based on an image disclosed in an embodiment of the present application;

[0049] Fig. 9 It is a structural schematic diagram of an automatic docking and positioning device for a flying car disclosed in an embodiment of the present application;

[0050] Fig.10 It is a schematic diagram of the structure of a road-going vehicle disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0052] The present application discloses an automatic docking positioning method, device and related equipment for a flying car, which are used to realize the positioning between a road vehicle and an aircraft through image positioning. Detailed descriptions are given below.

[0053] In order to better understand the embodiments of the present application, the flying car is first described below.

[0054] See also Figure 1 , Figure 1 Schematic diagram of the structure of a flying car disclosed in the embodiment of the present application. Figure 1 As shown, the flying car may include a road vehicle and an aircraft. The state of the flying car may be a combined state or a separated state. When the state of the flying car is a combined state, the road vehicle and the aircraft are combined together. Figure 1 The diagram shows the situation that the road vehicle and the aircraft are in a combined state. When the state of the flying car is a separated state, the road vehicle and the aircraft are separated. Figure 2 It shows the situation that the road vehicle and the aircraft are in a separated state.

[0055] like Figure 1 and Figure 2 As shown, the road vehicle may include an engine, a generator, a power distribution box, a land power battery, a land drive component, a contactor, a thermal management component, a power distribution management component, a communication management component, and a network management component.

[0056] The engine can control the generator to generate electricity, which can then charge the land power battery through the distribution box. The land power battery can supply power to the land drive component. The land drive component can drive the road vehicle to travel or move.

[0057] The thermal management component can control the vehicle to dissipate heat to prevent the vehicle from being overheated. Exemplarily, the vehicle may further include a heat dissipation component. When the temperature of the vehicle is detected to be higher than a first temperature, the thermal management component may control the heat dissipation component to dissipate heat. When the temperature of the vehicle is detected to be lower than a second temperature, the thermal management component may control the heat dissipation component to stop working. The second temperature is lower than the first temperature.

[0058] The power distribution management component can control the state of the engine according to the power of the land power battery. Exemplarily, when it is detected that the power of the land power battery is less than the first power, the engine can be controlled to work, so that the engine can control the generator to generate electricity to charge the land power battery. Exemplarily, when the engine is in a working state and it is detected that the power of the land power battery is full, the engine can be controlled to stop working, so as to avoid the situation of charging the land power battery all the time, thereby increasing the service life of the land power battery.

[0059] The communication management component can manage the communication components of the road vehicle. There can be one or more communication components. The communication management component can control the communication component to send information to an aircraft, a road vehicle, etc., and can also control the communication component to receive information from an aircraft, a road vehicle, etc. The communication component can be a component that can perform wireless communication, such as Bluetooth, Zigbee, WIFI, the fifth generation mobile communication technology (5G), and ultra-wideband (UWB).

[0060] The network management component can control the network components in the road vehicle to connect to the network. The network can be one or more of a data network, a WIFI network, etc.

[0061] like Figure 1 and Figure 2 As shown, the aircraft may include contactors, a power distribution box, a flight power battery, a flight drive component, a thermal management component, a power distribution management component, a communication management component, and a network management component.

[0062] The thermal management component can control the aircraft to dissipate heat to prevent the temperature of the aircraft from being too high. Exemplarily, the aircraft may further include a heat dissipation component. When it is detected that the temperature of the aircraft is higher than a third temperature, the thermal management component may control the heat dissipation component to dissipate heat. When it is detected that the temperature of the aircraft is lower than a fourth temperature, the thermal management component may control the heat dissipation component to stop working. The fourth temperature is lower than the third temperature.

[0063] The communication management component can manage the communication components of the aircraft. The communication component can be one or more. The communication management component can control the communication component to send information to road vehicles, road vehicles, etc., and can also control the communication component to receive information from road vehicles, road vehicles, etc. The detailed description of the communication component can refer to the relevant description above.

[0064] When the flying car is in the combined state, the power distribution management component can control the state of the engine in the road vehicle according to the power of the flying power battery. Exemplarily, when the flying car is combined with the road vehicle and it is detected that the power of the flying power battery is less than the second power, the engine can be controlled to work, and then the engine can control the generator to generate electricity to charge the flying power battery. Exemplarily, when the engine is in the working state and it is detected that the power of the flying power battery is full, the engine can be controlled to stop working, which can avoid the situation of charging the flying power battery all the time, thereby improving the service life of the flying power battery.

[0065] When the flying car is in a separated state and it is detected that the power level of the flight power battery is less than the third power level, the power distribution management component can control the communication component to send a low-power message to the road-going vehicle, or send a charging request or a recharging request to the road-going vehicle, so that the road-going vehicle can travel to the vicinity of the aircraft to charge the aircraft.

[0066] The flight power battery can supply power to the flight drive components, which can drive the rotors of the aircraft to work, thereby making the aircraft fly.

[0067] The network management component can control the network components of the aircraft to connect to the network. For detailed description, please refer to the relevant description above.

[0068] When the flying car is in a combined state, the network management component of the road vehicle is electrically coupled to the network management component of the aircraft, the communication management component of the road vehicle is electrically coupled to the communication management component of the aircraft, the power distribution management component of the road vehicle is electrically coupled to the power distribution management component of the aircraft, the thermal management component of the road vehicle is electrically coupled to the thermal management component of the aircraft, the contactor of the road vehicle is electrically coupled to the contactor of the aircraft, and at the same time, the road vehicle and the aircraft are locked together by a physical locking structure.

[0069] The locking structure may be provided on a road-going vehicle or on an aircraft, or may be provided partially on the road-going vehicle and partially on the aircraft. When the road-going vehicle and the aircraft are in a combined state, the aircraft may be fixed to the road-going vehicle by the locking structure, thereby preventing the aircraft from being separated from the road-going vehicle. Exemplarily, a partial locking structure may be provided on the road-going vehicle, and another partial locking structure corresponding to the partial locking structure may be provided on the aircraft. In the process of combining the road-going vehicle and the aircraft, the road-going vehicle and the aircraft may be physically locked together by the partial locking structure on the road-going vehicle and the other partial locking structure on the aircraft.

[0070] See also Figure 3 , Figure 3FIG. 1 is a schematic diagram of the structure of another flying car disclosed in the embodiment of the present application. Figure 3 As shown, the road vehicle may further include a positioning component, which can locate the position of the road vehicle. The positioning component may be one or more of a global positioning system (GPS), an assisted global satellite positioning system (AGPS), a radar, and the like.

[0071] like Figure 3 As shown, the road-going vehicle may further include one or more cameras, which may collect images. The cameras may be arranged behind the road-going vehicle, in front of the road-going vehicle, or at other locations of the road-going vehicle.

[0072] The road vehicle may also include sensors. The sensors may be various sensors required for use in the vehicle, such as gravity sensors, acceleration sensors, etc. Different sensors have different functions.

[0073] like Figure 3 As shown, the aircraft may further include a positioning component, which may locate the position of the aircraft. The detailed description of the positioning component may refer to the above description.

[0074] like Figure 3 As shown, a positioning pattern is provided on the aircraft. The positioning pattern can be provided in front of the aircraft, or in the back of the aircraft, or in the left side of the aircraft, or in the right side of the aircraft. A road vehicle can use a camera to capture an image including the positioning pattern, and then use the positioning pattern to determine the relative position between the road vehicle and the aircraft.

[0075] Flying cars can use independent decoupling to achieve the separation of road vehicles and aircraft, where the road vehicles can realize all the attributes of land cars and the aircraft can realize all the flight attributes of aircraft. Or they can use associated coupling to achieve the coupling of road vehicles and aircraft, where the road vehicles can "carry" the aircraft to a suitable flight site for flight and recharge the aircraft.

[0076] When the user needs the aircraft in the flying car to fly, the flying car can carry the aircraft to a suitable flying site. At this time, the road-going vehicle and the aircraft are in a combined state. While the road-going vehicle is driving to the flying site, the aircraft can be charged according to the power of the aircraft, so that the energy storage preparation before the aircraft flight can be made.

[0077] It should be understood that the above is an exemplary description of the structure of the flying car, and does not limit the structure of the flying car. For example, a road vehicle has all the properties of a land vehicle, and an aircraft has all the flight properties of an aircraft.

[0078] See also Figure 4 , Figure 4 Schematic diagram of automatic docking, energy replenishment and separation of a flying car disclosed in an embodiment of the present application. Figure 4 As shown, after the flying car drives to the flying site, the user can trigger the separation switch for separating the road-going vehicle and the aircraft through the user interface (UI) set on the road-going vehicle or the aircraft, or through the separation button set on the road-going vehicle or the aircraft. After the flying car receives the separation operation, it can start the separation program to separate. The flying car is provided with an automatic docking structure, and the flying car can control the separation of the automatic docking structure through the separation program to achieve automatic separation between the road-going vehicle and the aircraft.

[0079] When the road vehicle of the flying car is separated from the aircraft, the road vehicle can automatically drive to a safe area to wait, that is, the road vehicle can drive away from the aircraft to ensure that the aircraft has enough space to fly normally, that is, the road vehicle drives to an area that will not affect the flight of the aircraft. The safe area is an area whose distance from the current location of the aircraft is greater than or equal to the safe distance. The safe distance is the minimum distance that can ensure the normal flight of the aircraft.

[0080] After the road vehicle leaves the aircraft, the aircraft can perform a flight mission, that is, it can make corresponding take-off preparations according to the flight mission and fly.

[0081] During the flight mission, if the aircraft detects that the power of the aircraft is insufficient, that is, the aircraft needs to be recharged, the aircraft can land to prepare for recharging. The landing position of the aircraft can be the previous take-off position or a nearby position, that is, near the flight position of the aircraft.

[0082] When the aircraft is low on power and the flight mission has not been completed, the aircraft can send a request for energy replenishment or charging to a road-based vehicle after landing. The request for energy replenishment or charging can carry or include the current location of the aircraft. After receiving the request for energy replenishment or charging from the aircraft, the road-based vehicle can drive to the vicinity of the aircraft according to the current location of the aircraft and charge the aircraft. The road-based vehicle can charge the aircraft by wire or by wireless charging.

[0083] In the case where a road-going vehicle can wirelessly charge an aircraft, in one case, after the aircraft detects that the distance between the road-going vehicle and the aircraft is less than or equal to the charging distance, the aircraft can send a charging request to the road-going vehicle, and after the road-going vehicle receives the charging request, the aircraft can be wirelessly charged. In another case, after the road-going vehicle detects that the distance between the road-going vehicle and the aircraft is less than or equal to the charging distance, the aircraft can be wirelessly charged.

[0084] When the road-going vehicle charges the aircraft by wire, the road-going vehicle can realize automatic docking between the charging cable and the charging port of the aircraft, and then charge the aircraft through the connecting cable. The road-going vehicle can also output a prompt message for prompting the user to connect the charging cable, so that the user can connect the charging cable according to the prompt message and then charge the aircraft.

[0085] After the road-going vehicle detects that the aircraft is fully charged, it can drive away from the aircraft and wait in a safe area. In the case where the road-going vehicle charges the aircraft by wireless charging, after the road-going vehicle detects that the aircraft is fully charged, it can automatically drive away from the aircraft, and can also output a prompt message to remind the user that the aircraft has been charged. In the case where the aircraft does not require manual control, the aircraft can make a go-around according to the previous flight mission to continue to complete the unfinished mission. In the case where the aircraft requires manual control, the user can control the aircraft to make a go-around according to the prompt message to complete the unfinished mission. In the case where the road-going vehicle charges the aircraft by wired charging, after the road-going vehicle detects that the aircraft is fully charged, it can output a prompt message to remind the user that the aircraft has been charged, so that the user can retract the charging cable according to the prompt message.

[0086] After the aircraft is fully charged, it can continue to fly, that is, perform a go-around. After the flight mission is completed (that is, the go-around), the aircraft can land. In addition, the aircraft can also output a prompt message for prompting the user to land the aircraft.

[0087] After the aircraft lands, if the user needs to combine the road-based vehicle with the aircraft, the user can input a combination coupling request, a combination request or a combination instruction to the road-based vehicle through the UI interface, a button or a terminal device. After receiving the combination coupling request, the combination request or the combination instruction, the road-based vehicle can approach the aircraft according to the position of the aircraft to complete automatic docking.

[0088] After the automatic docking is completed, the automatic docking structure of the road-going vehicle and the aircraft can be used to pull the aircraft into the receiving cavity of the road-going vehicle to achieve coupling. Since the road-going vehicle may be uneven, the attitude of the road-going vehicle can be adjusted to be level first, and then the automatic docking structure of the road-going vehicle and the aircraft can be used to pull the aircraft into the receiving cavity of the road-going vehicle to achieve coupling.

[0089] See also Figure 5 , Figure 5 1 is a flow chart of an automatic docking and positioning method for a flying car disclosed in an embodiment of the present application. The flying car includes a road-going vehicle and an aircraft, and the automatic docking and positioning method for the flying car can be applied to a road-going vehicle. Figure 5 As shown, the automatic docking and positioning method of the flying car may include the following steps.

[0090] 501. Determine a first image.

[0091] The first image includes an aircraft and a positioning pattern, and the positioning pattern is arranged on the aircraft.

[0092] The aircraft may be provided with multiple positioning patterns, and the multiple positioning patterns are different. The multiple positioning patterns may be X patterns, circular patterns, or other recognizable patterns. For example, please refer to Figure 6 , Figure 6 is a schematic diagram of an X pattern disclosed in the embodiment of the present application. These multiple positioning patterns can be Figure 6 Part or all of the X pattern in .

[0093] The number of positioning patterns set on the aircraft may be greater than or equal to 3. Exemplarily, the number of positioning patterns set on the aircraft may be 3, 5, 6, or other values ​​greater than 3. It should be understood that the specific number of positioning patterns set on the aircraft is not limited, and only needs to be greater than or equal to 3.

[0094] Different positioning patterns among the multiple positioning patterns can be arranged on the same plane or on different planes. Therefore, the positioning pattern is arranged in front of (or behind, on the left or on the right of) the aircraft, which can be understood as the positioning pattern being arranged in front of the aircraft, that is, the positioning pattern on the aircraft can be seen in front of (or behind, on the left or on the right of) the aircraft.

[0095] For example, see Figure 7 , Figure 7 Schematic diagram of a positioning pattern set on a target object disclosed in an embodiment of the present application. Figure 7As shown, 8 different X patterns are arranged on the target object. The 8 X patterns are arranged in front of the target object and on different planes.

[0096] The road vehicle may store the setting surfaces of multiple positioning patterns on the aircraft and the setting position of each positioning pattern in the multiple positioning patterns. The setting position of the positioning pattern is the specific position of the positioning pattern on the aircraft.

[0097] A first image acquisition device is installed on the road vehicle. In the process of combining the road vehicle with the flying object, in one case, when the road vehicle is traveling toward the flying object, the road vehicle can use the acquisition device to acquire images in real time or periodically, and when it is recognized that the acquired image includes the flying object and the positioning pattern, it can be determined that the acquired image is the first image; in another case, the road vehicle can use the acquisition device to acquire images at a first position, and when it is recognized that the acquired image includes the flying object and the positioning pattern, it can be determined that the acquired image is the first image. The first position is a position at a preset distance from the flying object determined according to the position of the flying object. The position of the flying object is determined by the flying object according to the positioning component and sent to the road vehicle. The acquisition device can be a camera or other sensor device capable of acquiring images.

[0098] After the road vehicle collects an image through the collection device, it can first identify whether the collected image includes an aircraft. If it is identified that the collected image includes an aircraft, it can continue to identify whether the area where the aircraft is located in the collected image includes a positioning pattern set on the aircraft. If it is identified that the area where the aircraft is located in the collected image includes the positioning pattern set on the aircraft, it can be determined that the collected image is the first image. If it is identified that the collected image does not include an aircraft, or if it is identified that the area where the aircraft is located in the collected image does not include the positioning pattern set on the aircraft, it can be determined that the collected image is not the first image.

[0099] The road vehicle can use the first network model to identify the aircraft in the collected image. The first network model can be trained by the image including the aircraft. The first network model can be a network model capable of target detection. The first network model can be a two-stage network model, a one-stage network model, an anchor-based network model, an anchor-free network model and other target detection network models. The two-stage network model can be a network model such as a faster region-based convolutional neural network (RCNN) model, a cascade RCNN model, a mask RCNN model, etc. The first-order network model can be a Yolo series network model, a single shot multibox detector (SSD) model, a retina network (RetinaNet) model, a full convolutional one-stage object detection (FCOS) model, a corner (CornerNet) model and other network models.

[0100] The road vehicle can use the second network model to identify the positioning pattern in the first image. The second network model can be trained by the image including the positioning pattern. The second network model can be a network model capable of target detection, and a detailed description can refer to the first network model.

[0101] The first network model and the second network model may be the same or different. When the first network model and the second network model are the same, the first network model and the second network model may be trained by images including the aircraft and the positioning pattern. The first network model and the second network model may be different in that both the initial network model (i.e., the network model before training) and the training data are different, or the training data are different.

[0102] 502. Crop a region corresponding to the aircraft in the first image to obtain a second image.

[0103] After the road vehicle determines the first image, the area corresponding to the aircraft in the first image can be cropped to obtain the second image. Cropping the area corresponding to the aircraft in the first image can be understood as extracting a region of interest (ROI) (i.e., the area corresponding to the aircraft) from the first image, or can be understood as subtracting the aircraft from the first image.

[0104] 503. Identify the positioning pattern in the second image to obtain a pattern to be positioned.

[0105] After the road vehicle crops the area corresponding to the aircraft in the first image to obtain the second image, the second network model can be used to identify the positioning pattern in the second image to obtain the pattern to be positioned. The pattern to be positioned can be a partial positioning image or all patterns among the multiple positioning patterns set on the aircraft.

[0106] 504. Determine the relative position between the road vehicle and the aircraft according to the pattern to be positioned.

[0107] The road-going vehicle may first extract the sub-pixel corner points corresponding to the pattern to be positioned in the second image to obtain the first sub-pixel corner points, and then determine the relative position between the road-going vehicle and the aircraft according to the first sub-pixel corner points to obtain the relative position.

[0108] The road vehicle can first determine the sub-pixel corner point corresponding to the first sub-pixel corner point in the first image to obtain the second sub-pixel corner point, that is, map the sub-pixel corner point in the second image to the first image, and then determine the relative position between the road vehicle and the aircraft according to the second sub-pixel corner point. The second sub-pixel corner point can be converted from a two-dimensional coordinate to a three-dimensional coordinate to obtain the relative position between the road vehicle and the aircraft, that is, the second sub-pixel corner point can be solved for multi-point perspective imaging (Perspective-n-Point, PNP) to obtain the relative position between the road vehicle and the aircraft.

[0109] The vehicle on the road may first determine the pixel corner points of the pattern to be positioned in the second image, and then perform sub-pixel corner point detection on the second image according to the pixel corner points of the positioning pattern in the second image to obtain first sub-pixel corner points.

[0110] For example, the pixel corner of the pattern to be located may be (90, 200), while the sub-pixel corner of the pattern to be located may be (90.23, 199.98). It can be seen that the sub-pixel corner is more accurate than the pixel corner, so the detection accuracy can be improved according to the sub-pixel corner detection.

[0111] For example, see Figure 8 , Figure 8 is a schematic diagram of determining a relative position based on an image disclosed in an embodiment of the present application. Figure 8As shown, target detection can be performed on the first image to obtain the position or area of ​​the target object in the first image, and then ROI extraction can be performed on the first image to obtain the second image. Target detection can be performed on the second image to obtain the position or area of ​​the positioning pattern in the second image, and ROI extraction can be performed on the second image to obtain the pattern to be positioned. Then, sub-pixel corner point extraction can be performed on the second image according to the pattern to be positioned to obtain the sub-pixel corner points of the image to be positioned, and then the sub-pixel corner points can be solved PnP to obtain the relative distance to the target object.

[0112] It should be understood that Figure 8 This is an exemplary description of determining the relative position based on the image and does not limit it.

[0113] It can be seen that the road vehicle uses a two-stage deep learning detection method. The first stage detects the aircraft, and the second stage detects the positioning pattern, which can greatly improve the detection efficiency and accuracy of the positioning pattern.

[0114] In addition, since different positioning patterns are set at different positions on the aircraft, different positioning patterns have unique identifiers, which can deal with missed detections, making the positioning result more robust, and there are no special requirements for the layout position (or setting position) of the positioning pattern. Further, since the positioning pattern has a unique identifier and multiple positioning patterns are set on the aircraft, there will be no mismatching, resulting in the inability to calculate the correct position.

[0115] exist Figure 5 In the automatic docking and positioning method of the flying car described, the road-going vehicle in the flying car determines the first image, the area corresponding to the aircraft in the first image is cropped to obtain the second image, the positioning pattern in the second image is identified to obtain the pattern to be positioned, and the relative position between the road-going vehicle and the aircraft is determined according to the pattern to be positioned. It can be seen that the relative position between the road-going vehicle and the aircraft can be determined according to the first image, so the positioning between the road-going vehicle and the aircraft can be achieved through image positioning. In addition, the area corresponding to the aircraft in the first image is first cropped to obtain the second image, and then the positioning pattern in the second image is identified. The two-stage target detection is used, which can greatly improve the accuracy of the positioning pattern detection, and thus improve the accuracy of the image positioning.

[0116] It should be understood that the above-mentioned automatic docking and positioning method of the flying car can be applied to the process in which the road-going vehicle approaches the aircraft after the road-going vehicle and the aircraft detect a request for merging.

[0117] In some embodiments, the above-mentioned automatic docking and positioning method of the flying car may further include the following steps:

[0118] Determine a first path according to the relative position;

[0119] Travel toward the aircraft according to the first path.

[0120] The road-going vehicle may determine a first path according to the relative position between the road-going vehicle and the aircraft, and then travel toward the aircraft according to the first path.

[0121] For example, when the road vehicle is traveling toward the aircraft according to the planned path, after the road vehicle determines the relative position, it can determine a first path between the current position and the aircraft, and then continue to travel toward the aircraft according to the first path. The first path can be a partial path between the initial position of the road vehicle and the aircraft.

[0122] In some embodiments, the above-mentioned automatic docking and positioning method of the flying car may further include the following steps:

[0123] Determine a second path between the road-going vehicle and the target position according to the relative position, the target position being the position between the road-going vehicle and the aircraft;

[0124] Travel toward the target location according to the second route.

[0125] The road-going vehicle can determine a second path between the road-going vehicle and the aircraft according to the relative position between the road-going vehicle and the aircraft, and then travel to the target position according to the second path. The target position is the position determined by the road-going vehicle according to the relative position.

[0126] For example, the vehicle can determine the target position based on the relative position, and then determine a second path between the current position of the vehicle and the target position, and then travel to the target position according to the second path. The target position is the position between the land vehicle and the aircraft.

[0127] In some embodiments, the above-mentioned automatic docking and positioning method of the flying car may further include the following steps:

[0128] The position of the vehicle on the road is adjusted according to the relative position.

[0129] The road-going vehicle can adjust the position of the road-going vehicle according to the relative position between the road-going vehicle and the aircraft.

[0130] Exemplarily, the road vehicle can calibrate the first position according to the relative position.

[0131] It should be understood that the same or corresponding information in the above different embodiments can be referenced to each other.

[0132] See also Fig. 9 , Fig. 91 is a schematic diagram of the structure of an automatic docking and positioning device for a flying car disclosed in an embodiment of the present application. The flying car includes a road-going vehicle and an aircraft, and the automatic docking device for the flying car is applied to the road-going vehicle. Fig. 9 As shown, the automatic docking and positioning device of the flying car may include:

[0133] A first determining unit 901 is used to determine a first image, where the first image includes an aircraft and a positioning pattern, the positioning pattern is set on the aircraft, and a first image acquisition device is installed on the road vehicle;

[0134] A cropping unit 902 is used to crop a region corresponding to the target aircraft in the first image to obtain a second image;

[0135] The recognition unit 903 is used to recognize the positioning pattern in the second image to obtain the pattern to be positioned;

[0136] The second determining unit 904 is used to determine the relative position between the road vehicle and the aircraft according to the pattern to be located.

[0137] In some embodiments, the second determining unit 904 is specifically configured to:

[0138] Extracting sub-pixel corner points corresponding to the pattern to be located in the second image to obtain first sub-pixel corner points;

[0139] The relative position between the road vehicle and the aircraft is determined according to the first sub-pixel corner point.

[0140] In some embodiments, the second determining unit 904 determines the relative position between the road vehicle and the aircraft according to the first sub-pixel corner point, including:

[0141] Determine a sub-pixel corner point corresponding to the first sub-pixel corner point in the first image to obtain a second sub-pixel corner point;

[0142] The relative position between the road vehicle and the aircraft is determined according to the second sub-pixel corner point.

[0143] In some embodiments, the second determining unit 904 is further configured to determine a pixel corner point of the pattern to be located in the second image;

[0144] The second determination unit 904 extracts the sub-pixel corner point corresponding to the pattern to be located in the second image, and obtains the first sub-pixel corner point including:

[0145] According to the pixel corner point, sub-pixel corner point detection is performed on the second image to obtain a first sub-pixel corner point.

[0146] In some embodiments, the second determining unit 904 determines the relative position between the road vehicle and the aircraft according to the second sub-pixel corner point, including:

[0147] The second sub-pixel corner point is converted from a two-dimensional coordinate to a three-dimensional coordinate to obtain the relative position between the road vehicle and the aircraft.

[0148] In some embodiments, a plurality of positioning patterns are arranged on the aircraft, and the plurality of positioning patterns are different.

[0149] In some embodiments, the automatic docking and positioning device of the flying car may also include:

[0150] a third determining unit, configured to determine a first path according to the relative position;

[0151] A driving unit is used to drive toward the aircraft according to the first path.

[0152] In some embodiments, the automatic docking and positioning device of the flying car may also include:

[0153] a fourth determining unit, configured to determine a second path between the road-going vehicle and a target position according to the relative position, the target position being a position between the road-going vehicle and the aircraft;

[0154] A driving unit is used to drive toward a target location according to the second path.

[0155] In some embodiments, the automatic docking and positioning device of the flying car may also include:

[0156] The adjusting unit is used to adjust the position of the vehicle on the road according to the relative position.

[0157] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the automatic docking and positioning device of the flying car described above, the first determination unit 901, the cutting unit 902, the identification unit 903, the second determination unit 904, the third determination unit, the driving unit, the fourth determination unit and the adjustment unit can refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0158] In several embodiments provided in the present application, the coupling between the units may be electrical, mechanical or other forms of coupling.

[0159] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0160] Please refer to Fig.10 , Fig.10 Schematic diagram of the structure of a road vehicle disclosed in the embodiment of the present application. Fig.10As shown, the road vehicle may include a processor 1001 and a memory 1002. The memory 1002 may store one or more computer programs. The one or more computer programs are configured to execute the method described in the above method embodiment. The memory 1002 may exist independently or be integrated with the processor 1001.

[0161] The processor 1001 may include one or more processing cores. The processor 1001 may use various interfaces and lines to connect various parts of the entire road vehicle, and may execute various functions of the road vehicle and process data by running or executing instructions, programs, code sets or instruction sets stored in the memory 1002, and calling data stored in the memory 1002. Optionally, the processor 1001 may be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 1001 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 1001, but may be implemented separately through a communication chip.

[0162] The memory 1002 may include a random access memory (RAM) or a read-only memory (ROM). The memory 1002 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 1002 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also store data created by a road vehicle during use (such as a phone book, audio and video data, chat record data), etc.

[0163] When the computer program instructions stored in the memory 1002 are executed, the processor 1001 can be used to execute various operations performed by the road-going vehicle in the above method embodiments. The specific implementation of these operations can be referred to the previous embodiments, and will not be described in detail here.

[0164] The present application also discloses a computer-readable storage medium, in which a computer program code is stored, and the computer program code can be called by a processor to perform various operations in the above method embodiment. The specific implementation of each of the above operations can be referred to the previous embodiment, and will not be repeated here.

[0165] The computer readable storage medium may be an electronic memory such as a flash memory, an electrically erasable programmable read only memory (EEPROM), an erasable programmable read only memory (EPROM), a hard disk or a ROM. Alternatively, the computer readable storage medium may include a non-transitory computer-readable storage medium. The computer readable storage medium has storage space for program codes that execute any of the method steps in the above method. These computer program codes may be read from or written to one or more computer program products. The computer program code may be compressed, for example, in an appropriate form.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for automatic docking and positioning of a flying car, characterized in that: The flying car includes a road-going vehicle and an aircraft, and the method is applied to the road-going vehicle, and the method includes: Determine a first image, wherein the first image includes the aircraft and a positioning pattern, the positioning pattern is set on the aircraft, and a device for collecting the first image is installed on the road vehicle; Cropping a region corresponding to the aircraft in the first image to obtain a second image; identifying a positioning pattern in the second image to obtain a pattern to be positioned; The relative position between the road vehicle and the aircraft is determined according to the pattern to be positioned.

2. The method according to claim 1, characterized in that The determining the relative position between the road vehicle and the aircraft according to the pattern to be located comprises: Extracting a sub-pixel corner point corresponding to the pattern to be located in the second image to obtain a first sub-pixel corner point; The relative position between the road vehicle and the aircraft is determined according to the first sub-pixel corner point.

3. The method according to claim 2, characterized in that The determining the relative position between the road vehicle and the aircraft according to the first sub-pixel corner point comprises: Determine a sub-pixel corner point corresponding to the first sub-pixel corner point in the first image to obtain a second sub-pixel corner point; The relative position between the road vehicle and the aircraft is determined according to the second sub-pixel corner point.

4. The method according to claim 2, characterized in that: The determining the relative position between the road vehicle and the aircraft according to the pattern to be located further comprises: Determine pixel corner points of the pattern to be located in the second image; The extracting the sub-pixel corner point corresponding to the pattern to be located in the second image to obtain the first sub-pixel corner point comprises: According to the pixel corner point, sub-pixel corner point detection is performed on the second image to obtain a first sub-pixel corner point.

5. The method according to claim 3, characterized in that: The determining the relative position between the road vehicle and the aircraft according to the second sub-pixel corner point comprises: The second sub-pixel corner point is converted from a two-dimensional coordinate to a three-dimensional coordinate to obtain a relative position between the road vehicle and the aircraft.

6. The method according to claim 1, characterized in that The aircraft is provided with a plurality of positioning patterns, and the plurality of positioning patterns are different.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: determining a first path according to the relative position; Travel toward the aircraft according to the first path.

8. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Determine a second path between the road vehicle and a target position according to the relative position, wherein the target position is a position between the road vehicle and the aircraft; Travel toward the target location according to the second path.

9. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The position of the vehicle on the road is adjusted according to the relative position.

10. An automatic docking and positioning device for a flying car, characterized in that: The flying car includes a road-going vehicle and an aircraft, and the device is applied to the road-going vehicle, and the device includes: A first determining unit is used to determine a first image, wherein the first image includes the aircraft and a positioning pattern, the positioning pattern is arranged on the aircraft, and a device for collecting the first image is installed on the road vehicle; A cropping unit, used for cropping a region corresponding to the aircraft in the first image to obtain a second image; an identification unit, configured to identify a positioning pattern in the second image to obtain a pattern to be positioned; The second determining unit is used to determine the relative position between the road vehicle and the aircraft according to the pattern to be positioned.

11. A road-going vehicle, characterized in that: The method comprises a processor and a memory, wherein the processor calls a computer program stored in the memory to implement the method according to any one of claims 1 to 9.

12. A flying car, characterized in that: It comprises an aircraft and a road-going vehicle as claimed in claim 11.

Images