Aircraft cabin door recognition and positioning method and device and related equipment

Abstract

The invention provides an aircraft cabin door recognition and positioning method and device and related equipment, and relates to the technical field of cabin door positioning, and the method comprises the steps: obtaining an aircraft cabin door image; inputting the acquired aircraft cabin door image into a pre-trained cabin door identification model, and outputting an identification result of the aircraft cabin door; determining whether the cabin door threshold of the airplane cabin door is successfully identified according to the identification result of the airplane cabin door; under the condition that the cabin door threshold of the airplane cabin door is successfully identified, extracting an edge point set of an airplane cabin door entity; according to the edge point set of the cabin door entity, fitting to obtain a first line passing through the upper edge of the cabin door threshold entity, a second line passing through the left edge of the cabin door slot entity and a third line passing through the right edge of the cabin door slot entity; and obtaining a first angular point coordinate and a second angular point coordinate through the first line, the second line and the third line.

Description

Background Art

[0002] Traditional boarding bridges require bridge operators to manually operate the boarding bridge to dock the aircraft, which is inefficient and has safety hazards. The boarding bridge automatic docking system can achieve fully automatic docking without manual operation, which increases the efficiency of docking and improves the safety of the docking system. However, to achieve accurate and fully reliable automatic docking, accurately identifying the aircraft door and locating the position deviation between the aircraft door and the center of the boarding bridge docking port is one of the key technologies. In the existing automatic docking system, traditional machine learning methods are used to manually analyze and pre-analyze the aircraft door features, manually design algorithms, segment the aircraft door threshold, and then extract the model of the aircraft door threshold, and calculate the position coordinates of the left and right corners of the threshold.

[0003] However, the defect in the related art is that when the gap between the doors of the aircraft cabin door and the door sill is not clear enough in the picture, it is easy to fail to segment or segment incorrectly.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present invention provides an aircraft cabin door identification and positioning method, device and related equipment, which at least to a certain extent overcome the technical problem in the related art that the automatic boarding bridge receiving system cannot accurately identify the position of the aircraft cabin door threshold.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, there is provided an aircraft door recognition and positioning method, including: obtaining an aircraft door image, where the aircraft door image includes: an aircraft door to be recognized; inputting the obtained aircraft door image into a pre-trained door recognition model, and outputting a recognition result of the aircraft door, where the door recognition model is used to recognize the following elements of the aircraft door from the aircraft door image and perform image entity segmentation: the door gap area, the door identification area, and the door sill area; judging and determining whether the door sill of the aircraft door is successfully recognized according to the recognition result of the aircraft door; in the case where the door sill of the aircraft door is successfully recognized, extracting a set of edge points of the aircraft door entity, where the set of edge points of the aircraft door entity includes: a set of points on the upper edge of the door sill entity, a set of points on the left edge of the door gap entity, and a set of points on the right edge of the door gap entity; fitting a first line passing through the upper edge of the door sill entity, a second line passing through the left edge of the door gap entity, and a third line passing through the right edge of the door gap entity according to the set of edge points of the aircraft door entity; obtaining a first corner point coordinate and a second corner point coordinate according to the first line, the second line, and the third line, where the first corner point is the intersection point of the first line and the second line, and the second corner point is the intersection point of the first line and the third line.

[0008] In some exemplary embodiments of the present disclosure, based on the foregoing solution, judging and determining whether the door sill of the aircraft door is successfully recognized according to the recognition result of the aircraft door includes: after the door recognition model recognizes that the aircraft door image includes: the door gap area, the door identification area, and the door sill area, determining whether the door identification area and the door sill area are in standard positions; if the door identification area and the door sill area are in standard positions, determining that the door sill of the aircraft door is successfully recognized.

[0009] In some exemplary embodiments of the present disclosure, based on the foregoing solution, if the door identification area and the door sill area are in standard positions, determining that the door sill of the aircraft door is successfully recognized includes: the door identification area is within the door gap area; and / or, the door sill area is below the door identification area; and / or, the deviation of the door sill area from the center line of the door identification area is less than or equal to a first preset threshold; and / or, the distance between the door sill area and the door identification area is less than or equal to a second preset threshold; and / or, the difference in the width in the left-right direction between the door identification area and the width in the left-right direction of the door sill area is less than or equal to a third preset threshold.

[0010] In some exemplary embodiments of the present disclosure, based on the foregoing solution, according to the set of edge points of the aircraft door entity, a first line passing through the upper edge of the door sill entity, a second line passing through the left edge of the door gap entity, and a third line passing through the right edge of the door gap entity are fitted, including: extracting the set of points on the upper edge of the door sill entity and fitting the first line passing through the upper edge of the door sill entity; extracting the set of points on the left edge of the door gap entity and fitting the second line passing through the left edge of the door gap entity; extracting the set of points on the right edge of the door gap entity and fitting the third line passing through the right edge of the door gap entity.

[0011] In some exemplary embodiments of the present disclosure, based on the foregoing solution, according to the first line, the second line, and the third line, the first corner point coordinates and the second corner point coordinates are obtained, including: obtaining the plane coordinates of the first corner point according to the simultaneous equations of the first line and the second line; obtaining the plane coordinates of the second corner point according to the simultaneous equations of the first line and the third line; based on the plane coordinates of the first corner point and the plane coordinates of the second corner point, and according to the positioning principle of binocular vision, calculating the three-dimensional coordinates of the first corner point and the second corner point of the door sill.

[0012] In some exemplary embodiments of the present disclosure, based on the foregoing solution, before inputting the acquired aircraft door image into a pre-trained door recognition model and outputting the recognition result of the aircraft door, the method further includes: obtaining an aircraft door image training set, where the aircraft door image training set includes multiple aircraft door images; annotating the door gap area, the door identification area, and the door sill area in the collected aircraft door images to obtain an aircraft door annotation data set corresponding to the aircraft door image training set; training a pre-constructed multi-task deep learning network model according to the aircraft door image training set and the aircraft door annotation data set to obtain a trained door recognition model.

[0013] In some exemplary embodiments of the present disclosure, based on the foregoing solution, annotating the door gap area, the door identification area, and the door sill area in the collected aircraft door images to obtain an aircraft door annotation data set corresponding to the aircraft door image training set includes: using annotation software to annotate the door gap area, the door identification area, and the door sill area in the collected aircraft door images to obtain a door gap area annotation set, a door identification area annotation set, and a door sill area annotation set in the aircraft door image.

[0014] According to another aspect of the present disclosure, there is also provided an aircraft door recognition and positioning device, including: a door image acquisition module for acquiring an aircraft door image, where the aircraft door image includes: the aircraft door to be recognized; a door recognition result output module for inputting the acquired aircraft door image into a pre-trained door recognition model and outputting the recognition result of the aircraft door, where the door recognition model is used to recognize the following elements of the aircraft door from the aircraft door image and perform image entity segmentation: the door gap area, the door identification area, and the door sill area; a door recognition result judgment module for judging and determining whether the door sill of the aircraft door is successfully recognized according to the recognition result of the aircraft door; a door entity edge point extraction module for extracting a set of edge points of the aircraft door entity when the door sill of the aircraft door is successfully recognized, where the set of edge points of the aircraft door entity includes: a set of points on the upper edge of the door sill entity, a set of points on the left edge of the door gap entity, and a set of points on the right edge of the door gap entity; a door entity edge point fitting module for fitting a first line passing through the upper edge of the door sill entity, a second line passing through the left edge of the door gap entity, and a third line passing through the right edge of the door gap entity according to the set of edge points of the aircraft door entity; a coordinate generation module for obtaining a first corner point coordinate and a second corner point coordinate according to the first line, the second line, and the third line, where the first corner point is the intersection of the first line and the second line, and the second corner point is the intersection of the first line and the third line.

[0015] According to still another aspect of the present disclosure, there is also provided an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein, the processor is configured to execute any of the above aircraft door recognition and positioning methods by executing the executable instructions.

[0016] According to yet another aspect of the present disclosure, there is also provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements any of the above aircraft door recognition and positioning methods.

[0017] In the embodiments of the present disclosure, the provided aircraft door recognition and positioning method, device and related equipment first input the obtained aircraft door image into a pre-trained door recognition model, output whether there are door gap areas, door identification areas and door threshold areas in the aircraft door image, and determine whether the door threshold of the aircraft door is successfully recognized according to the recognition result of the aircraft door; then, when the door threshold of the aircraft door is successfully recognized, extract the set of edge points of the aircraft door entity, and fit the first line passing through the upper edge of the door threshold entity, the second line passing through the left edge of the door gap entity and the third line passing through the right edge of the door gap entity according to the set of edge points of the aircraft door entity, and determine the first corner point coordinates and the second corner point coordinates.

[0018] In the embodiments of the present disclosure, after determining that the aircraft door is recognized, by extracting the set of edge points of the door entity, fitting the first line passing through the upper edge of the door threshold entity, the second line passing through the left edge of the door gap entity and the third line passing through the right edge of the door gap entity, and finally obtaining the first corner point coordinates and the second corner point coordinates according to the three lines. In the embodiments of the present disclosure, three targets, namely, the door gap area, the door identification area and the door threshold area, are determined simultaneously according to the recognition result of the aircraft door, and the corner point coordinates are inferred by comprehensively using the three targets. Compared with a single target, the success rate and credibility of inferring the corner point coordinates of the door threshold in the embodiments of the present disclosure are greatly improved.

[0019] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0021] Figure 1 Schematic diagram showing the principle of a method for recognizing an aircraft door in an embodiment of the present disclosure;

[0022] Figure 2 Schematic diagram showing a method for recognizing and positioning an aircraft door in an embodiment of the present disclosure;

[0023] Figure 3 Schematic diagram showing a method for recognizing the door threshold of an aircraft door in an embodiment of the present disclosure;

[0024] Figure 4 Schematic diagram showing a method for determining corner point coordinates in an embodiment of the present disclosure;

[0025] Figure 5 Shows a schematic diagram for determining corner coordinates in an embodiment of the present disclosure;

[0026] Figure 6 Shows a schematic diagram of a method for training a hatch identification model in an embodiment of the present disclosure;

[0027] Figure 7 Shows a schematic diagram of a method for target annotation of an aircraft hatch in an embodiment of the present disclosure;

[0028] Figure 8 Shows a schematic diagram of the target annotation result of an aircraft hatch in an embodiment of the present disclosure;

[0029] Figure 9 Shows a schematic diagram of a method for annotating the hatch door gap in an embodiment of the present disclosure;

[0030] Figure 10 Shows a schematic diagram of the hatch door gap annotation result in an embodiment of the present disclosure;

[0031] Figure 11 Shows a schematic diagram of a method for annotating aircraft hatch markings in an embodiment of the present disclosure;

[0032] Figure 12 Shows a schematic diagram of the aircraft hatch marking annotation result in an embodiment of the present disclosure;

[0033] Figure 13 Shows a schematic diagram of a method for annotating the aircraft hatch sill in an embodiment of the present disclosure;

[0034] Figure 14 Shows a schematic diagram of the aircraft hatch sill annotation result in an embodiment of the present disclosure;

[0035] Figure 15 Shows a schematic diagram of a method using a constructed multi-task learning network in an embodiment of the present disclosure;

[0036] Figure 16 Shows a schematic diagram of a multi-task learning process in an embodiment of the present disclosure;

[0037] Figure 17 Shows a schematic diagram of the identification result of an aircraft hatch in an embodiment of the present disclosure;

[0038] Figure 18 Shows a schematic diagram of a device for identifying the sill area of an aircraft hatch in an embodiment of the present disclosure. Detailed implementation manners

[0039] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments.

[0040] In addition, the features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be used. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

[0041] The flowcharts shown in the accompanying drawings are merely illustrative and not necessarily include all the content and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0042] First, in the embodiments of the present disclosure, a method for aircraft door recognition and positioning that can be applied but is not limited to can be applied to any device with a door. Among them, in the case where the door sill of the aircraft door is successfully recognized in the embodiments of the present disclosure, a set of edge points of the aircraft door entity is extracted, and then a first line passing through the upper edge of the door sill entity, a second line passing through the left edge of the door gap entity, and a third line passing through the right edge of the door gap entity are fitted, and based on the first line, the second line, and the third line, the first corner point coordinates and the second corner point coordinates are finally determined, avoiding the inaccuracy of the aircraft door sill part segmented due to unclear images in the related art.

[0043] In some embodiments, such as Figure 1As shown in the figure, the aircraft door recognition method based on multi-task deep learning in the embodiments of the present disclosure generally includes three major steps: S102, training and extracting a door recognition model; S104, based on the trained and extracted door recognition model, recognizing the obtained aircraft door image, segmenting the door gap area, the door identification area, and the door sill area of the aircraft door, and comprehensively judging whether the door sill of the aircraft door is successfully recognized; S106, when the door sill of the aircraft door is successfully recognized, based on the segmented entities, extracting a set of edge points, fitting a straight line or a curve, and solving the three-dimensional coordinates of the left corner point, that is, the first corner point, and the right corner point, that is, the second corner point. The first corner point is the intersection of the left extension line of the door identification area and the aircraft door sill or its extension line, and the second corner point is the intersection of the right extension line of the door identification area and the aircraft door sill or its extension line. Due to different aircraft models, the first corner point and the second corner point may be on the aircraft door sill or on the extension line of the aircraft door sill.

[0044] Figure 2 As shown in the figure, a schematic diagram of an aircraft door recognition and positioning method in the embodiments of the present disclosure is as follows Figure 2 As shown in the figure, the aircraft door recognition and positioning method provided in the embodiments of the present disclosure includes the following steps:

[0045] S202, obtaining an aircraft door image, which includes: the aircraft door to be recognized.

[0046] It should be noted that the aircraft door image obtained in the embodiments of the present disclosure can be an image including the aircraft door taken by any photographing device. For example, when the boarding bridge reaches in front of the aircraft door at the aircraft parking apron, the camera group continuously takes pictures of the aircraft door image.

[0047] S204, inputting the obtained aircraft door image into a pre-trained door recognition model, and outputting the recognition result of the aircraft door. The door recognition model is used to recognize the following elements of the aircraft door according to the aircraft door image and perform image entity segmentation: the door gap area, the door identification area, and the door sill area.

[0048] It should be noted that the door recognition model in the embodiments of the present disclosure can be a model pre-trained by machine learning for various artificial intelligence algorithm models (for example, multi-task deep learning neural network models) or their combined models. This model can automatically recognize and match the aircraft door image to determine whether the following elements are included in the aircraft door image: the door gap area, the door identification area, and the door sill area.

[0049] S206, judging and determining whether the door sill of the aircraft door is successfully recognized according to the recognition result of the aircraft door.

[0050] It should be noted that in the embodiments of the present disclosure, the recognition result of the aircraft cabin door is whether the aircraft cabin door image includes a cabin door gap area, a cabin door identification area, and a cabin door threshold area. If the recognition result of the aircraft cabin door is that the aircraft cabin door image includes a cabin door gap area, a cabin door identification area, and a cabin door threshold area, it is determined that the cabin door threshold of the aircraft cabin door is successfully recognized; if the recognition result of the aircraft cabin door is that the aircraft cabin door image includes a cabin door gap area and a cabin door identification area, but does not include a cabin door threshold area, it is determined that the cabin door threshold of the aircraft cabin door is not successfully recognized; if the recognition result of the aircraft cabin door is that the aircraft cabin door image includes a cabin door gap area and a cabin door threshold area, but does not include a cabin door identification area, it is determined that the cabin door threshold of the aircraft cabin door is not successfully recognized; if the recognition result of the aircraft cabin door is that the aircraft cabin door image includes a cabin door threshold area and a cabin door identification area, but does not include a cabin door gap area, it is determined that the cabin door threshold of the aircraft cabin door is not successfully recognized.

[0051] S208. When the cabin door threshold of the aircraft cabin door is successfully recognized, extract the set of edge points of the aircraft cabin door entity. The set of edge points of the aircraft cabin door entity includes: the set of points on the upper edge of the cabin door threshold entity, the set of points on the left edge of the cabin door gap entity, and the set of points on the right edge of the cabin door gap entity.

[0052] It should be noted that the aircraft cabin door entity in the embodiments of the present disclosure is the actual physical aircraft cabin door corresponding to the acquired aircraft cabin door image. Among them, the set of points on the upper edge of the cabin door threshold entity is the set of all points at the upper edge position of the cabin door threshold entity; the set of points on the left edge of the cabin door gap entity is the set of all points at the left edge position of the cabin door gap entity; the set of points on the right edge of the cabin door gap entity is the set of all points at the right edge position of the cabin door gap entity.

[0053] S210. According to the set of edge points of the aircraft cabin door entity, fit the first line passing through the upper edge of the cabin door threshold entity, the second line passing through the left edge of the cabin door gap entity, and the third line passing through the right edge of the cabin door gap entity.

[0054] It should be noted that the first line passing through the upper edge of the cabin door threshold entity in the embodiments of the present disclosure is a line fitted based on the set of points on the upper edge of the cabin door threshold entity, which can be a straight line or a curve. The second line passing through the left edge of the cabin door gap entity is a line fitted based on the set of points on the left edge of the cabin door gap entity, which can be a straight line or a curve. The third line passing through the right edge of the cabin door gap entity is a line fitted based on the set of points on the right edge of the cabin door gap entity, which can be a straight line or a curve.

[0055] S212. According to the first line, the second line, and the third line, obtain the first corner point coordinates and the second corner point coordinates, where the first corner point is the intersection point of the first line and the second line, and the second corner point is the intersection point of the first line and the third line.

[0056] It should be noted that in the embodiments of the present disclosure, the first corner point coordinate and the second corner point coordinate are determined by the intersection point of the first line passing through the upper edge of the cabin door sill entity and the second line passing through the left edge of the cabin door gap entity, and the intersection point of the first line passing through the upper edge of the cabin door sill entity and the third line passing through the right edge of the cabin door gap entity. Then, according to the positioning principle of binocular vision, the three-dimensional coordinates of the first corner point and the second corner point are deduced.

[0057] For the aircraft cabin door recognition and positioning method provided in the embodiments of the present disclosure, first, an image including the aircraft cabin door to be recognized is obtained; second, the obtained aircraft cabin door image is input into a pre-trained cabin door recognition model to determine whether there are a cabin door gap area, a cabin door identification area, and a cabin door sill area in the aircraft cabin door image; then, it is judged according to the recognition result of the aircraft cabin door to determine whether the cabin door sill of the aircraft cabin door is successfully recognized; after that, when the cabin door sill of the aircraft cabin door is successfully recognized, a set of points on the upper edge of the cabin door sill entity, a set of points on the left edge line of the cabin door gap entity, and a set of points on the right edge line of the cabin door gap entity are extracted; then, the first line passing through the upper edge of the cabin door sill entity, the second line passing through the left edge of the cabin door gap entity, and the third line passing through the right edge of the cabin door gap entity are fitted according to the set of edge points of the aircraft cabin door entity; finally, the left and right corner point coordinates are determined according to the three lines.

[0058] In the embodiments of the present disclosure, after determining that the aircraft cabin door is recognized, by extracting the set of edge points of the cabin door entity, the first line passing through the upper edge of the cabin door sill entity, the second line passing through the left edge of the cabin door gap entity, and the third line passing through the right edge of the cabin door gap entity are fitted. Finally, two intersection points, namely the first corner point and the second corner point, are obtained according to the three lines. In the embodiments of the present disclosure, three targets, namely the cabin door gap area, the cabin door identification area, and the cabin door sill area, are simultaneously determined according to the recognition result of the aircraft cabin door. By comprehensively using the three targets to infer the corner point coordinates, compared with a single target, the success rate and credibility of the corner point coordinate inference in the embodiments of the present disclosure are greatly improved.

[0059] In some embodiments, judging and determining whether the cabin door sill of the aircraft cabin door is successfully recognized according to the recognition result of the aircraft cabin door includes: after the cabin door recognition model recognizes that the aircraft cabin door image includes a cabin door gap area, a cabin door identification area, and a cabin door sill area, determining whether the cabin door identification area and the cabin door sill area are in standard positions; if the cabin door identification area and the cabin door sill area are in standard positions, it is determined that the cabin door sill of the aircraft cabin door is successfully recognized. By determining whether the aircraft cabin door image includes a cabin door gap area, a cabin door identification area, and a cabin door sill area, and determining the positions of the cabin door gap area, the cabin door identification area, and the cabin door sill area to comprehensively infer the position of the cabin door sill of the aircraft cabin door, compared with a single target, the success rate and credibility of the inference in the embodiments of the present disclosure are greatly improved.

[0060] In some embodiments, if the door identification area and the door threshold area are in the standard position, it is determined that the door threshold of the aircraft door is successfully identified, including: the door identification area is in the door gap area; and / or the door threshold area is below the door identification area; and / or the deviation between the door threshold area and the center line of the door identification area is less than or equal to a first preset threshold; and / or the distance between the door threshold area and the door identification area is less than or equal to a second preset threshold; and / or the difference between the left and right widths of the door identification area and the left and right widths of the door threshold area is less than or equal to a third preset threshold. By determining the positions of the door gap area, the door identification area and the door threshold area in the aircraft door image, the accuracy of the position of the aircraft door threshold is further ensured.

[0061] Specifically, in the embodiment of the present disclosure, the first preset threshold can be set to 100 mm, the second preset threshold can be set to 100 mm, and the third preset threshold can be set to 10%. After determining that the aircraft door image includes a door identification area, a door threshold area and a door gap area, the positional relationship between the door identification area, the door threshold area and the door gap area is judged. If the door identification area is within the door gap area, the door threshold area is below the door identification area, the deviation between the center line of the door threshold area and the door identification area does not exceed 100 mm, and the distance between the door threshold area and the door identification area is within 100 mm, and the width difference between the door gap area, the door identification area and the door threshold area does not exceed 10%, then it is determined that the door threshold of the aircraft door is successfully identified.

[0062] In more detail, if the door identification area is not within the door gap area, an alarm will be issued: the aircraft door identification failed and the position of the door identification area is abnormal; if the door threshold area is not below the door identification area, or the distance between the door threshold area and the door identification area is not within the second preset threshold (100mm), an alarm will be issued: the aircraft door identification failed and the position of the door threshold area is abnormal; if the centerline deviation between the door threshold area and the door identification area exceeds the third preset threshold (100mm), or the width difference between the door gap area, the door identification area and the door threshold area exceeds the third preset threshold (10%), an alarm will be issued: the aircraft door identification failed and the position of the door threshold area is abnormal.

[0063] In some embodiments, Figure 3 As shown, in the embodiment of the present disclosure, judging whether the door threshold of the aircraft cabin door is successfully identified according to the identification result of the aircraft cabin door specifically includes the following steps:

[0064] S302, acquiring an image of an aircraft door, after the boarding bridge's pick-up gate reaches the front of the aircraft door, starting aircraft door recognition, starting the camera group to take pictures continuously, and storing them in an image cache area.

[0065] S304, aircraft door area detection, read the latest image in the image buffer, perform target detection, and extract the image within the target detection frame as the ROI (Region Of Interest) image.

[0066] S306, reading the door recognition model established through training.

[0067] S308, multi-task segmentation, segmenting the door threshold entity, the door entity and the door gap entity.

[0068] S310, determining whether the aircraft door area is successfully identified.

[0069] S3102, whether the door gap entity is successfully segmented from the acquired image, if yes, go to S3104, otherwise an alarm is given: the aircraft door recognition failed, and the door gap is not found.

[0070] S3104, whether the cabin door entity is successfully segmented from the collected image, if yes, go to S3106, otherwise an alarm prompts: aircraft cabin door recognition failed, the cabin door was not found.

[0071] S3106, whether the door threshold entity is successfully segmented from the collected image, if yes, go to S3108, otherwise an alarm prompts: aircraft door recognition failed, and the door threshold is not found.

[0072] S3108, whether the door identification area is within the border area where the door threshold area is located, if yes, go to S3110, otherwise, an alarm prompts: aircraft door recognition failed, and the door identification position is abnormal.

[0073] S3110: Is the upper frame of the door threshold area below the frame of the door identification area and within 100mm relative to the lower frame of the door identification area? If so, go to S3112; otherwise, an alarm prompts: aircraft door recognition failed and the door threshold position is abnormal.

[0074] S3112, the center line deviation between the frame of the door threshold area and the frame of the door identification area does not exceed 100mm, and the difference between the left and right widths of the door identification area and the left and right widths of the door threshold area does not exceed 10%. If so, the aircraft door recognition and segmentation are successful, otherwise an alarm prompts: aircraft door recognition failed, and the door threshold position is abnormal.

[0075] S312, aircraft door recognition and segmentation are successful.

[0076] It should be noted that the first preset threshold, the second preset threshold, and the third preset threshold in the embodiments of the present disclosure are not limited to 100 mm, 100 mm, and 10% exemplified above. Those skilled in the art can flexibly set the first preset threshold, the second preset threshold, and the third preset threshold according to the actual situation.

[0077] In some embodiments, according to the set of edge points of the aircraft door entity, the first line passing through the upper edge of the door sill entity, the second line passing through the left edge of the door gap entity, and the third line passing through the right edge of the door gap entity are fitted, including: extracting the set of points on the upper edge of the door sill entity and fitting the first line passing through the upper edge of the door sill entity; extracting the set of points on the left edge of the door gap entity and fitting the second line passing through the left edge of the door gap entity; extracting the set of points on the right edge of the door gap entity and fitting the third line passing through the right edge of the door gap entity. By fitting the first line passing through the upper edge of the door sill entity, the second line passing through the left edge of the door gap entity, and the third line passing through the right edge of the door gap entity, the position of the upper edge of the door sill entity, the position of the left edge of the door gap, and the position of the right edge of the door gap can be obtained more accurately, further ensuring that more accurate door sill information can be obtained based on the accurate door edge information subsequently.

[0078] More specifically, in actual situations, the aircraft door will be built into a rounded rectangular shape. Therefore, the second line passing through the left edge of the door gap entity and the third line passing through the right edge of the door gap entity fitted in the embodiments of the present disclosure can be curves.

[0079] In some embodiments, according to the first line, the second line, and the third line, the first corner point coordinates and the second corner point coordinates are obtained, including: obtaining the plane coordinates of the first corner point according to the simultaneous equations of the first line and the second line; obtaining the plane coordinates of the second corner point according to the simultaneous equations of the first line and the third line; based on the positioning principle of binocular vision, calculating the three-dimensional coordinates of the first corner point and the three-dimensional coordinates of the second corner point of the door sill according to the plane coordinates of the first corner point coordinates and the plane coordinates of the second corner point coordinates. In the embodiments of the present disclosure, it is determined whether the aircraft door image includes three targets: the door gap area, the door identification area, and the door sill area. When it is determined that all three targets exist in the obtained aircraft door image, the coordinate information of the door sill of the aircraft door is inferred and determined by comprehensively using the three targets, which can greatly improve the inference success rate and credibility of the coordinate information of the door sill of the aircraft door.

[0080] More specifically, by solving the simultaneous equations of the upper edge of the cabin door sill entity, the left edge line of the cabin door gap entity, and the right edge line of the cabin door gap entity, two intersection points between the three lines are obtained. Since it is binocular vision, the same method is used to obtain the plane coordinates of two different intersection points in two cameras respectively. Then, according to the positioning principle of binocular vision, the stereo coordinates of the first corner point and the second corner point in the three-dimensional space are deduced.

[0081] Specifically, as Figure 4 shown, the steps for the embodiments of the present disclosure to determine the first corner point and the second corner point include the following:

[0082] S402, extract the point set of the upper edge of the cabin door sill entity;

[0083] S404, fit the first line A passing through the upper edge of the cabin door sill entity based on the point set of the upper edge of the cabin door sill entity;

[0084] S406, extract the point set of the left edge of the cabin door gap entity;

[0085] S408, fit the second line B passing through the left edge of the cabin door gap entity based on the point set of the left edge of the cabin door gap entity;

[0086] S410, extract the point set of the right edge of the cabin door gap entity;

[0087] S412, fit the third line C passing through the right edge of the cabin door gap entity based on the point set of the right edge of the cabin door gap entity;

[0088] S414, solve the simultaneous equations of the first line A passing through the upper edge of the cabin door sill entity, the second line B passing through the left edge of the cabin door gap entity, and the third line C passing through the right edge of the cabin door gap entity to obtain the plane coordinates of the intersection points between the three lines. Using the same method, the plane coordinates of two different intersection points are obtained in two cameras respectively: D 11 (X 11 , Y 11 ), D 12 (X 12 , Y 12 ), D 21 (X 21 , Y 21 ), D 22 (X 22 , Y 22 );

[0089] S416, according to the positioning principle of binocular vision, deduce the stereo coordinates D1(X1, Y1, Z1), D2(X2, Y2, Z2) of D1 and D2 in the three-dimensional space.

[0090] More specifically, through the aircraft door recognition and positioning method in the embodiments of the present disclosure, the first corner point coordinates and the second corner point coordinates as shown in Figure 5 are obtained.

[0091] In some embodiments, if the door gap entity is not successfully segmented from the acquired aircraft door image in the embodiments of the present disclosure, an alarm prompt is given: aircraft door recognition fails, and the door gap entity is not found; if the door entity is not successfully segmented from the acquired aircraft door image, an alarm prompt is given: aircraft door recognition fails, and the door entity is not found; if the door sill entity is not successfully segmented from the acquired aircraft door image, an alarm prompt is given: aircraft door recognition fails, and the door sill entity is not found.

[0092] In some embodiments, the aircraft door recognition and positioning method in the embodiments of the present disclosure further includes: before inputting the acquired aircraft door image into a pre-trained door recognition model and outputting the recognition result of the aircraft door, obtaining an aircraft door image training set, where the aircraft door image training set includes multiple aircraft door images; annotating the door gap area, the door identification area, and the door sill area in the collected aircraft door images to obtain an aircraft door annotation data set corresponding to the aircraft door image training set; training a pre-constructed multi-task deep learning network model according to the aircraft door image training set and the aircraft door annotation data set to obtain a trained door recognition model. By using the multi-task network detection method, the training efficiency and generalization ability of the door recognition model can be improved.

[0093] In some embodiments, annotating the door gap area, the door identification area, and the door sill area in the collected aircraft door images to obtain an aircraft door annotation data set corresponding to the aircraft door image training set includes: using annotation software to annotate the door gap area, the door identification area, and the door sill area in the collected aircraft door images to obtain a door gap area annotation set, a door identification area annotation set, and a door sill area annotation set in the aircraft door image.

[0094] More specifically, the embodiments of the present disclosure obtain an aircraft door image training set, manually annotate the door gap area, the door identification area, and the door sill area in the aircraft door image, construct a multi-task deep learning neural network, and train a door recognition model based on the original aircraft door image training set and the aircraft door annotation data set, as shown in Figure 6 below. Specifically, it includes the following steps:

[0095] S602, obtain an aircraft door image training set. The acquired aircraft door images should cover as many aircraft models in service as possible, cover the liveries of all airlines, and cover images in different lighting scenarios such as sunny backlight, sunny frontlight with shadow, rain and snow weather, and weak light at night.

[0096] S604, Hatch Target Area Annotation Dataset: Open the images in the aircraft hatch image training set one by one, use annotation software to mark the aircraft hatch target in the image with a rectangular box, and save the annotation result as the aircraft hatch target detection label. The saved parameters include the picture path, border type, border position information (top-left and bottom-right pixel coordinates), label name, flag, etc. For example, according to the Figure 7 code shown, the image of the ROI is as Figure 8 shown.

[0097] S606, Hatch Gap Area Annotation Dataset: Open the images in the aircraft hatch image training set one by one, as Figure 9 shown, use annotation software to mark the hatch gap in the picture with multiple line segments, and save the annotation result as the hatch gap area label. The saved parameters include the picture path, binary mask of the aircraft hatch gap, label name, etc. Among them, the annotation result is as Figure 10 shown.

[0098] S608, Hatch Identification Area Annotation Dataset: Open the images in the aircraft hatch image training set one by one, as Figure 11 shown, use annotation software to mark the aircraft hatch identification in the picture with multiple line segments, and save the annotation result as the hatch identification area label. The saved parameters include the picture path, binary mask of the aircraft hatch identification, label name, etc. Among them, the annotation result is as Figure 12 shown.

[0099] S610, Hatch Threshold Area Annotation Dataset: Open the images in the aircraft hatch image training set one by one, as Figure 13 shown, use annotation software to mark the aircraft hatch threshold in the picture with multiple line segments, and save the annotation result as the hatch threshold area label. The saved parameters include the picture path, binary mask of the aircraft hatch threshold, label name, etc. Among them, the annotation result is as Figure 14 shown.

[0100] S612, Construct Hatch Recognition Model: Construct a multi-task learning network based on a deep neural network, and use data from multiple tasks for training at the same time. Use the constructed multi-task learning network, as Figure 15 shown, to train and learn to extract the hatch recognition model from the original image dataset and the annotation dataset.

[0101] It should be noted that multi-task learning is a machine learning method that, based on shared representations, learns multiple related tasks together. Multi-task learning is a type of derivative transfer learning method. The main task uses the domain-related information possessed by the training signals of related tasks as a derivation bias to improve the generalization effect of the main task. Multi-task learning involves multiple related tasks learning in parallel simultaneously, with gradients backpropagating in reverse. Multiple tasks help each other learn through underlying shared representations to improve the generalization effect. Simply put: multi-task learning learns multiple related tasks together, as Figure 16 shown, during the learning process, domain-related information learned is shared and complemented with each other through a shared representation in the shallow layer, promoting learning from each other and improving the generalization effect.

[0102] In some design examples, through the hatch recognition model of the present disclosure embodiment, the recognition result of the aircraft hatch as shown in Figure 17 can be obtained.

[0103] Based on the same inventive concept, the present disclosure embodiment also provides a threshold area recognition device for an aircraft hatch, as in the following embodiments. Since the principle of solving problems in this device embodiment is similar to that of the above method embodiment, the implementation of this device embodiment can refer to the implementation of the above method embodiment, and repeated parts will not be elaborated.

[0104] Figure 18 The following shows a schematic diagram of a threshold area recognition device for an aircraft hatch in the present disclosure embodiment, as Figure 18 shown, the device includes:

[0105] A hatch image acquisition module 180, configured to acquire an aircraft hatch image, where the aircraft hatch image includes: the aircraft hatch to be recognized;

[0106] A hatch recognition result output module 182, configured to input the acquired aircraft hatch image into a pre-trained hatch recognition model, and output the recognition result of the aircraft hatch. The hatch recognition model is used to recognize the following elements of the aircraft hatch and perform image entity segmentation: the hatch gap area, the hatch identification area, and the hatch threshold area;

[0107] A hatch recognition result determination module 184, configured to determine whether the hatch threshold of the aircraft hatch is successfully recognized according to the recognition result of the aircraft hatch;

[0108] A hatch entity edge point extraction module 186, configured to extract a set of edge points of the aircraft hatch entity when the hatch threshold of the aircraft hatch is successfully recognized. The set of edge points of the aircraft hatch entity includes: a set of points on the upper edge of the hatch threshold entity, a set of points on the left edge of the hatch gap entity, and a set of points on the right edge of the hatch gap entity;

[0109] The door entity edge point fitting module 188 is used to fit a first line passing through the upper edge of the door threshold entity, a second line passing through the left edge of the door seam entity, and a third line passing through the right edge of the door seam entity according to the edge point set of the aircraft door entity;

[0110] The coordinate generating module 190 is used to obtain the first corner point coordinates and the second corner point coordinates according to the first line, the second line and the third line, wherein the first corner point is the intersection point of the first line and the second line, and the second corner point is the intersection point of the first line and the third line.

[0111] The device for identifying the threshold area of an aircraft cabin door provided in an embodiment of the present disclosure inputs the acquired aircraft cabin door image into a pre-trained cabin door recognition model through a cabin door image acquisition module and a cabin door entity edge point extraction module, outputs whether there is a cabin door gap area, a cabin door identification area and a cabin door threshold area in the aircraft cabin door image, and determines whether the cabin door threshold of the aircraft cabin door is successfully identified according to the identification result of the aircraft cabin door through a cabin door recognition result judgment module; then, through a cabin door entity edge point extraction module and a cabin door entity edge point fitting module, if the cabin door threshold of the aircraft cabin door is successfully identified, an edge point set of the aircraft cabin door entity is extracted, and a first line passing through the upper edge of the cabin door threshold entity, a second line passing through the left edge of the cabin door gap entity and a third line passing through the right edge of the cabin door gap entity are fitted according to the edge point set of the aircraft cabin door entity, and finally, the coordinates of the first corner point and the second corner point are determined through a coordinate generation module.

[0112] Compared with the related art, when the gap between the doors and the door sill of the aircraft cabin door in the picture is not clear enough, it is easy to have the problem of being unable to segment or segmenting incorrectly. In the embodiment of the present disclosure, after determining and identifying the aircraft cabin door, by extracting the edge point set of the cabin door entity, a first line passing through the upper edge of the cabin door sill entity, a second line passing through the left edge of the cabin door gap entity and a third line passing through the right edge of the cabin door gap entity are fitted, and finally two left and right intersection points are obtained according to the three lines. The embodiment of the present disclosure simultaneously determines three targets of the cabin door gap area, the cabin door identification area and the cabin door sill area according to the recognition result of the aircraft cabin door, and comprehensively uses the three targets to infer the corner point coordinates. Compared with a single target, the success rate and credibility of the inference of the corner point coordinates of the cabin door sill in the embodiment of the present disclosure are greatly improved.

[0113] In some embodiments, the door recognition result judgment module in the embodiments of the present disclosure is also used to determine whether the door identification area and the door threshold area are in the standard position after the door recognition model recognizes that the aircraft door image includes: the door gap area, the door identification area and the door threshold area; if the door identification area and the door threshold area are in the standard position, it is determined that the door threshold of the aircraft door is successfully recognized.

[0114] In some embodiments, the hatch identification result judgment module in the embodiments of the present disclosure is further configured to: the hatch identification area is within the hatch gap area; and / or, the hatch sill area is below the hatch identification area; and / or, the deviation between the center line of the hatch sill area and the hatch identification area is less than or equal to a first preset threshold; and / or, the distance between the hatch sill area and the hatch identification area is less than or equal to a second preset threshold; and / or, the difference in the width in the left-right direction of the hatch identification area and the width in the left-right direction of the hatch sill area is less than or equal to a third preset threshold.

[0115] In some embodiments, the hatch physical edge point fitting module in the embodiments of the present disclosure is further configured to extract the set of points on the upper edge of the hatch sill entity, and fit to obtain a first line passing through the upper edge of the hatch sill entity; extract the set of points on the left edge of the hatch gap entity, and fit to obtain a second line passing through the left edge of the hatch gap entity; extract the set of points on the right edge of the hatch gap entity, and fit to obtain a third line passing through the right edge of the hatch gap entity.

[0116] In some embodiments, the coordinate generation module in the embodiments of the present disclosure is further configured to obtain the planar coordinates of the first corner point according to the simultaneous equations of the first line and the second line; obtain the planar coordinates of the second corner point according to the simultaneous equations of the first line and the third line; based on the planar coordinates of the first corner point and the planar coordinates of the second corner point, calculate the three-dimensional coordinates of the first corner point and the three-dimensional coordinates of the second corner point of the hatch sill based on the positioning principle of binocular vision.

[0117] In some embodiments, the threshold area identification device for an aircraft hatch in the embodiments of the present disclosure further includes: a training set image acquisition module, configured to acquire an aircraft hatch image training set before inputting the acquired aircraft hatch image into a pre-trained hatch identification model and outputting the identification result of the aircraft hatch, where the aircraft hatch image training set includes multiple aircraft hatch images; a labeling set generation module, configured to label the hatch gap area, the hatch identification area, and the hatch sill area in the collected aircraft hatch images to obtain an aircraft hatch labeling data set corresponding to the aircraft hatch image training set; a model training module, configured to train a pre-constructed multi-task deep learning network model according to the aircraft hatch image training set and the aircraft hatch labeling data set to obtain a trained hatch identification model.

[0118] In some embodiments, the labeling set generation module in the embodiments of the present disclosure is further configured to use labeling software to label the hatch gap area, the hatch identification area, and the hatch sill area in the collected aircraft hatch images to obtain a hatch gap area labeling set, a hatch identification area labeling set, and a hatch sill area labeling set in the aircraft hatch image.

[0119] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method, or a program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "system" here.

[0120] In an exemplary embodiment of the present disclosure, an electronic device is further provided. The electronic device includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the aircraft door identification and positioning method according to any one of the above by executing the executable instructions.

[0121] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described here can be implemented by software, or can be implemented by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0122] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer program product, which includes: a computer program, and when the computer program is executed by a processor, it implements the above aircraft door identification and positioning method.

[0123] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is further provided. The computer-readable storage medium can be a readable signal medium or a readable storage medium. A program product capable of implementing the above method of the present disclosure is stored thereon. In some possible implementation manners, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of this specification.

[0124] More specific examples of the computer-readable storage medium in the present disclosure may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0125] In the present disclosure, a computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0126] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0127] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, executed as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0128] It should be noted that although several modules or units of a device for action execution are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more of the above-described modules or units may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided and embodied by multiple modules or units.

[0129] In addition, although the steps of the methods in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in that specific order, or that all of the steps shown must be performed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0130] From the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (such as a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0131] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the appended claims.

Claims

1. An aircraft cabin door identification and positioning method, characterized in that, Including: Obtain an image of an aircraft cabin door, where the image of the aircraft cabin door includes: the aircraft cabin door to be recognized; Input the obtained image of the aircraft cabin door into a pre-trained cabin door recognition model, and output the recognition result of the aircraft cabin door. The cabin door recognition model is used to recognize the following elements of the aircraft cabin door and perform image entity segmentation: the cabin door gap area, the cabin door identification area, and the cabin door threshold area; Judge and determine whether the cabin door threshold of the aircraft cabin door is successfully recognized according to the recognition result of the aircraft cabin door; When the cabin door threshold of the aircraft cabin door is successfully recognized, extract the set of edge points of the aircraft cabin door entity. The set of edge points of the aircraft cabin door entity includes: the set of points on the upper edge of the cabin door threshold entity, the set of points on the left edge of the cabin door gap entity, and the set of points on the right edge of the cabin door gap entity; According to the set of edge points of the aircraft cabin door entity, fit to obtain a first line passing through the upper edge of the cabin door threshold entity, a second line passing through the left edge of the cabin door gap entity, and a third line passing through the right edge of the cabin door gap entity; According to the first line, the second line, and the third line, obtain the first corner point coordinates and the second corner point coordinates, where the first corner point is the intersection of the first line and the second line, and the second corner point is the intersection of the first line and the third line.

2. The method for identifying and positioning an aircraft cabin door according to claim 1, wherein, Judging and determining whether the cabin door threshold of the aircraft cabin door is successfully recognized according to the recognition result of the aircraft cabin door includes: After the cabin door recognition model recognizes that the aircraft cabin door image includes: the cabin door gap area, the cabin door identification area, and the cabin door threshold area, determine whether the cabin door identification area and the cabin door threshold area are in the standard positions; If the cabin door identification area and the cabin door threshold area are in the standard positions, determine that the cabin door threshold of the aircraft cabin door is successfully recognized.

3. The aircraft cabin door recognition and positioning method according to claim 2, characterized in that, If the cabin door identification area and the cabin door threshold area are in the standard positions, determining that the cabin door threshold of the aircraft cabin door is successfully recognized includes: The cabin door identification area is within the cabin door gap area; And / or, the cabin door threshold area is below the cabin door identification area; And / or, the deviation of the center line of the cabin door threshold area from the cabin door identification area is less than or equal to a first preset threshold; And / or, the distance between the cabin door threshold area and the cabin door identification area is less than or equal to a second preset threshold; And / or, the difference in the width in the left-right direction between the cabin door identification area and the width in the left-right direction of the cabin door threshold area is less than or equal to a third preset threshold.

4. The aircraft cabin door recognition and positioning method according to claim 1, wherein According to the set of edge points of the aircraft cabin door entity, fitting to obtain a first line passing through the upper edge of the cabin door threshold entity, a second line passing through the left edge of the cabin door gap entity, and a third line passing through the right edge of the cabin door gap entity, includes: Extract the set of points on the upper edge of the cabin door threshold entity, and fit to obtain a first line passing through the upper edge of the cabin door threshold entity; Extract the set of points on the left edge of the cabin door gap entity, and fit to obtain a second line passing through the left edge of the cabin door gap entity; Extract the set of points on the right edge of the cabin door gap entity, and fit to obtain a third line passing through the right edge of the cabin door gap entity.

5. The method for identifying and positioning an aircraft cabin door according to claim 4, wherein Obtaining the first corner point coordinates and the second corner point coordinates according to the first line, the second line, and the third line includes: Obtaining the planar coordinates of the first corner point according to the simultaneous equations of the first line and the second line; Obtaining the planar coordinates of the second corner point according to the simultaneous equations of the first line and the third line; Calculating the three-dimensional coordinates of the first corner point and the three-dimensional coordinates of the second corner point based on the positioning principle of binocular vision according to the planar coordinates of the first corner point and the planar coordinates of the second corner point.

6. The aircraft cabin door identification and positioning method according to claim 1, wherein, Before inputting the obtained aircraft door image into a pre-trained door recognition model and outputting the recognition result of the aircraft door, the method further includes: Obtaining an aircraft door image training set, where the aircraft door image training set includes a plurality of aircraft door images; Labeling the door gap area, the door identification area, and the door threshold area in the collected aircraft door images to obtain an aircraft door annotation data set corresponding to the aircraft door image training set; Training a pre-constructed multi-task deep learning network model according to the aircraft door image training set and the aircraft door annotation data set to obtain a trained door recognition model.

7. The aircraft cabin door recognition and positioning method according to claim 6, wherein Labeling the door gap area, the door identification area, and the door threshold area in the collected aircraft door images to obtain an aircraft door annotation data set corresponding to the aircraft door image training set, including: Using annotation software to label the door gap area, the door identification area, and the door threshold area in the collected aircraft door images to obtain a door gap area annotation set, a door identification area annotation set, and a door threshold area annotation set in the aircraft door image.

8. An aircraft cabin door recognition and positioning device, characterized in that, Including: An aircraft door image acquisition module, configured to acquire an aircraft door image, where the aircraft door image includes: an aircraft door to be recognized; An aircraft door recognition result output module, configured to input the obtained aircraft door image into a pre-trained door recognition model and output the recognition result of the aircraft door. The door recognition model is used to recognize the following elements of the aircraft door and perform image entity segmentation: the door gap area, the door identification area, and the door threshold area; An aircraft door recognition result judgment module, configured to judge and determine whether the door threshold of the aircraft door is successfully recognized according to the recognition result of the aircraft door; An aircraft door entity edge point extraction module, configured to extract a set of edge points of the aircraft door entity when the door threshold of the aircraft door is successfully recognized. The set of edge points of the aircraft door entity includes: a set of points on the upper edge of the door threshold entity, a set of points on the left edge of the door gap entity, and a set of points on the right edge of the door gap entity; An aircraft door entity edge point fitting module, configured to fit, according to the set of edge points of the aircraft door entity, to obtain a first line passing through the upper edge of the door threshold entity, a second line passing through the left edge of the door gap entity, and a third line passing through the right edge of the door gap entity. A coordinate generation module, configured to obtain first corner coordinates and second corner coordinates according to the first line, the second line, and the third line, where the first corner is the intersection point of the first line and the second line, and the second corner is the intersection point of the first line and the third line.

9. An electronic device, characterized in that, Comprising: A processor; And A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the aircraft door recognition and positioning method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the aircraft door recognition and positioning method according to any one of claims 1 to 7.

Images