Object recognition method and system based on deep learning

Abstract

The application discloses a kind of object identification method and system based on deep learning, it is related to artificial intelligence, target identification and other technical fields.Object identification method based on deep learning includes: obtaining building drawing file;The building drawing file is analyzed and processed, and drawing analysis data is obtained;Based on the first object area segmentation line, object area segmentation processing is carried out to drawing analysis data, and the first object area is marked in drawing analysis data, and drawing mark data is obtained;Based on the second object area mark information, image processing is carried out to drawing mark data, and the second object candidate area is determined;The second object candidate area is identified and processed by deep learning, and the second object identification result is determined.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence, object recognition and other technical fields, and in particular to an object recognition method and system based on deep learning. Background Technology

[0002] In the process of building digitization and building information modeling (BIM), it is necessary to automatically identify and classify different functional spaces in the building floor plan. The accurate division of public areas (such as stairwells, elevator halls, corridors, lobbies, etc.) is a key preliminary step for subsequent building reconstruction, property management, energy consumption analysis, and safety evacuation simulation. Therefore, how to accurately identify public areas has become the focus of research.

[0003] Related technologies typically involve manual identification and labeling of common areas within a modeling platform by observing elements such as walls, doorways, symbols, and text annotations in CAD (Computer-Aided Design) drawings. This process is inefficient and prone to omissions and misjudgments. Another related technology uses deep learning for image processing, converting CAD drawings into images and then automatically identifying them using general object detection or semantic segmentation models. However, general models often simultaneously identify multiple objects such as rooms, walls, and openings, lacking specific optimization for common area objects, resulting in lower recognition accuracy. Furthermore, the training set requires detailed annotation of the entire drawing, leading to high costs. Summary of the Invention

[0004] The embodiments of this application aim to at least partially solve one of the technical problems in the related art. Therefore, the purpose of the embodiments of this application is to propose a deep learning-based object recognition method, system, device, and medium, which improves the accuracy and efficiency of object recognition.

[0005] This application provides a deep learning-based object recognition method, comprising: acquiring architectural drawing files; parsing the architectural drawing files to obtain drawing parsing data; performing object region segmentation processing on the drawing parsing data based on a first object region segmentation line, and marking the first object region in the drawing parsing data to obtain drawing marking data; performing image processing on the drawing marking data based on second object region marking information to determine second object candidate regions; and performing deep learning recognition processing on the second object candidate regions to determine the second object recognition result.

[0006] For example, based on the second object region marking information, image processing is performed on the drawing marking data to determine the second object candidate region, including: removing the first object region marked in the drawing marking data, and merging or splitting the object regions initially marked by the second object region marking information to obtain the processed drawing marking data; and cropping the processed drawing marking data to obtain the second object candidate region. For example, performing deep learning recognition processing on the second object candidate region to determine the second object recognition result includes: performing multi-channel encoding processing on the second object candidate region to obtain encoded second object candidate regions; performing feature extraction processing on the encoded second object candidate region to obtain feature data; performing feature fusion processing on the feature data to obtain fused feature data; performing attention processing on the fused feature data to obtain attention feature data; and performing recognition processing on the attention feature data to determine the second object recognition result.

[0007] For example, multi-channel encoding processing is performed on the second object candidate region to obtain the encoded second object candidate region, including: performing first-channel encoding processing on the second object candidate region to obtain structural data; performing second-channel encoding processing on the second object candidate region to obtain symbolic data; performing third-channel encoding processing on the second object candidate region to obtain semantic data; performing fourth-channel encoding processing on the second object candidate region to obtain mask data; and obtaining the encoded second object candidate region based on the structural data, symbolic data, semantic data, and mask data. For example, performing feature fusion processing on feature data to obtain fused feature data includes: performing feature fusion processing on feature data to obtain intermediate feature data; and performing feature connection processing based on the intermediate feature data and the feature data to obtain fused feature data. For example, attention processing is performed on the fused feature data to obtain attention feature data, including: mapping the mask data to obtain attention weight data of the same size as the fused feature data; and multiplying the fused feature data based on the attention weight data to obtain attention feature data. For example, the recognition processing of attention feature data to determine the second object recognition result includes: performing classification processing and bounding box regression processing on the attention feature data to obtain candidate boxes, candidate categories and confidence data; performing non-maximum suppression processing based on the candidate boxes and confidence data to obtain target candidate boxes; performing mask calculation on each pixel of the target candidate box to obtain a mask image; and performing smoothing and mapping processing on the mask image to obtain the second object recognition result.

[0008] For example, the deep learning-based object recognition method further includes: in response to receiving region labeling feedback information for the second object recognition result, determining local incremental data from the drawing labeling data based on the region labeling feedback information; and performing deep learning recognition processing on the local incremental data to correct the second object recognition result.

[0009] For example, the object recognition method based on deep learning is applied to a deep learning model. The object recognition method based on deep learning further includes: using the second object candidate region with the candidate category in the second object candidate region as the target category as a positive sample, and using the second object candidate region with the candidate category in the second object candidate region as the non-target category as a negative sample, and training the deep learning model based on the classification loss function, the bounding box regression loss function, the mask loss function, the boundary loss function and the constraint rules.

[0010] Another embodiment of this application provides a deep learning-based object recognition system, comprising: an acquisition module for acquiring architectural drawing files; a parsing module for parsing the architectural drawing files to obtain drawing parsing data; a segmentation module for segmenting the drawing parsing data into object regions based on a first object region segmentation line, and marking the first object region in the drawing parsing data to obtain drawing marking data; a processing module for performing image processing on the drawing marking data based on second object region marking information to determine second object candidate regions; and a recognition module for performing deep learning recognition processing on the second object candidate regions to determine the second object recognition result.

[0011] Another embodiment of this application provides an electronic device having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0012] Another embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0013] In the above embodiments, the object recognition method based on deep learning includes: acquiring architectural drawing files; parsing the architectural drawing files to obtain drawing parsing data; segmenting the drawing parsing data into object regions based on a first object region segmentation line, and marking the first object region in the drawing parsing data to obtain drawing marking data; performing image processing on the drawing marking data based on second object region marking information to determine second object candidate regions; and performing deep learning recognition processing on the second object candidate regions to determine the second object recognition result. By acquiring and parsing architectural drawing files to obtain structured drawing data, achieving accurate primary object region segmentation and marking based on the first object region segmentation line, combining the second object region marking information for image processing to quickly locate the second object candidate regions, and finally accurately determining the type of the second object through deep learning recognition processing, the search space is reduced while maintaining the model complexity, significantly reducing the false recognition rate, improving the recall and accuracy of object recognition, and increasing the efficiency of object recognition. Attached Figure Description

[0014] Figure 1 A flowchart of a deep learning-based object recognition method provided for embodiments of this application; Figure 2 A flowchart for deep learning recognition of a second object candidate region provided in an embodiment of this application; Figure 3 A block diagram of a deep learning-based object recognition system provided for another embodiment of this application; Figure 4 Flowchart of another object recognition method based on deep learning provided for the implementation of this application; Figure 5 A block diagram of an electronic device provided for another embodiment of this application. Detailed Implementation

[0015] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0016] In the process of building digitization and building information modeling (BIM), it is necessary to automatically identify and classify different functional spaces in the building floor plan. The accurate division of public areas (such as stairwells, elevator halls, corridors, lobbies, etc.) is a key preliminary step for subsequent building reconstruction, property management, energy consumption analysis, and safety evacuation simulation. Therefore, how to accurately identify public areas has become the focus of research.

[0017] In building digitization applications, public area identification and extraction is one of the fundamental data sources for various business scenarios such as space management, asset management, fire evacuation, and energy consumption simulation. A typical process involves users uploading architectural CAD drawings (such as DWG (Drawing) or DXF (Drawing Exchange Format)) to the system, followed by operators manually defining and marking the locations of public areas within the modeling platform. This process requires repeated observation of various elements, including wall lines, doors and windows, risers, elevator symbols, stair symbols, and text annotations, and inferring the space's purpose and boundaries based on experience. Due to significant differences in drawing standards among different design institutes and projects, public area objects are often intertwined with room, structural component, and equipment symbols. Relying solely on manual identification is not only complex and time-consuming but also prone to omissions and misjudgments, making it difficult to support the automated processing needs of large-scale, multi-batch drawings.

[0018] The above-mentioned manual labeling of public areas is problematic because CAD drawing practices and layer specifications vary significantly between different design firms and projects. The same type of public area may be represented by different layer names, colors, line types, and symbols on different drawings. Therefore, manual interpretation often requires considerable experience, and the average time to identify public areas on a complex architectural drawing can reach tens of minutes or even longer. For scenarios involving a large number of buildings or continuously updated drawings, relying solely on manual operation is insufficient to meet efficiency requirements.

[0019] Meanwhile, with the popularization of deep learning in the field of image processing, related methods attempt to convert CAD drawings into images and then use general object detection or semantic segmentation models for automatic recognition. However, general models often recognize multiple objects such as rooms, walls, and openings simultaneously, lacking specific optimization for common areas, and the training set requires detailed annotation of the entire drawing, which is costly.

[0020] Related methods rely on identifying public areas from the entire image before filtering. This involves converting the entire building plan into an image, using image processing models (such as general object detection or segmentation models) to infer the entire image, outputting candidate regions for various spaces or components, and then filtering public area objects from the inference results through rules or post-processing logic. However, this method is not specifically optimized for public area objects: the model needs to learn multiple object types simultaneously (rooms, public areas, equipment, etc.), and public area objects constitute a limited proportion of the overall sample, making them susceptible to interference from other categories, resulting in insufficient recall and precision for public area identification. Since the model needs to search for public areas across the entire drawing, it is prone to misclassifying some large rooms and areas with concentrated equipment as public areas, requiring extensive manual review and filtering, leading to a high false recognition rate. Even if the model can provide candidate regions, operators still need to judge each one individually to determine if it is a true public area, resulting in limited overall efficiency improvement and time-consuming manual filtering.

[0021] Another related method involves classifying and labeling all objects in the entire drawing before filtering out public areas. This method labels all spaces or objects in the entire drawing as categories, such as rooms, corridors, and stairwells, and then selects objects labeled as public areas as the public area identification results. However, this method requires detailed labeling of all space or object types in the entire drawing, which is a huge amount of work and makes it difficult to continuously accumulate high-quality training data. The cost of building the training set is also high. Furthermore, the uses of some spaces in different projects are not clearly expressed, and it is difficult to distinguish whether they are public areas or rooms based on local geometry alone. This can lead to confusion in the classification model and limit the accuracy of object classification. In addition, the method is not "focused" on public area identification: the training objective of the model is biased towards the overall object classification rather than optimizing for the specific goal of "how to better identify public areas".

[0022] The core technical problems to be solved in this application include: (1) how to automatically and accurately identify and extract public areas and public objects in architectural CAD drawings under the premise of differences in drawing specifications; (2) how to allow users to guide and supplement the recognition results with minimal interaction while reducing manual intervention, thereby improving the overall recognition accuracy and usability; (3) how to effectively reduce the training cost of the public area object recognition model by using specially designed training sample construction methods and model usage methods under the condition of limited training data acquisition costs.

[0023] The purpose of this application is to: (1) provide an object recognition method that combines graphical interface interaction with deep learning image processing. By having the user define the approximate range of the apartment area and public area in a low-cost manner before recognition, the model can only perform recognition within the candidate range of the public area, thereby improving the accuracy and recall rate of public area object recognition. (2) Through a local interactive secondary recognition mechanism, the user can initiate incremental recognition only on the local public area when missing or inaccurate areas are found, avoiding repeated reasoning of the whole image, improving overall efficiency and reducing computing power consumption. (3) Through a dedicated training set construction method for public area objects, only the public area and related objects are labeled, avoiding full labeling of the whole image, significantly reducing the cost of training data construction; at the same time, the results generated during user interaction are used to further optimize model performance.

[0024] In view of this, the embodiments of this application provide a deep learning-based object recognition method that takes architectural CAD drawings as input and automatically completes the following with minimal user intervention: segmentation of apartment layouts and public areas, determination of candidate public areas, automatic recognition and extraction of the outer contours of objects within the public areas, local incremental optimization and interactive correction of the recognition results, and standardized output of the recognition results.

[0025] Figure 1 A flowchart of a deep learning-based object recognition method provided for an embodiment of this application.

[0026] like Figure 1 As shown, the object recognition method 100 based on deep learning provided in this application includes, for example, steps S110-S150.

[0027] Step S110: Obtain architectural drawing files. For example, architectural drawing files, such as CAD files, include formats such as DWG (Drawing) and DXF (Drawing Exchange Format). Users can upload architectural drawing files by selecting or dragging and dropping files.

[0028] Step S120: Parse the architectural drawing file to obtain the drawing parsing data. For example, architectural drawing files are parsed into internal data structures, extracting geometric elements (such as structural elements like wall lines, axes, and floor slab outlines, and symbols like elevator symbols, stair symbols, and door and window symbols), layer information (such as layer names (WALL, AXIS, DOOR, TEXT, etc.), colors, line types, line widths, and layer visibility), and text annotations (such as element labels (used to describe the attributes of geometric elements), element dimensions, etc.). A planar image preview is generated for display, preserving the original scale and coordinate information of the drawing. The planar image preview is used as the drawing parsing data, and the validity of the drawing parsing data is checked (whether the drawing parsing data contains the necessary layer information, dimension annotations, and structural integrity) and coordinate mapping relationships are established (establishing the correspondence between CAD coordinates and the pixel coordinates of the planar image preview).

[0029] Step S130: Based on the first object region dividing line, the drawing parsing data is processed to divide the object region, and the first object region is marked in the drawing parsing data to obtain the drawing marking data. For example, the first object area includes the apartment layout area. The user draws the dividing line of the first object area on the drawing parsing data, performs calculations based on the dividing line of the first object area, marks the area enclosed by the dividing line of the first object area as the first object area, and uses the marked drawing parsing data as the drawing marking data. The dividing line may include multiple polylines or closed polygons.

[0030] Step S140: Based on the second object region marking information, perform image processing on the drawing marking data to determine the second object candidate region. For example, the second object area includes a public area. The user uses a selection tool to select one or more areas in the drawing mark data by dragging a rectangle or using a polygon selection tool to obtain the second object area mark information. Based on the second object area mark information, the drawing mark data is processed to remove the apartment areas within the selected area and retain only the areas that are actually located in the public area. Multiple areas are then merged, split, or cropped to obtain the second object candidate area. Step S150: Perform deep learning recognition processing on the candidate region of the second object to determine the recognition result of the second object. For example, deep learning recognition processing is implemented based on an instance segmentation model, which includes a convolutional neural network model. Deep learning recognition processing includes encoding, feature extraction, feature fusion, attention, and recognition processes. The second object recognition result includes the second object type, confidence data, geometric contour of the second object, and object ID (Identifier), etc.

[0031] In the above embodiments, structured drawing data is obtained by acquiring and parsing architectural drawing files. Based on the first object region segmentation line, accurate primary object region segmentation and marking are achieved. Then, image processing is performed in combination with the second object region marking information to quickly locate the second object candidate region. Finally, deep learning recognition processing is used to accurately determine the type of the second object. While keeping the model complexity unchanged, the search space is reduced, the false recognition rate is significantly reduced, and the recall rate and accuracy of object recognition are improved, as well as the efficiency of object recognition. In one example, image processing is performed on the drawing mark data based on the second object region marking information to determine the second object candidate region. This includes: removing the first object region marked in the drawing mark data, and merging or splitting the object regions initially marked by the second object region marking information to obtain the processed drawing mark data; and cropping the processed drawing mark data to obtain the second object candidate region. Specifically, users can drag a rectangle or use a polygon selection tool to select one or more areas on the drawing mark data and mark them as "public area outer contour candidate areas" (object areas initially marked by the second object area mark information). The selected area will automatically eliminate the apartment area (first object area) and only retain the part that is actually within the public area. Multiple areas can be merged or split to obtain public area candidate areas (processed drawing mark data). For each public area candidate area, it is cropped according to the bounding rectangle to generate a standard-sized image block (second object candidate area), and its coordinate position and scaling ratio in the CAD file are recorded at the same time.

[0032] In one example, deep learning recognition processing is performed on the candidate region of the second object to determine the recognition result of the second object, including: multi-channel encoding processing of the candidate region of the second object to obtain encoded candidate region of the second object; feature extraction processing of the encoded candidate region of the second object to obtain feature data; feature fusion processing of the feature data to obtain fused feature data; attention processing of the fused feature data to obtain attention feature data; and recognition processing of the attention feature data to determine the recognition result of the second object.

[0033] Specifically, deep learning recognition processing is implemented based on deep learning models, such as instance segmentation models. Instance segmentation models include: a feature extraction backbone (Backbone) + a Feature Pyramid Network (FPN) + a common area attention module + an instance segmentation head (Mask Head). Multi-channel encoding processing includes first-channel encoding, second-channel encoding, and possibly third-channel, fourth-channel encoding, etc., to obtain multi-channel tensors. Feature extraction processing is implemented based on the feature extraction backbone, which can employ residual network structures such as ResNet and ResNeXt. The feature extraction backbone performs convolution, non-linear activation, and normalization on the encoded second object candidate regions to obtain feature data. Feature fusion processing is implemented based on the Feature Pyramid Network, including top-down fusion and horizontal connection processing of the feature data to obtain fused feature data (feature pyramid). Attention processing is implemented based on the common area attention module, including multiplication of the fused feature data based on attention weights. Recognition processing is based on the instance segmentation head (Mask Head). The instance segmentation head is implemented by a detection head and a masking head, thereby obtaining the second object recognition result.

[0034] For example, convolution, normalization, and nonlinear activation are performed on the multi-channel tensor (the encoded second object candidate region) to extract semantic features at different scales, resulting in multi-layer feature maps {F1, F2, F3, F4} (feature data).

[0035] In the above embodiments, the instance segmentation model adopts a structure of feature extraction backbone + feature pyramid network (FPN) + common area attention module + instance segmentation head (mask head). The second object candidate region after multi-channel encoding is fed into the feature extraction backbone and feature pyramid network to obtain multi-scale feature maps. After being weighted by the common area attention module, it enters the instance segmentation head, which can simultaneously complete "common area object localization" and "object contour segmentation" within the same network.

[0036] In one example, multi-channel encoding is performed on the second object candidate region to obtain the encoded second object candidate region, including: performing first-channel encoding on the second object candidate region to obtain structural data; performing second-channel encoding on the second object candidate region to obtain symbolic data; performing third-channel encoding on the second object candidate region to obtain semantic data; performing fourth-channel encoding on the second object candidate region to obtain mask data; and obtaining the encoded second object candidate region based on the structural data, symbolic data, semantic data, and mask data. Specifically, the multi-channel encoding process includes first-channel encoding, second-channel encoding, third-channel encoding, and fourth-channel encoding. The first-channel encoding is the structure line channel, the second-channel encoding is the symbol channel, the third-channel encoding is the text channel, and the fourth-channel encoding is the mask channel.

[0037] For example, for each candidate region of the second object, the CAD data is first encoded into a multi-channel tensor: Structural Line Channel (C1): Structural primitives such as wall lines, axes, and floor outlines are rendered into the same channel, retaining only line type information related to spatial boundaries (structural data). Symbol Channel (C2): Public area-related symbols such as elevator symbols, stair symbols, and door / window symbols are rendered into independent channels to obtain symbol data, used to enhance the recognition of "functional cues" in public areas. Text / Annotation Channel (C3): Text, elevations, numbers, etc., are encoded into channels in the form of point density or heatmaps to provide semantic cues such as "corridor" and "elevator lobby" (semantic data). Public Area Candidate Mask Channel (C4): The candidate region of the second object is encoded as a 0 / 1 mask channel as attention guidance information, focusing on features only within the candidate region of the public area. Finally, each candidate region of the second object is encoded into a multi-channel tensor of size H×W×4, which serves as the input to the instance segmentation network (instance segmentation head).

[0038] In one example, feature fusion processing is performed on feature data to obtain fused feature data, including: performing feature fusion processing on feature data to obtain intermediate feature data; and performing feature concatenation processing based on the intermediate feature data and the feature data to obtain fused feature data. Specifically, the feature fusion process includes top-down fusion and lateral connection processing of the feature data to obtain fused feature data (Feature Pyramid Network (FPN)). Top-down fusion includes upsampling processing, and lateral connection includes feature alignment and element addition.

[0039] For example, top-down fusion passes high-level semantic information to low-level feature maps, improving the semantic expressiveness of low-level features. For instance, upsampling: starting from the top of the multi-layer feature maps (e.g., F4, the deepest and lowest-resolution feature map), upsampling (e.g., bilinear interpolation) enlarges feature map F4 to the same size as the next layer F3. The enlarged high-level features (the features obtained after upsampling F4) are then fused with the features of the next layer F3. This process of upsampling and fusion continues until the lowest layer F1.

[0040] Lateral connection refers to preserving the original spatial location information (such as edges and corners) of shallow features during the fusion process, preventing the blurring operation (upsampling) of higher-level features from destroying the details of lower-level features. For example, for feature alignment, a 1x1 convolution is performed on the shallow feature map F3, and its number of channels is adjusted to match the number of channels of the upsampled higher-level features (features obtained after upsampling F4). The adjusted shallow features and the upsampled higher-level features are then added element-wise to obtain the fused feature data (feature pyramid).

[0041] For example, top-down fusion and lateral connection of multi-layer features can be performed to generate a set of feature pyramids {P1, P2, P3, P4} with increasing resolution and decreasing semantic information from top to bottom, so as to take into account the recognition needs of large-scale public areas (such as lobbies and long corridors) and small-scale structures (such as stairwell entrances).

[0042] In one example, attention processing is performed on the fused feature data to obtain attention feature data, including: mapping the mask data to obtain attention weight data of the same size as the fused feature data; and multiplying the fused feature data based on the attention weight data to obtain attention feature data. Specifically, the public region attention module maps the public region candidate mask channel (C4) to an attention weight map (attention weight data) with the same resolution as {P1–P4}. Through element-wise multiplication or gating, it obtains attention feature data to suppress responses in the pyramid features located outside the public region, thus achieving location prior guidance. This module ensures that the public region is "known" at the feature level, thereby reducing misidentification of non-public region areas.

[0043] In one example, the attention feature data is processed to determine the second object recognition result, including: classifying and regressing the attention feature data to obtain candidate boxes, candidate categories, and confidence data; performing non-maximum suppression based on the candidate boxes and confidence data to obtain target candidate boxes; performing mask calculation on each pixel of the target candidate boxes to obtain a mask image; and performing smoothing and mapping processing on the mask image to obtain the second object recognition result.

[0044] Specifically, the instance segmentation head generates public area object candidate regions (candidate boxes) on each scale feature map (attention feature data) and outputs instance-level segmentation results (second object recognition results), which include: (1) Region proposal branch: predicting whether each candidate anchor box (candidate box) is a public area object, the object category (candidate category) and the bounding box offset; (2) Non-Maximum Suppression (NMS) is applied to the candidate boxes to remove highly overlapping and low-confidence candidate boxes, and retain a set of high-confidence candidate boxes as target candidate boxes; (3) Mask branch: predicting pixel-level masks within the target candidate boxes to obtain fine object outlines (mask maps). Smoothing processing includes morphological filtering, connected component analysis and boundary smoothing, and mapping processing includes converting to geometric outlines in CAD coordinate system according to coordinate mapping relationship to obtain second object recognition results. The mask map includes a binary map for boundary outline extraction.

[0045] For example, the instance segmentation head generates several candidate boxes at various scales and predicts the category (candidate category) (e.g., stairwell, elevator hall, corridor, lobby, etc.) and confidence score (confidence data) for each candidate box. Non-maximum suppression is applied to the candidate boxes to remove highly overlapping candidates with low confidence scores, resulting in target candidate boxes. For each remaining candidate box (target candidate box), a mask branch predicts a pixel-level mask within that candidate region. Morphological filtering, connected component analysis, and boundary smoothing are applied to the masks to remove small noise regions, and adjacent masks are merged to restore the complete shape of the common area object.

[0046] The boundary contour of the mask image in the image coordinate system is extracted into polygons (by processing the edge points in the mask image to obtain polygons), and converted into geometric contours in the CAD coordinate system according to the established coordinate mapping relationship; combined with the recognition category (candidate category) and confidence data, structured "public area object" data (second object recognition result) is constructed, including: object ID, category (candidate category), geometric contour, confidence data, etc.

[0047] Ultimately, all the recognition results (second object recognition results) are aggregated to form a set of public area objects on the entire drawing, and then overlaid and displayed on the interface.

[0048] Figure 2 A flowchart for deep learning recognition of a second object candidate region provided for an embodiment of this application is shown below. Figure 2 As shown, the deep learning recognition of the second object candidate region includes S201-S210.

[0049] S201, Obtain the candidate region for the second object.

[0050] S202, perform multi-channel coding of public areas.

[0051] For example, multi-channel encoding includes first-channel encoding, second-channel encoding processing, third-channel encoding processing, and fourth-channel encoding processing, wherein the first-channel encoding processing is a structure line channel, the second-channel encoding processing is a symbol channel, the third-channel encoding processing is a text channel, and the fourth-channel encoding processing is a mask channel.

[0052] S203 is the backbone network for feature extraction.

[0053] For example, feature extraction is implemented based on a feature extraction backbone network. The feature extraction backbone network can adopt residual network structures such as ResNet and ResNeXt. The feature extraction backbone network can obtain feature data by performing convolution, non-linear activation and normalization on the encoded second object candidate region.

[0054] S204, Construct the feature pyramid.

[0055] For example, feature fusion processing of feature data can yield a feature pyramid. Feature fusion processing is implemented based on a feature pyramid network, including top-down fusion of feature data and horizontal connection processing, thereby obtaining fused feature data (feature pyramid).

[0056] S205, applying public area attention weighting.

[0057] For example, public area attention weighting is applied to the fused feature data to obtain attention feature data.

[0058] S206, Perform instance segmentation inference.

[0059] For example, the instance segmentation head performs recognition processing on attention feature data.

[0060] S207, perform candidate box generation and filtering.

[0061] For example, the instance segmentation head generates candidate regions (candidate boxes) of common objects on each scale feature map (attention feature data), and performs non-maximum suppression processing based on the candidate boxes and confidence data to obtain the target candidate boxes. S208 completes mask generation and post-processing.

[0062] For example, a mask calculation is performed on each pixel of the target candidate box to obtain a mask image. S209, Perform coordinate mapping and object construction.

[0063] For example, the boundary contour of the mask image in the image coordinate system is extracted into a polygon (by processing the edge points in the mask image to obtain the polygon), and converted into a geometric contour in the CAD coordinate system according to the established coordinate mapping relationship; combined with the recognition category (candidate category) and confidence data, a structured "public area object" data (second object recognition result) is constructed, including: object ID, category (candidate category), geometric contour, confidence data, etc. S210, output the public area object recognition result.

[0064] For example, all the recognition results (second object recognition results) are uniformly summarized to form a set of public area objects on the entire drawing, and then overlaid and displayed on the interface. In the above embodiments, a specific instance segmentation model for public area objects is used for recognition. The model as a whole adopts a structure of "multi-channel CAD feature encoding + public area attention guidance + instance segmentation network" to improve the recognition accuracy and robustness in CAD public area scenarios.

[0065] In one example, the deep learning-based object recognition method further includes: in response to receiving region label feedback information for the second object recognition result, determining local incremental data from the drawing label data based on the region label feedback information; and performing deep learning recognition processing on the local incremental data to correct the second object recognition result.

[0066] Specifically, the interface overlays the outlines of identified public area objects (second object recognition results), allowing users to visually assess the recognition effectiveness. When a user discovers a missed or misidentified area, they can select that local area using a box or polygon selection (regional marker feedback information from the second object recognition results). The deep learning model is re-inferred only for the user-selected local area (local incremental data), and the original results are replaced or merged within that local area, without recalculating the entire drawing. Users can manually delete obviously misidentified objects (second object recognition results), or supplement attribute information (such as usage descriptions, numbers, or labels) for identified objects, and then save the corrected results again.

[0067] In the above embodiments, the user-guided local incremental recognition mechanism allows users to efficiently correct local results with minimal interaction cost while maintaining overall automation, significantly reducing the workload of manual global selection and individual labeling.

[0068] After the user confirms the recognition results, the data is exported in a unified manner: (1) All selected public area objects and their outer contours (second object recognition results) are exported in standard formats such as JSON and SVG. (2) Each object records its geometric outer contour, category, confidence level, and business attributes supplemented by the user. (3) The exported files can be directly called by subsequent building reconstruction modules, BIM modeling modules, spatial statistical analysis modules, and other systems.

[0069] Figure 3 A block diagram of a deep learning-based object recognition system provided for another embodiment of this application, such as... Figure 3 As shown, the deep learning-based object recognition system 300 includes: an acquisition module 310 (drawing import and preprocessing module), a parsing module 320 (drawing import and preprocessing module), a segmentation module 330 (apartment type-public area segmentation module), a processing module 340 (second object candidate region generation module), and a recognition module 350 (a deep learning-based public area object recognition module, a user feedback-based local incremental recognition and result correction module, and a recognition result display and standard format export module). Specifically, the acquisition module 310 acquires architectural drawing files; the parsing module 320 parses the architectural drawing files to obtain drawing parsing data; the segmentation module 330 segments the drawing parsing data into object regions based on a first object region segmentation line and marks the first object region in the drawing parsing data to obtain drawing mark data; the processing module 340 performs image processing on the drawing mark data based on the second object region mark information to determine second object candidate regions; and the recognition module 350 performs deep learning recognition processing on the second object candidate regions to determine the second object recognition result.

[0070] Figure 4 Another flowchart of a deep learning-based object recognition method provided for the implementation of this application is shown below. Figure 4 As shown, the object recognition methods based on deep learning include S401-S406.

[0071] S401, Drawing Import and Preprocessing.

[0072] For example, users upload architectural drawing files by selecting or dragging and dropping files. The architectural drawing file is parsed into an internal data structure, extracting geometric elements (such as structural elements like wall lines, axes, and floor outlines, and symbols like elevator symbols, stair symbols, and door and window symbols), layer information (such as layer names (WALL, AXIS, DOOR, TEXT, etc.), colors, line types, line widths, and layer visibility), and text annotations (such as element labels (used to describe the attributes of geometric elements), element dimensions, etc.). A planar image preview is generated for display, preserving the original scale and coordinate information of the drawing. The planar image preview is used as the drawing parsing data, and the validity of the drawing parsing data is checked (whether the drawing parsing data contains the necessary layer information, dimension annotations, and structural integrity) and coordinate mapping relationships are established (establishing the correspondence between CAD coordinates and the pixel coordinates of the planar image preview).

[0073] S402, Apartment Type - Public Area Division.

[0074] Specifically, users use the drawing tool on the graphical interface to draw dividing lines on the drawing, defining the boundaries between apartment layouts and public areas. These dividing lines can be polylines or closed polygons. Based on the user-drawn dividing lines, the system performs topological calculations on the drawing plane, automatically marking the areas enclosed by these lines as apartment layout areas and excluding them from the subsequent public area identification range. Areas marked as apartment layouts are highlighted on the interface with different colors or transparency for easy user confirmation and correction. Through these segmentation steps, the subsequent deep learning model can operate within non-apartment layout areas, significantly reducing the search space.

[0075] S403, Generation of the second object candidate region.

[0076] Specifically, users can drag a rectangle or use a polygon selection tool to select one or more areas on the drawing mark data and mark them as "public area outer contour candidate areas" (object areas initially marked by the second object area mark information). The selected area will automatically eliminate the apartment area (first object area) and only retain the part that is actually within the public area. Multiple areas can be merged or split to obtain public area candidate areas (processed drawing mark data). For each public area candidate area, it is cropped according to the bounding rectangle to generate a standard-sized image block (second object candidate area), and its coordinate position and scaling ratio in the CAD file are recorded at the same time.

[0077] S404, Public Area Object Recognition Based on Deep Learning.

[0078] Specifically, public area object recognition is implemented based on a deep learning model, such as an instance segmentation model. The instance segmentation model includes: a feature extraction backbone, a feature pyramid network (FPN), a public area attention module, and an instance segmentation head (Mask Head). Before feature extraction, the candidate regions of the second object are encoded using multi-channel encoding to obtain multi-channel tensors. The feature extraction backbone can employ residual network structures such as ResNet or ResNeXt. The backbone performs convolution, non-linear activation, and normalization on the encoded candidate regions of the second object (multi-channel tensors) to obtain feature data. The feature pyramid network then performs feature fusion on the feature data, including top-down fusion and horizontal connection processing, resulting in fused feature data (feature pyramid). The public area attention module performs attention processing on the fused feature data, including multiplication based on attention weights, to obtain attention feature data. The instance segmentation head (Mask Head) then identifies the attention feature data to obtain the second object recognition result.

[0079] S405, Local incremental identification and result correction based on user feedback.

[0080] Specifically, the system interface overlays the outlines of identified public area objects (second object recognition results), allowing users to visually assess the recognition effectiveness. When a user discovers a missed or misidentified area, they can select that local area using a box selection or polygon selection (regional marker feedback information from the second object recognition results). The deep learning model is re-inferred only for the user-selected local area (local incremental data), and the original results are replaced or merged within that local area, without needing to recalculate the entire drawing. Users can manually delete obviously misidentified objects (second object recognition results), or supplement attribute information (such as usage descriptions, numbers, or labels) for identified objects, and then save the corrected results again.

[0081] S406, Recognition Results Display and Standard Format Export.

[0082] Specifically, after the user confirms the recognition results, a unified data export is performed: (1) All selected public area objects and their outer contours (second object recognition results) are exported in standard formats such as JSON and SVG. (2) Each object records its geometric outer contour, category, confidence level, and business attributes supplemented by the user. (3) The exported files can be directly called by subsequent building reconstruction modules, BIM modeling modules, spatial statistical analysis modules, and other systems.

[0083] In one example, a deep learning-based object recognition method is applied to a deep learning model. The deep learning-based object recognition method further includes: using a second object candidate region with the candidate category in the second object candidate region as the target category as a positive sample, and using a second object candidate region with the candidate category in the second object candidate region as the non-target category as a negative sample, and training a deep learning model based on a classification loss function, a bounding box regression loss function, a mask loss function, a boundary loss function, and constraint rules.

[0084] Specifically, deep learning models, such as instance segmentation models, consist of: a feature extraction backbone, a feature pyramid network (FPN), a common area attention module, and an instance segmentation mask head. During model training, only labeled samples (target categories) located within the common area (second object candidate region) are used. Negative samples are derived from "non-target areas" (non-target categories) within the common area (such as equipment rooms, storage rooms, etc.), achieving focused learning of common area objects. Samples obtained from historical projects and user interactions can be continuously added to the training set for incremental training and iterative model optimization.

[0085] For example, in order to make the model better adapt to the public area scene, the training phase adopts multi-task joint loss and public area topology constraint. Among them, multi-task joint loss includes: (1) classification loss: constrain the category (candidate category) prediction of the candidate box, so that the model can correctly distinguish different public area types; (2) bounding box regression loss: improve the fitting accuracy of the candidate box to the geometric position and range of the public area object; (3) mask loss (used to train the mask branch to predict the pixel-level mask in the target candidate box, and obtain the fine object outer contour (mask map)): constrain the degree of overlap between the mask and the real annotation through pixel-level cross entropy or Dice loss; (4) boundary loss (used for the boundary refinement branch, perform additional convolution or boundary-aware loss constraint on the mask boundary, so that the contour is more in line with the wall line and other structural lines): apply additional weights to the pixels near the mask boundary, so that the predicted boundary is more in line with the structural lines in CAD.

[0086] The public area topological constraints include: utilizing prior knowledge such as corridor continuity and the adjacency relationship between stairwells and stair symbols to construct simple topological constraints or post-processing rules; merging connected components of corridor masks that must be connected using graph theory methods; and screening or merging regions adjacent to stair symbols but with too small an area. These constraints can be implemented through rule post-processing or by embedding loss functions to further improve the structural rationality of the public area recognition results.

[0087] Compared with the traditional "general detection / segmentation model for whole images", the public area object recognition model (deep learning model) of this application has the following significant technical features: (1) Input side: It clearly utilizes the structural information of multiple layers and elements of CAD and public area candidate masks to realize prior constraints and information enhancement of public area regions; (2) Model structure side: It introduces a public area attention module and boundary refinement mechanism in the instance segmentation network to specifically optimize the morphological features of public area objects; (3) Training side: By focusing on public area samples, the model can still achieve good generalization ability and recognition accuracy under limited training data.

[0088] The object recognition method and system based on deep learning proposed in this application adopt the following: (1) A public area recognition process design of "first human-computer partitioning, then local intelligent recognition". The user draws lines to divide the apartment type and public area, and selects the candidate area of ​​the public area, thus limiting the recognition range of the deep learning model to the public area area that is not the apartment type. While keeping the model complexity unchanged, the search space is reduced, the false recognition rate is significantly reduced, and the recall rate and accuracy of public area object recognition are improved. (2) A dedicated deep learning recognition model and training set construction method for public area objects. Only objects in the public area are labeled and modeled. The model output focuses on public area related objects and combines CAD layers, line types and other information for multi-channel encoding. The dedicated recognition of public area objects is achieved with limited labeling cost. Compared with the full-category labeling scheme of the whole map, the training set construction cost is greatly reduced and the recognition performance of the model in the public area scene is improved. (3) A local incremental recognition mechanism based on user feedback. When the user finds that the local area recognition is inaccurate, he / she only needs to select the area on the interface. The system will rerun the recognition model and update the results in the local area without having to re-infer the whole drawing. While ensuring recognition accuracy, the system response speed and interactive experience are significantly improved, the waste of computing power caused by repeated calculations is reduced, and manual error correction is controlled within a small range, thereby improving overall efficiency. (4) The unified mapping and standard format output of the recognition results with the CAD coordinate system maintains the correspondence between the image pixel coordinates and the actual CAD coordinates during the recognition process, so that the outer contour of the output public area object can be directly used in subsequent CAD / BIM systems. This reduces the complexity of format conversion and secondary processing, and facilitates the direct reuse of recognition results in various downstream systems such as building reconstruction and spatial statistical analysis.

[0089] Figure 5 A block diagram of an electronic device provided for another embodiment of this application.

[0090] Another embodiment of this application provides an electronic device having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0091] like Figure 5As shown, for ease of understanding, an embodiment of this application illustrates a specific electronic device 500.

[0092] Electronic device 500 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0093] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0094] Multiple components in electronic device 500 are connected to input / output (I / O) interface 505. These components include: input unit 506, such as a keyboard or mouse; output unit 507, such as various types of displays or speakers; storage unit 508, such as a hard disk or optical disk; and communication unit 509, such as a network interface card (NIC), modem, or wireless transceiver. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0095] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods described above. For example, in some embodiments, any one or more of the various methods described above can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of any one or more of the various methods described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform any one or more of the various methods described above by any other suitable means (e.g., by means of firmware).

[0096] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0097] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this application, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0098] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0099] In the description of this application, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this application, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0100] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0101] Furthermore, the terms "first," "second," etc., used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance, or implicitly specifying the number of technical features indicated in this embodiment. Therefore, features defined with terms such as "first" and "second" in the embodiments of this application can explicitly or implicitly indicate that the embodiment includes at least one of those features. In the description of this application, the word "multiple" means at least two or more, such as two, three, four, etc., unless otherwise explicitly and specifically defined in the embodiments.

[0102] In this application, unless otherwise explicitly specified or limited in the embodiments, the terms "installation," "connection," "joining," and "fixing" appearing in the embodiments should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral part; it can also be a mechanical connection, an electrical connection, etc. Of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication between two components, or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific implementation.

[0103] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0104] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A deep learning-based object recognition method, characterized in that, The method includes: Obtain architectural drawing files; The architectural drawing file is parsed to obtain drawing parsing data; Based on the first object region segmentation line, the drawing parsing data is processed by object region segmentation, and the first object region is marked in the drawing parsing data to obtain drawing marking data; Based on the second object region marking information, image processing is performed on the drawing marking data to determine the second object candidate region; Deep learning recognition processing is performed on the candidate region of the second object to determine the recognition result of the second object.

2. The method according to claim 1, characterized in that, The step of performing image processing on the drawing marking data based on the second object region marking information to determine the second object candidate region includes: Remove the first object region marked in the drawing mark data, and merge or split the object regions initially marked by the second object region mark information to obtain the processed drawing mark data; The processed drawing mark data is cropped to obtain the second object candidate region.

3. The method according to claim 1, characterized in that, The step of performing deep learning recognition processing on the second object candidate region to determine the second object recognition result includes: The second object candidate region is subjected to multi-channel encoding processing to obtain the encoded second object candidate region; Feature extraction processing is performed on the encoded second object candidate region to obtain feature data; The feature data is subjected to feature fusion processing to obtain fused feature data; Attention processing is applied to the fused feature data to obtain attention feature data; The attention feature data is processed to determine the recognition result of the second object.

4. The method according to claim 3, characterized in that, The step of performing multi-channel encoding processing on the second object candidate region to obtain the encoded second object candidate region includes: The candidate region of the second object is encoded using the first channel to obtain structural data; The second candidate region of the object is processed by second-channel encoding to obtain symbolic data; The second object candidate region is encoded using the third channel to obtain semantic data; The candidate region of the second object is encoded using the fourth channel to obtain mask data. Based on the structural data, the symbolic data, the semantic data, and the mask data, the encoded second object candidate region is obtained.

5. The method according to claim 3, characterized in that, The step of performing feature fusion processing on the feature data to obtain fused feature data includes: The feature data is subjected to feature fusion processing to obtain intermediate feature data; The fused feature data is obtained by performing feature concatenation processing based on the intermediate feature data and the feature data.

6. The method according to claim 4, characterized in that, The attention processing of the fused feature data to obtain attention feature data includes: The mask data is mapped to obtain attention weight data of the same size as the fused feature data; The attention feature data is obtained by multiplying the fused feature data based on the attention weight data.

7. The method according to claim 3, characterized in that, The step of processing the attention feature data to determine the second object recognition result includes: The attention feature data is classified and bounding box regression is performed to obtain candidate bounding boxes, candidate categories, and confidence data. Based on the candidate boxes and the confidence data, non-maximum suppression processing is performed to obtain the target candidate boxes; Mask calculations are performed on each pixel of the target candidate box to obtain a mask image; The mask image is smoothed and mapped to obtain the second object recognition result.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: In response to receiving region marker feedback information for the identification result of the second object, local incremental data is determined from the drawing marker data based on the region marker feedback information; The local incremental data is subjected to deep learning recognition processing to correct the second object recognition result.

9. The method according to claim 7, characterized in that, The method is applied to a deep learning model, and the method further includes: The deep learning model is trained based on classification loss function, bounding box regression loss function, mask loss function, boundary loss function and constraint rules, with the candidate categories in the candidate regions of the second object being the target category as positive samples and the candidate categories in the candidate regions of the second object being the non-target categories as negative samples.

10. A deep learning-based object recognition system, characterized in that, The system includes: The acquisition module is used to acquire architectural drawing files; The parsing module is used to parse the architectural drawing file to obtain parsed drawing data; The segmentation module is used to perform object region segmentation processing on the drawing parsing data based on the first object region segmentation line, and to mark the first object region in the drawing parsing data to obtain drawing mark data; The processing module is used to perform image processing on the drawing marking data based on the second object region marking information to determine the second object candidate region; The recognition module is used to perform deep learning recognition processing on the candidate region of the second object to determine the recognition result of the second object.

Images