Component identification method and device, electronic equipment and storage medium

Abstract

The invention provides a component identification method and device, electronic equipment and a storage medium, relates to the technical field of computers, and solves the technical problem that the use efficiency of a low-code platform is affected due to the fact that the low-code platform comprises a large number of components, the styles of the components are various, and configuration items corresponding to each component are generally complex. The method comprises the following steps: acquiring a to-be-recognized image; determining a target feature of the to-be-recognized image, wherein the target feature is used for representing layout information of each to-be-recognized component in the to-be-recognized image; and inputting the target feature into a trained component identification model to obtain component information of the to-be-identified image, the component information including the type of each to-be-identified component and the position of each to-be-identified component in the to-be-identified image.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a component identification method, device, electronic device and storage medium. Background Art

[0002] Currently, for low-code platforms, users can create and design large visual screens by performing simple operations on components (such as dragging and dropping, etc.).

[0003] However, since the low-code platform includes a large number of components, the styles of these components are diverse, and the configuration items corresponding to each component are usually more complex, users need to spend a lot of time matching components in the process of creating and designing large visualization screens, which affects the efficiency of using the low-code platform. Summary of the invention

[0004] The present application provides a component identification method, device, electronic device and storage medium, which solves the technical problem that the low-code platform includes a large number of components, the styles of these components are diverse, and the configuration items corresponding to each component are usually complex, which affects the efficiency of using the low-code platform.

[0005] In a first aspect, the present application provides a component identification method, comprising: acquiring an image to be identified, the image to be identified comprising a plurality of components to be identified; determining a target feature of the image to be identified, the target feature being used to characterize layout information of each component to be identified among the plurality of components to be identified in the image to be identified, the layout information of each component to be identified in the image to be identified comprising the shape of each component to be identified and the relationship between each component to be identified and other components to be identified; inputting the target feature into a trained component identification model to obtain component information of the image to be identified, the component information comprising the type of each component to be identified and the position of each component to be identified in the image to be identified.

[0006] In a second aspect, the present application provides a component identification device, comprising: an acquisition module, a determination module and a processing module; the acquisition module is used to acquire an image to be identified, the image to be identified comprising multiple components to be identified; the determination module is used to determine a target feature of the image to be identified, the target feature is used to characterize layout information of each component to be identified among the multiple components to be identified in the image to be identified, the layout information of each component to be identified in the image to be identified includes the shape of each component to be identified and the relationship between each component to be identified and other components to be identified; the processing module is used to input the target feature into a trained component identification model to obtain component information of the image to be identified, the component information includes the type of each component to be identified and the position of each component to be identified in the image to be identified.

[0007] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory configured to store processor-executable instructions; wherein the processor is configured to execute the instructions to implement any one of the optional component identification methods in the first aspect above.

[0008] In a fourth aspect, the present application provides a computer-readable storage medium having instructions stored thereon. When the instructions in the computer-readable storage medium are executed by an electronic device, the electronic device is enabled to execute any one of the optional component identification methods in the first aspect described above.

[0009] In a fifth aspect, the present application provides a computer program product, including a computer program or instructions, which, when executed on an electronic device, enables the electronic device to execute any one of the optional component identification methods in the first aspect.

[0010] The component identification method, device, electronic device and storage medium provided by the present application can determine the target features of the image to be identified after the electronic device obtains the image to be identified, and input the target features into the trained component identification model to obtain the component information of the image to be identified, including the type of each component to be identified and the position of each component to be identified in the image to be identified. Since the target features can characterize the layout information of each component to be identified in the image to be identified, specifically the shape of each component to be identified and the relationship between each component to be identified and other components to be identified, the electronic device can accurately and effectively determine the type of each component to be identified and the position of each component to be identified in the image to be identified based on the shape of each component to be identified, the relationship between each component to be identified and other components to be identified, and the trained component identification model. This can assist users in quickly matching the type and position of components in the process of creating and designing large visualization screens, thereby improving the use efficiency of low-code platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0012] Figure 1 A schematic diagram of a flow chart of a component identification method provided in an embodiment of the present application;

[0013] Figure 2 A flowchart of another component identification method provided in an embodiment of the present application;

[0014] Figure 3 A flowchart of another component identification method provided in an embodiment of the present application;

[0015] Figure 4 A flowchart of another component identification method provided in an embodiment of the present application;

[0016] Figure 5 A flowchart of another component identification method provided in an embodiment of the present application;

[0017] Figure 6 A schematic diagram of the structure of a component identification device provided in an embodiment of the present application;

[0018] Figure 7 A schematic diagram of the structure of another component identification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The component identification method, device, electronic device and storage medium provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0020] The terms "first" and "second" and so on in the specification and drawings of this application are used to distinguish different objects rather than to describe the specific order of the objects. For example, the first basic feature and the second basic feature and so on are used to distinguish different basic features rather than to describe the specific order of the basic features.

[0021] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0022] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0023] The term "and / or" used in the present application includes using either or both of the two methods.

[0024] In the description of the present application, unless otherwise specified, “plurality” means two or more.

[0025] The visual large screen low-code platform (or low-code platform) mainly demonstrates its powerful functions in a visual and interactive form. In today's digital age, low-code platforms play a vital role. Usually, only front-end developers who have mastered front-end code development skills can do the visual large screen design work.

[0026] However, based on the low-code platform, business and operations personnel who do not understand code at all can also actively participate in the creation and design of visualization large screens. Specifically, users can easily create and design visualization large screens by performing simple operations on components (such as dragging and dropping, etc.). In this process, there is no need to have a lot of front-end design technical knowledge, nor is it necessary to spend time and effort writing a lot of code. In this way, users can focus more on the specific logic implementation of the business, greatly improving work efficiency.

[0027] However, although the low-code platform has simplified most of the professional code design links, due to the existence of a variety of component styles and the complicated configuration item design of each component, platform users still need to invest a lot of time and experience in component matching and large-screen design layout, which undoubtedly brings certain troubles to users.

[0028] Based on the description in the background technology, since in the related technology, the low-code platform includes a large number of components, the styles of these components are various, and the configuration items corresponding to each component are usually more complicated, so the user needs to spend a lot of time to match the components in the process of creating and designing a large visualization screen, which affects the use efficiency of the low-code platform. Based on this, the embodiment of the present application provides a component recognition method, device, electronic device and storage medium. Since the target feature can characterize the layout information of each component to be recognized in the image to be recognized, specifically the shape of each component to be recognized and the relationship between each component to be recognized and other components to be recognized, the electronic device can accurately and effectively determine the type of each component to be recognized and the position of each component to be recognized in the image to be recognized based on the shape of each component to be recognized, the relationship between each component to be recognized and other components to be recognized, and the trained component recognition model. Thereby, it can assist users in quickly matching the type and position of components in the process of creating and designing a large visualization screen, and improve the use efficiency of the low-code platform.

[0029] Exemplarily, the electronic device that executes the component identification method provided in the embodiment of the present application can be a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, and a cellular phone, a personal digital assistant (personal digital assistant, PDA), an augmented reality (augmented reality, AR)\virtual reality (virtual reality, VR) device, and the embodiment of the present application does not impose any special restrictions on the specific form of the electronic device. It can interact with the user through one or more methods such as a keyboard, a touchpad, a touch screen, a remote control, voice interaction or a handwriting device.

[0030] Optionally, the above-mentioned electronic device can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (content delivery network, CDN), as well as big data and artificial intelligence platforms.

[0031] like Figure 1 As shown, the component identification method provided in the embodiment of the present application may include S101-S103.

[0032] S101: Obtain an image to be recognized.

[0033] The image to be identified includes multiple components to be identified.

[0034] In an embodiment of the present application, the image to be identified may be a large-screen image to be identified, specifically an image corresponding to a low-code platform (or a visual low-code platform), which may include multiple components to be identified.

[0035] S102: Determine target features of the image to be recognized.

[0036] Among them, the target feature is used to characterize the layout information of each component to be identified among the above-mentioned multiple components to be identified in the image to be identified, and the layout information of each component to be identified in the image to be identified includes the shape of each component to be identified and the relationship between each component to be identified and other components to be identified.

[0037] Optionally, the relationship between a component to be identified and other components to be identified may include a positional relationship between the component to be identified and the other components to be identified.

[0038] Exemplarily, the shape of a component to be identified may be circular, rectangular, triangular, etc. The embodiment of the present application does not specifically limit the shape of the component to be identified.

[0039] S103: Input the target features of the image to be identified into the trained component identification model to obtain component information of the image to be identified.

[0040] The component information includes the type of each component to be identified and the position of each component to be identified in the image to be identified.

[0041] In the embodiment of the present application, the position of a component to be identified in the image to be identified can also be understood as the position of the component to be identified in the visualization large screen.

[0042] Optionally, the type of a component to be identified may be a text box, an icon, a drop-down box, a background block, a number flipper, or a decorative component, etc. The embodiment of the present application does not specifically limit the type of the component to be identified.

[0043] The technical solution provided by the above embodiment can at least bring the following beneficial effects: It can be known from S101-S103 that after the electronic device acquires the image to be identified, it can determine the target features of the image to be identified, and input the target features into the trained component recognition model to obtain the component information of the image to be identified, including the type of each component to be identified and the position of each component to be identified in the image to be identified. Since the target features can characterize the layout information of each component to be identified in the image to be identified, specifically the shape of each component to be identified and the relationship between each component to be identified and other components to be identified, the electronic device can accurately and effectively determine the type of each component to be identified and the position of each component to be identified in the image to be identified based on the shape of each component to be identified, the relationship between each component to be identified and other components to be identified, and the trained component recognition model. This can assist users in quickly matching the type and position of components in the process of creating and designing large visualization screens, thereby improving the use efficiency of low-code platforms.

[0044] In one implementation of the embodiment of the present application, in combination Figure 1 ,like Figure 2 As shown, the above-mentioned determination of the target features of the image to be identified may specifically include S1021-S1022.

[0045] S1021. Input the image to be identified into a feature extraction network to obtain image features of the image to be identified.

[0046] The image feature is used to characterize the shape of each component to be identified.

[0047] S1022: Input the image features of the image to be identified into a feature fusion network to determine the target features of the image to be identified.

[0048] In the embodiment of the present application, the feature extraction network may be a backbone network in a you only look once version 8 (YOLOv8) model, and the feature fusion network may be a neck network in a YOLOv8 model.

[0049] In an optional implementation, combining Figure 2 ,like Figure 3 As shown, the above-mentioned inputting the image to be identified into the feature extraction network to obtain the image features of the image to be identified can be specifically implemented through S1021a-S1021c.

[0050] S1021a, inputting the image to be identified into a feature extraction network to obtain a first basic feature of the image to be identified.

[0051] The first basic feature is used to characterize the edge contour of each component to be identified.

[0052] It should be understood that the edge contour of a component to be identified can also be understood as the boundary of the component to be identified.

[0053] Optionally, the first basic feature may also characterize the shape of each component to be identified.

[0054] S1021b. Perform linear transformation processing on the first basic feature to obtain a supplementary feature of the image to be identified.

[0055] The supplementary features are used to characterize the differences between the shape of each component to be identified and the shapes of other components to be identified.

[0056] Optionally, the compensation feature may also characterize specific detail information of each component to be identified, such as the difference in some detail information between a bar graph and a bar line graph.

[0057] It should be understood that the electronic device may perform linear transformation processing on the first basic feature by specifically acquiring a linear transformation matrix, and determining the product of the linear transformation matrix and the first basic feature as the supplementary feature.

[0058] Optionally, the number of the supplementary features is twice the number of the first basic features.

[0059] In an embodiment of the present application, the electronic device performs a linear transformation on the first basic feature to extract detailed features of each component to be identified, increase the diversity and richness of the features, and improve the component identification model's ability to identify various components in the visualization large screen and understand the layout.

[0060] S1021c. Obtain image features of the image to be identified based on the first basic features and the supplementary features.

[0061] Optionally, the electronic device may perform splicing processing on the first basic feature and the supplementary feature to obtain the image feature.

[0062] Exemplarily, it is assumed that the feature extraction network is the backbone network in the YOLOv8 model. The embodiment of the present application can improve the backbone network, specifically by replacing the original standard convolution with group shuffle convolution (GSConv).

[0063] Specifically, we can start from the convolution layer after the third residual block in the backbone network of the YOLOv8 model and gradually use GSConv lightweight convolution to replace the original standard convolution. GSConv first uses 8 3*3 standard convolution kernels to perform ordinary convolution operations to generate the above-mentioned first basic features. Then, the linear transformation matrix is ​​multiplied with the first basic feature to obtain the above-mentioned supplementary feature (also called the first ghost feature). Finally, the first basic feature and the supplementary feature are spliced ​​to obtain the above-mentioned image feature.

[0064] In an optional implementation, combining Figure 2 ,like Figure 4 As shown, the above-mentioned inputting the image features of the image to be identified into the feature fusion network to determine the target features of the image to be identified can be specifically implemented through S1022a-S1022c.

[0065] S1022a, inputting the image features of the image to be identified into a feature fusion network to obtain the second basic features of the image to be identified.

[0066] The second basic feature is used to characterize the relationship between each component to be identified and other components to be identified.

[0067] It should be understood that the second basic feature can also characterize the overall layout pattern of the above-mentioned multiple components to be identified.

[0068] It can be understood that the above-mentioned feature fusion network (or the obtained second basic feature) can fuse or extract higher-level semantic information and more abstract features in the image to be identified.

[0069] S1022b. Perform a pooling operation on the second basic feature based on at least two pooling windows to obtain pooling features corresponding to the at least two pooling windows.

[0070] The at least two pooling windows have different sizes.

[0071] Exemplarily, the sizes of the at least two pooling windows may include 2*2, 3*3, 4*4, etc.

[0072] S1022c: Determine target features of the image to be identified based on the pooling features corresponding to the at least two pooling windows.

[0073] Specifically, the electronic device may perform splicing and fusion processing on the pooling features corresponding to the at least two pooling windows respectively to determine the target feature.

[0074] Exemplarily, it is assumed that the feature fusion network is the neck network in the YOLOv8 model. The embodiment of the present application can improve the neck network, specifically by replacing the crossstage partial connections (CSP) module structure with the VoV-GSCSP module structure.

[0075] Specifically, in the neck network of the YOLOv8 model, we can find the location of the original CSP module, which is usually located in the feature pyramid network (FPN) and path aggregation network (PAN) parts, and replace them all with VoV-GSCSP modules.

[0076] The VoV-GSCSP module can first use 16 3*3 convolution kernels to perform ordinary convolution to generate the above-mentioned second basic features. Then, more supplementary features related to the second basic features (also called ghost features) are generated through linear operations, and the supplementary features related to the second basic features are shuffled. Then, the features obtained after the shuffle operation are pooled using the maximum pooling windows of 2*2, 3*3 and 4*4, respectively, to obtain feature maps of multiple different scales (i.e., the pooling features corresponding to the at least two pooling windows mentioned above). Multiple feature maps of different scales are spliced, and then fused through a convolution layer with a convolution kernel size of 1*1 to obtain a fused feature. The fused features are then passed through an output convolution layer to obtain the output result of the VoV-GSCSP module, which is the above-mentioned target feature.

[0077] It should be noted that the features in the embodiments of the present application (including target features, image features, basic features, supplementary features, pooling features, etc.) can also exist and be used in the form of feature maps.

[0078] In some embodiments, the above-mentioned component identification method may further include step A.

[0079] Step A: Generate a text file corresponding to the image to be identified based on the component information of the image to be identified.

[0080] The text file is used to represent the component information of the image to be identified.

[0081] It should be understood that the above text file can contain the component type, unique identifier, location information, style information, and data content, etc., which can provide a detailed component description for the low-code platform. Specifically, according to the different types of components, such as icons, text boxes, etc., appropriate fields can be filled in to ensure that the low-code platform can correctly parse these components. For example, for an icon component, a specific icon attribute field can be mentioned in the text file; for a text box component, the relevant attribute fields of the text box can be added.

[0082] Exemplarily, the text file may be in a (javascript object notation, JSON) structure or format.

[0083] The following is an example of the JSON structure provided in the embodiment of the present application:

[0084] {

[0085] "widgets":[

[0086] {

[0087] "type":"widget-paragraph",

[0088] "id":"368",

[0089] "position":{

[0090] "width":389.0,

[0091] "height":42.0,

[0092] "left":29.0,

[0093] "top":26.0,

[0094] "zIndex":50

[0095] },

[0096] "style":{

[0097] "fontSize":"8",

[0098] "color":"#FFF",

[0099] "background":"rgba(115,170,229,.0)",

[0100] "textAlign":"center"

[0101] },

[0102] "data":{

[0103] "staticData":"Example"

[0104] }

[0105] } ]

[0107] }

[0108] In this JSON structure, "type":"widget-paragraph" indicates that the type of the component is a paragraph component. The position of the component (i.e., position) includes the width of the component (width), the height of the component (height), the distance between the component and the left of the large screen (left), the distance between the component and the top of the large screen (top), and the layer or hierarchy information of the component (zIndex). The style of the component (i.e., style) includes the font size of the component (fontSize), the color of the component (color), the background color of the component (background), and the text alignment in the component (textAlign). The data content of the component (i.e., data) is static data (staticData).

[0109] In some other embodiments, the above-mentioned component identification method may further include steps B to C.

[0110] Step B: parse the text file corresponding to the image to be identified to obtain component information of the image to be identified.

[0111] In combination with the description of the above embodiment, it should be understood that the component information of the image to be identified includes the type of each component to be identified and the position of each component to be identified in the image to be identified.

[0112] Optionally, the above component information may also include the size of the component to be identified.

[0113] Step C: based on the priority of each component to be identified, adjusting the size of each component to be identified and / or the position of each component to be identified in the image to be identified.

[0114] Specifically, when the priority of a component to be identified is higher than the priority threshold, or the priority of the component is the same as the priority threshold, the electronic device can increase the size of the component to be identified or adjust the position of the component to be identified in the image to be identified to the center position of the image to be identified.

[0115] Correspondingly, when the priority of the component to be identified is lower than the priority threshold, the electronic device can reduce the size of the component to be identified or adjust the position of the component to be identified in the image to be identified to the edge position of the image to be identified.

[0116] It should be understood that core data display components with higher priority can be arranged in prominent positions to ensure that users can quickly obtain key information. Auxiliary illustrative components with lower priority can be placed around the core components to form a reasonable layout structure. For example, in a large screen of the sales management category, core data display components (such as sales data charts) can be placed in the center, and auxiliary components (such as description labels, drop-down boxes, etc.) can be placed at the edge, and can be distributed around the center.

[0117] It is understandable that important components with higher priority can be appropriately enlarged in size to highlight them, while taking into account the coordination of the overall layout to avoid a component being too large or too small to affect the overall aesthetics. By continuously iterating the size of each component, the entire layout is ensured to be more visually balanced. For example, for key data display components with higher priority, their size can be appropriately increased to make them more eye-catching; while for some auxiliary components with lower priority, their size can be relatively reduced to save space.

[0118] Optionally, the electronic device can also adjust the alignment of each component. Specifically, by calculating the bounding box and center point position of each component, a heuristic algorithm is used to gradually align. For horizontal alignment, a baseline is first determined, and then the center points of all components are adjusted as close to the baseline as possible in the horizontal direction; vertical alignment follows a similar idea. For example, the center line of the page or the edge line of a specific important component can be used as the baseline, and the center points of other components are moved closer to the baseline to achieve a neat layout effect.

[0119] The following is a complete example to illustrate the component identification method provided by the embodiment of the present application. Figure 5 As shown, the method specifically includes S201-S205.

[0120] S201: Obtain an image to be recognized.

[0121] S202: Obtain component information of the image to be identified based on the trained component identification model.

[0122] In combination with the description of the above embodiment, it should be understood that the component information includes the type of each component to be identified and the position of each component to be identified in the image to be identified. The component information is specifically obtained by the electronic device inputting the target feature of the image to be identified into the component recognition model. The target feature is used to characterize the layout information of each component to be identified in the image to be identified, and the layout information of each component to be identified in the image to be identified includes the shape of each component to be identified and the relationship between each component to be identified and other components to be identified.

[0123] S203: Generate a text file corresponding to the image to be identified based on the component information of the image to be identified.

[0124] S204. Control the low-code platform to parse the text file of the image to be identified.

[0125] S205. Fine-tune the generated visualization large screen according to requirements.

[0126] Specifically, the process of the electronic device fine-tuning the visualization large screen includes the process of the electronic device adjusting the size of each component to be identified, the position of each component to be identified in the image to be identified, and the alignment of each component in the above embodiment.

[0127] In some embodiments, the component identification method provided in the embodiments of the present application may also include a model training process of the component identification model, wherein the process specifically includes at least data collection and annotation, model training and testing, and performance evaluation and model optimization.

[0128] Specifically, in the process of data collection and standardization, electronic devices first need to collect various types of components to ensure the richness and diversity of the data and provide sufficient samples for subsequent model training. Afterwards, electronic devices can use the annotation tool in the YOLOv8 standard format to annotate the collected components. The standard content includes the type of component, the position of the component in the visualization screen (which can also be understood as the position of the component in the image), and the size of the component (ie, width and height). Accurate annotation provides clear learning goals for the model. Divide the annotated data into training set and validation set in a ratio of 8:2. The training set is used for model training, and the validation set is used to evaluate the performance of the model at different stages to ensure the high quality of model training and verification.

[0129] During model training and testing, the cross entropy loss function and the intersection over union loss (IoU) loss function can be used for model training. The cross entropy loss function can ensure the accuracy of component recognition (used to ensure the type of component), and the IoU loss function can ensure the accuracy of position regression (used to ensure the position of the component). The combination of the two can comprehensively improve the performance of the model in component recognition and position regression. Model training is performed in a graphics processing unit (GPU) environment, and the parallel computing power of the GPU is used to increase the training speed. Model performance is optimized by adjusting hyperparameters such as learning rate, batch size, regularization parameters, etc. The trained model is tested using a validation set to evaluate the performance of the model in actual applications and ensure that the model can stably identify components in different scenarios.

[0130] In the process of performance evaluation and model optimization, indicators such as mean average precision (mAP), frame rate (FPS), and model parameter quantity can be used to evaluate the performance of the model in different components and layout scenarios. Mean average precision can measure the detection accuracy of the model for different types of components; frame rate can reflect the running speed of the model; model parameter quantity can reflect the complexity and resource usage of the model. Through the comprehensive evaluation results of these indicators, the hyperparameters of the model are adjusted to further improve the performance of the model when detecting components of different sizes. For example, if the mean average precision is low, the learning rate can be adjusted, the number of training rounds can be increased, etc.; if the frame rate is low, the model structure can be optimized, unnecessary calculations can be reduced, etc.

[0131] The embodiment of the present application can divide the functional modules of the electronic device etc. according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0132] In the case of dividing each functional module into corresponding functional modules, Figure 6 A possible structural diagram of the component identification device involved in the above embodiment is shown. Figure 6 As shown, the component identification device 10 may include: an acquisition module 101 , a determination module 102 and a processing module 103 .

[0133] The acquisition module 101 is used to acquire an image to be recognized, where the image to be recognized includes a plurality of components to be recognized.

[0134] The determination module 102 is used to determine the target feature of the image to be identified, and the target feature is used to characterize the layout information of each component to be identified among the multiple components to be identified in the image to be identified. The layout information of each component to be identified in the image to be identified includes the shape of each component to be identified and the relationship between each component to be identified and other components to be identified.

[0135] The processing module 103 is used to input the target feature into the trained component recognition model to obtain component information of the image to be recognized, where the component information includes the type of each component to be recognized and the position of each component to be recognized in the image to be recognized.

[0136] Optionally, the processing module 103 is further used to input the image to be identified into a feature extraction network to obtain image features of the image to be identified, and the image features are used to characterize the shape of each component to be identified.

[0137] The determination module 102 is specifically configured to input the image features of the image to be identified into a feature fusion network to determine the target features of the image to be identified.

[0138] Optionally, the processing module 103 is specifically configured to input the image to be identified into the feature extraction network to obtain a first basic feature of the image to be identified, wherein the first basic feature is used to characterize an edge contour of each component to be identified.

[0139] The processing module 103 is further specifically used to perform a linear transformation process on the first basic feature to obtain a supplementary feature of the image to be identified, wherein the supplementary feature is used to characterize the difference between the shape of each component to be identified and the shapes of other components to be identified.

[0140] The processing module 103 is further specifically configured to obtain the image feature based on the first basic feature and the supplementary feature.

[0141] Optionally, the processing module 103 is further used to input the image feature into the feature fusion network to obtain a second basic feature of the image to be identified, and the second basic feature is used to characterize the relationship between each component to be identified and other components to be identified.

[0142] Optionally, the processing module 103 is further used to perform a pooling operation on the second basic feature based on at least two pooling windows to obtain pooling features corresponding to the at least two pooling windows respectively, and the at least two pooling windows have different sizes.

[0143] The determination module 102 is specifically configured to determine the target feature based on the pooling features corresponding to each of the at least two pooling windows.

[0144] Optionally, the processing module 103 is further used to generate a text file corresponding to the image to be identified based on the component information of the image to be identified, and the text file is used to represent the component information.

[0145] Optionally, the processing module 103 is further configured to parse the text file to obtain component information of the image to be identified.

[0146] The processing module 103 is further configured to adjust the size of each component to be identified and / or the position of each component to be identified in the image to be identified based on the priority of each component to be identified.

[0147] In the case of an integrated unit, Figure 7 FIG. 1 is a schematic diagram showing a possible structure of the component identification device involved in the above embodiment. Figure 7 As shown, the component identification device 20 may include: a processing module 201 and a communication module 202. The processing module 201 may be used to control and manage the actions of the component identification device 20. The communication module 202 may be used to support the communication between the component identification device 20 and other entities. Figure 7 As shown, the component identification device 20 may further include a storage module 203 for storing program codes and data of the component identification device 20 .

[0148] The processing module 201 may be a processor or a controller. The communication module 202 may be a transceiver, a transceiver circuit or a communication interface, etc. The storage module 203 may be a memory.

[0149] When the processing module 201 is a processor, the communication module 202 is a transceiver, and the storage module 203 is a memory, the processor, the transceiver, and the memory may be connected via a bus. The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0150] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0151] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0152] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0153] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0154] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing a computer program instruction on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or may contain one or more servers, data centers and other data storage devices that can be integrated with a medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0155] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A component identification method, characterized in that: The method comprises: Acquire an image to be identified, wherein the image to be identified includes a plurality of components to be identified; Determine a target feature of the image to be identified, wherein the target feature is used to characterize layout information of each component to be identified among the multiple components to be identified in the image to be identified, wherein the layout information of each component to be identified in the image to be identified includes a shape of each component to be identified and a relationship between each component to be identified and other components to be identified; The target feature is input into a trained component recognition model to obtain component information of the image to be recognized, wherein the component information includes the type of each component to be recognized and the position of each component to be recognized in the image to be recognized.

2. The method according to claim 1, characterized in that The determining the target feature of the image to be recognized includes: Inputting the image to be identified into a feature extraction network to obtain image features of the image to be identified, wherein the image features are used to characterize the shape of each component to be identified; The image features of the image to be identified are input into a feature fusion network to determine the target features of the image to be identified.

3. The method according to claim 2, characterized in that The step of inputting the image to be identified into a feature extraction network to obtain image features of the image to be identified includes: Inputting the image to be identified into the feature extraction network to obtain a first basic feature of the image to be identified, wherein the first basic feature is used to characterize the edge contour of each component to be identified; Performing a linear transformation process on the first basic feature to obtain a supplementary feature of the image to be identified, wherein the supplementary feature is used to characterize the difference between the shape of each component to be identified and the shapes of other components to be identified; The image feature is obtained based on the first basic feature and the supplementary feature.

4. The method according to claim 2, characterized in that: The step of inputting the image features of the image to be identified into a feature fusion network to determine the target features of the image to be identified includes: Inputting the image feature into the feature fusion network to obtain a second basic feature of the image to be identified, wherein the second basic feature is used to characterize the relationship between each component to be identified and other components to be identified; Performing a pooling operation on the second basic feature based on at least two pooling windows to obtain pooling features corresponding to the at least two pooling windows, respectively, where the at least two pooling windows have different sizes; The target feature is determined based on the pooling features corresponding to each of the at least two pooling windows.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Based on the component information of the image to be identified, a text file corresponding to the image to be identified is generated, and the text file is used to represent the component information.

6. The method according to claim 5, characterized in that The method further comprises: Parsing the text file to obtain component information of the image to be identified; Based on the priority of each to-be-recognized component, the size of each to-be-recognized component and / or the position of each to-be-recognized component in the to-be-recognized image is adjusted.

7. A component identification device, characterized in that: include: Acquisition module, determination module and processing module; The acquisition module is used to acquire an image to be identified, wherein the image to be identified includes a plurality of components to be identified; The determination module is used to determine a target feature of the image to be identified, wherein the target feature is used to characterize layout information of each component to be identified among the multiple components to be identified in the image to be identified, wherein the layout information of each component to be identified in the image to be identified includes a shape of each component to be identified and a relationship between each component to be identified and other components to be identified; The processing module is used to input the target feature into the trained component recognition model to obtain component information of the image to be recognized, wherein the component information includes the type of each component to be recognized and the position of each component to be recognized in the image to be recognized.

8. An electronic device, characterized in that: The electronic device comprises: processor; a memory configured to store instructions executable by the processor; The processor is configured to execute the instructions to implement the component identification method according to any one of claims 1 to 6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions in the computer-readable storage medium are executed by an electronic device, the electronic device is enabled to perform the component identification method according to any one of claims 1 to 6.

10. A computer program product comprising instructions, characterized in that When the instruction is executed on an electronic device, the electronic device is caused to execute the component identification method as described in any one of claims 1 to 6.

Images