Page UI element recognition method and system based on target detection

Abstract

The present invention relates to the field of target detection technology, and provides a method and system for identifying page UI elements based on target detection, the method comprising: obtaining a page image, constructing a data set, and dividing the data set into a training set and a test set; modifying a feature extraction network, a feature fusion network, and a loss function of a YOLOv8s network model to form an OD_YOLOv8s model based on a full-dimensional dynamic convolution kernel; inputting the training set into the OD_YOLOv8s model for training to obtain a target detection model; inputting the test set into the target detection model, and outputting a UI element detection result of a digital large-screen screenshot. According to the solution of the present invention, the present invention can accurately identify page UI elements, effectively improve missed detection caused by variable scales of target images, and effectively improve the stability of image description generation technology to meet the quality requirements of encoding.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a method and system for identifying page UI elements based on target detection. Background Art

[0002] The current software can be divided into the front-end and the back-end according to the system structure. The front-end is responsible for user interaction and data visualization, while the back-end implements functional logic and database interaction. The main task of the front-end developer is to encode the page based on the UI prototype diagram and the existing component library to realize the style, layout, and function of the page. Complex style matching and frequent layout adjustments make this work cumbersome and labor-intensive. If the process from UI prototype to page code generation can be optimized, the workload of front-end developers can be greatly reduced. This patent report aims to optimize this process and realize the automated process from UI prototype diagram to front-end page code generation, which reduces the burden of manual coding and reduces the resource consumption and time cost of front-end work.

[0003] In the research on the automatic conversion of UI prototypes to front-end applications, the code generation methods can be divided into code conversion based on specified constraints, code conversion based on templates, and code conversion based on deep learning. Code conversion based on specified constraints relies on software engineering technology. Developers formulate a set of specifications including code, interface, and mapping relationship for front-end development to realize the conversion between design and code. This method requires a complete formal specification as a premise, which makes the whole process extremely cumbersome and complicated, and the work efficiency is much lower than directly writing code. Template-based code conversion uses pre-defined component templates to quickly generate code. This method can improve development efficiency in specific scenarios, but it is difficult to handle complex requirements and is highly dependent on manual work. Code conversion based on deep learning, the core of this method is the image description generation model. Inputting the front-end screenshot or design prototype into the model can directly generate the target code. This technology effectively improves the efficiency of front-end development, but is limited by the general limitations of image description generation technology. In the small data set scenario, it is difficult for the model to achieve good generalization ability, and the fine identification of components in the UI prototype is unsatisfactory. On the other hand, the stability of image description generation technology is difficult to meet the quality requirements of coding. The front-end coding follows strict grammatical rules, and even slight deviations may cause compilation errors. Summary of the invention

[0004] The purpose of the present invention is to solve at least one technical problem in the background technology and to provide a method and system for identifying page UI elements based on target detection.

[0005] To achieve the above object, the present invention provides a method for identifying page UI elements based on target detection, comprising:

[0006] Get page images, build a dataset, and divide the dataset into a training set and a test set;

[0007] Modify the feature extraction network, feature fusion network and loss function of the YOLOv8s network model to form the OD_YOLOv8s model based on the full-dimensional dynamic convolution kernel;

[0008] Input the training set into the OD_YOLOv8s model for training to obtain the target detection model;

[0009] The test set is input into the target detection model, and the UI element detection result of the digital large screen screenshot is output.

[0010] According to one aspect of the present invention, it also includes:

[0011] Use LabelImg software to label the images of the training set and test set;

[0012] Use CutMix, scaling, shearing, or translation to augment the training set.

[0013] According to one aspect of the present invention, the YOLOv8s network model includes an input layer, a feature extraction network, a feature fusion network and an output layer;

[0014] The feature extraction network includes a CBS module, a C2f module and an SPPF module responsible for extracting key features of the image;

[0015] The modification of the feature extraction network of the YOLOv8s network model is as follows:

[0016] The standard convolution kernel of the CBS module in the feature extraction network is replaced with a full-dimensional dynamic convolution kernel.

[0017] According to one aspect of the present invention, the feature fusion network of the YOLOv8s network model is modified as follows:

[0018] The PANET network in the feature fusion network is replaced by the RepGFPN network with cross-scale connections.

[0019] According to one aspect of the present invention, the YOLOv8s network model uses the sum of DFL Loss and CIoU Loss as the positioning loss function;

[0020] The loss function of the YOLOv8s network model is modified as follows:

[0021] Replace CIoU Loss with SIoU Loss, and use the sum of DFL Loss and SIoU Loss as the positioning loss function.

[0022] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a page UI element recognition system based on target detection, comprising:

[0023] The data acquisition module acquires page images, builds a data set, and divides the data set into a training set and a test set;

[0024] The model building module modifies the feature extraction network, feature fusion network and loss function of the YOLOv8s network model to form the OD_YOLOv8s model based on the full-dimensional dynamic convolution kernel;

[0025] The model training module inputs the training set into the OD_YOLOv8s model for training to obtain the target detection model;

[0026] The detection result output module inputs the test set into the target detection model and outputs the UI element detection result of the digital large screen screenshot.

[0027] Furthermore, to achieve the above-mentioned purpose, the present invention also provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the page UI element recognition method based on target detection as described above.

[0028] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the page UI element recognition method based on target detection as described above is implemented.

[0029] According to the solution of the present invention, in order to break through the inherent limitations of the image description task, the present invention regards the automatic generation of front-end applications as a small sample target detection task. The deep learning model no longer directly extracts feature descriptions from the image to generate front-end code, but focuses on identifying and locating component elements in the image, and generates a configuration file for the front-end page through the detection results. This can effectively avoid the high model cost of the image description task and the low accuracy of the generated results, but the task still has challenges such as diverse targets, complex scenes, and variable scales. In response to these challenges, the present invention proposes the above-mentioned page UI element recognition method based on target detection, which first uses a full-dimensional dynamic convolution kernel (OD-Conv) to replace the standard convolution kernel to cope with the challenges of diverse targets and high-density scenes. Then, the original PANET (Path Aggregation Network) structure is replaced by a re-parameterized generalized feature pyramid network (RepGFPN), and feature maps of different levels are fused by cross-scale connections and multiple downsampling, thereby capturing feature information of different levels of the target, and effectively improving the missed detection caused by variable target scales. Finally, SIoU Loss (Soft Intersection Over Union Loss) is used to replace CIoU Loss (Complete Intersection Over Union Loss) as the loss function of the bounding box to accelerate the convergence speed of network training and improve detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The flowchart schematically shows a method for identifying page UI elements based on target detection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention will now be discussed with reference to exemplary embodiments. It should be understood that the embodiments discussed are only to enable those skilled in the art to better understand and thus implement the present invention, rather than implying any limitation on the scope of the present invention.

[0032] As used herein, the term “including” and variations thereof are to be interpreted as open-ended terms meaning “including, but not limited to.” The term “based on” is to be interpreted as “based, at least in part, on.” The terms “one embodiment” and “an embodiment” are to be interpreted as “at least one embodiment.”

[0033] Figure 1 The flowchart of the method for identifying page UI elements based on target detection according to an embodiment of the present invention is schematically shown. Figure 1 As shown, in this embodiment, the page UI element recognition method based on target detection includes:

[0034] Get page images (such as digital screenshots), build a dataset, and divide the dataset into a training set and a test set;

[0035] Modify the feature extraction network, feature fusion network and loss function of the YOLOv8s network model to form the OD_YOLOv8s model based on the full-dimensional dynamic convolution kernel;

[0036] Input the training set into the OD_YOLOv8s model for training to obtain the target detection model;

[0037] The test set is input into the target detection model, and the UI element detection result of the digital large screen screenshot is output.

[0038] Further, according to an embodiment of the present invention, the page UI element recognition method based on target detection of the present invention further includes:

[0039] Use LabelImg software to label the images of the training set and test set;

[0040] Use CutMix, scaling, shearing, or translation to augment the training set.

[0041] In this embodiment, LabelImg software is used to annotate the images, and the annotation types include 22 categories such as charts, buttons, and navigation bars. CutMix, zoom, shear, and translate are selected as the data expansion methods. The four methods are cross-randomly used to expand, for example, 200 training sets to 2000. Model training is completed under the experimental conditions of 2000 training sets, and the model performance is tested on the test set.

[0042] Furthermore, the YOLOv8s network model consists of four main parts: input layer, feature extraction network, feature fusion network and output layer.

[0043] The input layer loads the image and performs preprocessing operations such as cropping and scaling to ensure pixel uniformity. The feature extraction network includes CBS module, C2f module and SPPF module, which are responsible for extracting key features of the image. The feature fusion network adopts the feature pyramid network architecture to fuse feature information at different levels to achieve multi-scale learning. The output end consists of a detection head and a classification head, which predicts the bounding box coordinates and category of the target based on the fused features.

[0044] According to one embodiment of the present invention, the feature extraction network of the YOLOv8s network model is modified as follows:

[0045] The standard convolution kernel of the CBS module in the feature extraction network is replaced with a full-dimensional dynamic convolution kernel.

[0046] In this embodiment, in the feature extraction network, the method uses a full-dimensional dynamic convolution kernel (OD-Conv) to replace the standard convolution kernels of the first and third layers of YOLOv8s. In view of the complex background and many interference factors in the UI prototype image, the use of a single convolution operation can only capture partial features of the image, and cannot effectively capture the multi-dimensional information of the target, resulting in the model being difficult to adapt to local changes in the same type of targets. In particular, considering that UI components have obvious multi-scale characteristics, the lack of detailed information will lead to decreased model accuracy and low generalization ability. The full-dimensional dynamic convolution kernel enables the model to more effectively capture the multi-dimensional features of the target by parallel learning the attention of the four dimensions of input channel, output channel, space and kernel, helping the model to better learn the key features and contextual information of the target.

[0047] Further, according to an embodiment of the present invention, the feature fusion network of the YOLOv8s network model is modified as follows:

[0048] The PANET network in the feature fusion network is replaced by the RepGFPN network with cross-scale connections.

[0049] In this embodiment, in the feature fusion network, PANET is replaced with a RepGFPN network with cross-scale connections. PANET constructs a multi-scale feature map from top to bottom, and then completes path fusion from bottom to top to generate multi-scale features. This method can effectively reduce the information loss of features during the top-down transmission process, but it is difficult to balance the importance of features at different levels during the path fusion process, which may cause the model to fail to capture the key features of the target and thus overfit. RepGFPN uses a CSP structure to replace the ordinary 3×3 convolution kernel, uses different channel dimensions for feature maps at different levels, and introduces a reparameterization mechanism and an efficient layer aggregation network (ELAN) mechanism. These optimizations help the model accept more node outputs, more efficiently fuse high-level semantic information and low-level spatial information, and effectively solve the overfitting problem of the model.

[0050] Further, according to one embodiment of the present invention, the loss function of the YOLOv8s network model is modified as follows:

[0051] Replace CIoU Loss with SIoU Loss, and use the sum of DFL Loss and SIoU Loss as the positioning loss function.

[0052] In this implementation, YOLOv8s uses the sum of DFL Loss (Distribution Focal Loss) and CIoULoss as the positioning loss function. Since CIoU Loss has a certain ambiguity in the relative value described by the criss-cross and parallel ratio, it does not consider the problem of direction mismatch between the real box and the predicted box, which affects the convergence speed of the network. Therefore, SIoULoss is used to replace CIoU Loss. SIoU Loss introduces directionality in the cost function, considers the angle of the vector between the expectation and the regression, and redefines the angle penalty term. Compared with CIoU Loss, it can effectively improve the training speed of the model and the accuracy of reasoning.

[0053] Further, according to an embodiment of the present invention, under the same experimental conditions, using the full-dimensional dynamic convolution kernel, the model accuracy, recall rate, F1 score and average precision increased by 0.7%, 5.3%, 3.0% and 1.4% respectively. Using the RepGFPN structure to replace the PANet structure, the model accuracy, recall rate, F1 score and average precision increased by 0.5%, 3.9%, 2.2% and 0.6% respectively. After changing the loss function to SIoU, the model accuracy, recall rate, F1 score and average precision increased by 0.2%, 4.3%, 2.3% and 0.9% respectively. Compared with the YOLOv8s model, the accuracy, recall rate, F1 score and average precision of the improved model (OD_YOLOv8s model) increased by 5.6%, 7.0%, 6.3% and 3.3% respectively. The above conclusions prove that the proposed algorithm is superior to YOLOv8s in the front-end application automatic generation task.

[0054] According to the above scheme of the present invention, in order to break through the inherent limitations of the image description task, the present invention regards the automatic generation of front-end applications as a small sample target detection task. The deep learning model no longer directly extracts feature descriptions from the image to generate front-end code, but focuses on identifying and locating component elements in the image, and generates a configuration file for the front-end page through the detection results. This can effectively avoid the high model cost of the image description task and the low accuracy of the generated results, but the task still has challenges such as diverse targets, complex scenes, and variable scales. In response to these challenges, the present invention proposes the above-mentioned page UI element recognition method based on target detection, which first replaces the standard convolution kernel with a full-dimensional dynamic convolution kernel (OD-Conv) to cope with the challenges of diverse targets and high-density scenes. Then, the original PANET (Path Aggregation Network) structure is replaced by a re-parameterized generalized feature pyramid network (RepGFPN), and feature maps of different levels are fused by cross-scale connections and multiple downsampling, thereby capturing feature information of different levels of the target, and effectively improving the missed detection caused by variable target scales. Finally, SIoU Loss (Soft Intersection Over Union Loss) is used to replace CIoU Loss (Complete Intersection Over Union Loss) as the loss function of the bounding box to accelerate the convergence speed of network training and improve detection performance.

[0055] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a page UI element recognition system based on target detection, comprising:

[0056] The data acquisition module obtains digital screenshots, builds a data set, and divides the data set into a training set and a test set;

[0057] The model building module modifies the feature extraction network, feature fusion network and loss function of the YOLOv8s network model to form the OD_YOLOv8s model based on the full-dimensional dynamic convolution kernel;

[0058] The model training module inputs the training set into the OD_YOLOv8s model for training to obtain the target detection model;

[0059] The detection result output module inputs the test set into the target detection model and outputs the UI element detection result of the digital large screen screenshot.

[0060] Further, according to an embodiment of the present invention, the page UI element recognition method based on target detection of the present invention further includes:

[0061] Use LabelImg software to label the images of the training set and test set;

[0062] Use CutMix, scaling, shearing, or translation to augment the training set.

[0063] In this embodiment, LabelImg software is used to annotate the images, and the annotation types include 22 categories such as charts, buttons, and navigation bars. CutMix, zoom, shear, and translate are selected as the data expansion methods. The four methods are cross-randomly used to expand, for example, 200 training sets to 2000. Model training is completed under the experimental conditions of 2000 training sets, and the model performance is tested on the test set.

[0064] Furthermore, the YOLOv8s network model consists of four main parts: input layer, feature extraction network, feature fusion network and output layer.

[0065] The input layer loads the image and performs preprocessing operations such as cropping and scaling to ensure pixel uniformity. The feature extraction network includes CBS module, C2f module and SPPF module, which are responsible for extracting key features of the image. The feature fusion network adopts the feature pyramid network architecture to fuse feature information at different levels to achieve multi-scale learning. The output end consists of a detection head and a classification head, which predicts the bounding box coordinates and category of the target based on the fused features.

[0066] According to one embodiment of the present invention, the feature extraction network of the YOLOv8s network model is modified as follows:

[0067] The standard convolution kernel of the CBS module in the feature extraction network is replaced with a full-dimensional dynamic convolution kernel.

[0068] In this embodiment, in the feature extraction network, the method uses a full-dimensional dynamic convolution kernel (OD-Conv) to replace the standard convolution kernels of the first and third layers of YOLOv8s. In view of the complex background and many interference factors in the UI prototype image, the use of a single convolution operation can only capture partial features of the image, and cannot effectively capture the multi-dimensional information of the target, resulting in the model being difficult to adapt to local changes in the same type of targets. In particular, considering that UI components have obvious multi-scale characteristics, the lack of detailed information will lead to decreased model accuracy and low generalization ability. The full-dimensional dynamic convolution kernel enables the model to more effectively capture the multi-dimensional features of the target by parallel learning the attention of the four dimensions of input channel, output channel, space and kernel, helping the model to better learn the key features and contextual information of the target.

[0069] Further, according to an embodiment of the present invention, the feature fusion network of the YOLOv8s network model is modified as follows:

[0070] The PANET network in the feature fusion network is replaced by the RepGFPN network with cross-scale connections.

[0071] In this embodiment, in the feature fusion network, PANET is replaced with a RepGFPN network with cross-scale connections. PANET constructs a multi-scale feature map from top to bottom, and then completes path fusion from bottom to top to generate multi-scale features. This method can effectively reduce the information loss of features during the top-down transmission process, but it is difficult to balance the importance of features at different levels during the path fusion process, which may cause the model to fail to capture the key features of the target and thus overfit. RepGFPN uses a CSP structure to replace the ordinary 3×3 convolution kernel, uses different channel dimensions for feature maps at different levels, and introduces a reparameterization mechanism and an efficient layer aggregation network (ELAN) mechanism. These optimizations help the model accept more node outputs, more efficiently fuse high-level semantic information and low-level spatial information, and effectively solve the overfitting problem of the model.

[0072] Further, according to one embodiment of the present invention, the loss function of the YOLOv8s network model is modified as follows:

[0073] Replace CIoU Loss with SIoU Loss, and use the sum of DFL Loss and SIoU Loss as the positioning loss function.

[0074] In this implementation, YOLOv8s uses the sum of DFL Loss (Distribution Focal Loss) and CIoULoss as the positioning loss function. Since CIoU Loss has a certain ambiguity in the relative value described by the criss-cross and parallel ratio, it does not consider the problem of direction mismatch between the real box and the predicted box, which affects the convergence speed of the network. Therefore, SIoULoss is used to replace CIoU Loss. SIoU Loss introduces directionality in the cost function, considers the angle of the vector between the expectation and the regression, and redefines the angle penalty term. Compared with CIoU Loss, it can effectively improve the training speed of the model and the accuracy of reasoning.

[0075] Further, according to an embodiment of the present invention, under the same experimental conditions, using a full-dimensional dynamic convolution kernel, the model accuracy and recall rate increased by 0.7% and 1.3%, respectively. Using the RepGFPN structure to replace the PANet structure, the model accuracy and recall rate increased by 0.5% and 1.4%, respectively. After changing the loss function to SIoU, the model accuracy and recall rate increased by 0.2% and 1.3%, respectively. Compared with the YOLOv8s model, the accuracy and recall rate of the improved model (OD_YOLOv8s model) increased by 0.8% and 1.5%, respectively. The above conclusions prove that the proposed algorithm is superior to YOLOv8s in the front-end application automatic generation task.

[0076] According to the above scheme of the present invention, in order to break through the inherent limitations of the image description task, the present invention regards the automatic generation of front-end applications as a small sample target detection task. The deep learning model no longer directly extracts feature descriptions from the image to generate front-end code, but focuses on identifying and locating component elements in the image, and generates a configuration file for the front-end page through the detection results. This can effectively avoid the high model cost of the image description task and the low accuracy of the generated results, but the task still has challenges such as diverse targets, complex scenes, and variable scales. In response to these challenges, the present invention proposes the above-mentioned page UI element recognition method based on target detection, which first replaces the standard convolution kernel with a full-dimensional dynamic convolution kernel (OD-Conv) to cope with the challenges of diverse targets and high-density scenes. Then, the original PANET (Path Aggregation Network) structure is replaced by a re-parameterized generalized feature pyramid network (RepGFPN), and feature maps of different levels are fused by cross-scale connections and multiple downsampling, thereby capturing feature information of different levels of the target, and effectively improving the missed detection caused by variable target scales. Finally, SIoU Loss (Soft Intersection Over Union Loss) is used to replace CIoU Loss (Complete Intersection Over Union Loss) as the loss function of the bounding box to accelerate the convergence speed of network training and improve detection performance.

[0077] Furthermore, to achieve the above-mentioned purpose, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the page UI element recognition method based on target detection as described above is implemented.

[0078] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the page UI element recognition method based on target detection as described above is implemented.

[0079] Those skilled in the art will appreciate that the modules and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0080] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and equipment can refer to the corresponding processes in the aforementioned method implementation methods, and will not be repeated here.

[0081] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0082] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0083] In addition, each functional module in the embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0084] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the energy-saving signal sending / receiving method of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0085] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other.

[0086] It should be understood that the size of the serial numbers of each step in the content of the invention and the implementation methods of the present invention does not absolutely mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation methods of the present invention.

Claims

1. A page UI element recognition method based on target detection, characterized in that: include: Get page images, build a dataset, and divide the dataset into a training set and a test set; Modify the feature extraction network, feature fusion network and loss function of the YOLOv8s network model to form the OD_YOLOv8s model based on the full-dimensional dynamic convolution kernel; Input the training set into the OD_YOLOv8s model for training to obtain the target detection model; The test set is input into the target detection model, and the UI element detection result of the digital large screen screenshot is output.

2. The method for identifying page UI elements based on target detection according to claim 1, characterized in that: Also includes: Use LabelImg software to label the images of the training set and test set; Use CutMix, scaling, shearing, or translation to augment the training set.

3. The method for identifying page UI elements based on target detection according to claim 1, characterized in that: The YOLOv8s network model includes an input layer, a feature extraction network, a feature fusion network and an output layer; The feature extraction network includes a CBS module, a C2f module and an SPPF module responsible for extracting key features of the image; The modification of the feature extraction network of the YOLOv8s network model is as follows: The standard convolution kernel of the CBS module in the feature extraction network is replaced with a full-dimensional dynamic convolution kernel.

4. The method for identifying page UI elements based on target detection according to claim 1, characterized in that: The feature fusion network of the YOLOv8s network model is modified as follows: The PANET network in the feature fusion network is replaced by the RepGFPN network with cross-scale connections.

5. The method for identifying page UI elements based on target detection according to any one of claims 1 to 4, characterized in that: The YOLOv8s network model uses the sum of DFL Loss and CIoU Loss as the positioning loss function; The loss function of the YOLOv8s network model is modified as follows: Replace CIoU Loss with SIoU Loss, and use the sum of DFL Loss and SIoU Loss as the positioning loss function.

6. The page UI element recognition system based on target detection is characterized by: include: The data acquisition module acquires page images, builds a data set, and divides the data set into a training set and a test set; The model building module modifies the feature extraction network, feature fusion network and loss function of the YOLOv8s network model to form the OD_YOLOv8s model based on the full-dimensional dynamic convolution kernel; The model training module inputs the training set into the OD_YOLOv8s model for training to obtain the target detection model; The detection result output module inputs the test set into the target detection model and outputs the UI element detection result of the digital large screen screenshot.

7. An electronic device, characterized in that The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the method for identifying page UI elements based on target detection as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for identifying page UI elements based on target detection as described in any one of claims 1 to 5 is implemented.

Images