Semi-automatic labeling method and device based on BS architecture and machine learning

Abstract

The invention provides a semi-automatic labeling method and device based on BS architecture and machine learning. The method comprises the following steps: acquiring to-be-labeled data uploaded by a user through a user terminal; inputting the to-be-labeled data into a trained machine learning model running in the application server, and obtaining a first labeling result output by the machine learning model; the first labeling result is a preliminary labeling result obtained through a machine learning model; the application server adopts a distributed architecture; sending the to-be-labeled data and the first labeling result to a user terminal, so that a user confirms the first labeling result; obtaining a second labeling result uploaded by the user through the user terminal; the second labeling result is obtained after the user performs correction based on the to-be-labeled data and the first labeling result. According to the semi-automatic labeling method and device based on the BS architecture and the machine learning, the BS architecture and the machine learning model are combined to achieve semi-automatic labeling of the data, and the efficiency and quality of data expression are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly to a semi-automatic annotation method and device based on a BS architecture and machine learning. Background Art

[0002] With the rapid development of artificial intelligence and machine learning technologies, data annotation has become a key link in training high-quality models. Data annotation refers to the process of adding labels to the target objects in a dataset. In existing data annotation practices, manual annotation is still the main method. However, traditional manual annotation methods are not only time-consuming and laborious, but also prone to introducing human errors. To overcome this problem, some solutions have been proposed to solve it by querying publicly available annotated datasets or using data crowdsourcing, etc. However, data crowdsourcing is difficult to ensure data security, especially for confidential data, and there are problems such as poor annotation quality of target objects due to individual differences in multi-person annotation. Summary of the Invention

[0003] The present invention provides a semi-automatic annotation method and device based on a BS architecture and machine learning to solve the technical problem of poor data annotation quality in the prior art.

[0004] The present invention provides a semi-automatic annotation method based on a BS architecture and machine learning, including: Obtaining the data to be annotated uploaded by a user through a browser in a user terminal; Inputting the data to be annotated into a trained machine learning model running in an application server, and obtaining a first annotation result corresponding to the data to be annotated output by the machine learning model; the first annotation result is a preliminary annotation result obtained by the machine learning model; the application server adopts a distributed architecture; Sending the data to be annotated and the first annotation result to the user terminal for the user to confirm the first annotation result; Obtaining a second annotation result uploaded by the user through a browser in the user terminal; the second annotation result is obtained by the user after correcting based on the data to be annotated and the first annotation result.

[0005] In some embodiments, the method further includes: Training the machine learning model based on the data to be annotated, the first annotation result and the second annotation result, and updating the parameters of the machine learning model.

[0006] In some embodiments, the training of the machine learning model based on the data to be annotated, the first annotation result and the second annotation result includes: Performing data augmentation on the data to be annotated; Train the machine learning model based on the enhanced data to be labeled, the first labeling result, and the second labeling result.

[0007] In some embodiments, the method further includes: Store the data to be labeled, the first labeling result, and the second labeling result in a database.

[0008] In some embodiments, the database includes a relational database and a non-relational database.

[0009] In some embodiments, the data to be labeled includes videos and pictures; The machine learning model includes a YOLO model, a DETR model, and an SDD model.

[0010] The present invention also provides a semi-automatic labeling device based on a BS architecture and machine learning, including: An acquisition module for acquiring, through a user terminal, data to be labeled uploaded by a user; An automatic labeling module for inputting the data to be labeled into a trained machine learning model running in an application server to obtain a first labeling result corresponding to the data to be labeled output by the machine learning model; the first labeling result is a preliminary labeling result obtained by the machine learning model; the application server adopts a distributed architecture; A sending module for sending the data to be labeled and the first labeling result to the user terminal for the user to confirm the first labeling result; A correction module for acquiring, through a user terminal, a second labeling result uploaded by the user; the second labeling result is obtained after the user corrects based on the data to be labeled and the first labeling result. The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the semi-automatic labeling method based on a BS architecture and machine learning as described in any one of the above when executing the computer program.

[0011] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the semi-automatic labeling method based on a BS architecture and machine learning as described in any one of the above.

[0012] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the semi-automatic labeling method based on a BS architecture and machine learning as described in any one of the above.

[0013] The semi-automatic annotation method and device based on the BS architecture and machine learning provided by the present invention combine the BS architecture and the machine learning model to achieve semi-automatic annotation of data, improving the efficiency and quality of data representation. Brief Description of the Drawings

[0014] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0015] Figure 1 It is a schematic flow chart of the semi-automatic annotation method based on the BS architecture and machine learning provided by the present invention.

[0016] Figure 2 It is a schematic work flow chart of the semi-automatic annotation system provided by the present invention.

[0017] Figure 3 It is a schematic flow chart of the self-driven update of the machine learning model provided by the present invention.

[0018] Figure 4 It is a schematic structural diagram of the semi-automatic annotation device based on the BS architecture and machine learning provided by the present invention.

[0019] Figure 5 It is a schematic structural diagram of the electronic device provided by the present invention. Detailed Embodiments

[0020] With the emergence of large-scale annotated data sets, object detection technology has also achieved leapfrog development. The quality of the data set seriously affects the accuracy and recall rate of the object detection model. Therefore, data annotation has gradually received attention. Data annotation is a huge project. In the traditional data annotation process, a large amount of manual participation is usually required. Annotators mark data (such as videos, images, texts, audios, etc.) manually to generate labels for use in the training of machine learning models. Although this method is straightforward, it has the following main problems: high cost, requiring a large number of annotators to be hired, resulting in high labor costs and time costs; low efficiency, the manual annotation speed is relatively slow, especially in the case of large-scale data sets, the annotation time is long; error-prone, the individual differences and fatigue of annotators may introduce human errors, affecting the annotation quality. Currently, crowdsourcing means are mainly used for annotation, that is, contracting the data annotation task to other groups through the Internet. Since the data set is annotated by different people, the generated data annotation quality is uneven, and the data set quality cannot be guaranteed in tasks with high precision requirements. Therefore, although the emergence of the crowdsourcing model has alleviated the problem of data annotation, the data annotation quality has not been improved.

[0021] With the continuous development of technology, data annotation has gradually shifted from manual annotation to semi-automatic annotation with human-machine collaboration. At the same time, semi-automatic annotation tools have emerged continuously. However, looking at the annotated data, manual annotation still dominates, with humans accounting for 70% and machines only accounting for 30% as an auxiliary. Therefore, the development trend of data annotation is to develop semi-automatic annotation technology with machine annotation as an auxiliary.

[0022] Machine learning, as a powerful technology, has made remarkable progress in various fields in recent years. In data annotation tasks, machine learning algorithms can automatically or semi-automatically generate data labels, greatly improving the annotation efficiency and accuracy. The main methods to solve the data annotation problem include transfer learning, semi-supervised learning, and active learning, etc. Among them, transfer learning is to fine-tune a small amount of annotated data through a pre-trained model or directly use it on the dataset to be annotated; semi-supervised learning is to train a model through the annotated dataset, then predict the unannotated dataset, and then use the data with good prediction results as the training set for iterative training until the optimal model is generated; active learning is also to train a model through the annotated dataset, then predict the dataset, and add the new dataset after prediction to the annotated dataset to continue training the model. Therefore, semi-supervised learning focuses on exploring the known part of the unannotated dataset, and active learning focuses on mining the unknown part of the dataset. Active learning technology is more commonly used in the field of data annotation.

[0023] The browser / server (Browser / Server, BS) architecture is a common network design pattern widely used in web applications, allowing users to access and operate applications located on the server through a web browser. Under this architecture, while leveraging the computing power and storage resources of the server to process large-scale data, a friendly user experience can be provided through the browser. The BS architecture has the following significant advantages: ease of use, users do not need to install any software and can access the application only through a web browser, greatly simplifying user operations; flexibility, users can access the application anytime and anywhere through any device (such as a computer, mobile phone, tablet, etc.); scalability, by reasonably configuring server resources, the system can be easily expanded to handle larger-scale data and more users. The human-machine collaborative interaction strategy can assist users in quick annotation, thereby reducing the number of interactions and difficulty.

[0024] The semi-automatic annotation technology of the present invention based on the BS architecture and machine learning provides an efficient and accurate data annotation solution by making full use of the distributed processing advantages of the BS network and the intelligent characteristics of machine learning algorithms, opening up a new path for the further development of the data annotation field. From the perspective of the development of artificial intelligence and machine learning, the improvement of data annotation quality and efficiency is the evolutionary direction of the future artificial intelligence field.

[0025] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the scope of protection of the present invention.

[0026] Figure 1 is a schematic flowchart of the semi-automatic annotation method based on the BS architecture and machine learning provided by the present invention. As Figure 1 shown, the method includes the following: Step 101: Obtain the data to be annotated uploaded by the user through the browser in the user terminal.

[0027] Specifically, Figure 2 is a schematic working flowchart of the semi-automatic annotation system provided by the present invention. As Figure 2 shown, the semi-automatic annotation system involved in the present invention mainly includes a user terminal (browser), an application server, and a database. The user terminal accesses the system through the browser. The application server is responsible for data processing and the operation of the machine learning model. The database is used to store the original data, annotation results, and annotation correction results.

[0028] User terminal (browser): The user accesses the system through the browser to upload data, view annotation results, and annotation corrections. The browser supports multiple devices (such as computers, mobile phones, tablets, etc.), with a user-friendly interface and simple operations.

[0029] Application server: The application server is responsible for receiving the data uploaded by the user, then running the machine learning model for automated processing and optimization, generating annotation results, and returning the results to the user. The server adopts a distributed architecture and can handle large-scale data and multi-user concurrent requests.

[0030] Database: The database is used to store the original data, automated annotation results, and annotation correction results. The database combines a relational database (such as MySQL) and a non-relational database (such as MongoDB) to ensure the flexibility and efficiency of data storage.

[0031] First, the user needs to upload data, and the data types include videos and pictures. If the data is a picture, it can be directly imported and stored. If the data is a video, the video is first saved, and then pictures are evenly extracted according to the frame extraction interval set by the user and saved.

[0032] Step 102: Input the data to be labeled into the trained machine learning model running in the application server to obtain the first labeling result corresponding to the data to be labeled output by the machine learning model; the first labeling result is the preliminary labeling result obtained through the machine learning model; the application server adopts a distributed architecture.

[0033] Specifically, after receiving the picture data, the server side runs the pre-trained machine learning model to automatically process and label the data. For the convenience of users, the server supports multiple machine learning models such as YOLO, DETR, and SDD, and the model used for this labeling can be selected on the page.

[0034] The automatic processing process of the server for the data includes the following steps: Data preprocessing: Preprocess the preliminarily labeled data, such as image scaling, image cropping, etc., to meet the input requirements of the machine learning model.

[0035] Model prediction: Run the pre-trained machine learning model to predict the preprocessed data. The types of machine learning models include but are not limited to YOLO, RT-DETR, etc. The prediction result adds automatic annotation information to the data to be labeled and stores the prediction result as files in various formats such as txt and xml to be applicable to various model trainings.

[0036] Result optimization: Based on the automatic labeling result of the machine learning model, perform optimization processing. The optimization algorithms include but are not limited to Bayesian optimization, genetic algorithms, etc., to improve the accuracy and consistency of the labeling.

[0037] Finally, store the generated labeling result in the database.

[0038] Step 103: Send the data to be labeled and the first labeling result to the user terminal for the user to confirm the first labeling result.

[0039] Specifically, in order to further improve the labeling quality, the present invention introduces a user modification feedback mechanism. The user can view the labeling result after automatic processing through the browser and correct it.

[0040] Step 104: Obtain the second labeling result uploaded by the user through the browser in the user terminal; the second labeling result is obtained after the user corrects based on the data to be labeled and the first labeling result.

[0041] Specifically, in order to further improve the annotation quality, the present invention introduces a user modification feedback mechanism. Users can view the annotation results after automated processing through a browser and correct them. During the correction process, users can re-annotate the areas or categories of interest to provide more accurate annotation information.

[0042] Finally, the final annotation data is obtained and stored in the database.

[0043] The semi-automatic annotation method based on the BS architecture and machine learning provided by the present invention is time-consuming and laborious compared with the traditional manual annotation method. Especially when dealing with large-scale data sets, the annotation time is long. By introducing a machine learning model, the present invention can automate most of the annotation tasks, significantly reducing the workload of manual annotation and thus greatly improving the annotation efficiency.

[0044] Manual annotation is easily affected by factors such as individual differences and fatigue of annotators, resulting in inconsistent annotation results or introducing errors. The present invention uses a machine learning model for automated annotation and further improves the accuracy and consistency of annotation by optimizing the algorithm in combination with user feedback, ensuring the generation of high-quality annotation data.

[0045] Employing a large number of annotators for manual annotation is costly and requires long-term training and management. By means of automated processing and optimization, the present invention reduces the need for manual annotation, thereby reducing the annotation cost. Especially in large-scale annotation tasks, the cost advantage is more significant.

[0046] The present invention adopts the BS architecture. Users do not need to install any software and can access the system for annotation only through a web browser, greatly simplifying the user operation. At the same time, the system interface is user-friendly, and users can conveniently perform preliminary annotation and view the results, enhancing the user experience.

[0047] In some embodiments, the method further includes: Training the machine learning model based on the data to be annotated, the first annotation result, and the second annotation result, and updating the parameters of the machine learning model.

[0048] In some embodiments, the training of the machine learning model based on the data to be annotated, the first annotation result, and the second annotation result includes: Performing data augmentation on the data to be annotated; Training the machine learning model based on the augmented data to be annotated, the first annotation result, and the second annotation result.

[0049] Specifically, Figure 3It is a schematic diagram of the self-driven update process of the machine learning model provided by the present invention. As Figure 3 shown, based on the preprocessed data and the manually corrected data, data augmentation processing is performed on the manually corrected and feedback data, such as image flipping, image stitching, etc., to expand the training data set. Then, the model is automatically trained. After the training is completed, it is compared with the existing model, and the model with better prediction effect is retained to improve the generalization ability and annotation accuracy of the model, complete the automatic update of the model, and ensure the forward-looking and autonomy of the algorithm model.

[0050] The present invention uploads the corrected annotation data as feedback to the server for continuously optimizing the machine learning model and improving the generalization ability and annotation accuracy of the model. Through user feedback and continuous optimization, the present invention continuously improves the accuracy and consistency of annotation to ensure the generation of high-quality annotation data. In addition, users can export pictures and annotation files in different formats according to their needs.

[0051] In addition, the present invention can further improve the annotation accuracy and efficiency by combining the machine learning model with technologies such as graph neural networks and reinforcement learning.

[0052] The semi-automatic annotation technology based on the BS architecture and machine learning provided by the present invention is an efficient and accurate data annotation solution, which improves the data annotation efficiency and promotes the development of target detection capabilities.

[0053] Next, a semi-automatic annotation device based on the BS architecture and machine learning provided by the present invention will be described. The semi-automatic annotation device based on the BS architecture and machine learning described below can be mutually corresponding and referred to the semi-automatic annotation method based on the BS architecture and machine learning described above.

[0054] Figure 4 It is a schematic diagram of the structure of the semi-automatic annotation device based on the BS architecture and machine learning provided by the present invention. As Figure 4 shown, the present invention provides a semi-automatic annotation device based on the BS architecture and machine learning, including: An acquisition module 401 is configured to acquire the data to be annotated uploaded by the user through the user terminal; An automatic annotation module 402 is configured to input the data to be annotated into a trained machine learning model running in the application server, and obtain a first annotation result corresponding to the data to be annotated output by the machine learning model; the first annotation result is a preliminary annotation result obtained by the machine learning model; the application server adopts a distributed architecture; A sending module 403 is configured to send the data to be annotated and the first annotation result to the user terminal for the user to confirm the first annotation result; The correction module 404 is used to obtain the second annotation result uploaded by the user through the user terminal; the second annotation result is obtained by the user after correcting based on the data to be annotated and the first annotation result.

[0055] Specifically, the semi-automatic annotation device based on the BS architecture and machine learning provided in the embodiments of the present application can implement all the method steps implemented by the embodiments of the semi-automatic annotation method based on the BS architecture and machine learning, and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described in this embodiment.

[0056] Figure 5 An example of the physical structure diagram of an electronic device is shown as Figure 5 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 complete mutual communication through the communication bus 840. The processor 810 can call the logical instructions in the memory 830 to execute the semi-automatic annotation method based on the BS architecture and machine learning. The method includes: Obtain the data to be annotated uploaded by the user through the browser in the user terminal; Input the data to be annotated into the trained machine learning model running in the application server, and obtain the first annotation result corresponding to the data to be annotated output by the machine learning model; the first annotation result is the preliminary annotation result obtained by the machine learning model; the application server adopts a distributed architecture; Send the data to be annotated and the first annotation result to the user terminal for the user to confirm the first annotation result; Obtain the second annotation result uploaded by the user through the browser in the user terminal; the second annotation result is obtained by the user after correcting based on the data to be annotated and the first annotation result.

[0057] In addition, when the logical instructions in the above-mentioned memory 830 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0058] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the semi-automatic annotation method based on the BS architecture and machine learning provided by the above-mentioned various methods. The method includes: Obtain the data to be annotated uploaded by the user through a browser in the user terminal; Input the data to be annotated into a trained machine learning model running in the application server to obtain a first annotation result corresponding to the data to be annotated output by the machine learning model; the first annotation result is a preliminary annotation result obtained by the machine learning model; the application server adopts a distributed architecture; Send the data to be annotated and the first annotation result to the user terminal for the user to confirm the first annotation result; Obtain a second annotation result uploaded by the user through a browser in the user terminal; the second annotation result is obtained by the user after correcting based on the data to be annotated and the first annotation result.

[0059] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the semi-automatic annotation method based on the BS architecture and machine learning provided by the above-mentioned various methods. The method includes: Obtain the data to be annotated uploaded by the user through a browser in the user terminal; Input the data to be labeled into a trained machine learning model running in an application server to obtain a first labeling result corresponding to the data to be labeled output by the machine learning model; the first labeling result is a preliminary labeling result obtained by the machine learning model; the application server adopts a distributed architecture; Send the data to be labeled and the first labeling result to a user terminal for the user to confirm the first labeling result; Obtain a second labeling result uploaded by the user through a browser in the user terminal; the second labeling result is obtained by the user after correction based on the data to be labeled and the first labeling result.

[0060] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0061] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solutions, in essence, or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0062] In addition, it should be noted that: in the embodiments of the present application, terms such as "target", "first", "second", etc. are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are usually of the same type, and the number of objects is not limited. For example, the first object can be one or multiple.

[0063] "Determining B based on A" in the embodiments of this application means that the factor A should be considered when determining B. It is not limited to "determining B only based on A", but also includes: "determining B based on A and C", "determining B based on A, C, and E", "determining C based on A and further determining B based on C", etc. Additionally, it can also include using A as a condition for determining B. For example, "when A meets the first condition, use the first method to determine B"; for another example, "when A meets the second condition, determine B"; for yet another example, "when A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be that A is used as a condition for the factor of determining B. For example, "when A meets the first condition, use the first method to determine C and further determine B based on C", etc.

[0064] In the embodiments of this application, the term "a plurality of" means two or more, and other quantifiers are similar.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semi-automatic annotation method based on BS architecture and machine learning, characterized in that: include: Obtain the data to be annotated uploaded by the user through the browser in the user terminal; Inputting the data to be annotated into a trained machine learning model running in an application server, and obtaining a first annotation result corresponding to the data to be annotated output by the machine learning model; The first annotation result is a preliminary annotation result obtained by the machine learning model; the application server adopts a distributed architecture; Sending the data to be annotated and the first annotation result to a user terminal, so that the user can confirm the first annotation result; Acquiring, through a browser in a user terminal, a second annotation result uploaded by a user; The second annotation result is obtained after the user makes corrections based on the data to be annotated and the first annotation result.

2. The semi-automatic annotation method based on BS architecture and machine learning according to claim 1 is characterized in that: The method further comprises: Based on the data to be labeled, the first labeling result and the second labeling result, the machine learning model is trained and the parameters of the machine learning model are updated.

3. The semi-automatic annotation method based on BS architecture and machine learning according to claim 2 is characterized in that: The training of the machine learning model based on the data to be labeled, the first labeling result, and the second labeling result includes: Performing data enhancement on the data to be labeled; The machine learning model is trained based on the enhanced data to be labeled, the first labeling result and the second labeling result.

4. The semi-automatic annotation method based on BS architecture and machine learning according to claim 1 is characterized in that: The method further comprises: The data to be labeled, the first labeling result and the second labeling result are stored in a database.

5. The semi-automatic annotation method based on BS architecture and machine learning according to claim 4 is characterized in that: The database includes a relational database and a non-relational database.

6. The semi-automatic annotation method based on BS architecture and machine learning according to any one of claims 1 to 5, characterized in that: The data to be annotated includes videos and pictures; The machine learning models include the YOLO model, the DETR model and the SDD model.

7. A semi-automatic annotation device based on BS architecture and machine learning, characterized in that: include: An acquisition module is used to acquire the data to be annotated uploaded by the user through the user terminal; An automatic labeling module, used for inputting the data to be labeled into a trained machine learning model running in an application server, and obtaining a first labeling result corresponding to the data to be labeled output by the machine learning model; The first annotation result is a preliminary annotation result obtained by the machine learning model; the application server adopts a distributed architecture; A sending module, used for sending the data to be annotated and the first annotation result to a user terminal, so that the user can confirm the first annotation result; A correction module, used for obtaining a second annotation result uploaded by a user through a user terminal; The second annotation result is obtained after the user makes corrections based on the data to be annotated and the first annotation result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the semi-automatic labeling method based on BS architecture and machine learning is implemented as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the semi-automatic annotation method based on BS architecture and machine learning is implemented as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the semi-automatic annotation method based on BS architecture and machine learning is implemented as described in any one of claims 1 to 6.

Images