Intelligent image annotation system and method based on BS architecture

Abstract

The invention belongs to the technical field of aircraft artificial intelligence, and discloses a BS architecture-based intelligent image annotation system, which comprises a browser front-end interaction module, an intelligent annotation background module and an intelligent annotation model generation module, the browser front-end interaction module provides an agent page through Nginx, and the page comprises a to-be-labeled image data list, an intelligent labeling switch, a model selection setting and a labeling information display area; the intelligent labeling background module receives an intelligent labeling instruction transmitted by the browser front-end interaction module and executes a corresponding intelligent labeling core function; the intelligent labeling model generation module is obtained based on Yolov5m detection and recognition network training, and training data are obtained in a model simulation mode. According to the method, a user can intelligently label the specific target image data through the browser page. Compared with a traditional manual labeling mode, the labeling mode is easy and flexible to operate, labeling efficiency is greatly improved, and labeling cost is reduced. Compared with an existing labeling system in the Internet, the labeling system can be deployed in an internal local area network, and the privacy of labeling data is good; meanwhile, a labeling model in the intelligent labeling system is trained by simulating specific target group data, and has an intelligent labeling function on typical targets of specific target group images, so that the vacancy that the targets of the specific target group cannot be labeled by an existing labeling system is filled.

Description

Technical Field

[0001] The present invention belongs to the technical field of aircraft artificial intelligence, and relates to an intelligent system based on a Browser / Server browser server architecture, namely, a BS architecture, and specifically to an image intelligent annotation system and method based on the BS architecture. Background Art

[0002] With the rapid development of artificial intelligence theory and the continuous improvement of hardware computing power, more and more artificial intelligence algorithms have been widely used in the field of image processing, such as target detection and recognition, target segmentation, behavior recognition, etc. However, most of these methods are supervised learning methods, which require a large amount of data with labeled information for training. Traditional manual labeling methods are inefficient and costly, and cannot meet the needs of fast training and iteration of deep learning models. Therefore, it is necessary to develop an intelligent labeling system to replace manual labeling.

[0003] At present, the widely used intelligent labeling systems on the market are mainly divided into two categories. One is the labeling plug-in tool developed based on QT. The interface is similar to labelimg and labelme. Before performing intelligent labeling, you need to install and deploy the plug-in tool locally in advance, and the labeling function is relatively simple. The other is the intelligent labeling system based on BS architecture. The system is deployed in the server of the local area network. Users do not need to deploy software on the access terminal. They can directly complete the intelligent labeling function interaction by accessing the labeling system page to realize the intelligent labeling function of image data.

[0004] Compared with the intelligent annotation system developed based on QT, the intelligent annotation system based on BS architecture is more convenient and flexible. However, the existing BS architecture intelligent annotation systems on the market all need to connect to the Internet, which is not suitable for the annotation of sensitive data that should not be made public, and cannot complete the annotation task normally. Therefore, it is necessary to develop a LAN image intelligent annotation system based on BS architecture to meet the specific network model training needs. Summary of the invention

[0005] In order to solve the above problems, the present invention provides an image intelligent labeling system and method based on BS architecture, which can be deployed in an internal local area network. Based on the trained intelligent labeling model, it can effectively detect and identify images of specific target groups, and save the inferred rectangular boxes and category results in xml tag format, thereby realizing the intelligent labeling function of images of specific target groups.

[0006] The technical solution of the present invention is as follows:

[0007] An image intelligent annotation system based on BS architecture includes a browser front-end interaction module, an intelligent annotation background module and an intelligent annotation model generation module; the browser front-end interaction module provides a proxy page through Nginx, and the page includes a list of image data to be annotated, an intelligent annotation switch, a model selection setting, and an annotation information display area; the intelligent annotation background module receives the intelligent annotation instructions transmitted by the browser front-end interaction module and executes the corresponding intelligent annotation core functions; the intelligent annotation model generation module is obtained based on Yolov5m detection and recognition network training, and obtains training data through model simulation.

[0008] Furthermore, in the browser front-end interactive module, the list of image data to be annotated can be used to select the data to be intelligently annotated by clicking, and the intelligent annotation switch and model selection settings can perform manual annotation, intelligent annotation switching, and switching between different intelligent annotation models; the annotation information display area supports users to manually modify the annotation results.

[0009] Furthermore, when the smart labeling switch of the browser front-end interaction module is turned on, a smart labeling instruction will be sent each time an image data is selected, and the smart labeling instruction is transmitted to the smart labeling background module in the form of a POST request; the smart labeling background module receives and parses the POST request through Django, selects the corresponding smart labeling model, and obtains the smart labeling result through pytorch inference.

[0010] Furthermore, the intelligent labeling model uses the yolov5m model, and its reasoning process is as follows:

[0011] T=f(θ,I)

[0012]

[0013] Where I is the input image to be annotated, θ is the backbone and feature pyramid parameters of Yolov5m, f(*) represents the inference function of Yolov5m backbone and feature pyramid, T represents the features output by the feature pyramid, φ l are the parameters of the position regression network layer in the Yolov5m detection head, is the function of position regression in the Yolov5m detection head, φ t are the parameters of the category regression network layer in the Yolov5m detection head, is the function of category regression in the Yolov5m detection head, P l , P tare the position regression and category regression results of the Yolov5m network, respectively. R is the real number domain. H, W, and C represent the width, height, and number of channels of the Yolov5m network input layer. h, w, and c represent the width, height, and number of channels of the Yolov5m network feature pyramid output. l 、w l 、c l represents the width, height and number of channels of the output of the position regression part of the Yolov5m network detection head, where c l =12,h t 、w t 、c t represents the width, height and number of channels of the output of the position regression part of the Yolov5m network detection head, c t =N+1, where N is the maximum number of categories that the intelligent labeling network can output.

[0014] Furthermore, after the intelligent labeling network outputs the position regression information and the category regression information, a post-processing operation is performed to output the final position information. The position post-processing calculation process is as follows:

[0015] bx ij =scale*[2σ(P l (i,j,g+1))-0.5]+j*L g

[0016] by ij =scale*[2σ(P l (i,j,g+2))-0.5]+i*L g

[0017]

[0018] g∈[0,2]

[0019]

[0020] Where P l (i, j, g) represents the value of the g-th channel in the i-th row, j-th column, and g represents the number of feature pyramid layers corresponding to the regression channel. g Indicates the receptive field size of the g-th layer feature pyramid, bx ij 、by ij 、bw ij , bh ij They represent the abscissa, ordinate, width and height of the final output target box center, σ(*) is the growth curve function, scale is the image compression ratio of the input layer of the intelligent annotation model, and A x is the width of the network anchor box, A y is the height of the network anchor box, and e is the natural exponent.

[0021] Furthermore, after the intelligent annotation network outputs the position regression information and the category regression information, a post-processing operation is performed to output the final category confidence information. The category confidence calculation process is as follows:

[0022] bc p (i,j)=σ(P t (i,j,p))*σ(P t (i,j,N+5))

[0023] p∈[1,N]

[0024] Where P t (i,j,p) represents the value of the pth channel in the i-th row, j-th column in the position regression output, bc p (i, j) represents the confidence that the regression output target in the i-th row and j-th column is the p-th category, and N is the maximum number of categories output by the intelligent labeling network.

[0025] Furthermore, in the intelligent annotation model generation module, the training data is rendered by a virtual engine, and the 3D specific target model and simulation environment data are input to construct a specific target scene. Then, by setting different viewing angle parameters, 2D sample data of different distances, positions, and angles are obtained. The grayscale value of visible light data can be automatically obtained through the virtual engine, and the grayscale value of infrared data is obtained based on Planck radiation theorem.

[0026] Furthermore, the grayscale value of infrared data is obtained as follows:

[0027]

[0028] where ε avg is the infrared emissivity of the object material, λ is the wavelength (the upper and lower limits of the integral here are the upper and lower limits of the sensor sensitive wavelength (8 to 12 um for long-wave infrared and 3 to 5 um for medium-wave infrared), R λ is the camera's response value to wavelength λ (between 0 and 1), currently set to 1, and h is Planck's constant (6.62607×10 -34 J·s, c is the speed of light (2.99792458×10 14 um / s), T is Kelvin temperature, K is the Boltzmann constant (1.380649×10 -23 J / K), exp(*) is the natural exponential function.

[0029] A method for intelligent image annotation based on BS architecture, using the aforementioned intelligent image annotation system based on BS architecture, comprises the following steps:

[0030] S1: Provides browser proxy services through the browser front-end interaction module;

[0031] S2: Build a specific target simulation scene based on the virtual engine and render and generate 2D simulation data;

[0032] S3: Use the intelligent annotation model generation module to train the yolov5m intelligent annotation model based on the simulation training data, and modify the background configuration file to add the trained annotation model to the annotation system;

[0033] S4: Open the smart annotation button in the browser front-end interactive module and select the smart annotation model and category;

[0034] S5: Click on the image data to be annotated, generate intelligent annotation results, manually fine-tune the annotation results, and click Save Data to complete the annotation.

[0035] The beneficial effects of the present invention are as follows:

[0036] The present invention designs a specific target image intelligent labeling system based on BS architecture, and users can realize intelligent labeling of specific target image data through browser pages. Compared with the traditional manual labeling method, this labeling method is simple and flexible to operate, greatly improves the labeling efficiency, and reduces the labeling cost. Compared with the existing labeling system on the Internet, this labeling system can be deployed in an internal LAN, and the privacy of the labeling data is good; at the same time, the labeling model in the intelligent labeling system is trained by simulating the data of a specific target group, and has the function of intelligently labeling typical targets of the specific target group image, filling the gap that the existing labeling system cannot label the targets of the specific target group. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a schematic diagram of the intelligent annotation interface of the present invention;

[0038] Figure 2 A schematic diagram for selecting the intelligent annotation model of the present invention;

[0039] Figure 3 This is a schematic diagram of the intelligent labeling result of the present invention;

[0040] Figure 4 This is an example diagram of the XML annotation file format of the present invention;

[0041] Figure 5 This is a schematic diagram of an example of visible light simulation data of the present invention;

[0042] Figure 6 This is a schematic diagram of an infrared simulation data sample of the present invention;

[0043] Figure 7 This is the intelligent marking flow chart of the present invention. DETAILED DESCRIPTION

[0044] This section is an embodiment of the present invention, which is used to explain and illustrate the technical solution of the present invention. In the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0045] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate directions or positional relationships based on the orientations or positional relationships in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or case referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implying the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0046] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be interpreted in a broad sense, for example, it can be a fixed connection, a detachable connection or an integrated connection; it can be a mechanical connection or a point connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0047] Embodiment 1:

[0048] An image intelligent annotation system based on BS architecture includes a browser front-end interaction module, an intelligent annotation background module and an intelligent annotation model generation module; the browser front-end interaction module provides a proxy page through Nginx, and the page includes a list of image data to be annotated, an intelligent annotation switch, a model selection setting, and an annotation information display area; the intelligent annotation background module receives the intelligent annotation instructions transmitted by the browser front-end interaction module and executes the corresponding intelligent annotation core functions; the intelligent annotation model generation module is obtained based on Yolov5m detection and recognition network training, and obtains training data through model simulation.

[0049] In the browser front-end interactive module, the list of image data to be annotated can be used to select the data to be intelligently annotated by clicking on it. The intelligent annotation switch and model selection settings can be used for manual annotation, intelligent annotation switching, and switching between different intelligent annotation models; the annotation information display area supports users to manually modify the annotation results.

[0050] When the smart labeling switch of the browser front-end interactive module is turned on, a smart labeling instruction will be sent each time an image data is selected. The smart labeling instruction is transmitted to the smart labeling background module in the form of a POST request; the smart labeling background module receives and parses the POST request through Django, selects the corresponding smart labeling model, and obtains the smart labeling result through pytorch inference.

[0051] The intelligent labeling model uses the yolov5m model, and its reasoning process is as follows:

[0052] T=f(θ,I)

[0053]

[0054] Where I is the input image to be annotated, θ is the backbone and feature pyramid parameters of Yolov5m, f(*) represents the inference function of Yolov5m backbone and feature pyramid, T represents the features output by the feature pyramid, φ l are the parameters of the position regression network layer in the Yolov5m detection head, is the function of position regression in the Yolov5m detection head, φ t are the parameters of the category regression network layer in the Yolov5m detection head, is the function of category regression in the Yolov5m detection head, P l , P t are the position regression and category regression results of the Yolov5m network, respectively. R is the real number domain. H, W, and C represent the width, height, and number of channels of the Yolov5m network input layer. h, w, and c represent the width, height, and number of channels of the Yolov5m network feature pyramid output. l 、w l 、c l represents the width, height and number of channels of the output of the position regression part of the Yolov5m network detection head, where c l =12,h t 、w t 、c t represents the width, height and number of channels of the output of the position regression part of the Yolov5m network detection head, c t =N+1, where N is the maximum number of categories that the intelligent labeling network can output.

[0055] After the intelligent labeling network outputs the position regression information and category regression information, post-processing operations are performed to output the final position information. The position post-processing calculation process is as follows:

[0056] bx ij =scale*[2σ(P l(i,j,g+1))-0.5]+j*L g

[0057] by ij =scale*[2σ(P l (i,j,g+2))-0.5]+i*L g

[0058]

[0059] g∈[0,2]

[0060]

[0061] Where P l (i, j, g) represents the value of the g-th channel in the i-th row, j-th column, and g represents the number of feature pyramid layers corresponding to the regression channel. g Indicates the receptive field size of the g-th layer feature pyramid, bx ij 、by ij 、bw ij , bh ij They represent the abscissa, ordinate, width and height of the final output target box center, σ(*) is the growth curve function, scale is the image compression ratio of the input layer of the intelligent annotation model, and A x is the width of the network anchor box, A y is the height of the network anchor box, and e is the natural exponent.

[0062] After the intelligent annotation network outputs the position regression information and category regression information, post-processing operations are performed to output the final category confidence information. The category confidence calculation process is as follows:

[0063] bc p (i,j)=σ(P t (i,j,p))*σ(P t (i,j,N+5))

[0064] p∈[1,N]

[0065] Where P t (i,j,p) represents the value of the pth channel in the i-th row, j-th column in the position regression output, bc p (i, j) represents the confidence that the regression output target in the i-th row and j-th column is the p-th category, and N is the maximum number of categories output by the intelligent labeling network.

[0066] In the intelligent annotation model generation module, the training data is rendered by a virtual engine, and the 3D specific target model and simulation environment data are input to construct a specific target scene. Then, by setting different viewing angle parameters, 2D sample data of different distances, positions, and angles are obtained. The grayscale value of visible light data can be automatically obtained through the virtual engine, and the grayscale value of infrared data is obtained based on Planck radiation theorem.

[0067] The grayscale value of infrared data is obtained as follows:

[0068]

[0069] where ε avg is the infrared emissivity of the object material, λ is the wavelength (the upper and lower limits of the integral here are the upper and lower limits of the sensor sensitive wavelength (8 to 12 um for long-wave infrared and 3 to 5 um for medium-wave infrared), R λ is the camera's response value to wavelength λ (between 0 and 1), currently set to 1, and h is Planck's constant (6.62607×10 -34 J·s, c is the speed of light (2.99792458×10 14 um / s), T is Kelvin temperature, K is the Boltzmann constant (1.380649×10 -23 J / K), exp(*) is the natural exponential function.

[0070] A method for intelligent image annotation based on BS architecture, using the aforementioned intelligent image annotation system based on BS architecture, comprises the following steps:

[0071] S1: Provides browser proxy services through the browser front-end interaction module;

[0072] S2: Build a specific target simulation scene based on the virtual engine and render and generate 2D simulation data;

[0073] S3: Use the intelligent annotation model generation module to train the yolov5m intelligent annotation model based on the simulation training data, and modify the background configuration file to add the trained annotation model to the annotation system;

[0074] S4: Open the smart annotation button in the browser front-end interactive module and select the smart annotation model and category;

[0075] S5: Click on the image data to be annotated, generate intelligent annotation results, manually fine-tune the annotation results, and click Save Data to complete the annotation.

[0076] Embodiment 2:

[0077] The specific target image intelligent annotation system based on BS architecture is mainly divided into three parts: browser front-end interaction part, intelligent annotation background part and intelligent annotation model generation part.

[0078] (1) Browser front-end interaction

[0079] The static page of the intelligent annotation system is developed based on react, and Nginx is used to provide proxy page proxy service. The basic layout of the page is as follows: Figure 1 The left side of the page contains a list of image data to be annotated. You can click to select the data to be intelligently annotated. The top of the page contains the intelligent annotation switch, model selection settings, etc. Figure 2 , you can perform manual annotation, intelligent annotation switching, and different intelligent annotation models switching; the middle part is the annotation information display area, such as the attached Figure 3 , users can see the annotation results in this area, and users can manually modify the annotation results.

[0080] (2) Intelligent labeling background

[0081] The main function of the smart annotation backend software is to receive the smart annotation instructions from the frontend of the page and execute the corresponding smart annotation core functions. When the smart annotation switch is turned on, the frontend page will send a smart annotation request for each selected image data. The smart annotation instructions are transmitted to the backend in the form of POST requests. The backend software receives and parses the POST requests through Django, selects the corresponding smart annotation model, and obtains the smart annotation results through pytorch inference. The smart annotation model uses the yolov5m model, and its inference process is as follows:

[0082] T=f(θ,I)

[0083]

[0084] Where I is the input image to be annotated, θ is the backbone and feature pyramid parameters of Yolov5m, f(*) represents the inference function of Yolov5m backbone and feature pyramid, T represents the features output by the feature pyramid, φ l are the parameters of the position regression network layer in the Yolov5m detection head, is the function of position regression in the Yolov5m detection head, φ t are the parameters of the category regression network layer in the Yolov5m detection head, is the function of category regression in the Yolov5m detection head, P l , P tare the position regression and category regression results of the Yolov5m network, respectively. R is the real number domain. H, W, and C represent the width, height, and number of channels of the Yolov5m network input layer. h, w, and c represent the width, height, and number of channels of the Yolov5m network feature pyramid output. l 、w l 、c l represents the width, height and number of channels of the output of the position regression part of the Yolov5m network detection head, where c l =12,h t 、w t 、c t represents the width, height and number of channels of the output of the position regression part of the Yolov5m network detection head, c t =N+1, where N is the maximum number of categories that the intelligent labeling network can output.

[0085] After the intelligent annotation network outputs the position regression information and category regression information, post-processing operations are required to output the final position and category confidence information. The position post-processing calculation process is as follows:

[0086] bx ij =scale*[2σ(P l (i,j,g+1))-0.5]+j*L g

[0087] by ij =scale*[2σ(P l (i,j,g+2))-0.5]+i*L g

[0088]

[0089] g∈[0,2]

[0090]

[0091] Where P l (i, j, g) represents the value of the g-th channel in the i-th row, j-th column, and g represents the number of feature pyramid layers corresponding to the regression channel. g Indicates the receptive field size of the g-th layer feature pyramid, bx ij 、by ij 、bw ij , bh ij They represent the abscissa, ordinate, width and height of the final output target box center, σ(*) is the growth curve function, scale is the image compression ratio of the input layer of the intelligent annotation model, and A x is the width of the network anchor box, A y is the height of the network anchor box, and e is the natural exponent.

[0092] The category confidence calculation process is as follows:

[0093] bc p (i,j)=σ(P t (i,j,p))*σ(P t (i,j,N+5))

[0094] p∈[1,N]

[0095] Where P t (i,j,p) represents the value of the pth channel in the i-th row, j-th column in the position regression output, bc p (i, j) represents the confidence that the regression output target in the i-th row and j-th column is the p-th category, and N is the maximum number of categories output by the intelligent labeling network.

[0096] The final output location information and category information will be written into the specified annotation file in XML format to realize the intelligent annotation function. The XML annotation file is as follows: Figure 4 .

[0097] (3) Intelligent annotation model generation

[0098] The intelligent labeling model is based on the Yolov5m detection and recognition network training. Since the specific target image data is usually sensitive and the real training data is lacking, it is planned to obtain the training data through model simulation. The current training data is rendered by the Unreal Engine virtual engine. The specific target scene is constructed by 3D specific target model and simulated terrain data, and then 2D sample data of different distances, positions and angles is obtained by setting different viewing angle parameters. The grayscale value of visible light data can be automatically obtained through the virtual engine, and the grayscale value of infrared data is obtained based on Planck radiation theorem, as follows:

[0099]

[0100] where ε avg is the infrared emissivity of the object material, λ is the wavelength (the upper and lower limits of the integral here are the upper and lower limits of the sensor sensitive wavelength (8 to 12 um for long-wave infrared and 3 to 5 um for medium-wave infrared), R λ is the camera's response value to wavelength λ (between 0 and 1), currently set to 1, and h is Planck's constant (6.62607×10 -34 J·s, c is the speed of light (2.99792458×10 14 um / s), T is Kelvin temperature, K is the Boltzmann constant (1.380649×10 -23 J / K), exp(*) is the natural exponential function.

[0101] Some examples of simulation data are as follows: Figure 5 , 6 After obtaining the simulation training samples, a training project will be built through the pytorch framework to train the Yolov5m detection and recognition model. The trained model will be added to the intelligent labeling system by adding a new configuration file list. The user will finally choose to use the specified intelligent labeling model through the intelligent labeling model switch button.

[0102] The implementation process of intelligent annotation of specific target group image intelligent annotation system based on BS architecture is as follows: Figure 7 As shown, it is mainly divided into the following steps:

[0103] Step 1: Develop the browser page for users through React and provide browser proxy service through nginx;

[0104] Step 2: Build a specific target simulation scene based on Unreal Engine, and render and generate 2D simulation data (with self-annotation);

[0105] Step 3: Train the yolov5m intelligent annotation model based on the simulation training data, and modify the background configuration file to add the trained annotation model to the annotation system;

[0106] Step 4: Open the smart labeling button in the browser page and select the smart labeling model and category;

[0107] Step 5: Click the image data to be annotated, generate intelligent annotation results, manually fine-tune the annotation results, and click Save Data to complete the annotation.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the field can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. An image intelligent annotation system based on BS architecture, characterized in that: It includes a browser front-end interaction module, an intelligent annotation background module and an intelligent annotation model generation module; the browser front-end interaction module provides a proxy page through Nginx, and the page includes a list of image data to be annotated, an intelligent annotation switch, a model selection setting, and an annotation information display area; The intelligent labeling backend module receives the intelligent labeling instructions transmitted by the browser front-end interaction module and executes the corresponding intelligent labeling core functions; the intelligent labeling model generation module is based on the Yolov5m detection and recognition network training and obtains training data through model simulation.

2. According to the BS architecture-based image intelligent annotation system of claim 1, it is characterized in that: In the browser front-end interactive module, the list of image data to be annotated can be used to select the data to be intelligently annotated by clicking on it. The intelligent annotation switch and model selection settings can be used for manual annotation, intelligent annotation switching, and switching between different intelligent annotation models; the annotation information display area supports users to manually modify the annotation results.

3. According to the BS architecture-based image intelligent annotation system of claim 1, it is characterized in that: When the smart annotation switch of the browser front-end interactive module is turned on, a smart annotation instruction will be sent every time a picture data is selected. The smart annotation instruction is transmitted to the smart annotation back-end module in the form of a POST request; The intelligent labeling backend module receives and parses the POST request through Django, selects the corresponding intelligent labeling model, and obtains the intelligent labeling result through pytorch inference.

4. According to claim 3, the image intelligent annotation system based on BS architecture is characterized in that: The intelligent labeling model uses the yolov5m model, and its reasoning process is as follows: T=f(θ,I) Where I is the input image to be annotated, θ is the backbone and feature pyramid parameters of Yolov5m, f(*) represents the inference function of Yolov5m backbone and feature pyramid, T represents the features output by the feature pyramid, φ1 is the parameter of the position regression network layer in the Yolov5m detection head, is the function of position regression in the Yolov5m detection head, φ t are the parameters of the category regression network layer in the Yolov5m detection head, is the function of category regression in the Yolov5m detection head, P l , P t are the position regression and category regression results of the Yolov5m network, respectively. R is the real number domain. H, W, C represent the width, height, and number of channels of the Yolov5m network input layer. h, w, c represent the width, height, and number of channels of the Yolov5m network feature pyramid output. h1, w1, c1 represent the width, height, and number of channels of the output of the position regression part of the Yolov5m network detection head. Among them, c1=12, h t 、w t 、c t represents the width, height and number of channels of the output of the position regression part of the Yolov5m network detection head, c t =N+1, where N is the maximum number of categories that the intelligent labeling network can output.

5. According to claim 4, the image intelligent annotation system based on BS architecture is characterized in that: After the intelligent labeling network outputs the position regression information and category regression information, post-processing operations are performed to output the final position information. The position post-processing calculation process is as follows: bx ij =scale*[2σ(P l (i,j,g+1))-0.5]+j*L g by ij =scale*[2σ(P l (i,j,g+2))-0.5]+i*L g Where P1(i, j, g) represents the value of the g-th channel in the i-th row, j-th column, and g represents the number of feature pyramid layers corresponding to the regression channel. g Indicates the receptive field size of the g-th layer feature pyramid, bx ij 、by ij 、bw i，j , bh ij They represent the abscissa, ordinate, width and height of the final output target box center, σ(*) is the growth curve function, scale is the image compression ratio of the input layer of the intelligent annotation model, and A x is the width of the network anchor box, A y is the height of the network anchor box, and e is the natural exponent.

6. The image intelligent annotation system based on BS architecture according to claim 5 is characterized in that: After the intelligent annotation network outputs the position regression information and category regression information, post-processing operations are performed to output the final category confidence information. The category confidence calculation process is as follows: b.c. p (i,j)=σ(P t (i,j,p))*σ(P t (i, j, N+5)) p∈[1,N] Where P t (i, j, p) represents the value of the pth channel in the i-th row, j-th column in the position regression output, bc p (i, j) represents the confidence that the regression output target in the i-th row and j-th column is the p-th category, and N is the maximum number of categories output by the intelligent labeling network.

7. The image intelligent annotation system based on BS architecture according to claim 1 is characterized in that: In the intelligent annotation model generation module, the training data is rendered by a virtual engine, and the 3D specific target model and simulation environment data are input to construct a specific target scene. Then, by setting different viewing angle parameters, 2D sample data of different distances, positions, and angles are obtained. The grayscale value of visible light data can be automatically obtained through the virtual engine, and the grayscale value of infrared data is obtained based on Planck radiation theorem.

8. The image intelligent annotation system based on BS architecture according to claim 7 is characterized in that: The grayscale value of infrared data is obtained as follows: where ε avg is the infrared emissivity of the object material, λ is the wavelength, R λ is the response value of the camera to the wavelength λ, h is Planck's constant, c is the speed of light, T is the Kelvin temperature, K is the Boltzmann constant, and exp(*) is the natural exponential function.

9. A method for intelligent image annotation based on BS architecture, using an intelligent image annotation system based on BS architecture as claimed in any one of claims 1 to 8, comprising the following steps: S1: Provides browser proxy services through the browser front-end interaction module; S2: Build a specific target simulation scene based on the virtual engine and render and generate 2D simulation data; S3: Use the intelligent annotation model generation module to train the yolov5m intelligent annotation model based on the simulation training data, and modify the background configuration file to add the trained annotation model to the annotation system; S4: Open the smart annotation button in the browser front-end interactive module and select the smart annotation model and category; S5: Click on the image data to be annotated, generate intelligent annotation results, manually fine-tune the annotation results, and click Save Data to complete the annotation.

Images