Weak supervision industrial equipment detection method for carrying out annotation enhancement on synthetic scene constructed based on Internet images
By constructing synthetic scenes and real scenes based on Internet images, combined with the proposed prototype feature alignment module method, the problem of low accuracy in existing weakly supervised industrial equipment detection methods in complex scenarios is solved, and higher detection accuracy and robustness.
Patent Information
- Application Number
- CN202510021495.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-07
- Publication Date
- 2025-02-18
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing weakly supervised industrial equipment detection methods do not perform well in complex industrial scenarios, especially when the equipment is densely distributed, the lighting conditions are extreme or the background is extremely complicated, the detection accuracy is limited and the feature representation is not distinguishable enough.
The method of labeling enhancement of synthetic scenes constructed based on Internet images is adopted. By converting the industrial equipment images retrieved by the Internet into three-dimensional shapes, a three-dimensional shape point cloud library is constructed, a synthetic scene is generated, and a target detection network is trained using synthetic scenes and real scenes. ,Introduce a proposed prototype feature alignment module to reduce domain difference.
It improves the accuracy and stability of industrial equipment inspection, overcomes the problems of traditional high annotation cost and insufficient distinction of feature representation, and significantly improves the detection accuracy and robustness.
Smart Images
Figure CN119478750A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an equipment detection method, in particular to a weakly supervised industrial equipment detection method based on annotation enhancement of a synthetic scene constructed based on Internet images. Background Art
[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.
[0003] In the industrial field, especially in industries such as petrochemicals, manufacturing, and energy, equipment detection and monitoring are essential to ensure production safety, reduce equipment failures, and improve operational efficiency. Key equipment in these industries - such as pumps, tanks, valves, and instruments - are often in complex and dynamic environments. Such equipment is usually widely distributed, involving not only a large geographical layout, but also mutual occlusion between equipment and the influence of environmental factors such as light changes, dust, and humidity. Therefore, timely and accurate detection and identification of these devices to ensure that they always operate in the best condition is one of the core challenges of industrial automation and safety management. In practical applications, 3D object detection in industrial scenes is extremely complex. Traditional 3D detection methods usually rely on a large amount of annotated data for model training and feature extraction. However, it is very difficult to obtain such large-scale, high-quality annotated data. The variety of equipment types and complex shapes in industrial environments, coupled with frequent equipment occlusion and complex backgrounds, make manual annotation work huge and prone to mislabeling or inaccurate problems. When annotators perform 3D annotation on equipment, they must not only understand the appearance and structure of the equipment, but also face the complex industrial scene layout, which further increases the difficulty and cost of data annotation. Due to these reasons, traditional high-precision detection methods are severely limited in practical industrial applications.
[0004] With the advancement of deep learning technology, weakly supervised learning has gradually become a potential solution to the large-scale labeling problem. Weakly supervised learning technology can detect targets when labeled data is scarce, reducing the reliance on manual labeling and the cost of data preparation. However, existing weakly supervised methods still have significant limitations. Due to the sparsity of weakly supervised signals, detection accuracy is limited and feature representations are often not discriminative enough. Existing methods perform poorly in complex industrial scenarios, especially when equipment is densely distributed, lighting conditions are extreme, or the background is extremely complex.
[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0006] Purpose of the invention: The technical problem to be solved by the present invention is to provide a weakly supervised industrial equipment detection method based on annotation enhancement of synthetic scenes constructed based on Internet images in response to the shortcomings of the prior art.
[0007] In order to solve the above technical problems, the present invention discloses a weakly supervised industrial equipment detection method based on annotation enhancement of synthetic scenes constructed based on Internet images, comprising the following steps: Step 1: convert the industrial equipment image retrieved from the Internet into a three-dimensional shape and construct a three-dimensional shape point cloud library; Step 2, collecting real scenes related to the industrial equipment, and generating synthetic scenes based on the real scenes and a three-dimensional shape point cloud library; Step 3: Train the target detection network using synthetic scenes and real scenes; Step 4: During the training process of step 3, a proposed prototype feature alignment module is introduced to reduce the domain difference between the synthetic scene and the real scene; Step 5: Use the trained target detection network to perform the industrial equipment detection.
[0008] Furthermore, the industrial equipment described in step 1 includes four categories, namely: pumps, tanks, valves and instruments.
[0009] Furthermore, the generation of the synthetic scene described in step 2 includes the following steps: Step 2-1, collecting real scenes related to the industrial equipment, the real scenes include location-level annotations, and extracting a rough layout of the real scenes according to the location-level annotations of the real scenes, wherein the rough layout L is expressed as follows: ; in, represents the category of the i-th object in the real scene, represents the coordinates of the center point of the object, and N represents the number of objects in the real scene; Step 2-2, based on the rough layout L, calculate the initial layout in the synthetic scene ; Step 2-3, based on the initial layout in step 2-2 Assembling a corresponding synthetic object in a synthetic scene, wherein the synthetic object selects a three-dimensional shape of the same category as the object in the real scene from the three-dimensional shape point cloud library; Step 2-4, adjust the positional relationship between the synthesized objects to optimize the scene layout, including using gravity constraints and collision constraints.
[0010] Further, the initial layout of the calculated synthetic scene described in step 2-2 , the method is as follows: ; in, , and is random jitter, subject to distributed.
[0011] Further, the training of the target detection network using the synthetic scene and the real scene described in step 3 includes the following steps: Step 3-1, obtain the point-level annotation of the synthetic scene, input the synthetic scene into the target detection network, and calculate the strong supervision loss of the target detection network , as follows: ; in, is the center point loss, is the bounding box loss, is the semantic classification loss; Step 3-2: Obtain the position-level annotation of the real scene, input the real scene into the target detection network, and calculate the weak supervision loss of the target detection network. , including center point loss, size category loss and semantic classification loss; Step 3-3, use strong supervision loss and weak supervision loss to train the target detection network to obtain a trained target detection network.
[0012] Furthermore, the bounding box loss described in step 3-1 includes: Size loss and orientation angle estimation loss.
[0013] Furthermore, the size loss includes: Size category loss and size residual loss; among them, Each category of industrial equipment is equipped with a predefined size template, each template corresponds to a size, the target detection network predicts the size category to which the target belongs, and obtains preliminary size information, and the size category loss uses cross entropy loss to complete the category classification task; The size residual is used to describe the deviation between the actual size of the target and the size of its category template.
[0014] Furthermore, the orientation angle estimation loss includes: Direction angle category loss and direction angle residual loss; wherein the direction angle category loss is used to classify the target according to the direction angle category; the direction angle residual loss is used to regress and predict the deflection angle of the target orientation relative to its direction angle category direction.
[0015] Furthermore, the proposed prototype feature alignment module described in step 4 is introduced to reduce the domain difference between the synthetic scene and the real scene, that is, to reduce the domain difference between the synthetic scene and the real scene by minimizing the cross-domain distribution distance between the synthetic scene and the real scene in the proposed prototype feature space.
[0016] Furthermore, the proposed prototype feature alignment module described in step 4 is introduced to reduce the domain difference between the synthetic scene and the real scene, specifically including: Step 4-1: Extract the prototype features of the synthetic industrial scene and the real industrial scene, that is, extract the prototype feature vector of each category of industrial equipment ,as follows: ; in, is the number of samples of category c, is the feature representation of the candidate box of the i-th object of category c; Step 4-2 provides domain migration from synthetic scenes to real scenes, that is, by setting the domain migration loss Minimize the distribution difference between the prototype features of the synthetic scene and the real scene, the domain transfer loss , the calculation method is as follows: ; in, For the prototype features of the real scene, Prototype features for synthetic scenes; Step 4-3, improve the training loss function of the target detection network to , which is expressed as follows: .
[0017] Beneficial effects: 1. The present invention introduces arbitrary images as additional supervisory information. By utilizing the readily available 2D image Internet resources, 2D images are converted into 3D shapes, thereby introducing additional supervisory information. Based on position-level annotation, this method provides more effective annotations through the assistance of arbitrary images, thereby improving the accuracy and stability of detection.
[0018] 2. The present invention uses a synthetic scene generation module to enhance supervision information. By generating a synthetic scene with a real scene layout, the method provides a strong supervision signal for training the detection model. This method can overcome the problems of high traditional annotation costs and insufficient distinguishability of feature representation, and effectively improve the detection accuracy and effect.
[0019] 3. The present invention narrows the domain gap by aligning the prototype distribution in the real scene and the synthetic scene constructed based on the retrieved image. This allows the model to learn more discriminative features, further improving the accuracy and robustness of detection. Therefore, the knowledge obtained from the synthetic scene can be better transferred to the real scene, improving the performance of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.
[0021] Figure 1 It is a flow chart of the present invention.
[0022] Figure 2 It is a schematic diagram of the network structure of the present invention. DETAILED DESCRIPTION
[0023] The present invention aims to construct a synthetic 3D scene by retrieving and using 2D images of industrial equipment on the Internet, thereby providing additional annotation information and improving the performance of weakly supervised 3D object detection. The present invention can reduce the reliance on manual annotation, solve the domain difference problem, and significantly improve the generalization ability of the detector.
[0024] Technical solution: A weakly supervised industrial equipment detection method based on synthetic scenes constructed from Internet images for annotation enhancement. The core is to use any two-dimensional images retrieved from the Internet as auxiliary information to enhance the annotation information of weakly supervised three-dimensional object detection (position-level annotation, i.e., the category and center point coordinates of the object), thereby improving the detection accuracy, such as Figure 1 As shown, the following steps are included: Step 1: 3D shape construction based on retrieved images: Random 2D images collected from the Internet are transformed into 3D shapes through a specific pixel-to-point conversion network and algorithm, and a 3D shape point cloud library is created.
[0025] Step 2: Construction of synthetic scenes: By applying the generated 3D shapes to the scene synthesis module, synthetic scenes can be generated. The synthetic scenes contain objects and backgrounds similar to the real scenes, providing additional supervision information for training detectors.
[0026] Step 3: Supervised training of the object detection network: Both synthetic scenes constructed based on the retrieved images and real point cloud scenes are used to train the object detection network. Objects in the synthetic scenes can generate automatic point-level annotations to indicate the location and bounding box information of the objects, and the training loss is the same as the fully supervised detection target. For real scenes, the supervision information used to guide network training is the position-level annotation of the weakly supervised setting, which can indicate the location and category information of the objects.
[0027] Step 4: Propose prototype feature alignment module: In order to bridge the domain gap between synthetic scenes and real scenes, a proposed prototype feature alignment module is introduced. This module aligns the prototype features in synthetic scenes and real scenes to make them more consistent in feature representation, thereby improving the generalization ability and robustness of the detector.
[0028] Step 1 is to construct 3D shapes using a large amount of 2D image information available on the Internet. Compared with a limited number of 3D shape datasets, the 3D shape library generated from 2D images is free from quantity and variety restrictions. The process is divided into the following steps: Step 1-1: Use automated crawler tools to retrieve 2D images related to industrial equipment from the Internet, specifically pumps, tanks, valves, and instruments. Filter based on specific keywords when crawling, and ensure that the image resolution and quality are sufficient for the subsequent 3D reconstruction process. Through data cleaning and deduplication, remove blurry and duplicate images to build a high-quality industrial equipment image set covering different angles and backgrounds. .
[0029] Step 1-2: Use the pixel-to-point conversion network to convert the image randomly grabbed from the network in step 1-1 Convert to 3D shape , build a 3D shape repository. To improve the conversion accuracy, the network is optimized using a supervised learning method with a known 3D model during training, ensuring that the generated 3D shapes have high fidelity.
[0030] In steps 1-3, a uniform sampling strategy is used to extract point clouds from the generated 3D shape mesh. The density of the point clouds is adjusted according to the requirements of the target detection model. These 3D shape point clouds can be used for subsequent scene synthesis and annotation information enhancement. In order to ensure the availability of subsequent scene synthesis, each 3D point cloud data is structured and stored, including information such as point coordinates, normal vectors and other features, to build a complete 3D shape point cloud library.
[0031] Step 2 is to select appropriate 3D shapes from the pre-generated 3D shapes with hundreds of similar images in step 1 for each object of a given weakly annotated ground truth scene. These selected objects are then combined to form a new scene based on the coarse layout information extracted from the box-level annotations of the real scene. The process is divided into the following steps: Step 2-1: Extract the rough layout based on the location-level annotations of the real scene. Extract location-level annotations (such as the category of each object) from the real industrial scene. , center point coordinates In order to achieve a reasonable and realistic layout, the initial position of each object in the synthetic scene is the original rough layout of the real scene. Generated by using random jitter The positions are fine-tuned to ensure similarity to the real scene while increasing the diversity of the synthetic scene. The calculated layout is: ; In step 2-2, each object in the synthetic scene is assembled according to the initial position in step 2-1, and the scene layout is optimized by adjusting the positional relationship between the shapes. Each object in the synthetic scene is selected from the generated 3D shapes of the same category as the objects in the real scene. The layout of the synthetic scene is adjusted according to two main principles. First, in order to meet the gravity constraint, the supported shape falls on the supporting surface of the supporting body, and the bottom of the supported shape is aligned with the supporting surface of the supporting body. Second, there is no collision between the 3D shapes. Therefore, the updated layout can further ensure the rationality and authenticity of the final generated synthetic scene.
[0032] Step 3 is to provide supervision information guidance of synthetic scenes and real scenes for the training of the target detection network. The process is divided into the following steps: Step 3-1 provides point-level annotations for the synthetic scene. Unlike real scenes that only have position-level annotations, the supervision information for synthetic scenes is complete. Since the generated 3D shapes have explicit knowledge, the complete annotations for the synthetic scene can be automatically generated. Since the synthetic scene has point-level supervision information, its loss function in the proposal and classification stages includes the target loss (center point loss), bounding box loss (size and orientation estimation loss) and semantic classification loss For the bounding box loss , Its size loss includes size category loss and size residual loss. Each category of objects may have a predefined size template (e.g., car, pedestrian, cyclist, etc.), each template corresponds to a size, each category may have multiple size templates, and a template can be represented by a size category. The model obtains the preliminary size information of an object by predicting which size category an object belongs to. The size category loss can use cross entropy loss to complete the category classification task. However, even objects belonging to the same size category may still have subtle individual differences in size (e.g., different models of cars may have different lengths or widths). For this reason, the size residual is used to model the deviation between the actual size of the object and the size of the category template to further improve the accuracy of size estimation. Similarly, in the orientation angle estimation, the loss function consists of the orientation angle category loss and the orientation angle residual loss. The output of the network includes two parts: the orientation angle category and the orientation angle residual. The orientation angle category task is a classification problem, which is usually used to divide the object orientation into discrete direction categories (e.g., east, south, west, north or more refined partitions). The prediction of the residual error of the orientation angle is a regression task, which is used to characterize the slight deviation of the orientation of an object relative to the direction of its category template (for example, a deflection of a certain degree on the basis of the predetermined direction). Splitting the orientation estimation into two subtasks, category classification and residual regression, can significantly reduce the difficulty of prediction. The classification task is relatively simple, and as long as the orientation category is predicted correctly, the final estimated angle error will usually be small. In contrast, direct regression prediction of specific angles of 0-360 degrees will significantly increase the complexity of the task and require a higher learning ability of the model.
[0033] The training loss function of the target detection network in this part is: ; Step 3-2 provides the position-level annotation of the real scene. The weak supervision information of the real scene includes the position and category information of the objects in the scene. Due to the lack of labels for heading angle and object size, the loss function in the proposal and classification stages is The center point loss, size category loss and semantic classification loss are still retained.
[0034] Step 4 is to facilitate weak object detection. A proposed prototype feature alignment method is introduced to utilize strongly supervised synthetic scenes. This method is based on the structural consistency of synthetic scenes and real scenes. By minimizing the distribution distance between synthetic scenes and real scenes across domains in the proposed prototype feature space, the object detection network can learn discriminative representations with only position-level annotations of real scenes.
[0035] Step 4-1 provides prototype features of synthetic industrial scenes and real industrial scenes. To reduce the domain difference between synthetic and real scenes, extract the prototype feature vector of each category: ; in, is the number of samples of category c, It is the feature representation of candidate box i of category c.
[0036] Step 4-2 provides domain migration from synthetic scenes to real scenes. By setting the domain migration loss Minimize the distribution difference between prototype features of synthetic and real scenes.
[0037] ; in, For the prototype features of the real scene, Prototype features for synthetic scenes.
[0038] Finally, the training loss function of the target detection network is improved to: ; Example
[0039] This embodiment proposes a weakly supervised industrial equipment detection method based on synthetic scenes constructed from Internet images for annotation enhancement, which specifically includes the following steps: Step 1: Convert any 2D image collected from the Internet into a 3D shape; Step 2: Generate a synthetic scene with a real point cloud scene layout; Step 3: Train the object detection network using the point-level annotations of the synthetic scene and the location-level annotations of the real scene; Step 4: A proposal prototype feature alignment module is introduced to bridge the domain gap between synthetic scenes and real scenes.
[0040] like Figure 2 As shown in Figure 1, step 1 is to construct 3D shapes using a large amount of 2D image information available on the Internet. Compared with a limited number of 3D shape datasets, the 3D shape library generated from 2D images is free from quantity and type restrictions. Using the rich 2D image resources on the Internet and generating 3D shapes through deep learning technology can greatly expand the diversity of 3D shapes and reduce dependence on manual annotation. The process is divided into the following steps: Step 1-1: Use a web crawler to crawl a large number of two-dimensional images of industrial equipment from the Internet. These images cover four common equipment categories: pumps, tanks, valves, and instruments. To ensure data quality, the collected images are cleaned and classified, and fuzzy or irrelevant content is removed, ultimately forming a high-quality image library covering common equipment types in industrial scenarios.
[0041] Step 1-2, through a simple pixel-to-point conversion network, the image randomly grabbed from the network in step 1-1 is used to build a 3D shape repository. The pixel-to-point conversion network is composed of an image feature network and a cascaded mesh deformation network. Given a single color image As input, the pixel-to-point conversion network directly outputs a 3D mesh P. Its image feature network uses a similar architecture to VGG16. In the forward path, it extracts perceptual image features 𝐹 through multiple layers of convolution. These features are then input into a series of cascaded mesh deformation networks consisting of 3 meshes to transform the initial template mesh into a high-precision 3D device mesh.
[0042] In steps 1-3, point clouds are obtained from the generated 3D shape mesh by uniform sampling. These 3D shape point clouds can be used for subsequent scene synthesis and annotation information enhancement.
[0043] Step 2: Generate a synthetic scene with a realistic scene layout. The complexity and diversity of the layout in industrial scenes place high demands on target detection. By generating realistic synthetic scenes, we can not only expand the training data, but also provide more complete annotation information for the model. The process is divided into the following steps: Step 2-1, extract the rough layout of the real scene based on its position-level annotation. In order to achieve a reasonable and realistic layout, the initial position of each object in the synthetic scene is the original rough layout of the real scene. Generated.
[0044] In step 2-2, each object in the synthetic scene is assembled according to the initial position in step 2-1, and the scene layout is optimized by adjusting the positional relationship between the shapes. Each object in the synthetic scene is selected from the generated 3D shapes of the same category as the objects in the real scene. The layout of the synthetic scene is adjusted according to two main principles. First, in order to meet the gravity constraint, the supported shape falls on the supporting surface of the supporting body, and the bottom of the supported shape is aligned with the supporting surface of the supporting body. Second, there is no collision between the 3D shapes. Therefore, the updated layout can further ensure the rationality and authenticity of the final generated synthetic scene.
[0045] Step 3 is to provide supervision information guidance of synthetic scenes and real scenes for the training of the target detection network. Synthetic scenes can provide accurate point-level annotations to make up for the sparse annotation problem of real scenes. At the same time, the robustness and accuracy of the detection model are improved through transfer learning by combining the two types of annotation information. The process is divided into the following steps: Step 3.1 provides point-level annotations for the synthetic scene. Unlike real scenes that only have position-level annotations, the supervision information for synthetic scenes is complete. Since the generated 3D shapes have explicit knowledge, the complete annotations for the synthetic scene can be automatically generated. Since the synthetic scene has point-level supervision, its loss function in the proposal and classification stages includes the target loss (center point loss), bounding box loss (size and orientation estimation loss) and semantic classification loss The training loss function of the target detection network in this part is: ; Step 3.2 provides the position-level annotation of the real scene. The weak supervision information of the real scene includes the position and category information of the objects in the scene. Due to the lack of labels for heading angle and object size, the loss function in the proposal and classification stages still retains the center point loss, size category loss and semantic classification loss.
[0046] Step 4 is to facilitate weak object detection. A proposed prototype feature alignment method is introduced to utilize strongly supervised synthetic scenes. This method is based on the structural consistency of synthetic scenes and real scenes. By minimizing the distribution distance between synthetic scenes and real scenes across domains in the proposed prototype feature space, the object detection network can learn discriminative representations with only position-level annotations of real scenes. Specifically, feature alignment is achieved by minimizing the L2 distance between the feature distributions of synthetic scenes and real scenes. The related loss is defined as: ; in, For the prototype features of the real scene, Prototype features for synthetic scenes.
[0047] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium can store a computer program, and when the computer program is executed by the data processing unit, the invention content of the weakly supervised industrial equipment detection method for labeling enhancement based on a synthetic scene constructed based on Internet images provided by the present invention and some or all of the steps in each embodiment. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
[0048] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on such understanding, the technical solutions in the embodiments of the present invention can be essentially or partly contributed to the prior art in the form of computer programs, i.e., software products, which can be stored in a storage medium and include several instructions for enabling a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, an MCU or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.
[0049] The present invention provides a method and idea for weakly supervised industrial equipment detection based on synthetic scenes constructed from Internet images for annotation enhancement. There are many methods and approaches to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention. All components not specified in this embodiment can be implemented using existing technologies.
Claims
1. A weakly supervised industrial equipment detection method based on annotation enhancement of synthetic scenes constructed based on Internet images, characterized in that: The steps include: Step 1: convert the industrial equipment image retrieved from the Internet into a three-dimensional shape and construct a three-dimensional shape point cloud library; Step 2, collecting real scenes related to the industrial equipment, and generating synthetic scenes based on the real scenes and a three-dimensional shape point cloud library; Step 3: Train the target detection network using synthetic scenes and real scenes; Step 4: During the training process of step 3, a proposed prototype feature alignment module is introduced to reduce the domain difference between the synthetic scene and the real scene; Step 5: Use the trained target detection network to perform the industrial equipment detection.
2. According to claim 1, a weakly supervised industrial equipment detection method for annotation enhancement based on synthetic scenes constructed based on Internet images is characterized in that: The industrial equipment described in step 1 includes four categories: pumps, tanks, valves, and instruments.
3. According to claim 2, a weakly supervised industrial equipment detection method for annotation enhancement based on synthetic scenes constructed based on Internet images is characterized in that: The generation of the synthetic scene described in step 2 includes the following steps: Step 2-1, collecting real scenes related to the industrial equipment, the real scenes include location-level annotations, and extracting a rough layout of the real scenes according to the location-level annotations of the real scenes, wherein the rough layout L is expressed as follows: ； in, represents the category of the i-th object in the real scene, represents the coordinates of the center point of the object, and N represents the number of objects in the real scene; Step 2-2, based on the rough layout L, calculate the initial layout in the synthetic scene ; Step 2-3, based on the initial layout in step 2-2 Assembling a corresponding synthetic object in a synthetic scene, wherein the synthetic object selects a three-dimensional shape of the same category as the object in the real scene from the three-dimensional shape point cloud library; Step 2-4, adjust the positional relationship between the synthesized objects to optimize the scene layout, including using gravity constraints and collision constraints.
4. According to claim 3, a weakly supervised industrial equipment detection method for annotation enhancement based on synthetic scenes constructed based on Internet images is characterized in that: The initial layout of the scene in the calculation synthesis described in step 2-2 , the method is as follows: ； in, , and is random jitter, subject to distributed.
5. According to claim 4, a weakly supervised industrial equipment detection method for annotation enhancement based on synthetic scenes constructed based on Internet images is characterized in that: The training of the target detection network using synthetic scenes and real scenes described in step 3 includes the following steps: Step 3-1, obtain the point-level annotation of the synthetic scene, input the synthetic scene into the target detection network, and calculate the strong supervision loss of the target detection network , as follows: ； in, is the center point loss, is the bounding box loss, is the semantic classification loss; Step 3-2: Obtain the position-level annotation of the real scene, input the real scene into the target detection network, and calculate the weak supervision loss of the target detection network. , including center point loss, size category loss and semantic classification loss; Step 3-3, use strong supervision loss and weak supervision loss to train the target detection network to obtain a trained target detection network.
6. The weakly supervised industrial equipment detection method for annotation enhancement based on synthetic scenes constructed based on Internet images according to claim 5 is characterized in that: The bounding box loss described in step 3-1, including: Size loss and orientation angle estimation loss.
7. The weakly supervised industrial equipment detection method for annotation enhancement based on synthetic scenes constructed based on Internet images according to claim 6 is characterized in that: The size loss includes: Size category loss and size residual loss; among them, Each category of industrial equipment is equipped with a predefined size template, each template corresponds to a size, the target detection network predicts the size category to which the target belongs, and obtains preliminary size information, and the size category loss uses cross entropy loss to complete the category classification task; The size residual is used to describe the deviation between the actual size of the target and the size of its category template.
8. The weakly supervised industrial equipment detection method for annotation enhancement based on synthetic scenes constructed based on Internet images according to claim 7 is characterized in that: The heading angle estimation loss includes: Direction angle category loss and direction angle residual loss; wherein the direction angle category loss is used to classify the target according to the direction angle category; the direction angle residual loss is used to regress and predict the deflection angle of the target orientation relative to its direction angle category direction.
9. The weakly supervised industrial equipment detection method for annotation enhancement based on synthetic scenes constructed based on Internet images according to claim 8 is characterized in that: The proposed prototype feature alignment module described in step 4 is introduced to reduce the domain difference between the synthetic scene and the real scene, that is, to reduce the domain difference between the synthetic scene and the real scene by minimizing the cross-domain distribution distance between the synthetic scene and the real scene in the proposed prototype feature space.
10. The weakly supervised industrial equipment detection method for annotation enhancement based on synthetic scenes constructed based on Internet images according to claim 9, characterized in that: The proposed prototype feature alignment module described in step 4 is introduced to reduce the domain difference between the synthetic scene and the real scene, specifically including: Step 4-1: Extract the prototype features of the synthetic industrial scene and the real industrial scene, that is, extract the prototype feature vector of each category of industrial equipment ,as follows: ； in, is the number of samples of category c, is the feature representation of the candidate box of the i-th object of category c; Step 4-2 provides domain migration from synthetic scenes to real scenes, that is, by setting the domain migration loss Minimize the distribution difference between the prototype features of the synthetic scene and the real scene, the domain transfer loss , the calculation method is as follows: ； in, For the prototype features of the real scene, Prototype features for synthetic scenes; Step 4-3, improve the training loss function of the target detection network to , which is expressed as follows: 。