Method for collecting personnel or article recognition dataset in industrial scene

Abstract

The application provides a collection method of personnel or article recognition data set in an industrial scene, comprising a carrier carrying a microwave radar module, an optical image sensor module and a laser radar module, and collecting through the following steps: step 1) in a specific scene, the carrier obtains visual angle data at different angles around the scene where the target personnel or article is located; step 2) according to the life signal of the personnel target or the pre-collected microwave signal of the object target, the target position and posture are confirmed, and the target in the original optical image sensor is labeled; step 3) an empty scene without the target is obtained; step 4) through the multi-view optical data of the empty scene and the target, new views of the target and the empty scene are generated; and step 5) the labeling of the target in the new view under the new scene with the target is generated. The method can generate a target recognition data set with view diversity, environment diversity and target posture diversity.

Description

Technical Field

[0001] This invention relates to the field of industrial application technology of artificial intelligence combined with microwave radar technology, specifically, to a method for collecting identification datasets for people or specific objects in industrial scenarios. Background Technology

[0002] In industrial settings, production safety issues are becoming increasingly prominent. With the maturation of artificial intelligence technology, methods using optical image sensors and target recognition algorithms to detect dangerous behaviors and environmental anomalies are being widely applied. Applying such methods requires constructing target recognition datasets with diverse perspectives and high generalization capabilities specific to particular scenarios.

[0003] In industrial settings, the main challenges include the following: The first issue is staff identification. To ensure staff safety or facilitate management, staff are often required to wear special protective clothing or work clothes. Due to the special nature of the materials used in these protective suits or work clothes, such as reflective strips, obscuring human feature points, blurring texture features, and optical features being easily affected by the environment, existing target recognition datasets trained on these models cannot properly identify personnel targets.

[0004] Secondly, there's the issue of industrial production components. The objects that need to be identified are often pre-installed industrial parts with specific sizes and shapes. For safety reasons, markings and protective materials are often sprayed onto the surface of these objects. However, for labeling work, these markings and protective materials can lead to mislabeling and omissions.

[0005] Thirdly, there is the issue of ambient lighting. Due to the influence of the production process, the ambient color temperature and brightness will change periodically. During the production process, momentary strong light, momentary darkness, and strong electromagnetic interference frequently occur. For these extreme environments, if the dataset does not contain corresponding samples, the prediction accuracy of the algorithm model will be reduced.

[0006] Fourthly, there's the issue of changes in production process arrangements. Production processes and environments constantly change as production progresses. The angles and orientations of the cameras used for monitoring also need to be adjusted continuously as production continues.

[0007] The reasons mentioned above make it quite difficult to obtain high-quality target recognition datasets in industrial environments.

[0008] Traditional dataset annotation requires collecting data using fixed cameras and then manually labeling it. This method is not only very labor-intensive and difficult to guarantee the accuracy of the annotations, but also results in data with poor generalization and perspective diversity.

[0009] With the development of radar technology, the technology of using microwave radar to achieve tasks such as object attitude estimation and target recognition has become increasingly mature. Although high-resolution radar is more expensive due to limitations in microwave radar performance, it is largely unaffected by smoke or changes in environmental optical conditions.

[0010] With the development of NeRF technology, techniques for 3D reconstruction and new perspective generation for specific scenes and targets based on existing data have become increasingly mature. How to utilize these technologies, combining the advantages of microwave radar with the characteristics of optical recognition, to solve various problems existing in traditional optical recognition technologies and generate high-quality target recognition datasets with diverse perspectives, environments, and targets has become a major problem that urgently needs to be solved.

[0011] In order to solve the above problems, people have been seeking an ideal technological solution. Summary of the Invention

[0012] The purpose of this invention is to address the shortcomings of existing technologies by providing a method for estimating the position and attitude of specific targets using microwave radar with minimal manpower, thereby generating labeled data from optical image sensors. By employing lidar and unmanned vehicles carrying sensors, the system cruises in specific environments and samples data. Using a NeRF (Neural Image Recognition) method, labeled data of targets (humans and objects) from new perspectives is synthesized. This method enhances the image data, generating high-quality datasets for identifying people or objects in industrial scenarios with diverse perspectives, environments, and targets.

[0013] To achieve the above objectives, the technical solution adopted by this invention is: a method for collecting personnel identification datasets in industrial scenarios, comprising a vehicle carrying a microwave radar module, an optical image sensing module, and a lidar module, and collecting data through the following steps: Step 1) The vehicle acquires sensor data from several different angles around the scene where the target person or object is located; Step 2) Obtain the position and posture of the personnel target: 2.1) Obtain the vital signs of the target personnel and determine the location of the target human body through microwave radar.

[0014] 2.2) Detect the posture of the target human body using microwave radar.

[0015] 2.3) The original optical images are labeled based on the position and posture of the human target.

[0016] Step 3) Acquire the raw optical data of the human target and generate a new perspective image of the human body using NeRF.

[0017] 3.1) Based on the target human pose in step 2.2), perform semantic segmentation of the data from the optical image sensor to obtain the pixels of the human target.

[0018] 3.2) Combine the target pose and the target pixels to generate a new perspective image of the target personnel using NeRF.

[0019] Step 4) Reconstruct empty scene image data without people: 4.1) Based on the original optical data obtained in step 1), the pixels of the human target are subtracted according to the human target pose in step 2) and the semantic segmentation information in step 3.1).

[0020] 4.2) Complete the deducted pixels using the PICNet model to obtain an empty scene image without a target.

[0021] Step 5) Based on the empty scene image generated in Step 4), generate new perspective data of the empty scene using NeRF.

[0022] Step 6) Based on the target human body model generated in Step 3) and the empty environment generated in Step 5), generate an image containing the target human body pose from a new perspective and the corresponding annotations.

[0023] Based on the above, in the aforementioned vehicle, the optical sensing module, microwave radar module, and lidar module are arranged from bottom to top, and are configured with corresponding calibration processes, including: A calibration rod is configured, on which are provided an optical image sensor calibration area, a microwave reflection area, and a lidar reflection area, which are respectively at the same height as the optical sensing module, the microwave radar module, and the lidar module. In a microwave anechoic chamber, multiple calibration rods are arranged in an isosceles apex shape from a top-down view, with the carrier located on the perpendicular bisector of the isosceles apex shape; Start each sensor module and adjust the assembly posture of the microwave radar module and the lidar module respectively so that the strongest signal points sensed are all on the horizontal center line of the sensor's field of view. This can be regarded as the registration of the microwave radar module and the lidar module. Adjusting the optical sensing module and detecting that the crosshairs of each calibration rod are on the horizontal center line of the field of view of the optical image sensor can be regarded as optical sensing module registration; To eliminate sensor assembly position offsets, specifically, the viewing angle information from different sensors is synthesized into a single viewing angle. The positional error of the same location within the field of view is calculated, and the average error data is obtained. After each sensor acquires the target position and the corresponding error is subtracted, field-of-view alignment is achieved.

[0024] A method for collecting a dataset for object recognition in an industrial setting includes a vehicle carrying a microwave radar module, an optical sensing module, and a lidar module, and the data is collected through the following steps: Step 1) In a microwave anechoic chamber, pre-acquire microwave features and a 3D model of the target object.

[0025] Step 2) During the vehicle's cruise, the position and attitude of the target object are confirmed by microwave radar.

[0026] Step 3) Based on the target's pose and position, and combined with the pre-acquired information from Step 1), the target object is subtracted from the original image.

[0027] Step 4) Use the PICNet model to complete the parts deducted in Step 3) and generate new perspective optical data of the empty scene using NeRF.

[0028] Step 5) Based on the 3D model of the target object pre-acquired in Step 1) and the empty scene information obtained in Step 4), synthesize an image of the target object in a new pose from a new perspective and generate corresponding annotations.

[0029] Based on the above, in the vehicle, the optical sensing module, microwave radar module, and lidar module are arranged from bottom to top, and are configured with corresponding calibration processes, including: A calibration rod is configured, on which are provided an optical image sensor calibration area, a microwave reflection area, and a lidar reflection area, which are respectively at the same height as the optical sensing module, the microwave radar module, and the lidar module. In a microwave anechoic chamber, multiple calibration rods are arranged in an isosceles apex shape from a top-down view, with the carrier located on the perpendicular bisector of the isosceles apex shape; Start each sensor module and adjust the assembly posture of the microwave radar module and the lidar module respectively so that the strongest signal points sensed are all on the horizontal center line of the sensor's field of view. This can be regarded as the registration of the microwave radar module and the lidar module. Adjusting the optical sensing module and detecting that the crosshairs of each calibration rod are on the horizontal center line of the optical image sensor's field of view can be considered as optical image sensor module registration; To eliminate sensor assembly position offsets, specifically, the viewing angle information from different sensors is synthesized into a single viewing angle. The positional error of the same location within the field of view is calculated, and the average error data is obtained. After each sensor acquires the target position and the corresponding error is subtracted, field-of-view alignment is achieved.

[0030] This invention has outstanding substantive features and significant progress compared to the prior art. Specifically, this invention has the following advantages: This invention uses the NeRF method to achieve 3D reconstruction of a scene containing both the environment and the target object. The target object and the empty scene are reconstructed separately, and then the two are combined to form a new scene image from any viewpoint. This generates new viewpoint data and corresponding annotations, ensuring data generalization. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the unmanned vehicle in this invention.

[0032] Figure 2 This is a schematic diagram of the structure of the correction rod in this invention.

[0033] Figure 3 This is a distribution diagram of the calibration rods during the calibration process in this invention.

[0034] Figure 4 This is one of the schematic diagrams for calibrating lidar and microwave radar in this invention.

[0035] Figure 5 This is the second schematic diagram of calibrating the lidar and microwave radar in this invention.

[0036] Figure 6 This is one of the schematic diagrams for calibrating the optical image sensor in this invention.

[0037] Figure 7 This is the second schematic diagram of calibrating the optical image sensor in this invention.

[0038] Figure 8 This is a schematic diagram of human posture recognition and 3D data modeling.

[0039] Figure 9 This is a schematic diagram of generating an empty scene with people.

[0040] Figure 10 This is a schematic diagram illustrating the generation of a new perspective scene with people.

[0041] Figure 11 This is a diagram showing the personnel annotations in the original scene.

[0042] Figure 12 This is a 3D data modeling diagram of the object.

[0043] Figure 13 This is a diagram illustrating the generation of an empty scene with items.

[0044] Figure 14 This is a schematic diagram illustrating the generation of a new perspective for an empty scene with items.

[0045] Figure 15 It is a generated and labeled image of a new perspective scene with objects. Detailed Implementation

[0046] The technical solution of the present invention will be further described in detail below through specific embodiments.

[0047] like Figure 1As shown, a method for collecting a target object recognition dataset in an industrial setting is mainly divided into two parts: one part is a method for collecting a recognition dataset for a scene containing people, and the other part is a method for collecting a recognition dataset for a scene containing specific items.

[0048] The embodiment also utilizes a vehicle, which is configured as a wheeled unmanned vehicle in this embodiment, including a microwave radar module, an optical sensing module, and a lidar module. Figure 1 The first one is the chassis of an unmanned vehicle. Figure 1 2 is the horizontal centerline of the optical image sensor module. Figure 1 In the middle, 3 is the distance d1 from the horizontal centerline of the optical sensing module to the sensing centerline of the millimeter-wave module. Figure 1 4 is the centerline of the microwave sensor module. Figure 1 5 is the distance d2 from the horizontal centerline of the millimeter-wave sensor to the horizontal centerline of the lidar. Figure 1 6 is the horizontal centerline of the lidar. Figure 1 The 7th one is lidar. Figure 1 The 8th one is a microwave radar module. Figure 1 The number 9 in the diagram represents the optical image sensor module.

[0049] Before use, each sensor in the unmanned vehicle needs to be calibrated and adjusted.

[0050] The specific process is as follows: Configure a correction rod, such as Figure 2 The calibration rod is mainly composed of a lidar reflection area 14, a microwave reflection area 13, and an optical image sensor calibration area 12. Figure 2 11 in the diagram represents the base of the calibration rod. The lidar reflection zone reflects laser light with a low loss rate when illuminated by lidar. The millimeter-wave reflection zone reflects microwaves with a low loss rate and high directional accuracy when illuminated by millimeter-wave radar. The optical image sensor calibration area provides horizontal and vertical reference lines for determining the camera's position. Item 15 in the diagram is the horizontal centerline of the lidar reflection zone. Item 17 is the horizontal centerline of the microwave radar reflection zone. Item 19 is the horizontal centerline of the optical image sensor calibration area. Item 16 is the distance d3 from the horizontal centerline of the lidar reflection zone to the horizontal centerline of the microwave radar reflection zone. Item 18 is the distance d4 from the centerline of the microwave radar reflection zone to the horizontal centerline of the optical image sensor calibration area. During sensor assembly, it is necessary to ensure that d1 equals d4 and d2 equals d3.

[0051] In the microwave anechoic chamber, the calibration rod is positioned as follows: Figure 3The situation is as shown. The unmanned vehicle's viewpoint 36 is directly facing correction lever 33, and correction lever 3 lies on the perpendicular bisector of the line connecting correction levers 2 (32) to 4 (34). The perpendicular bisector of the line connecting correction levers 1 (31) to 5 (35) coincides with the perpendicular bisector of the line connecting correction levers 2 (2) to 4 (4). After the environment setup is complete, the sensors are activated. This applies to microwave radar and lidar. Figure 4 As shown, if the sensor is not registered, the strongest signal point (rectangular block) within the sensor's field of view is not on the horizontal center line of the sensor's field of view. This can be achieved by adjusting the assembly orientation of the microwave radar and lidar. Figure 5 The effect shown can be considered as the sensor achieving the registration position. The strongest signal point within the field of view of microwave radar and lidar sensors lies on the horizontal midline of the sensor's field of view.

[0052] For optical image sensors, the horizontal line of the field of view needs to be aligned with the horizontal line of the crosshairs in the calibration area of ​​the optical image sensor. For example... Figure 6 As shown, without registration, in the field of view 37 of the optical image sensor, the horizontal line 38 of the crosshairs in the calibration area of ​​the optical image sensor for all calibration rods is not on the horizontal center line 39 of the field of view of the optical image sensor.

[0053] like Figure 7 As shown, after manual calibration, the horizontal center line 39 of the optical image sensor's field of view coincides with the horizontal line 38 of the crosshairs in the calibration area of ​​the optical image sensor for all calibration rods.

[0054] After manual calibration, the horizontal and vertical lines in the sensor's field of view are basically consistent. The next step is to eliminate the offset of the sensor's assembly position. To eliminate the offset of the sensor's assembly position, specifically, the viewing angle information of different sensors is combined into the same viewing angle, the positional error of the same position in the field of view is calculated, and the average error data is obtained. After each sensor acquires the target position and the corresponding error is subtracted, the field of view is aligned.

[0055] The method for collecting datasets containing personnel identification information is performed through the following steps: Step 1) Under manual control, within a specified range, acquire point cloud information of the environment using 3D LiDAR, and combine it with GNSS positioning data to generate a 3D point cloud of the specified area. Convert the 3D point cloud information of the specific area into a 3D octree map.

[0056] Step 2) Within the designated scene, the vehicle plans a path based on an octree map, generates a 3D point cloud using LiDAR and GNSS positioning, and achieves cruise control of the robot, cruising along the designated route. During the cruise, the sensor module activates to collect data. Figure 8 The text presents data from three different perspectives, namely... Figure 8The data from the three perspectives shown in numbers 1, 2, and 3 are as follows.

[0057] Step 3) Obtain the position and posture of the personnel target: 3.1) Obtain the vital signs of the target personnel and determine the location of the target human body through microwave radar.

[0058] 3.2) Using a microwave radar module, the posture of the human body under different viewing angles is measured, such as... Figure 9 The numbers 4, 5, and 6 are in the diagram, with red representing the obscured limbs; 3.3) Based on the measured posture and combined with the optical images acquired by the optical image sensor, the optical data of the human body is obtained through image semantic segmentation. For example... Figure 8 The serial numbers 7, 8, and 9 in the text; 3.4) The NeRF network is trained using camera pose information and human optical data. Based on the trained NeRF network, optical images of the human body from other perspectives are acquired, such as… Figure 8 As shown in the serial number 10; The training and usage process of NeRF is as follows: Step a) Obtain the position and attitude of the sensor modules based on the vehicle's IMU and GNSS, and generate 5D information for the NeRF network input (position x, y, z values ​​+ orientation parameter θ). ).

[0059] Step b) Obtain the output information of NeRF (color values ​​r, g, b + density) through an optical image sensor and a lidar.

[0060] Step c) Based on the existing input and output information, train a multi-layer MLP to fit the formula F. θ (x,d)=[σ,c].

[0061] Step d) After training is complete, convert the required camera pose parameters into corresponding x, y, z, θ. The parameters are input into the trained NeRF to obtain the corresponding output, and the corresponding optical image sensor data under this viewpoint is synthesized.

[0062] Step 4) Restore the empty scene data without people: 4.1) From the various viewpoint data acquired in step 2), the position data of the human body in the optical image is subtracted to generate an image with subtracted information, such as... Figure 9 The serial numbers 4, 5, and 6 in the text; 4.2) By using algorithms to complete semantic information, pure background scene images without target data are generated, such as... Figure 9 The serial numbers 7, 8, and 9 in the text; 4.3) Combining the original data from each viewpoint with the image data of the empty environment, generate an optical image of the empty environment excluding people. Based on the generated multi-view optical image of the empty environment, use the optical image information of the empty environment and the camera pose to train NeRF. Figure 9 Numbers 10, 11, and 12 in the sequence are used to generate one or more new perspectives via NeRF, such as Figure 9 The serial numbers 13 and 14 in the text; Step 5) Based on the generated empty scene perspective Figure 10 Images No. 2, 3 and new perspective images of the human body Figure 10 In section 1, synthesize multi-pose images and data of the target human body from a new scene perspective, such as... Figure 10 The numbers 4 and 5 in the sequence; Step 6) Generate annotations for the new perspective image based on the human pose parameters and the new perspective parameters, i.e., the annotation data required for the dataset, such as... Figure 10 The serial numbers 6 and 7 in the text.

[0063] Data from the original viewpoint also needs annotation, and the method for generating annotations is shown in the figure. For the original image data ( Figure 11 The numbers 1, 2, and 3 in the middle are... Figure 7 (Sections 1, 2, 3), based on the attitude data measured by the radar ( Figure 11 The annotations for numbers 4, 5, and 6 in the middle are generated directly. Figure 11 The red border of the targets in numbers 7, 8, and 9.

[0064] The method for collecting datasets containing object recognition features is performed through the following steps: like Figure 12 As shown, in step 1), in a microwave anechoic chamber, using a sensor module of the same type as the vehicle or with equivalent function to the sensors in the vehicle, microwave radar data of the target object is acquired from different perspectives, generating microwave radar data from multiple perspectives in a 3D view. For example... Figure 12 As shown, numbers 1, 2, 5, and 6 are sensor modules used for data acquisition; number 4 is a platform made of absorbing material; and number 3 is the target object.

[0065] Step 2) Under manual control, within a specified range, acquire point cloud information of the environment using 3D LiDAR, and combine it with GNSS positioning data to generate a 3D point cloud of the specified area. Convert the 3D point cloud information of the specific area into a 3D octree map.

[0066] Step 3) Within the designated scene, the vehicle generates 3D point clouds and GNSS positioning using LiDAR based on the path planned on the octree map, enabling cruise control of the robot. The vehicle collects data within the designated range.

[0067] Step 4) Figure 13As shown, based on the item information pre-collected in step 1), combined with the data collected in step 3), Figure 13 (Sections 1, 2, and 3) are used to calculate the pose information of specific objects in the scene obtained in step 3), and to generate data annotations for the original image. Figure 13 (Sections 4, 5, and 6) Simultaneously, combining pose data and original image information, the pixels representing the object's position in the optical image are subtracted to generate an image with subtracted information. Figure 13 (Middle serial numbers 7, 11, 12) Step 5), as follows Figure 13 As shown, by using an algorithm to complete the subtracted portion, pure background image data without target data is generated. Figure 13 (Middle serial numbers 8, 9, 10) Step 6), as follows Figure 14 As shown, by combining the original data from various perspectives and the semantic information of the empty environment, an optical image of the empty environment without objects is generated. Figure 14 (Sections 1, 2, and 3). Based on the generated multi-view spatial environment optical images and corresponding camera pose information, a NeRF network is trained. One or more new viewpoints are generated using the NeRF network. Figure 14 (Middle serial numbers 5 and 6) Step 7) Figure 15 As shown, based on the generated empty scene perspective ( Figure 15 (Sequence numbers 1 and 2) and 3D data of the items ( Figure 15 (Section 3) Synthesize multi-pose images and data of target objects from a new scene perspective ( Figure 15 (Middle numbers 4, 5, 6, 7) Step 8) Generate annotations for the new perspective image ( Figure 15 The positional parameters of the yellow border boxes in 8, 9, 10, and 11 are the annotation data required for the dataset.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation of the present invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in the present invention.

Claims

1. A method for collecting personnel identification datasets in industrial scenarios, characterized in that: A vehicle carrying a microwave radar module, an optical sensing module, and a lidar module collects data through the following steps: Step 1) The vehicle cruises within a specific area, acquiring sensor data from several different angles around the scene where the target personnel are located; Step 2) Obtain the location and posture of the target person: 2.1) Obtain vital signs of the target personnel through microwave radar and determine the location of the target personnel from different perspectives; 2.2) Detect the posture of target personnel from different perspectives using microwave radar; 2.3) Annotate the original optical images based on the position and posture of the target personnel; Step 3) Obtain the original optical data of the target person from different perspectives, and generate the optical data of the target person from the new perspective; 3.1) Based on the pose of the target person from different perspectives in step 2.2), perform semantic segmentation of the data from the optical sensing module to obtain the pixels of the target person; 3.2) The NeRF network is trained using the attitude information from the optical sensing module and the optical data of the target person as training data. Based on the trained NeRF network, optical images of the target person from other perspectives are obtained. Step 4) Restore the empty scene data without the target personnel: 4.1) Based on the raw optical data obtained in step 1), and according to the target person's pose in step 2) and the semantic segmentation information in step 3.1), subtract the pixels of the target person; 4.2) By using algorithms to complete the deducted pixels, empty scene image data without target personnel can be obtained; Step 5) Based on the empty scene image data generated in Step 4) and the pose of the optical sensing module, generate new perspective optical data of the empty scene using NeRF; Step 6) Based on the optical data of the target person from a new perspective generated in Step 3) and the empty scene generated in Step 5), generate an image of the target person's pose from the new perspective and the corresponding annotations.

2. The method for collecting personnel identification datasets in industrial scenarios according to claim 1, characterized in that: In the vehicle, the optical sensing module, microwave radar module, and lidar module are arranged from bottom to top, and are configured with corresponding calibration processes, including: A calibration rod is configured, on which are provided an optical sensing module calibration area, a microwave reflection area, and a lidar reflection area, which are respectively at the same height as the optical sensing module, the microwave radar module, and the lidar module. In a microwave anechoic chamber, multiple calibration rods are arranged in an isosceles apex shape from a top-down view, with the carrier located on the perpendicular bisector of the isosceles apex shape; Start each sensor module and adjust the assembly posture of the microwave radar module and the lidar module respectively so that the strongest signal points sensed are all on the horizontal center line of the sensor's field of view. This can be regarded as the registration of the microwave radar module and the lidar module. Adjust the optical sensing module and check if the crosshairs of each calibration rod are on the horizontal center line of the field of view of the optical sensing module. This can be regarded as the optical sensing module registration. To eliminate sensor assembly position offsets, specifically, the viewing angle information of different sensors is synthesized into the same viewing angle, the positional error of the same position in the field of view is calculated, and the average error data is obtained. After each sensor obtains the target position and the corresponding error is subtracted, the field of view alignment can be achieved.

3. A method for collecting object recognition datasets in industrial scenarios, characterized in that: The data is collected by a vehicle carrying a microwave radar module, an optical sensing module, and a lidar module through the following steps: Step 1) In a microwave anechoic chamber, microwave radar data of the target object is pre-acquired from different perspectives to generate microwave radar data from multiple perspectives in 3D view, and 3D data of the target object is generated. Step 2) The vehicle cruises within a specific area. During the cruise, the position and attitude of the target object are confirmed by combining the data from the microwave radar with the data obtained in Step 1). Step 3) Based on the pose and position of the target object, and combined with the information obtained in Step 1), the target object is subtracted from the original image; Step 4) The algorithm completes the parts deducted in Step 3). The completed image data is pure background image data without target objects. Combined with the original data from each viewpoint and the semantic information of the empty environment, an empty environment optical image without target objects is generated. Based on the generated empty environment optical images from multiple viewpoints and the corresponding pose information of the optical sensing module, a NeRF network is trained. The NeRF network generates new viewpoint optical data for one or more empty scenes. Step 5) Based on the 3D data of the target object pre-collected in Step 1) and the empty scene data obtained in Step 4), synthesize an image of the target object in a new pose from a new perspective and generate corresponding annotations.

4. The method for collecting object recognition datasets in industrial scenarios according to claim 3, characterized in that: In the vehicle, the optical sensing module, microwave radar module, and lidar module are arranged from bottom to top, and are configured with corresponding calibration processes, including: A calibration rod is configured, on which are provided an optical sensing module calibration area, a microwave reflection area, and a lidar reflection area, which are respectively at the same height as the optical sensing module, the microwave radar module, and the lidar module. In a microwave anechoic chamber, multiple calibration rods are arranged in an isosceles apex shape from a top-down view, with the carrier located on the perpendicular bisector of the isosceles apex shape; Start each sensor module and adjust the assembly posture of the microwave radar module and the lidar module respectively so that the strongest signal points sensed are all on the horizontal center line of the sensor's field of view. This can be regarded as the registration of the microwave radar module and the lidar module. Adjust the optical sensing module and check if the crosshairs of each calibration rod are on the horizontal center line of the field of view of the optical sensing module. This can be regarded as the optical sensing module registration. To eliminate sensor assembly position offsets, specifically, the viewing angle information of different sensors is synthesized into the same viewing angle, the positional error of the same position in the field of view is calculated, and the average error data is obtained. After each sensor obtains the target position and the corresponding error is subtracted, the field of view alignment can be achieved.

Images