Anti-collision method and system based on three-dimensional target detection

Abstract

The invention relates to the technical field of petroleum drilling and production, in particular to an anti-collision method and system based on three-dimensional target detection, and the method comprises the steps: obtaining an original image of a target region through an industrial camera, and obtaining the point cloud data of the target region through a laser radar; inputting the original image into an image segmentation model to obtain a segmented image corresponding to each target object; according to the target category corresponding to each target object and the corresponding relation table, performing color integration on all the segmented images to obtain a color integration image corresponding to each target object; mapping all the color integration images into the point cloud data to obtain corresponding point cloud comprehensive data; and inputting the point cloud comprehensive data into a three-dimensional detection network to obtain position information of all target objects in the target area. The method has the beneficial effects that the recognition accuracy is obviously improved, the positioning is accurate, and accurate collision prevention between personnel and equipment and between the equipment on the oil drilling platform is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of oil drilling and production, and particularly to an anti-collision method and system based on three-dimensional object detection. Background Art

[0002] On oil drilling platforms, the anti-collision technology for personnel and equipment is a key component to ensure safe operation. These platforms are usually located offshore or in remote areas with harsh environmental conditions and limited space, so special attention needs to be paid to avoiding accidents caused by misoperation or accidents.

[0003] Currently, for anti-collision between personnel and equipment on oil drilling platforms, there are mainly two methods: the first is to adopt advanced automation control technologies, including but not limited to automatic drilling systems, remote driller, etc., to reduce the risk of personnel injury by reducing the chance of direct human participation in dangerous operations; the second is to use high-definition cameras to design a two-dimensional image-based personnel and equipment object detection system, and delimit the infrared range in the image. Under certain working conditions of the drilling platform, it can detect in real time whether personnel enter the infrared range, identify potential threats, and send warning signals to relevant personnel in a timely manner, thus preventing the occurrence of collision accidents.

[0004] However, the adoption of advanced automation control technologies highly depends on the automation system. Once technical failures (such as software errors, hardware damage, etc.) occur, it may cause the entire operation process to pause or even lead to safety accidents. At the same time, the complexity of the automation system may also increase the maintenance difficulty; in addition, in order to ensure the effective use of these new technologies, operators and maintenance personnel need to receive specialized training. If the training is insufficient, it may lead to the misuse or abuse of automation functions, instead increasing the possibility of accidents.

[0005] For the camera target red zone detection, under harsh weather conditions (such as rain, snow, fog, etc.) or insufficient light, the quality of the images captured by the camera will be affected, thus reducing the accuracy of target recognition; secondly, for two-dimensional image-based object detection, the target positioning accuracy is poor. When a person appears at the boundary of the red zone, false alarms often occur, which may affect the trust of staff in the alarms in the long run. Summary of the Invention

[0006] (I) Technical Problems to be Solved

[0007] In view of the above-mentioned disadvantages and deficiencies of the prior art, the present invention provides an anti-collision method and system based on three-dimensional object detection, which solves the technical problems of poor target positioning accuracy and susceptibility to environmental influence in two-dimensional image-based object detection, and the technical problems of high dependence on the automation system, large maintenance difficulty, and waste of human and material resources in automation control technologies.

[0008] (2) Technical Solution

[0009] To achieve the above object, the main technical solutions adopted by the present invention include:

[0010] In a first aspect, an embodiment of the present invention provides a collision prevention method based on three-dimensional object detection, which is implemented based on an industrial camera and a lidar pre-deployed on a drilling platform. The method includes:

[0011] S11. Obtain the original image of the target area through the industrial camera, and obtain the point cloud data of the target area through the lidar; the target area includes the working area of the oil drilling platform;

[0012] S12. Input the original image into a pre-set image segmentation model, extract pixels of all target objects in the original image, and obtain a segmentation image corresponding to each target object;

[0013] The target object is a position monitoring object pre-set in the target area; each target object has a pre-set target category;

[0014] S13. According to the target category corresponding to each target object and a pre-set correspondence table, perform color integration on all segmentation images to obtain a color-integrated image corresponding to each target object;

[0015] The correspondence table is a correspondence table between the background color used for each target object during color integration and the target category to which the target object belongs;

[0016] S14. Map all the color-integrated images into the point cloud data to obtain corresponding point cloud comprehensive data;

[0017] S15. Input the point cloud comprehensive data into a pre-set three-dimensional detection network to obtain the position information of all target objects in the target area, so as to assist in judging whether there is a collision risk between any two target objects in the target area.

[0018] Optionally, S13 includes:

[0019] S13-1. Obtain the initial pixel value of each pixel position in each segmentation image according to all the segmentation images;

[0020] S13-2. Obtain the background color pixel value corresponding to each target object according to the target category corresponding to each target object and a pre-set correspondence table;

[0021] Both the initial pixel value and the background color pixel value include three channels of red, green, and blue;

[0022] S13-3. Integrate the colors of all the segmented images according to the initial pixel values of all pixel positions in each segmented image, the background color pixel values of the target object corresponding to each segmented image, and the pre-set formula 1; the formula 1 is:

[0023]

[0024] Wherein, I is the pixel value after color integration at any pixel position on the segmented image, (R0, G0, B0) is the initial pixel value of this pixel position on the segmented image, and (R1, G1, B1) is the background color pixel value of the target object corresponding to this segmented image.

[0025] Optionally, S12 further includes:

[0026] Obtain the pixel coordinate values corresponding to all pixel positions in each segmented image according to the pixel coordinate values corresponding to each pixel position in the original image, that is, the pixel coordinate values corresponding to all pixel positions in each color-integrated image; the pixel coordinate values include two-dimensional image coordinates and corresponding depth values;

[0027] S14 includes:

[0028] S14-1. Obtain the three-dimensional coordinates of all pixel positions in each color-integrated image in the camera coordinate system according to the pixel coordinate values corresponding to all pixel positions in each color-integrated image, the pre-set camera coordinate conversion parameters, and the pre-set formula 2; the formula 2 is:

[0029]

[0030] Wherein, c x 、c y 、f x and f y are all pre-set camera coordinate conversion parameters, (u, v) are the two-dimensional image coordinates corresponding to any pixel position, d is the depth value corresponding to this pixel position, and (X, Y, Z) are the three-dimensional coordinates of this pixel position in the camera coordinate system;

[0031] S14-2. Obtain the three-dimensional coordinates of all pixel positions in each color-integrated image in the world coordinate system according to the three-dimensional coordinates of each pixel position in each color-integrated image in the camera coordinate system, the pre-set world coordinate conversion matrix, and the pre-set formula 3; the formula 3 is:

[0032]

[0033] Wherein, (X w , Y w , Z wis the three-dimensional coordinate of any pixel position in the world coordinate system, is a pre-set world coordinate transformation matrix;

[0034] S14-3. Based on the three-dimensional coordinates of all pixel positions in each color integrated image in the world coordinate system, align the coordinates of all color integrated images with the point cloud data and map them into the point cloud data to obtain the corresponding comprehensive point cloud data;

[0035] Any point cloud position in the comprehensive point cloud data includes the corresponding first point cloud parameter; the first point cloud parameter includes six channels: red, green, blue, horizontal position, vertical position, and height.

[0036] Optionally, S14-3 further includes: mapping a pre-set target color to the background area in the point cloud data that is not coordinate-aligned with any color integrated image;

[0037] The target color is black.

[0038] Optionally, the image segmentation model includes: an image segmentation model obtained by training a pre-trained semantic segmentation model with a pre-set first training set;

[0039] The first training set includes: at least one first training image; each training image includes at least one target object that has been labeled;

[0040] The image segmentation model is used for:

[0041] Performing target detection on the received original image to obtain the corresponding target detection result; the target detection result includes the region of interest corresponding to the pre-set target object and the target category corresponding to each region of interest;

[0042] Segmenting and pixel-extracting each region of interest to obtain a segmentation image corresponding to each target object.

[0043] Optionally, when the image segmentation model segments and pixel-extracts each region of interest to obtain a segmentation image corresponding to each target object, it includes:

[0044] Based on all regions of interest in the original image, obtaining the labeled region and the unlabeled region corresponding to each target object; wherein, the labeled region is used to label the target object in the region of interest, and the labeled region and the unlabeled region are complementary;

[0045] Based on the labeled region and the unlabeled region in the region of interest corresponding to each target object, obtaining the occluded region corresponding to each target object;

[0046] Segment each of the regions of interest and extract pixels to obtain a process image corresponding to each target object;

[0047] According to the occlusion region corresponding to each target object and a preset adjacent point difference algorithm, supplement the process image corresponding to each target object to obtain a segmented image corresponding to each target object.

[0048] Optionally, the 3D detection network is used to:

[0049] According to the first point cloud parameters [R2, G2, B2, X2, Y2, Z2] corresponding to each point cloud position in the point cloud comprehensive data and a preset formula four, obtain the second point cloud parameters [C, X2, Y2, Z2] corresponding to each point cloud position; the formula four is:

[0050]

[0051] where C is the value of the comprehensive color channel in the second point cloud parameter, R2, G2, and B2 are the values of the red, green, and blue channels in the first point cloud parameter respectively, and X2, Y2, and Z2 are the values of the horizontal position, vertical position, and height channels respectively;

[0052] According to the value of the comprehensive color channel of the second point cloud parameter corresponding to each point cloud position in the point cloud comprehensive data and a preset region segmentation algorithm, obtain the point cloud boundary corresponding to each target object;

[0053] According to the point cloud boundary corresponding to each target and the values of the horizontal position, vertical position, and height channels in the second point cloud parameter corresponding to each point cloud position, obtain the position information of each target object in the target region to assist in determining whether there is a collision risk between any two target objects in the target region.

[0054] Optionally, the 3D detection network obtains the point cloud boundary corresponding to each target object according to the value of the comprehensive color channel of the second point cloud parameter corresponding to each point cloud position in the point cloud comprehensive data and a preset region segmentation algorithm, including:

[0055] According to the value of the comprehensive color channel of the second point cloud parameter corresponding to each point cloud position in the point cloud comprehensive data and a preset discrimination threshold, determine whether any two adjacent point cloud positions in the point cloud comprehensive data are neighbors;

[0056] Traverse all the point cloud positions in the point cloud comprehensive data. When any point cloud position has a neighbor, mark the point cloud position and its neighbor as the same potential object;

[0057] When any point cloud position has no unmarked neighbor, the point cloud position is another potential object;

[0058] Filter and discard the potential objects belonging to the background among all potential objects according to a preset color threshold;

[0059] Obtain the point cloud boundary of the corresponding target object according to the point cloud positions of all non-background potential objects.

[0060] Optionally, the 3D detection network filters and discards the potential objects belonging to the background among all potential objects according to a preset color threshold, including:

[0061] Obtain the average value of the comprehensive color channel corresponding to each potential object according to the value of the comprehensive color channel corresponding to the point cloud positions in each potential object;

[0062] When the average value of the comprehensive color channel corresponding to any potential object is less than or equal to the preset color threshold, then the potential object is the background and is discarded.

[0063] In a second aspect, an anti-collision system based on 3D object detection according to an embodiment of the present invention includes a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement an anti-collision method based on 3D object detection of the above method.

[0064] (III) Advantageous Effects

[0065] The beneficial effects of the present invention are as follows: For an anti-collision method based on 3D object detection of the present invention, since the original pixel values of the target object are integrated with a preset background color and then mapped to point cloud data, compared with the prior art, the recognition accuracy is significantly improved, and the positioning is accurate, realizing precise anti-collision between personnel and equipment and between equipment on an oil drilling platform. Description of the Drawings

[0066] Figure 1 It is a flowchart of an anti-collision method based on 3D object detection provided by an embodiment of the present invention;

[0067] Figure 2 It is a simplified flowchart of an anti-collision method based on 3D object detection on an oil drilling platform provided by an embodiment of the present invention. Detailed Embodiments

[0068] In order to better explain the present invention for easy understanding, the present invention will be described in detail below with reference to the drawings through specific embodiments.

[0069] A collision prevention method based on 3D object detection proposed in an embodiment of the present invention, after integrating the original pixel values of the target object with a preset background color and mapping them to point cloud data, compared with the prior art, the recognition accuracy is significantly improved, and the positioning is accurate, realizing precise collision prevention between personnel and equipment, and between equipment on an oil drilling platform.

[0070] To better understand the above technical solution, the exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more clear and thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0071] Embodiment 1

[0072] A collision prevention method based on 3D object detection, the method is implemented based on an industrial camera and a lidar pre-deployed on a drilling platform, and the method is as Figure 1 shown, including:

[0073] S11. Obtain the original image of the target area through the industrial camera, and obtain the point cloud data of the target area through the lidar; the target area includes the working area of the oil drilling platform;

[0074] S12. Input the original image into a preset image segmentation model, extract the pixels of all target objects in the original image, and obtain a segmentation image corresponding to each target object;

[0075] The target object is a position monitoring object preset in the target area; each target object has a preset target category;

[0076] S13. According to the target category corresponding to each target object and a preset correspondence table, perform color integration on all segmentation images to obtain a color integration image corresponding to each target object;

[0077] The correspondence table is a correspondence table between the background color used for each target object during color integration and the target category to which the target object belongs;

[0078] S14. Map all the color integration images into the point cloud data to obtain corresponding point cloud comprehensive data;

[0079] S15. Input the point cloud comprehensive data into a preset 3D detection network to obtain the position information of all target objects in the target area, so as to assist in judging whether there is a collision risk between any two target objects in the target area.

[0080] An anti-collision method based on three-dimensional object detection provided by an embodiment of the present invention. After integrating the original pixel values of the target object with the preset background color and mapping them to the point cloud data, compared with the prior art, the recognition accuracy is significantly improved, and the positioning is accurate, realizing precise anti-collision between personnel and equipment and between equipment on the oil drilling platform.

[0081] Embodiment 2

[0082] This embodiment provides an anti-collision method based on three-dimensional object detection on an oil drilling platform. The method is implemented based on industrial cameras and lidars pre-installed on the drilling platform. Generally, after the industrial cameras and lidars are installed, their positions are fixed and do not move. The fields of view of the industrial cameras and lidars include the main working areas of the staff on the platform. At the same time, mechanical equipment such as the iron roughneck on the drilling platform is also within this field of view. The method is as Figure 2 shown and includes:

[0083] Simultaneously collect multiple first training images and multiple second training data at different time periods within the field of view (target area) through the industrial cameras and lidars respectively;

[0084] Segment and label target objects such as personnel and iron roughnecks in the first training images respectively, distinguish the training set and the test set, and use the training set to train the pre-trained image segmentation model to obtain the segmentation model;

[0085] During the training, the image segmentation model segments the first training images and extracts the target objects therein. At this time, the target objects have initial pixel values. Set the background color pixel values of each target object in the first training images. For example, the background color of personnel is red, and the background color of the iron roughneck is yellow. Integrate the initial pixel values and the background color pixel values of the target objects. Both the initial pixel values and the background color pixel values have three channels of red, green, and blue. When red is used as the background color, the three-channel values of this background color pixel value are 255, 0, 0 respectively; for yellow, they are 127, 127, 0; and for black, they are 0, 0, 0. And the color integration follows the following formula:

[0086]

[0087] After the color integration, draw the target objects (segmented images) into the corresponding second training data, and draw the second training data of the background irrelevant to the target objects as black. Expand the second training data from three dimensions of X, Y, Z to six dimensions of R, G, B, X, Y, Z.

[0088] Input the mapped second training data into the 3D detection network for training to obtain a trained 3D detection network, and then use this network to perform real-time detection and target localization on the visible area of the drilling platform.

[0089] On the drilling platform, industrial cameras and lidars collect data in real time. Use the segmentation model to segment the segmentation image of each target object in the original image, and then perform color integration through a pre-set correspondence table. The integrated color integration image is drawn into the point cloud data. Detect the target from the point cloud data through the 3D detection network, and obtain the spatial position information and range of the target object.

[0090] Based on the target localization information of the 3D detection network, calculate the distances between various target objects to assist in determining whether there is a collision risk between any two target objects in the target area.

[0091] A collision prevention method based on 3D object detection on an oil drilling platform provided in this embodiment segments the target object through an image segmentation model, preset the target background color, perform color integration on the segmentation image and the background color, draw it into the point cloud data, draw the point cloud irrelevant to the target object in black, and expand the point cloud data from three data dimensions to six data dimensions.

[0092] Compared with the prior art, its recognition accuracy is significantly improved, and the positioning is accurate, which is beneficial to the precise collision prevention between personnel and equipment, and between equipment on the oil drilling platform.

[0093] Embodiment 3

[0094] This embodiment provides a collision prevention method based on 3D object detection, including:

[0095] This method is implemented based on industrial cameras and lidars pre-deployed on the drilling platform, and this method includes:

[0096] S11. Obtain the original image of the target area through the industrial camera, and obtain the point cloud data of the target area through the lidar; the target area includes the working area of the oil drilling platform;

[0097] S12. Input the original image into a pre-set image segmentation model, extract the pixels of all target objects in the original image, and obtain the segmentation image corresponding to each target object;

[0098] The target object is a position monitoring object pre-set in the target area; each target object has a pre-set target category;

[0099] S13. According to the target category corresponding to each target object and the pre-set corresponding relationship table, perform color integration on all the segmented images to obtain a color-integrated image corresponding to each target object;

[0100] The corresponding relationship table is a corresponding relationship table between the background color used for each target object during color integration and the target category to which the target object belongs;

[0101] S14. Map all the color-integrated images into the point cloud data to obtain corresponding point cloud comprehensive data;

[0102] S15. Input the point cloud comprehensive data into a pre-set three-dimensional detection network to obtain the position information of all target objects within the target area, so as to assist in judging whether there is a collision risk between any two target objects within the target area.

[0103] Further, the S13 includes:

[0104] S13-1. According to all the segmented images, obtain the initial pixel values at each pixel position in each segmented image;

[0105] S13-2. According to the target category corresponding to each target object and the pre-set corresponding relationship table, obtain the background color pixel values corresponding to each target object;

[0106] Both the initial pixel values and the background color pixel values include three channels of red, green, and blue;

[0107] S13-3. According to the initial pixel values at all pixel positions in each segmented image, the background color pixel values of the target object corresponding to each segmented image, and the pre-set formula 1, perform color integration on all the segmented images; the formula 1 is:

[0108]

[0109] where I is the pixel value after color integration at any pixel position on the segmented image, (R0, G0, B0) is the initial pixel value of this pixel position on the segmented image, and (R1, G1, B1) is the background color pixel value of the target object corresponding to this segmented image.

[0110] Further, the S12 also includes:

[0111] According to the pixel coordinate values corresponding to each pixel position in the original image, obtain the pixel coordinate values corresponding to all pixel positions in each segmented image, that is, the pixel coordinate values corresponding to all pixel positions in each color-integrated image; the pixel coordinate values include two-dimensional image coordinates and corresponding depth values;

[0112] Then, the S14 includes:

[0113] S14-1. Obtain the three-dimensional coordinates of all pixel positions in each color-integrated image in the camera coordinate system according to the pixel coordinate values corresponding to all pixel positions in each color-integrated image, the pre-set camera coordinate conversion parameters, and the pre-set formula two; the formula two is:

[0114]

[0115] where c x , c y , f x and f y are all pre-set camera coordinate conversion parameters, (u, v) is the two-dimensional image coordinate corresponding to any pixel position, d is the depth value corresponding to this pixel position, and (X, Y, Z) is the three-dimensional coordinate of this pixel position in the camera coordinate system;

[0116] S14-2. Obtain the three-dimensional coordinates of all pixel positions in each color-integrated image in the world coordinate system according to the three-dimensional coordinates of each pixel position in each color-integrated image in the camera coordinate system, the pre-set world coordinate conversion matrix, and the pre-set formula three; the formula three is:

[0117]

[0118] where (X w , Y w , Z w ) is the three-dimensional coordinate of any pixel position in the world coordinate system, is the pre-set world coordinate conversion matrix;

[0119] S14-3. Based on the three-dimensional coordinates of all pixel positions in each color-integrated image in the world coordinate system, align the coordinates of all color-integrated images with the point cloud data and map them into the point cloud data to obtain the corresponding comprehensive point cloud data;

[0120] Any point cloud position in the comprehensive point cloud data includes the corresponding first point cloud parameter; the first point cloud parameter includes six channels: red, green, blue, horizontal position, vertical position, and height.

[0121] The S14-3 further includes:

[0122] Map the pre-set target color to the background area in the point cloud data that is not coordinate-aligned with any color-integrated image;

[0123] The target color is black.

[0124] Furthermore,

[0125] The image segmentation model is used for:

[0126] Perform object detection on the received original image to obtain corresponding object detection results; the object detection results include regions of interest corresponding to preset target objects and target categories corresponding to each region of interest.

[0127] Obtain the labeled region and unlabeled region corresponding to each target object according to all regions of interest in the original image; wherein, the labeled region is used to label the target object in the region of interest, and the labeled region and the unlabeled region are complementary.

[0128] Obtain the occlusion region corresponding to each target object according to the labeled region and unlabeled region in the region of interest corresponding to each target object.

[0129] Segment and extract pixels for each of the regions of interest to obtain a process image corresponding to each target object.

[0130] Supplement the process image corresponding to each target object according to the occlusion region corresponding to each target object and a preset adjacent point difference algorithm to obtain a segmented image corresponding to each target object.

[0131] Further, the 3D detection network is used for:

[0132] Obtain the second point cloud parameter [C, X2, Y2, Z2] corresponding to each point cloud position according to the first point cloud parameter [R2, G2, B2, X2, Y2, Z2] corresponding to each point cloud position in the point cloud comprehensive data and a preset formula four; the formula four is:

[0133]

[0134] Wherein, C is the value of the comprehensive color channel in the second point cloud parameter, R2, G2, and B2 are the values of the red, green, and blue channels in the first point cloud parameter respectively, and X2, Y2, and Z2 are the values of the horizontal position, vertical position, and height channels respectively.

[0135] Judge whether any two adjacent point cloud positions in the point cloud comprehensive data are neighbors according to the value of the comprehensive color channel of the second point cloud parameter corresponding to each point cloud position in the point cloud comprehensive data and a preset discrimination threshold.

[0136] Traverse all point cloud positions in the point cloud comprehensive data. When any point cloud position has a neighbor, mark the point cloud position and its neighbor as the same potential object.

[0137] When any point cloud position has no unlabeled neighbor, the point cloud position is another potential object.

[0138] Obtain the average value of the comprehensive color channel corresponding to the potential object according to the values of the comprehensive color channels corresponding to all the point cloud positions in each potential object;

[0139] When the average value of the comprehensive color channel corresponding to any potential object is less than or equal to a pre-set color threshold, then the potential object is the background and is discarded.

[0140] Obtain the point cloud boundary of the corresponding target object according to all the point cloud positions of each potential object that does not belong to the background.

[0141] Obtain the position information of each target object in the target area according to the point cloud boundary corresponding to each target and the values of the lateral position, longitudinal position, and height channel in the second point cloud parameter corresponding to each point cloud position, so as to assist in judging whether there is a collision risk between any two target objects in the target area.

[0142] A collision prevention method based on three-dimensional object detection provided by an embodiment of the present invention, since the original pixel value of the target object is integrated with a pre-set background color and then mapped to the point cloud data, compared with the prior art, the recognition accuracy is significantly improved, and the positioning is accurate, realizing precise collision prevention between personnel and equipment and between equipment on the oil drilling platform.

[0143] Embodiment 4

[0144] A collision prevention system based on three-dimensional object detection provided by this embodiment includes a memory, a processor, and a computer program stored on the memory, and the processor executes the computer program to implement the method described in any one of Embodiments 1 to 3.

[0145] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality" means two or more, unless otherwise specifically defined.

[0146] In the present invention, unless otherwise clearly specified and limited, the terms "installation", "connection", "connection", "fixation" and other terms should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium; it can be the communication inside two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0147] In the present invention, unless otherwise clearly specified or limited, a first feature being "on" or "under" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact via an intermediate medium. Moreover, a first feature being "above", "over" and "on top of" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the horizontal height of the first feature is higher than that of the second feature. A first feature being "under", "below" and "beneath" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the horizontal height of the first feature is lower than that of the second feature.

[0148] In the description of this specification, the descriptions of terms such as "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples", etc., mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0149] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A collision avoidance method based on three-dimensional target detection, characterized in that: The method is implemented based on an industrial camera and a laser radar pre-deployed on a drilling platform, and the method includes: S11, acquiring an original image of a target area through the industrial camera, and acquiring point cloud data of the target area through a laser radar; the target area includes a working area of ​​an oil drilling platform; S12, inputting the original image into a preset image segmentation model, extracting pixels of all target objects in the original image, and obtaining a segmented image corresponding to each target object; The target object is a position monitoring object preset in the target area; each target object has a preset target category; S13, performing color integration on all segmented images according to the target category corresponding to each target object and a preset correspondence table to obtain a color integrated image corresponding to each target object; The correspondence table is a correspondence table between the background color used by each target object during color integration and the target category to which the target object belongs; S14, mapping all color integrated images to the point cloud data to obtain corresponding point cloud comprehensive data; S15, inputting the point cloud comprehensive data into a pre-set three-dimensional detection network to obtain the position information of all target objects in the target area to assist in determining whether there is a collision risk between any two target objects in the target area.

2. The anti-collision method based on three-dimensional target detection according to claim 1, characterized in that: The S13 includes: S13-1, obtaining an initial pixel value of each pixel position in each segmented image according to all the segmented images; S13-2, obtaining the background color pixel value corresponding to each target object according to the target category corresponding to each target object and a preset corresponding relationship table; The initial pixel value and the background pixel value both include three channels: red, green and blue; S13-3, color integration is performed on all segmented images according to the initial pixel values ​​of all pixel positions in each segmented image and the background pixel value of the target object corresponding to each segmented image, and a preset formula 1; the formula 1 is: Wherein, I is the pixel value after color integration of any pixel position on the segmented image, (R0, G0, B0) is the initial pixel value of the pixel position on the segmented image, and (R1, G1, B1) is the background color pixel value of the target object corresponding to the segmented image.

3. The anti-collision method based on three-dimensional target detection according to claim 1, characterized in that: The S12 further includes: According to the pixel coordinate value corresponding to each pixel position in the original image, the pixel coordinate values ​​corresponding to all pixel positions in each segmented image, that is, the pixel coordinate values ​​corresponding to all pixel positions in each color integrated image, are obtained; the pixel coordinate values ​​include two-dimensional image coordinates and corresponding depth values; The S14 includes: S14-1, according to the pixel coordinate values ​​corresponding to all pixel positions in each color integrated image and the preset camera coordinate conversion parameters, and the preset formula 2, obtain the three-dimensional coordinates of all pixel positions in each color integrated image in the camera coordinate system; the formula 2 is: Among them, c x 、c y 、f x and f y are all pre-set camera coordinate conversion parameters, (u, v) is the two-dimensional image coordinate corresponding to any pixel position, d is the depth value corresponding to the pixel position, and (X, Y, Z) is the three-dimensional coordinate of the pixel position in the camera coordinate system; S14-2, according to the three-dimensional coordinates of each pixel position in each color integrated image in the camera coordinate system and the preset world coordinate conversion matrix, and the preset formula three, obtain the three-dimensional coordinates of all pixel positions in each color integrated image in the world coordinate system; the formula three is: Among them, (X w ,Y w ,Z w ) is the three-dimensional coordinate of any pixel position in the world coordinate system, is the preset world coordinate transformation matrix; S14-3, based on the three-dimensional coordinates of all pixel positions in each color integrated image in the world coordinate system, aligning all color integrated images with the point cloud data, mapping them to the point cloud data, and obtaining corresponding integrated point cloud data; Any point cloud position in the comprehensive point cloud data includes corresponding first point cloud parameters; the first point cloud parameters include six channels: red, green, blue, horizontal position, vertical position and height.

4. The anti-collision method based on three-dimensional target detection according to claim 3, characterized in that: The S14-3 further includes: mapping the preset target color to a background area in the point cloud data that is not aligned with any color integrated image; The target color is black.

5. The anti-collision method based on three-dimensional target detection according to claim 1, characterized in that: The image segmentation model comprises: an image segmentation model obtained by training a pre-trained semantic segmentation model through a preset first training set; The first training set includes: at least one first training image; each training image includes at least one labeled target object; The image segmentation model is used to: Performing target detection on the received original image to obtain a corresponding target detection result; the target detection result includes a region of interest corresponding to a preset target object and a target category corresponding to each region of interest; Segmentation and pixel extraction are performed on each of the regions of interest to obtain a segmented image corresponding to each target object.

6. The anti-collision method based on three-dimensional target detection according to claim 5, characterized in that: The image segmentation model performs segmentation and pixel extraction on each region of interest to obtain a segmented image corresponding to each target object, including: According to all the regions of interest in the original image, a marked region and an unmarked region corresponding to each target object are obtained; wherein the marked region is used to mark the target object in the region of interest, and the marked region is complementary to the unmarked region; Obtaining the occluded area corresponding to each target object according to the marked area and the unmarked area in the region of interest corresponding to each target object; Segmenting and pixel extracting each of the regions of interest to obtain a process image corresponding to each target object; According to the occluded area corresponding to each target object and the preset adjacent point difference algorithm, the process image corresponding to each target object is supplemented to obtain the segmented image corresponding to each target object.

7. The anti-collision method based on three-dimensional target detection according to claim 1, characterized in that: The three-dimensional detection network is used to: According to the first point cloud parameters [R2, G2, B2, X2, Y2, Z2] corresponding to each point cloud position in the point cloud comprehensive data and the preset formula 4, the second point cloud parameters [C, X2, Y2, Z2] corresponding to each point cloud position are obtained; the formula 4 is: Among them, C is the value of the comprehensive color channel in the second point cloud parameter, R2, G2 and B2 are the values ​​of the red, green and blue channels in the first point cloud parameter, respectively, and X2, Y2 and Z2 are the values ​​of the horizontal position, vertical position and height channels respectively; According to the value of the integrated color channel of the second point cloud parameter corresponding to each point cloud position in the integrated point cloud data and a preset region segmentation algorithm, a point cloud boundary corresponding to each target object is obtained; According to the point cloud boundary corresponding to each target and the values ​​of the lateral position, longitudinal position and height channel in the second point cloud parameter corresponding to each point cloud position, the position information of each target object in the target area is obtained to assist in determining whether there is a risk of collision between any two target objects in the target area.

8. The anti-collision method based on three-dimensional target detection according to claim 7, characterized in that: The three-dimensional detection network obtains the point cloud boundary corresponding to each target object according to the value of the integrated color channel of the second point cloud parameter corresponding to each point cloud position in the point cloud integrated data and a preset region segmentation algorithm, including: According to the value of the integrated color channel of the second point cloud parameter corresponding to each point cloud position in the point cloud integrated data and a preset distinction threshold, determining whether any two adjacent point cloud positions in the point cloud integrated data are neighbors; Traverse all point cloud positions in the point cloud comprehensive data. When any point cloud position has neighbors, mark the point cloud position and its neighbors as the same potential object. When any point cloud location does not have an unlabeled neighbor, then the point cloud location is another potential object; According to the preset color threshold, the potential objects belonging to the background among all potential objects are screened and discarded; According to all point cloud positions of each potential object that does not belong to the background, the point cloud boundary corresponding to the target object is obtained.

9. The anti-collision method based on three-dimensional target detection according to claim 8, characterized in that: The three-dimensional detection network screens and discards potential objects belonging to the background among all potential objects according to a preset color threshold, including: According to the values ​​of the integrated color channels corresponding to all point cloud positions of each potential object, an average value of the integrated color channels corresponding to the potential object is obtained; When the average value of the comprehensive color channel corresponding to any potential object is less than or equal to the preset color threshold, the potential object is considered as the background and is discarded.

10. A collision avoidance system based on three-dimensional target detection, comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement a collision avoidance method based on three-dimensional target detection according to any one of claims 1 to 9.

Images