Valve body barycenter positioning method based on image processing

Abstract

A valve body barycenter positioning method based on image processing, comprising: training a semantic segmentation model capable of performing semantic segmentation on images by using a large number of pre-shot valve body sample images; obtaining a valve body color image of a valve body to be positioned in barycenter by a binocular depth camera; inputting the valve body color image into the semantic segmentation model to output a valve body semantic segmentation image of the valve body to be positioned in barycenter; performing image processing on the valve body semantic segmentation image and calculating a two-dimensional center pixel coordinate of the valve body to be positioned in barycenter; determining a depth coordinate of the valve body to be positioned in barycenter based on shooting parameters when the binocular depth camera shoots the valve body color image; and determining a three-dimensional coordinate composed of the two-dimensional center pixel coordinate and the depth coordinate as the barycenter coordinate of the valve body to be positioned in barycenter. The scheme can improve the determination accuracy of the valve body positioning reference, and further improve the cutting accuracy of the valve body, so that the quality of the cut casting is improved.

Description

Technical Field

[0001] This invention relates to the fields of valve body casting and intelligent control technology, and in particular to a valve center of gravity positioning method based on image processing. Background Technology

[0002] The valve body, a major component of valves, is primarily manufactured using casting processes due to its complex internal structure. The risers and gating systems used in casting inevitably result in excess material that needs to be cut off to facilitate subsequent machining. Currently, the cutting of risers in valve body castings is mainly done manually using hand-held abrasive wheels. This method is labor-intensive and inefficient. Furthermore, the large amounts of metal dust and fumes generated during the cutting process cause environmental pollution.

[0003] With the continuous development of intelligent manufacturing technology, industrial robots, as platforms, communicate with industrial cameras to perform functions such as identification, positioning, and grasping of target workpieces. This improves production efficiency while also protecting worker safety. Currently, when applying robots to the cutting of valve body castings, the common approach is to clamp the valve body casting onto a mechanical device, attach a cutting tool to the robot's end effector, determine the cutting trajectory through a teaching method, and then control the robot to move along the taught trajectory to complete the cutting operation. However, in practical applications, because the casting is a relatively complex blank, it is difficult to find a suitable positioning reference, resulting in low positioning accuracy. This leads to a large deviation between the actual cutting trajectory and the theoretical cutting trajectory, easily causing overcutting or undercutting, and consequently, the quality of the cut casting often fails to meet production requirements. Summary of the Invention

[0004] In view of this, and to address the above shortcomings, it is necessary to propose a valve body center-of-gravity positioning method based on image processing. This method can improve the accuracy of valve body positioning reference determination, thereby improving the accuracy of valve body cutting and thus enhancing the quality of the cut castings.

[0005] This invention provides a valve center of gravity localization method based on image processing, comprising:

[0006] Using a large number of pre-captured valve body sample images, a semantic segmentation model capable of semantic segmentation of images was trained.

[0007] Acquire a color image of the valve body to be located by the center of gravity, captured by a binocular depth camera;

[0008] The color image of the valve body is input into the semantic segmentation model, and the semantic segmentation image of the valve body to be located is output.

[0009] Image processing is performed on the semantic segmentation image of the valve body, and the two-dimensional center pixel coordinates of the valve body to be located are calculated.

[0010] Based on the shooting parameters when the binocular depth camera captures a color image of the valve body, the depth coordinates of the valve body to be located are determined.

[0011] The three-dimensional coordinates formed by the two-dimensional center pixel coordinates and the depth coordinates are determined as the center-of-gravity coordinates of the valve body to be positioned.

[0012] Preferably, the step of training a semantic segmentation model capable of semantic segmentation of images using a large number of pre-captured valve body sample images includes:

[0013] Acquire several valve body sample images captured by a binocular depth camera;

[0014] Perform the following for each valve body sample image:

[0015] The current valve body sample image is downsampled by N times to obtain a first feature map that simultaneously contains low semantic information and high semantic information; wherein, the low semantic information includes at least one of the valve body's edge information and vertex information, and the high semantic information includes the valve body's contour information;

[0016] The first feature map is upsampled by N times, and several second feature maps are added and fused to obtain the valve body semantic segmentation prediction image.

[0017] Based on the predicted semantic segmentation image of the valve body and the pre-labeled image, the cross-entropy loss function is iteratively calculated to optimize the model parameters of the semantic segmentation model; wherein, the valve body part in the label image is a positive sample, and the part outside the valve body is a negative sample.

[0018] Preferably, the image processing of the semantic segmentation image of the valve body includes:

[0019] Morphological processing is performed on the semantic segmentation image of the valve body to eliminate noise points outside the valve body portion and void points inside the valve body portion.

[0020] Preferably, the morphological processing of the semantic segmentation image of the valve body includes:

[0021] A first scanning matrix is ​​determined; wherein the dimensions of the first scan are all smaller than the matrix dimension corresponding to the semantic segmentation image of the valve body, the first scanning matrix includes a scanning origin, and the first scanning matrix is ​​a binary matrix;

[0022] The first scanning matrix is ​​used to scan the matrix of the binary valve body semantic segmentation image;

[0023] During each scan, when the elements of the first scan matrix completely correspond to the elements with a value of 1 in the matrix of the valve body semantic segmentation image it covers, the scan origin of this scan is output as 1; otherwise, it is output as 0.

[0024] After scanning such that the first scanning matrix covers every element value of the matrix of the semantic segmentation image of the valve body, the first matrix is ​​output, which is composed of the output values ​​of each scanning origin.

[0025] Determine a second scan matrix; wherein the dimensions of the second scan matrix are all smaller than the dimensions of the first matrix, the second scan matrix includes a scan origin, and the second scan matrix is ​​a binary matrix;

[0026] The first matrix is ​​scanned using the second scan matrix;

[0027] During each scan, if at least one element of the second scan matrix and the element of the first matrix it covers are both 1, then the scan origin at that time is output as 1; otherwise, it is output as 0.

[0028] After the second scanning matrix covers every element value of the first matrix, the matrix composed of the output values ​​of each scanning origin is determined as the semantic segmentation image of the valve body after image processing.

[0029] Preferably, calculating the two-dimensional center pixel coordinates of the valve body to be positioned includes:

[0030] The center pixel coordinates of the valve body to be positioned are calculated using the following formula group one:

[0031]

[0032] Where, x c and y c , i and j are the coordinates of the valve body to be located along the x-axis and y-axis, respectively, i and j are the row and column numbers of each pixel in the semantic segmentation image of the valve body, and pixel(i,j) is the pixel value of the semantic segmentation image of the valve body at position (i,j).

[0033] Preferably, determining the depth coordinates of the valve body to be located based on the shooting parameters when the binocular depth camera captures the color image of the valve body includes:

[0034] The depth coordinates are calculated using the following formula:

[0035]

[0036] Among them, zc Let be the coordinate value of the valve body to be positioned along the z-axis, B be the horizontal optical center distance between the two lenses of the binocular depth camera, f be the focal length of the binocular depth camera lens, and x1-x2 be the pixel difference between the images of the valve body to be positioned on the two lenses.

[0037] As can be seen from the above technical solution, the valve center of gravity localization method based on image processing provided in this embodiment of the invention first trains a semantic segmentation model using a large number of valve body sample images. Then, the semantic segmentation model is used to detect and output the valve body color image captured by a binocular depth camera to obtain a semantic segmentation image of the valve body to be located. Further, image processing is performed on the valve body semantic segmentation, and the two-dimensional center pixel coordinates of the valve body to be located are calculated. The depth coordinates of the valve body to be located are also calculated based on the shooting parameters when the binocular depth camera captures the color image of the valve body. Thus, the center of gravity coordinates of the valve body to be located can be obtained. Therefore, this solution determines the center of gravity coordinates of valve bodies with complex shapes through image processing. This method is not affected by the complexity of the blank casting shape and can greatly improve the accuracy of valve center of gravity determination. Furthermore, using this more accurate center of gravity coordinate as a positioning reference for trajectory planning can improve the accuracy of valve body cutting and result in higher quality cut castings. Attached Figure Description

[0038] Figure 1 A flowchart illustrating a valve center of gravity positioning method based on image processing, provided in an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of a valve center positioning device based on image processing, provided as an embodiment of the present invention. Detailed Implementation

[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] like Figure 1 As shown in the figure, this embodiment of the invention provides a valve center of gravity localization method based on image processing, which may include the following steps:

[0042] Step 101: Using a large number of pre-captured valve body sample images, train a semantic segmentation model capable of semantic segmentation of images;

[0043] Step 102: Obtain a color image of the valve body to be located by the binocular depth camera;

[0044] Step 103: Input the color image of the valve body into the semantic segmentation model, and output the semantic segmentation image of the valve body to be located.

[0045] Step 104: Perform image processing on the semantic segmentation image of the valve body and calculate the two-dimensional center pixel coordinates of the valve body to be located.

[0046] Step 105: Determine the depth coordinates of the valve body to be located based on the shooting parameters when capturing color images of the valve body using a binocular depth camera;

[0047] Step 106: Determine the three-dimensional coordinates formed by the two-dimensional center pixel coordinates and the depth coordinates as the center coordinates of the valve body to be located.

[0048] In this embodiment, image processing is used to determine the center-of-gravity coordinates of valves with complex shapes. This determination method is not affected by the complexity of the shape of the blank casting and can greatly improve the accuracy of the determination of the valve's center of gravity. Then, using this more accurate center-of-gravity coordinates as a positioning reference for trajectory planning can improve the accuracy of valve body cutting and make the quality of the cut casting higher.

[0049] In step 101, when training a semantic segmentation model capable of semantic segmentation of images using a large number of pre-captured valve body sample images, this can be achieved in the following way:

[0050] Acquire several valve body sample images captured by a binocular depth camera;

[0051] Perform the following for each valve body sample image:

[0052] The current valve body sample image is downsampled by N times to obtain a first feature map that contains both low semantic information and high semantic information; wherein, the low semantic information includes at least one of the edge information and vertex information of the valve body, and the high semantic information includes the contour information of the valve body.

[0053] The obtained first feature map is upsampled by N times, and several obtained second feature maps are fused by addition to obtain the valve body semantic segmentation prediction image;

[0054] Based on the predicted image of the valve body semantic segmentation and the pre-labeled image, the cross-entropy loss function is iteratively calculated to optimize the model parameters of the semantic segmentation model; where the valve body part in the label image is a positive sample, and the part outside the valve body is a negative sample.

[0055] In this embodiment, during model training, the labels are binary images containing only the valve body, with the valve body portion as positive samples and the rest as negative samples. When the semantic segmentation model trained in this way outputs a color image of the valve body, it produces a binary image that clearly shows the valve body's outline and structure. Furthermore, during model training, downsampling preserves low-semantic information with poor semantic meaning but rich detail, such as the valve body's edges and vertices, while simultaneously preserving high-semantic information with high semantic meaning but lacking shallow details, such as the overall valve body outline. Therefore, the semantic segmentation model trained using this method can adapt to semantic segmentation outputs for valve bodies of any complex shape, exhibiting high universality and enabling better cutting accuracy even for complex-shaped valve body castings.

[0056] For example, during model training, a large number of valve body YUV format binary streams captured by binocular depth cameras can be decoded and arranged into RGB three-channel color images to construct a semantic segmentation dataset, which can then be used to train the U-Net semantic segmentation model. The semantic segmentation network model can adopt an Encode-Decode structure, mainly composed of a ResNet50 feature extraction network, an FPN feature fusion method, and a BCE cross-entropy loss function. As shown in the U-Net model structure, after the valve body image is input into the semantic segmentation network, it is first downsampled by 16 times to obtain four low-semantic and high-semantic feature maps, then upsampled by 16 times for restoration and enhanced using an additive fusion method, finally obtaining a single-channel valve body semantic segmentation prediction image. Finally, the prediction image and the label image are iteratively calculated, and the cross-entropy loss function is calculated to update the semantic segmentation network model parameters. Specifically, the additive fusion method can be achieved by adding feature maps of the same magnification generated during the encoding and decoding process to obtain a fused feature map; of course, it can also be achieved by directly concatenating matrices.

[0057] For steps 102 and 103, consider using a binocular depth camera to capture a color image of the valve body to be located. Then, input this color image of the valve body into the semantic segmentation model trained in step 101 to output a semantic segmentation image of the valve body to be located. This semantic segmentation image of the valve body is a binary image; for example, the valve body portion is displayed in red, and the rest is displayed in black.

[0058] When processing the semantic segmentation image of the valve body in step 104, morphological processing is considered to eliminate noise points outside the valve body and void points inside the valve body, thereby eliminating the influence of noise and void points on the positioning results and improving the accuracy of determining the center of gravity of the valve body. Morphological processing methods may include dilation, erosion, opening operations, closing operations, white top cap transformation, and black top cap transformation.

[0059] Specifically, in one embodiment, morphological processing of the valve body semantic segmentation model can be achieved in the following way:

[0060] Determine a first scanning matrix; wherein the dimensions of the first scan are all smaller than the matrix dimensions corresponding to the semantic segmentation image of the valve body, the first scanning matrix includes a scanning origin, and the first scanning matrix is ​​a binary matrix;

[0061] The matrix of the binary valve body semantic segmentation image is scanned using the first scanning matrix;

[0062] During each scan, when the elements of the first scan matrix completely correspond to the elements with a value of 1 in the matrix of the valve body semantic segmentation image it covers, the scan origin of this scan is output as 1; otherwise, it is output as 0.

[0063] After scanning such that the first scanning matrix covers every element value of the matrix of the semantic segmentation image of the valve body, the output is the first matrix composed of the output values ​​of each scanning origin.

[0064] Determine the second scan matrix; wherein the dimensions of the second scan matrix are all smaller than the dimensions of the first matrix, the second scan matrix includes a scan origin, and the second scan matrix is ​​a binary matrix;

[0065] The first matrix is ​​scanned using the second scan matrix;

[0066] During each scan, if at least one element of the second scan matrix is ​​equal to 1 in the first matrix it covers, the scan origin at that time is output as 1; otherwise, it is output as 0.

[0067] After the second scanning matrix covers every element value of the first matrix, the matrix composed of the output values ​​of each scanning origin is determined as the semantic segmentation image of the valve body after image processing.

[0068] For example, the first scan matrix is And the origin of the scan is the position of the first element, and the matrix of the semantic segmentation image of the valve body is... Using the first scanning matrix, scanning is performed sequentially from the top left corner. Obviously, only when the scanning origin of the first scanning matrix is ​​located in the second row and second column of the semantic segmentation image matrix of the valve body, the element value of the semantic segmentation image matrix covered by the element with a value of 1 in the second scanning matrix is ​​also exactly 1. That is, the output of the second row and second column should be 1 at this time. Based on this, it can be seen that the output of other positions in this example is 0.

[0069] Similarly, for example, the second scan matrix is And the origin of the scan is the position of the first element, and the matrix of the semantic segmentation image of the valve body is... Using the first scanning matrix, scanning is performed sequentially from the top left corner. Obviously, the output of the semantic segmentation image matrix located at (1,1), (1,2), (1,3), (1,2), (2,1), (2,2), (2,3), (3,2), (3,3), (4,4) is 1. That is, when the scanning origin is located at these positions, there are elements covered by the second scanning matrix that have the same value of 1 as the elements in the second scanning matrix.

[0070] When calculating the two-dimensional center pixel coordinates of the valve body to be located in step 104, the following calculation formula can be used:

[0071]

[0072] For step 105, when determining the depth coordinates of the valve body to be located by the center of gravity when capturing a color image of the valve body using a binocular depth camera, the following calculation formula can be used:

[0073]

[0074] Among them, z c Let be the coordinates of the valve body to be positioned along the z-axis, B be the horizontal optical center distance between the two lenses of the binocular depth camera, f be the focal length of the binocular depth camera lens, and x1-x2 be the pixel difference between the images of the valve body to be positioned on the two lenses.

[0075] In this embodiment, the depth value is obtained using the depth acquisition principle of a stereo depth camera. Firstly, the valve body depth can be obtained from the stereo depth camera. Figure 2 The data is processed using a unit-8 encoding, which is then decoded and arranged to obtain a depth map of the valve body. The intrinsic and extrinsic parameters of the depth camera are then obtained based on the factory calibration, and the depth value of the valve body is calculated.

[0076] Specifically, the binocular depth camera has two lenses, sensor1 and sensor2, on the same horizontal plane. The optical center of sensor1 is o1, and the optical center of sensor2 is o2. The baseline distance between the two optical centers is B. P1 and P2 are the points where the valve body's location P intersects the image plane after coordinate transformation in the camera coordinate system of the two cameras, passing through the optical center. Since sensor1 and sensor2 are two lenses of the same specification, the focal length of both camera lenses is f. The lateral distance of point P1 from the edge on the image plane is x1, and the lateral distance of point P2 from the edge on the image plane is x2. Therefore, x1-x2 is the pixel difference between the images of the valve body's location P on the two lenses. Based on the principle of similar triangles, the vertical distance of point P from the camera's optical center is the required depth coordinate z. c .

[0077] Furthermore, the center pixel coordinates and depth coordinates have already been determined above. Based on these coordinates (x... c ,y c ,z c This refers to the center-of-gravity coordinates of the valve body to be positioned, which is to be determined in step 106.

[0078] Correspondingly, such as Figure 2 As shown, the present invention embodiment can also provide a valve center of gravity positioning device based on image processing, including: a model training module 201, an acquisition module 202, an output module 203, an image processing and calculation module 204, a first determination module 205, and a second determination module 206;

[0079] The model training module 201 is configured to train a semantic segmentation model capable of semantic segmentation of images using a large number of pre-captured valve body sample images.

[0080] The acquisition module 202 is configured to acquire a color image of the valve body to be located by a binocular depth camera.

[0081] Output module 203 is configured to input the color image of the valve body acquired by acquisition module 202 into the semantic segmentation model trained by model training module 201, and output the semantic segmentation image of the valve body to be located.

[0082] The image processing and calculation module 204 is configured to perform image processing on the semantic segmentation image of the valve body output by the output module 203, and calculate the two-dimensional center pixel coordinates of the valve body to be located.

[0083] The first determining module 205 is configured to determine the depth coordinates of the valve body to be located based on the shooting parameters when capturing a color image of the valve body using a binocular depth camera.

[0084] The second determining module 206 is configured to determine the three-dimensional coordinates formed by the two-dimensional center pixel coordinates obtained by the image processing and calculation module 204 and the depth coordinates determined by the first determining module 205 as the center coordinates of the valve body to be located.

[0085] In one embodiment, when the model training module 201 trains a semantic segmentation model capable of semantic segmentation of images using a large number of pre-captured valve body sample images, it is configured to perform the following operations:

[0086] Acquire several valve body sample images captured by a binocular depth camera;

[0087] Perform the following for each valve body sample image:

[0088] The current valve body sample image is downsampled by N times to obtain a first feature map that contains both low semantic information and high semantic information; wherein, the low semantic information includes at least one of the edge information and vertex information of the valve body, and the high semantic information includes the contour information of the valve body.

[0089] The obtained first feature map is upsampled by N times, and several obtained second feature maps are fused by addition to obtain the valve body semantic segmentation prediction image;

[0090] Based on the predicted image of the valve body semantic segmentation and the pre-labeled image, the cross-entropy loss function is iteratively calculated to optimize the model parameters of the semantic segmentation model; where the valve body part in the label image is a positive sample, and the part outside the valve body is a negative sample.

[0091] In one embodiment, when the image processing and calculation module 204 performs image processing on the semantic segmentation image of the valve body, it is configured to perform morphological processing on the semantic segmentation image of the valve body to eliminate noise points outside the valve body portion and void points inside the valve body portion in the semantic segmentation image of the valve body.

[0092] In one embodiment, the image processing and calculation module 204 is configured to perform the following operations when performing morphological processing on the semantic segmentation image of the valve body:

[0093] Determine a first scanning matrix; wherein the dimensions of the first scan are all smaller than the matrix dimensions corresponding to the semantic segmentation image of the valve body, the first scanning matrix includes a scanning origin, and the first scanning matrix is ​​a binary matrix;

[0094] The matrix of the binary valve body semantic segmentation image is scanned using the first scanning matrix;

[0095] During each scan, when the elements of the first scan matrix completely correspond to the elements with a value of 1 in the matrix of the valve body semantic segmentation image it covers, the scan origin of this scan is output as 1; otherwise, it is output as 0.

[0096] After scanning such that the first scanning matrix covers every element value of the matrix of the semantic segmentation image of the valve body, the output is the first matrix composed of the output values ​​of each scanning origin.

[0097] Determine the second scan matrix; wherein the dimensions of the second scan matrix are all smaller than the dimensions of the first matrix, the second scan matrix includes a scan origin, and the second scan matrix is ​​a binary matrix;

[0098] The first matrix is ​​scanned using the second scan matrix;

[0099] During each scan, if at least one element of the second scan matrix is ​​equal to 1 in the first matrix it covers, the scan origin at that time is output as 1; otherwise, it is output as 0.

[0100] After the second scanning matrix covers every element value of the first matrix, the matrix composed of the output values ​​of each scanning origin is determined as the semantic segmentation image of the valve body after image processing.

[0101] In one embodiment, when calculating the two-dimensional center pixel coordinates of the valve body to be located, the image processing and calculation module 204 is configured to perform the following operations:

[0102] The center pixel coordinates of the valve body to be positioned are calculated using the following formula group 1:

[0103]

[0104] Where, x c and y c , i and j are the coordinates of the valve body to be located along the x and y axes, respectively, i and j are the row and column numbers of each pixel in the semantic segmentation image of the valve body, and pixel(i,j) is the pixel value at position (i,j) in the semantic segmentation image of the valve body.

[0105] In one embodiment, when the first determining module 205 determines the depth coordinates of the valve body to be located based on the shooting parameters when capturing a color image of the valve body using a binocular depth camera, it is configured to perform the following operations:

[0106] Calculate the depth coordinates using the following formula:

[0107]

[0108] Among them, zc Let be the coordinates of the valve body to be positioned along the z-axis, B be the horizontal optical center distance between the two lenses of the binocular depth camera, f be the focal length of the binocular depth camera lens, and x1-x2 be the pixel difference between the images of the valve body to be positioned on the two lenses.

[0109] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method in any of the embodiments of the specification.

[0110] This specification also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the methods in any of the embodiments of the specification.

[0111] The information interaction and execution process between the various units in the above-mentioned device are based on the same concept as the method embodiments in this specification, and the specific details can be found in the descriptions in the method embodiments in this specification, so they will not be repeated here.

[0112] The modules or units in the device of this invention can be merged, divided, and deleted according to actual needs. The above-disclosed embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of the invention. Those skilled in the art will understand that implementing all or part of the processes of the above embodiments and making equivalent changes according to the claims of this invention still fall within the scope of the invention.

Claims

1. A valve center of gravity localization method based on image processing, characterized in that, include: Using a large number of pre-captured valve body sample images, a semantic segmentation model capable of semantic segmentation of images was trained. Acquire a color image of the valve body to be located by the center of gravity, captured by a binocular depth camera; The color image of the valve body is input into the semantic segmentation model, and the semantic segmentation image of the valve body to be located is output. Image processing is performed on the semantic segmentation image of the valve body, and the two-dimensional center pixel coordinates of the valve body to be located are calculated. Based on the shooting parameters when the binocular depth camera captures a color image of the valve body, the depth coordinates of the valve body to be located are determined. The three-dimensional coordinates formed by the two-dimensional center pixel coordinates and the depth coordinates are determined as the center of gravity coordinates of the valve body to be positioned. The image processing of the semantic segmentation image of the valve body includes: Morphological processing is performed on the semantic segmentation image of the valve body to eliminate noise points outside the valve body and void points inside the valve body in the semantic segmentation image of the valve body. The morphological processing of the semantic segmentation image of the valve body includes: A first scanning matrix is ​​determined; wherein the dimensions of the first scan are all smaller than the matrix dimension corresponding to the semantic segmentation image of the valve body, the first scanning matrix includes a scanning origin, and the first scanning matrix is ​​a binary matrix; The first scanning matrix is ​​used to scan the matrix of the binary valve body semantic segmentation image; During each scan, when the elements of the first scan matrix completely correspond to the elements with a value of 1 in the matrix of the valve body semantic segmentation image it covers, the scan origin of this scan is output as 1; otherwise, it is output as 0. After scanning such that the first scanning matrix covers every element value of the matrix of the semantic segmentation image of the valve body, the first matrix is ​​output, which is composed of the output values ​​of each scanning origin. Determine a second scan matrix; wherein the dimensions of the second scan matrix are all smaller than the dimensions of the first matrix, the second scan matrix includes a scan origin, and the second scan matrix is ​​a binary matrix; The first matrix is ​​scanned using the second scan matrix; During each scan, if at least one element of the second scan matrix and the element of the first matrix it covers are both 1, then the scan origin at that time is output as 1; otherwise, it is output as 0. After the second scanning matrix covers every element value of the first matrix, the matrix composed of the output values ​​of each scanning origin is determined as the semantic segmentation image of the valve body after image processing.

2. The valve center of gravity positioning method based on image processing according to claim 1, characterized in that, The semantic segmentation model, trained using a large number of pre-captured valve body sample images, capable of semantic segmentation of images, includes: Acquire several valve body sample images captured by a binocular depth camera; Perform the following for each valve body sample image: The current valve body sample image is downsampled by N times to obtain a first feature map that simultaneously contains low semantic information and high semantic information; wherein, the low semantic information includes at least one of the valve body's edge information and vertex information, and the high semantic information includes the valve body's contour information; The first feature map is upsampled by N times, and several second feature maps are added and fused to obtain the valve body semantic segmentation prediction image. Based on the predicted semantic segmentation image of the valve body and the pre-labeled image, the cross-entropy loss function is iteratively calculated to optimize the model parameters of the semantic segmentation model; wherein, the valve body part in the label image is a positive sample, and the part outside the valve body is a negative sample.

3. The valve center of gravity positioning method based on image processing according to claim 1, characterized in that, The calculation of the two-dimensional center pixel coordinates of the valve body to be positioned includes: The center pixel coordinates of the valve body to be positioned are calculated using the following formula group one: Where, x c and y c , i and j are the coordinates of the valve body to be located along the x-axis and y-axis, respectively, i and j are the row and column numbers of each pixel in the semantic segmentation image of the valve body, and pixel(i,j) is the pixel value of the semantic segmentation image of the valve body at position (i,j).

4. The valve center of gravity positioning method based on image processing according to any one of claims 1 to 3, characterized in that, The process of determining the depth coordinates of the valve body to be located based on the shooting parameters when capturing a color image of the valve body using the binocular depth camera includes: The depth coordinates are calculated using the following formula: Among them, z c Let be the coordinate value of the valve body to be positioned along the z-axis, B be the horizontal optical center distance between the two lenses of the binocular depth camera, f be the focal length of the binocular depth camera lens, and x1-x2 be the pixel difference between the images of the valve body to be positioned on the two lenses.

Images